Mantis Bugtracker
Main | My View | View Issues | Change Log | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ View Advanced ] [ Issue History ] [ Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0000113 [Resin] minor always 04-20-05 00:00 11-30-05 14:42
Reporter sam View Status public  
Assigned To
Priority urgent Resolution fixed  
Status closed   Product Version 3.0.12
Summary 0000113: cluster-store taking up too much space
Description RSN-104
> Having completed the upgrade, when the site is working, it's working
> wonderfully, however we're having a number of teething problems...
>
> Teething problem 1:
> -------------------
> We were using the old <tcp-store> in our old session config stuff and
> have migrated to the new persistant store.
>
> The config we have is:
> <server>
> <cluster>
> <srun server-id="a" host="10.0.1.1" port="6803" index="1"/>
> <srun server-id="b" host="10.0.1.2" port="6803" index="2"/>
> <srun server-id="c" host="10.0.1.3" port="6803" index="3"/>
> <srun server-id="d" host="10.0.1.4" port="6803" index="4"/>
> </cluster>
> <persistent-store type="cluster">
> <init path="/home/resin-sessions/cluster"/>
> </persistent-store>
> <web-app-default>
> <session-config>
> <session-max>4096</session-max>
> <session-timeout>240</session-timeout>
> <enable-cookies>true</enable-cookies>
> <enable-url-rewriting>true</enable-url-rewriting>
> <cookie-max-age>9999999999</cookie-max-age>
> <cookie-domain>.moneyam.com</cookie-domain>
> <use-persistent-store/>
> </session-config>
> </web-app-default>
> </server>
>
> (+ lots of other stuff, but that's the immediately relevent sections)
>
> Now, on each of our application servers we have the following:
>
> resin@appserv3:/usr/local/resin-3.0$ du -H /home/resin-sessions/
> 5.5G /home/resin-sessions/cluster
> 5.5G /home/resin-sessions
>
> 5.5 gigabytes of information.
>
> This is causing a number of problems:
> 1) disk space. 5.5Gb * 4 servers. We're running out of disk space for
> this data.
> 2) startup times. It's taking an inordinate amount of time to start
> resin, I guess due to it having to load in some of that session data.
>
> Comparing this to the session store that we had with our resin-2.1.14
> config:
>
> resin@appserv3:webapps/shares/WEB-INF$ du -H sessions/ | tail -1
> 37M sessions
>
> 37 Mb, vs 5.5Gb.
>
> What have we done wrong?
>
> What can I do to resolve this?
>
> Every few days I'm having to stop everything (web servers) trash the
> session store and start again, else it's taking an age if we have to
> restart an application server.
>
> Teething problem 2:
> -------------------
> This morning I had loads of errors like this in our log files:
> [08:15:58.030] at
> com.caucho.db.jdbc.PreparedStatementImpl.executeUpdate(PreparedStatementImpl.java:325)
> [08:15:58.030] at
> com.caucho.server.cluster.ClusterStore.updateAccess(ClusterStore.java:932)
> [08:15:58.030] at
> com.caucho.server.hmux.HmuxClusterRequest.accessObject(HmuxClusterRequest.java:536)
> [08:15:58.030] at
> com.caucho.server.hmux.HmuxClusterRequest.handleRequest(HmuxClusterRequest.java:269)
> [08:15:58.030] at
> com.caucho.server.hmux.HmuxRequest.scanHeaders(HmuxRequest.java:604)
> [08:15:58.030] at
> com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:337)
> [08:15:58.030] at
> com.caucho.server.port.TcpConnection.run(TcpConnection.java:341)
> [08:15:58.030] at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:467)
> [08:15:58.030] at com.caucho.util.ThreadPool.run(ThreadPool.java:408)
> [08:15:58.030] at java.lang.Thread.run(Thread.java:595)
> [08:15:58.032] java.sql.SQLException: transaction timed out waiting for
> lock 510
> [08:15:58.032] at com.caucho.db.store.Lock.queue(Lock.java:348)
> [08:15:58.032] at com.caucho.db.store.Lock.queue(Lock.java:315)
> [08:15:58.032] at
> com.caucho.db.store.Lock.lockReadAndUpgrade(Lock.java:190)
> [08:15:58.032] at
> com.caucho.db.store.Transaction.lockAutoCommitWrite(Transaction.java:237)
> [08:15:58.032] at
> com.caucho.db.sql.UpdateQuery.execute(UpdateQuery.java:106)
> [08:15:58.032] at
> com.caucho.db.jdbc.PreparedStatementImpl.execute(PreparedStatementImpl.java:349)
> [08:15:58.032] at
> com.caucho.db.jdbc.PreparedStatementImpl.executeUpdate(PreparedStatementImpl.java:325)
> [08:15:58.032] at
> com.caucho.server.cluster.ClusterStore.updateAccess(ClusterStore.java:932)
> [08:15:58.032] at
> com.caucho.server.hmux.HmuxClusterRequest.accessObject(HmuxClusterRequest.java:536)
Additional Information
Attached Files

- Relationships

- Notes
(0000131)
ferg
04-20-05 00:00

 db/002-
 
(0000132)
user201
04-20-05 00:00

 Hello.

I upgraded our main site to resin-pro-3.0.13 last night, came in the morning and I'm finding a number of errors like this:

[09:35:48.928] java.lang.NullPointerException
[09:35:48.928] at com.caucho.db.jdbc.AbstractResultSet.getBinaryStream(AbstractResultSet.java:481)
[09:35:48.928] at com.caucho.server.cluster.FileBacking.loadSelf(FileBacking.java:311)
[09:35:48.928] at com.caucho.server.cluster.ClusterStore.load(ClusterStore.java:404)
[09:35:48.928] at com.caucho.server.cluster.ClusterObject.load(ClusterObject.java:249)
[09:35:48.928] at com.caucho.server.session.SessionImpl.load(SessionImpl.java:705)
[09:35:48.928] at com.caucho.server.session.SessionManager.load(SessionManager.java:1150)
[09:35:48.928] at com.caucho.server.session.SessionManager.getSession(SessionManager.java:1012)
[09:35:48.928] at com.caucho.server.connection.AbstractHttpRequest.createSession(AbstractHttpRequest.java:1363)
[09:35:48.928] at com.caucho.server.connection.AbstractHttpRequest.getSession(AbstractHttpRequest.java:1173)
[09:35:48.928] at com.caucho.server.connection.AbstractHttpRequest.getSession(AbstractHttpRequest.java:1151)
[09:35:48.928] at org.apache.struts.util.SecureRequestUtils.reclaimRequestAttributes(SecureRequestUtils.java:369)
[09:35:48.928] at org.apache.struts.util.SecureRequestUtils.getRedirectString(SecureRequestUtils.java:432)
[09:35:48.928] at org.apache.struts.action.SecureActionServlet.checkSsl(SecureActionServlet.java:110)
[09:35:48.928] at org.apache.struts.action.SecureActionServlet.process(SecureActionServlet.java:93)
[09:35:48.928] at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:492)
[09:35:48.928] at javax.servlet.http.HttpServlet.service(HttpServlet.java:113)
[09:35:48.928] at javax.servlet.http.HttpServlet.service(HttpServlet.java:90)
[09:35:48.928] at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:99)
[09:35:48.928] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:163)
[09:35:48.928] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:208)
[09:35:48.928] at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:396)
[09:35:48.928] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:341)
[09:35:48.928] at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
[09:35:48.928] at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
[09:35:48.928] at java.lang.Thread.run(Thread.java:595)


Also, in the 12 hours since the upgrade occurred, the disk usage used by the session data currently stands at 300Mb.

This is not expected behaviour and I don't think the problem has been resolved.
 
(0000133)
flausa
04-20-05 00:00

 Is this issue realy solved???!! I'm runing 3.0.14 and exeperience the same behaviour. I have a cluster with two instances on the same machine and each of the session "database" is 6.1G ??!! What is the solution to this problem?

-rw-rw-r-- 1 resin resin 6.1G okt 6 09:22 srun_a.db
-rw-rw-r-- 1 resin resin 6.1G okt 6 09:22 srun_b.db
 

- Issue History
Date Modified Username Field Change
04-20-05 00:00 sam New Issue
11-30-05 00:00 administrator Fixed in Version  => 3.0.14
11-30-05 14:42 ferg Status resolved => closed

Mantis 1.0.0rc3[^]
Copyright © 2000 - 2005 Mantis Group
34 total queries executed.
31 unique queries executed.
Powered by Mantis Bugtracker