| Anonymous | Login | Signup for a new account | 04-25-2024 02:34 PDT |
| Main | My View | View Issues | Change Log | Docs |
| Viewing Issue Simple Details [ Jump to Notes ] | [ View Advanced ] [ Issue History ] [ Print ] | ||||||||
| ID | Category | Severity | Reproducibility | Date Submitted | Last Update | ||||
| 0004499 | [Resin] | minor | always | 04-12-11 14:26 | 06-02-11 17:37 | ||||
| Reporter | ferg | View Status | public | ||||||
| Assigned To | ferg | ||||||||
| Priority | normal | Resolution | fixed | ||||||
| Status | closed | Product Version | 4.0.14 | ||||||
| Summary | 0004499: bam/heartbeat synchronization | ||||||||
| Description |
(rep by William Kong) Last night, our system produced these where exception messages. We have two load balanced servers running. Here's what I did: 1. Login / Authenticate into our Web Application. (i.e. Our logic to authenticate with the user using Resin and Oracle Database.) 2. Authenticated successfully and redirected to our landing page. 3. <Error occurs here> Search for an account (i.e. Our logic to search accounts using Resin Oracle Database.) I waited and waited and eventually get a one liner Services Not Available (sorry, don't have a screen shot). 4. <Debugging Steps> I went to look at both server logs and found the exception below. 5. <Debugging Steps> I just stop only one of the Resin Web Server 6. <Debugging Steps> I go back to step 3 above to hit the refresh button to resubmit my post request. However, the application re-routes to be the login page. From the step, I assuming there was a session load balancing issue. 7. <Debugging Steps> I did not restart the other resin server instance and it seems like I have not gotten any new complaints similar to below: Is there a session load balancing issue? What can I do to prevent this issue from re-occurring again? [04/12/11 00:35:26] [11-04-12 00:35:26.155] {server://10.8.9.28:6800-431} [^] com.caucho.bam.TimeoutException: com.caucho.cloud.bam.TriadFuture@1a8c6fdb query timeout: 60000 to: cluster-cache@aaa.estmt.admin.resin query: CacheGet[key=88ca0aae,version=0,server=0] at com.caucho.cloud.bam.TriadFuture.get(TriadFuture.java:65) at com.caucho.cloud.bam.TriadFirstQuery.get(TriadFirstQuery.java:179) at com.caucho.cloud.bam.BamTriadSender.queryTriadFirstRemote(BamTriadSender.java:642) at com.caucho.server.distcache.CacheMnodeActor.get(CacheMnodeActor.java:126) at com.caucho.server.distcache.CacheMnodeManager.get(CacheMnodeManager.java:137) at com.caucho.server.distcache.CacheClusterBackingImpl.loadClusterValue(CacheClusterBackingImpl.java:53) at com.caucho.server.distcache.AbstractCacheManager.loadExpiredValue(AbstractCacheManager.java:311) at com.caucho.server.distcache.AbstractCacheManager.reloadValue(AbstractCacheManager.java:271) at com.caucho.server.distcache.AbstractCacheManager.getMnodeValue(AbstractCacheManager.java:247) at com.caucho.server.distcache.ProCacheEntry.getMnodeValue(ProCacheEntry.java:106) at com.caucho.distcache.AbstractCache.getExtCacheEntry(AbstractCache.java:496) at com.caucho.server.session.SessionImpl.load(SessionImpl.java:611) at com.caucho.server.session.SessionManager.createSession(SessionManager.java:1308) at com.caucho.server.http.AbstractCauchoRequest.createSession(AbstractCauchoRequest.java:255) at com.caucho.server.http.AbstractCauchoRequest.getSession(AbstractCauchoRequest.java:198) at _jsp._foo__jsp._jspService(_acctdetails__jsp.java:160) at com.caucho.jsp.JavaPage.service(JavaPage.java:64) at com.caucho.jsp.Page.pageservice(Page.java:542) at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194) at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:184) at com.caucho.server.cache.ProxyCacheFilterChain.doFilter(ProxyCacheFilterChain.java:162) at com.caucho.server.webapp.AccessLogFilterChain.doFilter(AccessLogFilterChain.java:95) at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:287) at com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468) at com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:373) at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:339) at com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:723) at com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:685) at com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:665) at com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:613) at com.caucho.network.listen.AcceptTask.doTask(AcceptTask.java:104) at com.caucho.network.listen.ConnectionReadTask.runThread(ConnectionReadTask.java:98) at com.caucho.network.listen.ConnectionReadTask.run(ConnectionReadTask.java:81) at com.caucho.network.listen.AcceptTask.run(AcceptTask.java:67) at com.caucho.env.thread.ResinThread.runTasks(ResinThread.java:164) at com.caucho.env.thread.ResinThread.run(ResinThread.java:130) [04/12/11 00:36:41] [11-04-12 00:36:41.191] {server://10.8.9.28:6800-466} [^] com.caucho.bam.TimeoutException: com.caucho.cloud.bam.TriadFuture@4ecf3c56 query timeout: 60000 to: cluster-cache@aaa.estmt.admin.resin query: CacheGet[key=88ca0aae,version=0,server=0] at com.caucho.cloud.bam.TriadFuture.get(TriadFuture.java:65) at com.caucho.cloud.bam.TriadFirstQuery.get(TriadFirstQuery.java:179) at com.caucho.cloud.bam.BamTriadSender.queryTriadFirstRemote(BamTriadSender.java:642) at com.caucho.server.distcache.CacheMnodeActor.get(CacheMnodeActor.java:126) at com.caucho.server.distcache.CacheMnodeManager.get(CacheMnodeManager.java:137) at com.caucho.server.distcache.CacheClusterBackingImpl.loadClusterValue(CacheClusterBackingImpl.java:53) at com.caucho.server.distcache.AbstractCacheManager.loadExpiredValue(AbstractCacheManager.java:311) at com.caucho.server.distcache.AbstractCacheManager.reloadValue(AbstractCacheManager.java:271) at com.caucho.server.distcache.AbstractCacheManager.getMnodeValue(AbstractCacheManager.java:247) at com.caucho.server.distcache.ProCacheEntry.getMnodeValue(ProCacheEntry.java:106) at com.caucho.distcache.AbstractCache.getExtCacheEntry(AbstractCache.java:496) at com.caucho.server.session.SessionImpl.load(SessionImpl.java:611) at com.caucho.server.session.SessionManager.createSession(SessionManager.java:1308) at com.caucho.server.http.AbstractCauchoRequest.createSession(AbstractCauchoRequest.java:255) at com.caucho.server.http.AbstractCauchoRequest.getSession(AbstractCauchoRequest.java:198) at _jsp._foo__jsp._jspService(_acctdetails__jsp.java:160) at com.caucho.jsp.JavaPage.service(JavaPage.java:64) at com.caucho.jsp.Page.pageservice(Page.java:542) at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194) at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:184) at com.caucho.server.cache.ProxyCacheFilterChain.doFilter(ProxyCacheFilterChain.java:162) at com.caucho.server.webapp.AccessLogFilterChain.doFilter(AccessLogFilterChain.java:95) at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:287) at com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468) at com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:373) at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:339) at com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:723) at com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:685) at com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:665) at com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:613) at com.caucho.network.listen.AcceptTask.doTask(AcceptTask.java:104) at com.caucho.network.listen.ConnectionReadTask.runThread(ConnectionReadTask.java:98) at com.caucho.network.listen.ConnectionReadTask.run(ConnectionReadTask.java:81) at com.caucho.network.listen.AcceptTask.run(AcceptTask.java:67) at com.caucho.env.thread.ResinThread.runTasks(ResinThread.java:164) at com.caucho.env.thread.ResinThread.run(ResinThread.java:130) [04/12/11 00:37:56] [11-04-12 00:37:56.227] {server://10.8.9.28:6800-467} [^] com.caucho.bam.TimeoutException: com.caucho.cloud.bam.TriadFuture@d2b24c7 query timeout: 60000 to: cluster-cache@aaa.foo.admin.resin query: CacheGet[key=88ca0aae,version=0,server=0] at com.caucho.cloud.bam.TriadFuture.get(TriadFuture.java:65) at com.caucho.cloud.bam.TriadFirstQuery.get(TriadFirstQuery.java:179) at com.caucho.cloud.bam.BamTriadSender.queryTriadFirstRemote(BamTriadSender.java:642) at com.caucho.server.distcache.CacheMnodeActor.get(CacheMnodeActor.java:126) at com.caucho.server.distcache.CacheMnodeManager.get(CacheMnodeManager.java:137) at com.caucho.server.distcache.CacheClusterBackingImpl.loadClusterValue(CacheClusterBackingImpl.java:53) at com.caucho.server.distcache.AbstractCacheManager.loadExpiredValue(AbstractCacheManager.java:311) at com.caucho.server.distcache.AbstractCacheManager.reloadValue(AbstractCacheManager.java:271) at com.caucho.server.distcache.AbstractCacheManager.getMnodeValue(AbstractCacheManager.java:247) at com.caucho.server.distcache.ProCacheEntry.getMnodeValue(ProCacheEntry.java:106) at com.caucho.distcache.AbstractCache.getExtCacheEntry(AbstractCache.java:496) at com.caucho.server.session.SessionImpl.load(SessionImpl.java:611) at com.caucho.server.session.SessionManager.createSession(SessionManager.java:1308) at com.caucho.server.http.AbstractCauchoRequest.createSession(AbstractCauchoRequest.java:255) at com.caucho.server.http.AbstractCauchoRequest.getSession(AbstractCauchoRequest.java:198) at _jsp._statements._acctdetails__jsp._jspService(_acctdetails__jsp.java:160) at com.caucho.jsp.JavaPage.service(JavaPage.java:64) at com.caucho.jsp.Page.pageservice(Page.java:542) at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194) at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:184) at com.caucho.server.cache.ProxyCacheFilterChain.doFilter(ProxyCacheFilterChain.java:162) at com.caucho.server.webapp.AccessLogFilterChain.doFilter(AccessLogFilterChain.java:95) at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:287) at com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:468) at com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:373) at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:339) at com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:723) at com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:685) at com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:665) at com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:613) at com.caucho.network.listen.AcceptTask.doTask(AcceptTask.java:104) at com.caucho.network.listen.ConnectionReadTask.runThread(ConnectionReadTask.java:98) at com.caucho.network.listen.ConnectionReadTask.run(ConnectionReadTask.java:81) |
||||||||
| Additional Information | |||||||||
| Attached Files | |||||||||
|
|
|||||||||
 Relationships |
|
|
Notes |
|
|
(0005290) ferg 06-02-11 17:37 |
cloud/60o1 |
|
Issue History |
|||
| Date Modified | Username | Field | Change |
| 04-12-11 14:26 | ferg | New Issue | |
| 06-02-11 17:37 | ferg | Note Added: 0005290 | |
| 06-02-11 17:37 | ferg | Assigned To | => ferg |
| 06-02-11 17:37 | ferg | Status | new => closed |
| 06-02-11 17:37 | ferg | Resolution | open => fixed |
| 06-02-11 17:37 | ferg | Fixed in Version | => 4.0.19 |
| Mantis 1.0.0rc3[^]
Copyright © 2000 - 2005 Mantis Group
28 total queries executed. 25 unique queries executed. |  |
