Mantis Bugtracker
  

Viewing Issue Simple Details Jump to Notes ] View Advanced ] Issue History ] Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0002728 [Resin] major always 06-09-08 11:59 06-11-08 21:35
Reporter paulberto View Status public  
Assigned To ferg
Priority normal Resolution fixed  
Status closed   Product Version 3.1.6
Summary 0002728: Running latest snapshot -- cache stops working after InterruptedIOException
Description After which, the load on the machine skyrockets since cache is disabled.

[21:50:17.369] Caused by: java.io.InterruptedIOException: timeout fd=1395 errno=11
[21:50:17.369]
[21:50:17.369] at com.caucho.vfs.JniSocketImpl.writeNative(Native Method)
[21:50:17.369] at com.caucho.vfs.JniSocketImpl.write(JniSocketImpl.java:288)
[21:50:17.369] at com.caucho.vfs.JniStream.write(JniStream.java:124)
[21:50:17.369] at com.caucho.vfs.WriteStream.nextBuffer(WriteStream.java:327)
[21:50:17.369] at com.caucho.server.connection.ResponseStream.writeNext(ResponseStream.java:480)
[21:50:17.369] at com.caucho.server.connection.ToByteResponseStream.write(ToByteResponseStream.java:336)
[21:50:17.369] at com.caucho.vfs.VfsStream.write(VfsStream.java:201)
[21:50:17.369] at com.caucho.vfs.WriteStream.print(WriteStream.java:553)
[21:50:17.369] at com.caucho.vfs.WriteStream.print(WriteStream.java:631)
[21:50:17.369] at com.caucho.quercus.program.TextStatement.execute(TextStatement.java:63)
[21:50:17.369] ... 26 more



Additional Information I'm being forced to monitor the machines in question and restart them when this is occuring. Please issue fix!!!! I am running resin-pro-3.1.s080503 SNAPSHOT in an attempt to get the fix from a previous bug ID 0002704.

Please advise
Attached Files

- Relationships

- Notes
(0003154)
paulberto
06-09-08 12:19

sorry here is the real log error:

[2008/06/09 22:12:08.800] java.io.InterruptedIOException: timeout fd=134 errno=11
[2008/06/09 22:12:08.800]
[2008/06/09 22:12:08.800] at com.caucho.vfs.JniSocketImpl.writeNative(Native Method)
[2008/06/09 22:12:08.800] at com.caucho.vfs.JniSocketImpl.write(JniSocketImpl.java:288)
[2008/06/09 22:12:08.800] at com.caucho.vfs.JniStream.write(JniStream.java:124)
[2008/06/09 22:12:08.800] at com.caucho.vfs.WriteStream.nextBuffer(WriteStream.java:327)
[2008/06/09 22:12:08.800] at com.caucho.server.connection.ResponseStream.nextBuffer(ResponseStream.java:324)
[2008/06/09 22:12:08.800] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:102)
[2008/06/09 22:12:08.800] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:79)
[2008/06/09 22:12:08.800] at com.caucho.server.cache.CacheFilterChain.fillFromCache(CacheFilterChain.java:530)
[2008/06/09 22:12:08.800] at com.caucho.server.cache.CacheFilterChain.useCache(CacheFilterChain.java:301)
[2008/06/09 22:12:08.800] at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:174)
[2008/06/09 22:12:08.800] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
[2008/06/09 22:12:08.800] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
[2008/06/09 22:12:08.800] at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:270)
[2008/06/09 22:12:08.800] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:678)
[2008/06/09 22:12:08.800] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
[2008/06/09 22:12:08.800] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
[2008/06/09 22:12:08.800] at java.lang.Thread.run(Thread.java:619)
[2008/06/09 22:12:08.801] cache disabled due to java.io.InterruptedIOException: timeout fd=134 errno=11
 
(0003155)
paulberto
06-09-08 15:26

This is a show stopper for my site. We rely on the cache heavily and once it stops working the site dies. It almost spirals out of control because of the amount of work it performs when the caching gets automatically disabled. So much so that even the new feature which attempts to kill resin on high loads doesn't occur.

Would it be possible to get the latest snapshot with the bug fix committed (reported in 0002704) or any suggestions as to how we can workaround it in the meantime ?
 
License #: 1007371
 
(0003156)
ferg
06-09-08 16:27

snapshots are not supposed to be used for production code. They're essentially early alpha code. It's possible to release a new snapshot, but it will still be an alpha snapshot.
 
(0003158)
paulberto
06-09-08 17:19

I cant revert to 3.1.5 because of a quercus issue and 3.1.6 seems to have this issue. I don't mind giving the snapshot a whirl. Unless of course, you can suggest a work around for this issue in 3.1.6 ?
 
(0003159)
ferg
06-09-08 18:51

You can try remove libresin.so. That would change the interruption to something the cache can handle.
 
(0003160)
paulberto
06-10-08 09:09

Thanks! Seems to have had a positive effect.
 
(0003161)
paulberto
06-10-08 14:33

Under low traffic removing libresin.so helped tremendously but under high load (right now) we're having the same issue and it's bringing us down every few minutes.

The current snapshot release i'm running seems to be much more stable than 3.1.5 --- we have a big issue with rewrites (rewrite-dispatch) not working properly in 3.1.6 so i have to stick with the snapshot.

In any case, would it be possible to get the alpha snapshot discussed yesterday -- i really need it at this point. Understanding it is alpha i can revert back to any version from a click of a button if i deem it to unstable.

Thanks Ferg!
 
(0003162)
paulberto
06-10-08 14:35

Sorry disregard -- i see that you have published new snapshot.. gona give it a whirl !!!! thanks!!!!!!
 
(0003164)
paulberto
06-10-08 16:42

Still happening using latest snapshot build (s080609):

[2008/06/11 02:28:03.054] java.io.InterruptedIOException: timeout fd=2494 errno=11
[2008/06/11 02:28:03.054]
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniSocketImpl.writeNative(Native Method)
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniSocketImpl.write(JniSocketImpl.java:288)
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniStream.write(JniStream.java:124)
[2008/06/11 02:28:03.054] at com.caucho.vfs.WriteStream.nextBuffer(WriteStream.java:327)
[2008/06/11 02:28:03.054] at com.caucho.server.connection.ResponseStream.nextBuffer(ResponseStream.java:324)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:102)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:79)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.fillFromCache(CacheFilterChain.java:530)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.useCache(CacheFilterChain.java:301)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:174)
[2008/06/11 02:28:03.054] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
[2008/06/11 02:28:03.054] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
[2008/06/11 02:28:03.054] at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:270)
[2008/06/11 02:28:03.054] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:678)
[2008/06/11 02:28:03.054] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
[2008/06/11 02:28:03.054] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
[2008/06/11 02:28:03.054] at java.lang.Thread.run(Thread.java:619)

Any suggestions ? Should i remove the JNI library again -- will that help ?
 
(0003165)
paulberto
06-10-08 16:43

Sorry was cut off:

[2008/06/11 02:28:03.054] java.io.InterruptedIOException: timeout fd=2494 errno=11
[2008/06/11 02:28:03.054]
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniSocketImpl.writeNative(Native Method)
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniSocketImpl.write(JniSocketImpl.java:288)
[2008/06/11 02:28:03.054] at com.caucho.vfs.JniStream.write(JniStream.java:124)
[2008/06/11 02:28:03.054] at com.caucho.vfs.WriteStream.nextBuffer(WriteStream.java:327)
[2008/06/11 02:28:03.054] at com.caucho.server.connection.ResponseStream.nextBuffer(ResponseStream.java:324)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:102)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheInode.writeToStream(CacheInode.java:79)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.fillFromCache(CacheFilterChain.java:530)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.useCache(CacheFilterChain.java:301)
[2008/06/11 02:28:03.054] at com.caucho.server.cache.CacheFilterChain.doFilter(CacheFilterChain.java:174)
[2008/06/11 02:28:03.054] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
[2008/06/11 02:28:03.054] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
[2008/06/11 02:28:03.054] at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:270)
[2008/06/11 02:28:03.054] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:678)
[2008/06/11 02:28:03.054] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721)
[2008/06/11 02:28:03.054] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643)
[2008/06/11 02:28:03.054] at java.lang.Thread.run(Thread.java:619)
[2008/06/11 02:28:03.055] not-modified: /camcom.css
[2008/06/11 02:28:03.056] cache disabled due to java.io.InterruptedIOException: timeout fd=2494 errno=11
 
(0003166)
ferg
06-11-08 08:06

There's a new snapshot. The previous one did not include the InterruptedIOException fixes.
 

- Issue History
Date Modified Username Field Change
06-09-08 11:59 paulberto New Issue
06-09-08 12:19 paulberto Note Added: 0003154
06-09-08 15:26 paulberto Note Added: 0003155
06-09-08 16:27 ferg Note Added: 0003156
06-09-08 17:19 paulberto Note Added: 0003158
06-09-08 18:51 ferg Note Added: 0003159
06-10-08 09:09 paulberto Note Added: 0003160
06-10-08 09:53 paulberto Issue Monitored: paulberto
06-10-08 14:33 paulberto Note Added: 0003161
06-10-08 14:35 paulberto Note Added: 0003162
06-10-08 16:42 paulberto Note Added: 0003164
06-10-08 16:43 paulberto Note Added: 0003165
06-11-08 08:06 ferg Note Added: 0003166
06-11-08 21:35 ferg Assigned To  => ferg
06-11-08 21:35 ferg Status new => closed
06-11-08 21:35 ferg Resolution open => fixed
06-11-08 21:35 ferg Fixed in Version  => 3.1.7
06-11-08 21:35 ferg Description Updated


Mantis 1.0.0rc3[^]
Copyright © 2000 - 2005 Mantis Group
51 total queries executed.
36 unique queries executed.
Powered by Mantis Bugtracker