Mantis - Quercus
Viewing Issue Advanced Details
1719 crash always 04-27-07 08:44 04-29-07 02:53
salishev  
nam  
normal  
closed 3.1.1  
fixed  
none    
none 3.1.1  
0001719: Thread race when com.caucho.quercus.env.DefinitionState
I've checked with latest resin-3.1.s070425 snapshot

The following exception is always thrown when requesting 64K pages from 2 or more threads simultanously

[19:31:33.820] java.util.ConcurrentModificationException
[19:31:33.820] at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
[19:31:33.820] at java.util.HashMap$EntryIterator.next(HashMap.java:833)
[19:31:33.820] at java.util.HashMap$EntryIterator.next(HashMap.java:831)
[19:31:33.820] at java.util.HashMap.putAll(HashMap.java:523)
[19:31:33.820] at com.caucho.quercus.env.DefinitionState.<init>(DefinitionState.java:98)
[19:31:33.820] at com.caucho.quercus.env.DefinitionState.copy(DefinitionState.java:357)
[19:31:33.820] at com.caucho.quercus.Quercus.putDefinitionCache(Quercus.java:673)
[19:31:33.820] at com.caucho.quercus.env.Env.importPage(Env.java:3317)
[19:31:33.820] at com.caucho.quercus.env.Env.include(Env.java:3295)
[19:31:33.820] at com.caucho.quercus.expr.IncludeExpr.eval(IncludeExpr.java:86)
[19:31:33.820] at com.caucho.quercus.program.ExprStatement.execute(ExprStatement.java:64)
[19:31:33.820] at com.caucho.quercus.program.BlockStatement.execute(BlockStatement.java:99)
[19:31:33.820] at com.caucho.quercus.program.Function.callImpl(Function.java:316)
[19:31:33.820] at com.caucho.quercus.program.Function.call(Function.java:264)
[19:31:33.820] at com.caucho.quercus.expr.FunctionExpr.evalImpl(FunctionExpr.java:180)
[19:31:33.820] at com.caucho.quercus.expr.FunctionExpr.eval(FunctionExpr.java:125)
[19:31:33.820] at com.caucho.quercus.program.ExprStatement.execute(ExprStatement.java:64)
[19:31:33.820] at com.caucho.quercus.program.BlockStatement.execute(BlockStatement.java:99)
[19:31:33.820] at com.caucho.quercus.program.QuercusProgram.execute(QuercusProgram.java:239)
[19:31:33.820] at com.caucho.quercus.page.InterpretedPage.execute(InterpretedPage.java:61)
[19:31:33.820] at com.caucho.quercus.page.QuercusPage.executeTop(QuercusPage.java:115)
[19:31:33.820] at com.caucho.quercus.servlet.ResinQuercusServlet.service(ResinQuercusServlet.java:125)
[19:31:33.820] at com.caucho.quercus.servlet.QuercusServlet.service(QuercusServlet.java:341)
[19:31:33.820] at javax.servlet.http.HttpServlet.service(HttpServlet.java:92)
[19:31:33.820] at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
[19:31:33.820] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:175)
[19:31:33.820] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:240)
[19:31:33.820] at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:263)
[19:31:33.820] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:477)
[19:31:33.820] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:600)
[19:31:33.820] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:522)
[19:31:33.820] at java.lang.Thread.run(Thread.java:619)
Can't provide the test case as it's proprietary software.

It uses DokuWiki as a backend and creates an exhaustive multi-threaded request stream through http with page size of 64K.

Synchronizing access to com.caucho.quercus.env.DefinitionState solves the problem.

Notes
(0001873)
salishev   
04-28-07 04:10   
The root cause is incorrect implementation of copyLazy. It supposes the original is immutable which is not true. So it's possible to concurrently mutate the original while creating non lazy copy in copy/copyOnWrite method.

So copyLazy should ensure the original is lazy.
(0001874)
salishev   
04-28-07 04:13   
Also Method findFunction mutates the object without creating the lazy copy.
(0001877)
nam   
04-29-07 02:53   
Thanks for the detailed report. DefinitionState.findFunction() was missing the copyOnWrite()'s, thus allowing a lazy copy to modify the original's HashMap.