0004300: Wachdog does not stop

Viewing Issue Simple Details [ Jump to Notes ]

[ View Advanced ] [ Issue History ] [ Print ]

Category

Severity

Reproducibility

Date Submitted

Last Update

0004300

[Resin]

major

always

11-18-10 09:54

11-18-10 12:38

Reporter

reza

View Status

public

Assigned To

ferg

Priority

high

Resolution

fixed

Status

closed

Product Version

4.0.12

Summary

0004300: Wachdog does not stop

Description

a) Watchdog cannot stop it's processes (well over 5 minutes after a 'stop')
b) Watchdog cannot start because of lingering watchdog processes

This is manifested as traffic increased to the site.

Here are some log examples:
================================================================================
[2010/11/17 05:41:50.875] Server[id=,cluster=] active
[2010/11/17 05:41:50.921] WebApp[http://resin-admin] [^] active
[2010/11/17 05:41:50.927] servlet-mapping /watchdog -> watchdog
[2010/11/17 05:41:50.957] WatchdogProcess[Watchdog[resin1],1] starting Resin
[2010/11/17 05:41:51.301] WatchdogProcess[Watchdog[web-entry-1],1] starting Resin
[2010/11/17 16:35:34.568] Watchdog stop: resin1
[2010/11/17 16:35:34.569] WatchdogProcess[Watchdog[resin1],1] stopping Resin
[2010/11/17 16:35:36.141] Watchdog stop: web-entry-1
[2010/11/17 16:35:36.142] WatchdogProcess[Watchdog[web-entry-1],1] stopping Resin
[2010/11/17 16:36:02.335]
[2010/11/17 16:36:02.336] SunOS 5.10 amd64
[2010/11/17 16:36:02.336] Java(TM) SE Runtime Environment 1.6.0_17-b04, ISO646-US, en
[2010/11/17 16:36:02.336] Java HotSpot(TM) 64-Bit Server VM 14.3-b01, 64, mixed mode, Sun Microsystems Inc.
[2010/11/17 16:36:02.337] user.name: root
[2010/11/17 16:36:02.340] resin.home = /opt/resin/
[2010/11/17 16:36:02.340]
[2010/11/17 16:36:02.356] Host[resin-admin] active
[2010/11/17 16:36:02.359] Loaded Socket JNI library.
[2010/11/17 16:36:02.361] hmux listening to localhost:6600
[2010/11/17 16:36:02.368] Server[id=,cluster=] active
[2010/11/17 16:36:02.376]
[2010/11/17 16:36:02.377] SunOS 5.10 amd64
[2010/11/17 16:36:02.377] Java(TM) SE Runtime Environment 1.6.0_17-b04, ISO646-US, en
[2010/11/17 16:36:02.377] Java HotSpot(TM) 64-Bit Server VM 14.3-b01, 64, mixed mode, Sun Microsystems Inc.
[2010/11/17 16:36:02.378] user.name: root
[2010/11/17 16:36:02.380] resin.home = /opt/resin/
[2010/11/17 16:36:02.380]
[2010/11/17 16:36:02.396] Host[resin-admin] active
[2010/11/17 16:36:02.399] Loaded Socket JNI library.
[2010/11/17 16:36:02.420] WebApp[http://resin-admin] [^] active
[2010/11/17 16:36:02.426] servlet-mapping /watchdog -> watchdog
[2010/11/17 16:36:02.456] WatchdogProcess[Watchdog[resin1],1] starting Resin
[2010/11/17 16:36:02.763] WatchdogProcess[Watchdog[web-entry-1],1] starting Resin
[2010/11/17 16:36:02.816] java.lang.IllegalStateException: Can't start new task because of old task 'WatchdogTask[Watchdog[web-entry-1]]'
[2010/11/17 16:36:02.816] at com.caucho.boot.Watchdog.start(Watchdog.java:296)
[2010/11/17 16:36:02.816] at com.caucho.boot.WatchdogManager.startServer(WatchdogManager.java:295)
[2010/11/17 16:36:02.816] at com.caucho.boot.WatchdogServlet.start(WatchdogServlet.java:75)
[2010/11/17 16:36:02.816] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[2010/11/17 16:36:02.816] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[2010/11/17 16:36:02.816] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[2010/11/17 16:36:02.816] at java.lang.reflect.Method.invoke(Method.java:597)
[2010/11/17 16:36:02.816] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180)
[2010/11/17 16:36:02.816] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109)
[2010/11/17 16:36:02.816] at com.caucho.hessian.server.HessianServlet.service(HessianServlet.java:396)
[2010/11/17 16:36:02.816] at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
[2010/11/17 16:36:02.816] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187)
[2010/11/17 16:36:02.816] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:265)
[2010/11/17 16:36:02.816] at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:436)
[2010/11/17 16:36:02.816] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:682)
[2010/11/17 16:36:02.816] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:743)
[2010/11/17 16:36:02.816] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:662)
[2010/11/17 16:36:02.816] at java.lang.Thread.run(Thread.java:619)
[2010/11/17 16:36:07.607] java.net.BindException: Address already in use
[2010/11/17 16:36:07.607] Can't bind to /127.0.0.1:6600.
[2010/11/17 16:36:07.607] Check for another server listening to that port.
[2010/11/17 16:36:07.607] at com.caucho.vfs.QJniServerSocket.create(QJniServerSocket.java:103)
[2010/11/17 16:36:07.607] at com.caucho.server.port.Port.bind(Port.java:964)
[2010/11/17 16:36:07.607] at com.caucho.server.cluster.Server.bindPorts(Server.java:1379)
[2010/11/17 16:36:07.607] at com.caucho.server.cluster.Server.start(Server.java:1325)
[2010/11/17 16:36:07.607] at com.caucho.boot.WatchdogManager.<init>(WatchdogManager.java:157)
[2010/11/17 16:36:07.607] at com.caucho.boot.WatchdogManager.main(WatchdogManager.java:354)
[2010/11/17 16:36:07.608] Server[id=,cluster=] error
Exception in thread "main" java.lang.RuntimeException: java.net.BindException: Address already in use
Can't bind to /127.0.0.1:6600.
Check for another server listening to that port.
        at com.caucho.server.cluster.Server.start(Server.java:1345)
        at com.caucho.boot.WatchdogManager.<init>(WatchdogManager.java:157)
        at com.caucho.boot.WatchdogManager.main(WatchdogManager.java:354)
Caused by: java.net.BindException: Address already in use
Can't bind to /127.0.0.1:6600.
Check for another server listening to that port.
        at com.caucho.vfs.QJniServerSocket.create(QJniServerSocket.java:103)
        at com.caucho.server.port.Port.bind(Port.java:964)
        at com.caucho.server.cluster.Server.bindPorts(Server.java:1379)
        at com.caucho.server.cluster.Server.start(Server.java:1325)
        ... 2 more
================================================================================

Additional Information

Attached Files

Relationships

Notes
(0004855) ferg 11-18-10 12:38	server/1al6

Issue History
Date Modified	Username	Field	Change
11-18-10 09:54	reza	New Issue
11-18-10 12:38	ferg	Note Added: 0004855
11-18-10 12:38	ferg	Assigned To	=> ferg
11-18-10 12:38	ferg	Status	new => closed
11-18-10 12:38	ferg	Resolution	open => fixed
11-18-10 12:38	ferg	Fixed in Version	=> 4.0.14

Mantis 1.0.0rc3[^]

29 total queries executed.
26 unique queries executed.