0006125: Resin shutdown from unexpected watchdog exit - exit reason: MEMORY (exit code=6)

6125

crash

random

01-03-18 05:07

03-12-18 17:11

Reporter:

venkatesan

Platform:

Oracle

Assigned To:

ferg

OS:

Solaris

Priority:

normal

OS Version:

5.11

Status:

closed

Product Version:

4.0.53

Product Build:

Resolution:

no change required

Projection:

none

ETA:

none

Fixed in Version:

Summary:

0006125: Resin shutdown from unexpected watchdog exit - exit reason: MEMORY (exit code=6)

Description:

Hi,

We are resin 4.0.53 in Solaris 5.11.
We are facing an issue due which resin is stopping automatically by the watch dog.After our analysis it is been observed that the resin getting stopped after afternoon one time (i.e. Around & After 2:00 PM ) and at mid night (i.e. around & After 10:00 PM) daily. At this time the resin will be mostly idle.

We are getting following logs in the watchdog-manager.log
--------------
[2017/12/27 19:22:59.197] {main} watchdog = 127.0.0.1:-1 (watchdog:default)
[2017/12/27 19:22:59.197] {main} stage = production
[2017/12/27 19:22:59.434] {resin-17} WebApp[production/webapp/admin.resin/ROOT] active
[2017/12/27 19:22:59.437] {main} Host[production/host/admin.resin] active
[2017/12/27 19:22:59.437] {main} ServletService[id=default,cluster=watchdog] active
[2017/12/27 19:22:59.437] {main}
[2017/12/27 19:22:59.445] {main} http listening to localhost:9001
[2017/12/27 19:22:59.450] {main}
[2017/12/27 19:22:59.518] {watchdog-app-b} WatchdogChild[app-b] starting
[2017/12/27 19:22:59.521] {watchdog-app-b} Watchdog starting Resin[app-b]
[2017/12/27 21:03:15.158] {watchdog-app-b} Watchdog detected close of Resin[app-b,pid=0]
exit reason: [2017/12/27 21:03:15.158] {watchdog-app-b} WatchdogChild[app-b] starting
[2017/12/27 21:03:15.159] {watchdog-app-b} Watchdog starting Resin[app-b]
[2017/12/27 22:06:26.957] {watchdog-app-b} Watchdog detected close of Resin[app-b,pid=0]
exit reason: MEMORY (exit code=6)
------------

And we are getting following log in the stdout.log
---------
[17-12-28 15:04:14.097] {resin-191} DataStore removing 3 orphans (remove=3,entry-count=474)
[17-12-28 15:04:20.436] {resin-main-link} WarningService: Shutdown: Resin shutdown from unexpected watchdog exit.
[17-12-28 15:04:20.448] {resin-shutdown} ServletService[id=app-b,cluster=app] stopping
[17-12-28 15:04:20.450] {resin-shutdown} Host[production/host/default] stopping
[17-12-28 15:04:20.450] {resin-shutdown} WebApp[production/webapp/default/myapp] stopping
[17-12-28 15:04:20.484] {resin-shutdown} WebApp[production/webapp/default/ROOT] stopping
[17-12-28 15:04:20.485] {resin-shutdown} WebApp[production/webapp/default/resin-admin] stopping
[17-12-28 15:04:20.606] {resin-shutdown} Shutdown Resin reason: WATCHDOG_EXIT
-------------------------------------------------------------

We have adequate RAM (24GB) in the server.

We have changed the following parameters to address this issue.

1. Reduced the stack size : 512kb to 256kb

2. Initial heap size defined as : 256 MB

3. Maximum extending of heap size defined as : 2GB

Even though we are getting the same error.Please let us know how to resolve this issue.

Steps To Reproduce:

Additional Information:

Relationships

Attached Files:

Mantis - Resin
Viewing Issue Advanced Details

ID:	Category:	Severity:	Reproducibility:	Date Submitted:	Last Update:
6125		crash	random	01-03-18 05:07	03-12-18 17:11

Reporter:	venkatesan	Platform:	Oracle
Assigned To:	ferg	OS:	Solaris
Priority:	normal	OS Version:	5.11
Status:	closed	Product Version:	4.0.53
Product Build:		Resolution:	no change required
Projection:	none
ETA:	none	Fixed in Version:

Summary:	0006125: Resin shutdown from unexpected watchdog exit - exit reason: MEMORY (exit code=6)
Description:	Hi, We are resin 4.0.53 in Solaris 5.11. We are facing an issue due which resin is stopping automatically by the watch dog.After our analysis it is been observed that the resin getting stopped after afternoon one time (i.e. Around & After 2:00 PM ) and at mid night (i.e. around & After 10:00 PM) daily. At this time the resin will be mostly idle. We are getting following logs in the watchdog-manager.log -------------- [2017/12/27 19:22:59.197] {main} watchdog = 127.0.0.1:-1 (watchdog:default) [2017/12/27 19:22:59.197] {main} stage = production [2017/12/27 19:22:59.434] {resin-17} WebApp[production/webapp/admin.resin/ROOT] active [2017/12/27 19:22:59.437] {main} Host[production/host/admin.resin] active [2017/12/27 19:22:59.437] {main} ServletService[id=default,cluster=watchdog] active [2017/12/27 19:22:59.437] {main} [2017/12/27 19:22:59.445] {main} http listening to localhost:9001 [2017/12/27 19:22:59.450] {main} [2017/12/27 19:22:59.518] {watchdog-app-b} WatchdogChild[app-b] starting [2017/12/27 19:22:59.521] {watchdog-app-b} Watchdog starting Resin[app-b] [2017/12/27 21:03:15.158] {watchdog-app-b} Watchdog detected close of Resin[app-b,pid=0] exit reason: [2017/12/27 21:03:15.158] {watchdog-app-b} WatchdogChild[app-b] starting [2017/12/27 21:03:15.159] {watchdog-app-b} Watchdog starting Resin[app-b] [2017/12/27 22:06:26.957] {watchdog-app-b} Watchdog detected close of Resin[app-b,pid=0] exit reason: MEMORY (exit code=6) ------------ And we are getting following log in the stdout.log --------- [17-12-28 15:04:14.097] {resin-191} DataStore removing 3 orphans (remove=3,entry-count=474) [17-12-28 15:04:20.436] {resin-main-link} WarningService: Shutdown: Resin shutdown from unexpected watchdog exit. [17-12-28 15:04:20.448] {resin-shutdown} ServletService[id=app-b,cluster=app] stopping [17-12-28 15:04:20.450] {resin-shutdown} Host[production/host/default] stopping [17-12-28 15:04:20.450] {resin-shutdown} WebApp[production/webapp/default/myapp] stopping [17-12-28 15:04:20.484] {resin-shutdown} WebApp[production/webapp/default/ROOT] stopping [17-12-28 15:04:20.485] {resin-shutdown} WebApp[production/webapp/default/resin-admin] stopping [17-12-28 15:04:20.606] {resin-shutdown} Shutdown Resin reason: WATCHDOG_EXIT ------------------------------------------------------------- We have adequate RAM (24GB) in the server. We have changed the following parameters to address this issue. 1. Reduced the stack size : 512kb to 256kb 2. Initial heap size defined as : 256 MB 3. Maximum extending of heap size defined as : 2GB Even though we are getting the same error.Please let us know how to resolve this issue.
Steps To Reproduce:
Additional Information:
Relationships
Attached Files: