0006132: Resin & JDK crashes (continuation to issue no. 0006131 )

Viewing Issue Simple Details [ Jump to Notes ]

[ View Advanced ] [ Issue History ] [ Print ]

Category

Severity

Reproducibility

Date Submitted

Last Update

0006132

[Resin]

crash

always

02-16-18 06:15

03-05-18 15:08

Reporter

punithan

View Status

public

Assigned To

ferg

Priority

normal

Resolution

no change required

Status

closed

Product Version

4.0.53

Summary

0006132: Resin & JDK crashes (continuation to issue no. 0006131 )

Description

Resin Crashes and jvm crashes

Hi,

We are using Resin pro 5.0.53 (Licensed) version. we are facing a peculiar issue as describes as follows.

1. Resin is neither connected any database(configuring <database> parameters) nor configured to run any application (by calling servlet program).

2. By simply running bare resin we getting the error "Shutdown Resin reason: WATCHDOG_EXIT" after the resin is idle for 1 to 2 hours.

3. We have just started the resin and kept idle to watch whether resin is getting shutdown automatically. Resin is getting shutdown automatically and JVM is also getting crashed.

4. In stdout.log message is coming as below:
=============================================
[18-02-13 14:41:32.970] {main} Using Resin(R) Open Source under the GNU Public License (GPL).

                         See http://www.caucho.com [^] [^] for information on Resin Professional,
                         including caching, clustering, JNI acceleration, and OpenSSL integration.

[18-02-13 14:41:33.193] {main} <proxy-cache> requires Resin Professional. Please see http://www.caucho.com [^] [^] for Resin Professional information and licensing.
[18-02-13 14:41:33.457] {main}
[18-02-13 14:41:33.457] {main} Resin-4.0.53 (built Fri, 16 Jun 2017 02:24:08 PDT)
[18-02-13 14:41:33.457] {main}
[18-02-13 14:41:33.458] {main} SunOS 5.11 sparc
[18-02-13 14:41:33.458] {main} Java(TM) SE Runtime Environment 1.7.0_161-b13, UTF-8, en
[18-02-13 14:41:33.458] {main} Java HotSpot(TM) Server VM 24.161-b13, 32, mixed mode, Oracle Corporation
[18-02-13 14:41:33.458] {main}
[18-02-13 14:41:33.458] {main} user.name = cbimig
[18-02-13 14:41:33.523] {main}
[18-02-13 14:41:33.529] {main} server listening to APP:9005
[18-02-13 14:41:33.533] {main}
[18-02-13 14:41:33.588] {main} Table[mnode:2,/RESIN/resin-4.0.53/resin-data/app-b/distcache/mnode.db] validating indexes due to unclean shutdown.
[18-02-13 14:41:33.615] {main} Table[data:3,/RESIN/resin-4.0.53/resin-data/app-b/distcache/data.db] validating indexes due to unclean shutdown.
[18-02-13 14:41:33.627] {main}
[18-02-13 14:41:33.627] {main} resin.home = /RESIN/resin-4.0.53/
[18-02-13 14:41:33.627] {main} resin.root = /RESIN/resin-4.0.53/
[18-02-13 14:41:33.628] {main} resin.conf = /RESIN/resin-4.0.53/conf/resin.xml
[18-02-13 14:41:33.628] {main}
[18-02-13 14:41:33.628] {main} server = 10.5.6.34:9005 (app:app-b)
[18-02-13 14:41:33.628] {main} stage = production
[18-02-13 14:41:34.213] {resin-34} WebApp[production/webapp/default/resin-admin] active
[18-02-13 14:41:34.272] {resin-36} WebApp[production/webapp/default/ROOT] active
[18-02-13 14:41:37.469] {resin-43} WebApp[production/webapp/default/gbmapp] active
[18-02-13 14:41:37.469] {main} Host[production/host/default] active
[18-02-13 14:41:37.470] {main} ServletService[id=app-b,cluster=app] active
[18-02-13 14:41:37.470] {main}
[18-02-13 14:41:37.470] {main} http listening to *:9003
[18-02-13 14:41:37.470] {main}
[18-02-13 14:41:37.471] {main} Resin[id=app-b] started in 5327ms
[18-02-13 15:33:11.485] {resin-main-link} WarningService: Shutdown: Resin shutdown from unexpected watchdog exit.
[18-02-13 15:33:11.497] {resin-shutdown} ServletService[id=app-b,cluster=app] stopping
[18-02-13 15:33:11.500] {resin-shutdown} Host[production/host/default] stopping
[18-02-13 15:33:11.500] {resin-shutdown} WebApp[production/webapp/default/gbmapp] stopping
[18-02-13 15:33:11.509] {resin-shutdown} WebApp[production/webapp/default/ROOT] stopping
[18-02-13 15:33:11.510] {resin-shutdown} WebApp[production/webapp/default/resin-admin] stopping
[18-02-13 15:33:11.628] {resin-shutdown} Shutdown Resin reason: WATCHDOG_EXIT
===============================================================
5. The below message coming again and again in watchdog-manager.log, but not starting the resin

[2018/02/13 14:41:30.417] {watchdog-app-b} Watchdog detected close of Resin[app-b,pid=0]
                                             exit reason: MEMORY (exit code=6)
[2018/02/13 14:41:30.418] {watchdog-app-b} WatchdogChild[app-b] starting
[2018/02/13 14:41:30.418] {watchdog-app-b} Watchdog starting Resin[app-b]
==================================================================
6.JDK Crash report is logging as below
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (output.cpp:1574), pid=25560, tid=0x0000001a
# guarantee((int)(blk_starts[i+1] - blk_starts[i]) >= (current_offset - blk_offset)) failed: shouldn't increase block size
#
# JRE version: Java(TM) SE Runtime Environment (7.0_161-b13) (build 1.7.0_161-b13)
# Java VM: Java HotSpot(TM) Server VM (24.161-b13 mixed mode solaris-sparc )
# Core dump written. Default location: /RESIN/resin-4.0.53/core or core.25560
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp [^] [^]
#

--------------- T H R E A D ---------------

Current thread (0x00276000): JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=26, stack(0xb3240000,0xb32c0000)]

Stack: [0xb3240000,0xb32c0000], sp=0xb32bde20, free space=503k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0xbd1ec4] void VMError::report_and_die()+0x7b4
V [libjvm.so+0x5a1e5c] void report_vm_error(const char*,int,const char*,const char*)+0x60
V [libjvm.so+0xa48450] void Compile::fill_buffer(CodeBuffer*,unsigned*)+0x1264
V [libjvm.so+0x20fcf8] void Compile::Code_Gen()+0x66c
V [libjvm.so+0x5496c4] Compile::Compile(ciEnv*,C2Compiler*,ciMethod*,int,bool,bool)+0x1200
V [libjvm.so+0x22ccb8] void C2Compiler::compile_method(ciEnv*,ciMethod*,int)+0xac
V [libjvm.so+0x22de14] void CompileBroker::invoke_compiler_on_method(CompileTask*)+0xafc
V [libjvm.so+0x2b1114] void CompileBroker::compiler_thread_loop()+0x9e0
V [libjvm.so+0xb60cc0] void JavaThread::thread_main_inner()+0x9c
V [libjvm.so+0x26fc70] void JavaThread::run()+0x4b4
V [libjvm.so+0xa30eb8] java_start+0x360

Current CompileTask:
C2:3902111 687 ! com.caucho.env.thread2.AbstractTaskWorker2::run (490 bytes)

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
  0x00491c00 JavaThread "resin-106" daemon [_thread_blocked, id=126, stack(0xb1fd0000,0xb2050000)]
  0x00790800 JavaThread "resin-105" daemon [_thread_blocked, id=125, stack(0xb2750000,0xb27d0000)]
  0x00d45000 JavaThread "resin-port-9003-launcher" daemon [_thread_blocked, id=124, stack(0xb2ac0000,0xb2b40000)]
  0x0048ec00 JavaThread "resin-102" daemon [_thread_blocked, id=122, stack(0xb25a0000,0xb2620000)]
  0x00795000 JavaThread "resin-101" daemon [_thread_blocked, id=121, stack(0xb1d00000,0xb1d80000)]
  0x00cf9800 JavaThread "resin-100" daemon [_thread_blocked, id=120, stack(0xb2a30000,0xb2ab0000)]
  0x0076c400 JavaThread "resin-99" daemon [_thread_blocked, id=119, stack(0xaf1e0000,0xaf260000)]
  0x00792000 JavaThread "resin-97" daemon [_thread_blocked, id=117, stack(0xb1c70000,0xb1cf0000)]
  0x0078b400 JavaThread "resin-93" daemon [_thread_blocked, id=113, stack(0xb2630000,0xb26b0000)]
  0x00d98c00 JavaThread "resin-90" daemon [_thread_blocked, id=110, stack(0xb2900000,0xb2980000)]
  0x00d97c00 JavaThread "resin-89" daemon [_thread_blocked, id=109, stack(0xb1ac0000,0xb1b40000)]
  0x008cc400 JavaThread "resin-88" daemon [_thread_blocked, id=108, stack(0xb1eb0000,0xb1f30000)]
  0x008ce400 JavaThread "resin-85" daemon [_thread_blocked, id=105, stack(0xaf0c0000,0xaf140000)]
  0x0076ac00 JavaThread "resin-81" daemon [_thread_blocked, id=101, stack(0xb0ee0000,0xb0f60000)]
  0x0079c800 JavaThread "resin-80" daemon [_thread_blocked, id=100, stack(0xb1b50000,0xb1bd0000)]
  0x00c3c400 JavaThread "resin-79" daemon [_thread_blocked, id=99, stack(0xb26c0000,0xb2740000)]
  0x00b1c000 JavaThread "resin-77" daemon [_thread_blocked, id=97, stack(0xb0e50000,0xb0ed0000)]
  0x007a9000 JavaThread "resin-74" daemon [_thread_blocked, id=94, stack(0xb1f40000,0xb1fc0000)]
  0x00768c00 JavaThread "resin-73" daemon [_thread_blocked, id=93, stack(0xaefa0000,0xaf020000)]
  0x00a51000 JavaThread "resin-69" daemon [_thread_blocked, id=89, stack(0xb1d90000,0xb1e10000)]
  0x00a7fc00 JavaThread "resin-67" daemon [_thread_blocked, id=87, stack(0xb0f70000,0xb0ff0000)]
  0x00a73c00 JavaThread "resin-65" daemon [_thread_blocked, id=85, stack(0xb1be0000,0xb1c60000)]
  0x0078cc00 JavaThread "resin-64" daemon [_thread_blocked, id=84, stack(0xb2240000,0xb22c0000)]
  0x00b3f000 JavaThread "resin-port-10.5.6.34:9005-launcher" daemon [_thread_blocked, id=82, stack(0xb0ca0000,0xb0d20000)]
  0x00b22800 JavaThread "resin-59" daemon [_thread_blocked, id=79, stack(0xb1860000,0xb18e0000)]
  0x0048dc00 JavaThread "resin-port-9003-55" daemon [_thread_in_native, id=75, stack(0xaf150000,0xaf1d0000)]
  0x00a64800 JavaThread "resin-53" daemon [_thread_blocked, id=73, stack(0xb01c0000,0xb0240000)]
  0x00b29800 JavaThread "resin-main-link" daemon [_thread_in_native, id=72, stack(0xb0250000,0xb02d0000)]
  0x00b37800 JavaThread "resin-port-9003-51" daemon [_thread_in_native, id=71, stack(0xb02e0000,0xb0360000)]
  0x00b36400 JavaThread "resin-port-9003-50" daemon [_thread_in_native, id=70, stack(0xb0370000,0xb03f0000)]
  0x00b33000 JavaThread "resin-port-9003-49" daemon [_thread_in_native, id=69, stack(0xb0c10000,0xb0c90000)]
  0x00aef400 JavaThread "resin-shutdown" daemon [_thread_blocked, id=61, stack(0xb1910000,0xb1990000)]
  0x00aaa800 JavaThread "resin-fail-safe-memory-free" daemon [_thread_blocked, id=60, stack(0xb19a0000,0xb1a20000)]
  0x00aa7800 JavaThread "resin-fail-safe-halt" daemon [_thread_blocked, id=59, stack(0xb1a30000,0xb1ab0000)]
  0x00a4f000 JavaThread "resin-32" daemon [_thread_blocked, id=52, stack(0xb1e20000,0xb1ea0000)]
  0x00a52c00 JavaThread "ThreadScheduleWorker[]-2" daemon [_thread_blocked, id=48, stack(0xb2060000,0xb20e0000)]
  0x00797800 JavaThread "resin-27" daemon [_thread_blocked, id=47, stack(0xb2120000,0xb21a0000)]
  0x00793400 JavaThread "resin-24" daemon [_thread_blocked, id=44, stack(0xb22d0000,0xb2350000)]
  0x00773400 JavaThread "resin-port-10.5.6.34:9005-23" daemon [_thread_in_native, id=43, stack(0xb2360000,0xb23e0000)]
  0x00772000 JavaThread "resin-port-10.5.6.34:9005-22" daemon [_thread_in_native, id=42, stack(0xb23f0000,0xb2470000)]
  0x0076fc00 JavaThread "resin-port-10.5.6.34:9005-21" daemon [_thread_in_native, id=41, stack(0xb2480000,0xb2500000)]
  0x0076ec00 JavaThread "resin-port-10.5.6.34:9005-20" daemon [_thread_in_native, id=40, stack(0xb2510000,0xb2590000)]
  0x0078dc00 JavaThread "resin-15" daemon [_thread_blocked, id=35, stack(0xb27e0000,0xb2860000)]
  0x0078f400 JavaThread "CoordinatorThread[]-3" daemon [_thread_blocked, id=30, stack(0xb2b50000,0xb2bd0000)]
  0x00787c00 JavaThread "ThreadLauncher2[ThreadPool[system]]-1" daemon [_thread_blocked, id=29, stack(0xb2be0000,0xb2c60000)]
  0x00277c00 JavaThread "Service Thread" daemon [_thread_blocked, id=27, stack(0xb31b0000,0xb3230000)]
=>0x00276000 JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=26, stack(0xb3240000,0xb32c0000)]
  0x00274000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=25, stack(0xb32d0000,0xb3350000)]
  0x0024a400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=24, stack(0xb3ae0000,0xb3b60000)]
  0x000b3400 JavaThread "Finalizer" daemon [_thread_blocked, id=23, stack(0xb3b70000,0xb3bf0000)]
  0x000a2400 JavaThread "Reference Handler" daemon [_thread_blocked, id=22, stack(0xb40a0000,0xb4120000)]
  0x000c1000 JavaThread "main" [_thread_blocked, id=2, stack(0xfdf90000,0xfe010000)]

Other Threads:
  0x00090400 VMThread [stack: 0xb4130000,0xb41b0000] [id=21]
  0x0028b400 WatcherThread [stack: 0xb3120000,0xb31a0000] [id=28]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
PSYoungGen total 148736K, used 64259K [0xe5000000, 0xf3800000, 0xfa800000)
  eden space 91392K, 17% used [0xe5000000,0xe5f9b580,0xea940000)
  from space 57344K, 84% used [0xee800000,0xf17258f0,0xf2000000)
  to space 64256K, 0% used [0xea940000,0xea940000,0xee800000)
ParOldGen total 212992K, used 110455K [0xba800000, 0xc7800000, 0xe5000000)
  object space 212992K, 51% used [0xba800000,0xc13ddc50,0xc7800000)
PSPermGen total 32768K, used 15809K [0xb6800000, 0xb8800000, 0xba800000)
  object space 32768K, 48% used [0xb6800000,0xb7770450,0xb8800000)

Card table byte_map: [0xfa8d0000,0xfaaf2000] byte_map_base: 0xfa31c000

Polling page: 0xfdedc000

Code Cache [0xfbc00000, 0xfc000000, 0xfdc00000)
total_blobs=760 nmethods=593 adapters=117 free_code_cache=31222Kb

Compilation events (10 events):
Event: 3115.262 Thread 0x00276000 nmethod 682 0xfbc935c8 code [0xfbc936a0, 0xfbc936fc]
Event: 3602.262 Thread 0x00274000 683 com.caucho.db.table.TableIterator::initRow (10 bytes)
Event: 3602.262 Thread 0x00274000 nmethod 683 0xfbd55308 code [0xfbd553e0, 0xfbd5543c]
Event: 3742.190 Thread 0x00276000 684 com.caucho.util.RingQueue::beginPoll (63 bytes)
Event: 3742.191 Thread 0x00276000 nmethod 684 0xfbd54fc8 code [0xfbd550a0, 0xfbd551bc]
Event: 3767.190 Thread 0x00274000 685 java.util.concurrent.atomic.AtomicLongFieldUpdater$CASUpdater::fullCheck (32 bytes)
Event: 3767.190 Thread 0x00276000 686 java.util.concurrent.atomic.AtomicLongFieldUpdater$CASUpdater::ensureProtectedAccess (82 bytes)
Event: 3767.191 Thread 0x00276000 nmethod 686 0xfbc6af08 code [0xfbc6afe0, 0xfbc6b0bc]
Event: 3767.191 Thread 0x00274000 nmethod 685 0xfbc6ab48 code [0xfbc6ac40, 0xfbc6ad7c]
Event: 3902.050 Thread 0x00276000 687 ! com.caucho.env.thread2.AbstractTaskWorker2::run (490 bytes)

GC Heap History (10 events):
Event: 2.840 GC heap before
{Heap before GC invocations=7 (full 2):
PSYoungGen total 78848K, used 3064K [0xe5000000, 0xee800000, 0xfa800000)
  eden space 75776K, 0% used [0xe5000000,0xe5000000,0xe9a00000)
  from space 3072K, 99% used [0xe9a00000,0xe9cfe388,0xe9d00000)
  to space 3072K, 0% used [0xee500000,0xee500000,0xee800000)
ParOldGen total 40960K, used 39580K [0xba800000, 0xbd000000, 0xe5000000)
  object space 40960K, 96% used [0xba800000,0xbcea7090,0xbd000000)
PSPermGen total 24576K, used 14849K [0xb6800000, 0xb8000000, 0xba800000)
  object space 24576K, 60% used [0xb6800000,0xb76806a8,0xb8000000)
Event: 3.052 GC heap after
Heap after GC invocations=7 (full 2):
PSYoungGen total 78848K, used 247K [0xe5000000, 0xee800000, 0xfa800000)
  eden space 75776K, 0% used [0xe5000000,0xe5000000,0xe9a00000)
  from space 3072K, 8% used [0xe9a00000,0xe9a3dff0,0xe9d00000)
  to space 3072K, 0% used [0xee500000,0xee500000,0xee800000)
ParOldGen total 98304K, used 40887K [0xba800000, 0xc0800000, 0xe5000000)
  object space 98304K, 41% used [0xba800000,0xbcfedf18,0xc0800000)
PSPermGen total 28672K, used 14848K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb76801d8,0xb8400000)
}
Event: 3.738 GC heap before
{Heap before GC invocations=8 (full 2):
PSYoungGen total 78848K, used 76023K [0xe5000000, 0xee800000, 0xfa800000)
  eden space 75776K, 100% used [0xe5000000,0xe9a00000,0xe9a00000)
  from space 3072K, 8% used [0xe9a00000,0xe9a3dff0,0xe9d00000)
  to space 3072K, 0% used [0xee500000,0xee500000,0xee800000)
ParOldGen total 98304K, used 40887K [0xba800000, 0xc0800000, 0xe5000000)
  object space 98304K, 41% used [0xba800000,0xbcfedf18,0xc0800000)
PSPermGen total 28672K, used 14874K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb7686960,0xb8400000)
Event: 3.765 GC heap after
Heap after GC invocations=8 (full 2):
PSYoungGen total 88576K, used 3056K [0xe5000000, 0xee800000, 0xfa800000)
  eden space 85504K, 0% used [0xe5000000,0xe5000000,0xea380000)
  from space 3072K, 99% used [0xee500000,0xee7fc2e0,0xee800000)
  to space 31744K, 0% used [0xeaa00000,0xeaa00000,0xec900000)
ParOldGen total 98304K, used 68887K [0xba800000, 0xc0800000, 0xe5000000)
  object space 98304K, 70% used [0xba800000,0xbeb45ed8,0xc0800000)
PSPermGen total 28672K, used 14874K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb7686960,0xb8400000)
}
Event: 4.841 GC heap before
{Heap before GC invocations=9 (full 2):
PSYoungGen total 88576K, used 88560K [0xe5000000, 0xee800000, 0xfa800000)
  eden space 85504K, 100% used [0xe5000000,0xea380000,0xea380000)
  from space 3072K, 99% used [0xee500000,0xee7fc2e0,0xee800000)
  to space 31744K, 0% used [0xeaa00000,0xeaa00000,0xec900000)
ParOldGen total 98304K, used 68887K [0xba800000, 0xc0800000, 0xe5000000)
  object space 98304K, 70% used [0xba800000,0xbeb45ed8,0xc0800000)
PSPermGen total 28672K, used 14875K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb7686fb0,0xb8400000)
Event: 4.898 GC heap after
Heap after GC invocations=9 (full 2):
PSYoungGen total 123904K, used 31728K [0xe5000000, 0xf2000000, 0xfa800000)
  eden space 92160K, 0% used [0xe5000000,0xe5000000,0xeaa00000)
  from space 31744K, 99% used [0xeaa00000,0xec8fc0f8,0xec900000)
  to space 57344K, 0% used [0xee800000,0xee800000,0xf2000000)
ParOldGen total 110592K, used 107837K [0xba800000, 0xc1400000, 0xe5000000)
  object space 110592K, 97% used [0xba800000,0xc114f700,0xc1400000)
PSPermGen total 28672K, used 14875K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb7686fb0,0xb8400000)
}
Event: 4.898 GC heap before
{Heap before GC invocations=10 (full 3):
PSYoungGen total 123904K, used 31728K [0xe5000000, 0xf2000000, 0xfa800000)
  eden space 92160K, 0% used [0xe5000000,0xe5000000,0xeaa00000)
  from space 31744K, 99% used [0xeaa00000,0xec8fc0f8,0xec900000)
  to space 57344K, 0% used [0xee800000,0xee800000,0xf2000000)
ParOldGen total 110592K, used 107837K [0xba800000, 0xc1400000, 0xe5000000)
  object space 110592K, 97% used [0xba800000,0xc114f700,0xc1400000)
PSPermGen total 28672K, used 14875K [0xb6800000, 0xb8400000, 0xba800000)
  object space 28672K, 51% used [0xb6800000,0xb7686fb0,0xb8400000)
Event: 5.042 GC heap after
Heap after GC invocations=10 (full 3):
PSYoungGen total 123904K, used 4592K [0xe5000000, 0xf2000000, 0xfa800000)
  eden space 92160K, 0% used [0xe5000000,0xe5000000,0xeaa00000)
  from space 31744K, 14% used [0xeaa00000,0xeae7c018,0xec900000)
  to space 57344K, 0% used [0xee800000,0xee800000,0xf2000000)
ParOldGen total 212992K, used 110451K [0xba800000, 0xc7800000, 0xe5000000)
  object space 212992K, 51% used [0xba800000,0xc13dcc50,0xc7800000)
PSPermGen total 32768K, used 14875K [0xb6800000, 0xb8800000, 0xba800000)
  object space 32768K, 45% used [0xb6800000,0xb7686ee0,0xb8800000)
}
Event: 2222.211 GC heap before
{Heap before GC invocations=11 (full 3):
PSYoungGen total 123904K, used 96752K [0xe5000000, 0xf2000000, 0xfa800000)
  eden space 92160K, 100% used [0xe5000000,0xeaa00000,0xeaa00000)
  from space 31744K, 14% used [0xeaa00000,0xeae7c018,0xec900000)
  to space 57344K, 0% used [0xee800000,0xee800000,0xf2000000)
ParOldGen total 212992K, used 110451K [0xba800000, 0xc7800000, 0xe5000000)
  object space 212992K, 51% used [0xba800000,0xc13dcc50,0xc7800000)
PSPermGen total 32768K, used 15800K [0xb6800000, 0xb8800000, 0xba800000)
  object space 32768K, 48% used [0xb6800000,0xb776e0c8,0xb8800000)
Event: 2222.262 GC heap after
Heap after GC invocations=11 (full 3):
PSYoungGen total 148736K, used 48278K [0xe5000000, 0xf3800000, 0xfa800000)
  eden space 91392K, 0% used [0xe5000000,0xe5000000,0xea940000)
  from space 57344K, 84% used [0xee800000,0xf17258f0,0xf2000000)
  to space 64256K, 0% used [0xea940000,0xea940000,0xee800000)
ParOldGen total 212992K, used 110455K [0xba800000, 0xc7800000, 0xe5000000)
  object space 212992K, 51% used [0xba800000,0xc13ddc50,0xc7800000)
PSPermGen total 32768K, used 15800K [0xb6800000, 0xb8800000, 0xba800000)
  object space 32768K, 48% used [0xb6800000,0xb776e0c8,0xb8800000)
}

Deoptimization events (10 events):
Event: 6.336 Thread 0x00b22800 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd4eb28 method=com.caucho.util.RingValueQueue.offer(Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;I)Z @ 102
Event: 6.390 Thread 0x00b10800 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd50128 method=com.caucho.util.RingValueQueue.offer(Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;I)Z @ 102
Event: 7.190 Thread 0x0078f400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbc6ab30 method=com.caucho.util.AlarmClock.updateNextAlarmTime(J)J @ 50
Event: 62.733 Thread 0x00b1d400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd508b0 method=com.caucho.util.RingValueQueue.offer(Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;I)Z @ 102
Event: 305.896 Thread 0x0078b400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd5ae3c method=com.caucho.util.LongKeyLruCache$ValueIterator.hasNext()Z @ 29
Event: 1202.050 Thread 0x00a52c00 Uncommon trap: reason=null_check action=make_not_entrant pc=0xfbd7a018 method=com.caucho.env.thread2.ThreadTaskRing2.takeAndSchedule(Lcom/caucho/env/thread2/ResinThread2;)Z @ 4
Event: 2222.277 Thread 0x000b3400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd59dcc method=java.lang.ref.Finalizer.remove()V @ 10
Event: 2222.277 Thread 0x000b3400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd35448 method=java.lang.ref.Finalizer.remove()V @ 10
Event: 2222.277 Thread 0x000b3400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd41398 method=java.lang.ref.Finalizer.remove()V @ 10
Event: 2222.277 Thread 0x000b3400 Uncommon trap: reason=unstable_if action=reinterpret pc=0xfbd34d8c method=java.lang.ref.Finalizer.remove()V @ 10

Internal exceptions (10 events):
Event: 5.906 Thread 0x000c1000 Threw 0xe94e0478 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.907 Thread 0x000c1000 Threw 0xe94e3f00 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.907 Thread 0x000c1000 Threw 0xe94e8110 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.908 Thread 0x00b29800 Threw 0xe9681b28 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.909 Thread 0x00b29800 Threw 0xe9683a60 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.909 Thread 0x00b29800 Threw 0xe9686c80 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.910 Thread 0x00b29800 Threw 0xe968e290 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.910 Thread 0x00b29800 Threw 0xe968f880 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 5.910 Thread 0x00b29800 Threw 0xe9693238 at /HUDSON/workspace/7u-2-build-solaris-sparc/jdk7u161/3725/hotspot/src/share/vm/prims/jvm.cpp:1323
Event: 1202.050 Thread 0x00a52c00 Implicit null exception at 0xfbd798ac to 0xfbd7a008

Events (10 events):
Event: 3605.891 Thread 0x0076dc00 Thread added: 0x0076dc00
Event: 3605.891 Thread 0x00d45000 Thread added: 0x00d45000
Event: 3605.891 Thread 0x00790800 Thread added: 0x00790800
Event: 3605.892 Thread 0x00491c00 Thread added: 0x00491c00
Event: 3666.336 Executing VM operation: RevokeBias
Event: 3666.337 Executing VM operation: RevokeBias done
Event: 3666.337 Thread 0x0076dc00 Thread exited: 0x0076dc00
Event: 3726.390 Thread 0x00b10800 Thread exited: 0x00b10800
Event: 3786.391 Thread 0x00796000 Thread exited: 0x00796000
Event: 3847.190 Thread 0x007d0800 Thread exited: 0x007d0800

Dynamic libraries:
0x00010000 /jdk1.7.0_161/bin/java
0xff36c000 /lib/libthread.so.1
0xff330000 /jdk1.7.0_161/bin/../lib/sparc/jli/libjli.so
0xff354000 /lib/libdl.so.1
0xfb3e0000 /lib/libc.so.1
0xfe400000 /jdk1.7.0_161/jre/lib/sparc/server/libjvm.so
0xff2e0000 /lib/libsocket.so.1
0xff30c000 /usr/lib/libsched.so.1
0xff2c0000 /lib/libm.so.1
0xfe3d0000 /usr/lib/libCrun.so.1
0xff3fa000 /lib/libdoor.so.1
0xfe380000 /usr/lib/libdemangle.so.1
0xfab00000 /lib/libkstat.so.1
0xfe290000 /lib/libm.so.2
0xfab90000 /lib/libnsl.so.1
0xfe150000 /lib/libucrypto.so.1
0xfe130000 /lib/libcryptoutil.so.1
0xfe0f0000 /lib/libelf.so.1
0xfe0b0000 /lib/libz.so.1
0xfe090000 /lib/libmp.so.2
0xfdf50000 /jdk1.7.0_161/jre/lib/sparc/libverify.so
0xfdf10000 /jdk1.7.0_161/jre/lib/sparc/libjava.so
0xfab20000 /lib/libscf.so.1
0xfab70000 /lib/libuutil.so.1
0xfdee0000 /lib/libgen.so.1
0xfac10000 /lib/libnvpair.so.1
0xfde70000 /jdk1.7.0_161/jre/lib/sparc/libzip.so
0xb3400000 /usr/lib/locale/en_US.UTF-8/en_US.UTF-8.so.3
0xb4080000 /usr/lib/locale/en_US.UTF-8/methods_unicode.so.3
0xb2e00000 /jdk1.7.0_161/jre/lib/sparc/libmanagement.so
0xb2dd0000 /jdk1.7.0_161/jre/lib/sparc/libnet.so
0xb2db0000 /jdk1.7.0_161/jre/lib/sparc/libnio.so
0xff3f6000 /lib/librt.so.1
0xb2d90000 /lib/libsendfile.so.1
0xb2d40000 /jdk1.7.0_161/jre/lib/sparc/libj2ucrypto.so
0xb2d3c000 /lib/libmd.so
0xb2d00000 /jdk1.7.0_161/jre/lib/sparc/libj2pkcs11.so
0xb2cc0000 /usr/lib/libpkcs11.so
0xb2c70000 /usr/lib/security/pkcs11_softtoken.so

VM Arguments:
jvm_args: -Dresin.server=app-b -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djava.system.class.loader=com.caucho.loader.SystemClassLoader -Djava.endorsed.dirs=/jdk1.7.0_161/jre/lib/endorsed:/RESIN/resin-4.0.53//endorsed:/RESIN/resin-4.0.53//endorsed -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Djava.awt.headlesslib=true -Dresin.home=/RESIN/resin-4.0.53/
java_command: com.caucho.server.resin.Resin --root-directory /RESIN/resin-4.0.53/ -conf /RESIN/resin-4.0.53/conf/resin.xml -server app-b -socketwait 43341 start --log-directory /RESIN/resin-4.0.53/log
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/jdk1.7.0_161/
PATH=/jdk1.7.0_161/bin:.:/usr/ccs/bin:/usr/bin:/usr/lib:/usr/sbin/:/usr/ucb:/usr/bin/perl:/oracle/app/oracle/product/11.2.0.2/db_1:/oracle/app/oracle/product/11.2.0.2/db_1/lib:/oracle/app/oracle/product/11.2.0.2/db_1/bin:/jdk1.7.0_161/lib/sparcv9:/usr/local/bin
LD_LIBRARY_PATH=/RESIN/resin-4.0.53/libexec:/RESIN/resin-4.0.53/libexec
SHELL=/bin/ksh
DYLD_LIBRARY_PATH=/RESIN/resin-4.0.53/libexec:/RESIN/resin-4.0.53/libexec

Signal Handlers:
SIGSEGV: [libjvm.so+0xbd3110], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGBUS: [libjvm.so+0xbd3110], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGFPE: [libjvm.so+0x22aab0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGPIPE: [libjvm.so+0x22aab0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGXFSZ: [libjvm.so+0x22aab0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGILL: [libjvm.so+0x22aab0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGQUIT: [libjvm.so+0xa34984], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGHUP: [libjvm.so+0xa34984], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGINT: [libjvm.so+0xa34984], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIGTERM: [libjvm.so+0xa34984], sa_mask[0]=0xffbffeff, sa_flags=0x00000004
SIG39: [libjvm.so+0xa39670], sa_mask[0]=0x00000000, sa_flags=0x00000008
SIG40: [libjvm.so+0x22aab0], sa_mask[0]=0xffbffeff, sa_flags=0x0000000c

--------------- S Y S T E M ---------------

OS: Oracle Solaris 11.3 SPARC
  Copyright (c) 1983, 2016, Oracle and/or its affiliates. All rights reserved.
                            Assembled 03 August 2016

uname:SunOS 5.11 11.3 sun4v
  (T2 libthread)
rlimit: STACK 8192k, CORE infinity, NOFILE 65536, AS infinity
load average:0.01 0.00 0.02

CPU:total 24 v9, popc, vis1, vis2, vis3, blk_init, cbcond, sun4v

Memory: 8k page, physical 28835840k(18681064k free)

vm_info: Java HotSpot(TM) Server VM (24.161-b13) for solaris-sparc JRE (1.7.0_161-b13), built on Sep 13 2017 10:36:26 by "java_re" with Sun Studio 12u1

time: Tue Feb 13 14:41:21 2018
elapsed time: 3902 seconds (0d 1h 5m 2s)

=====================================
I have installed JDK1.7.0_161-b13 32-bit JDK. I have doubtful observations as follows.

1. In few occurrence time of this issue are correlating with after the log file is zipped by resin and recreating a new log file.
2. Issues are occurring mostly at resin is idle.

Configured with JDK 1.8.0_131-b11 64-bit VM.
Observation after that is :

It is taking 8:30 hours to restart, whereas it is restarted within 1 or 2 hours with JDK 1.7 32 bit

We are using SunOS 64 bit ONLY.

output of unix command

$ isainfo -b
64
$ isainfo -v
64-bit sparcv9 applications
32-bit sparc applications

Please don't close this issue and provide solution/workaround at the earliest.

Thanks and regards
Punithan.P

Additional Information

Attached Files

Relationships

Notes
(0006827) ferg 03-05-18 15:08	The java is running in 32 bit mode see log: [18-02-13 14:41:33.458] {main} Java HotSpot(TM) Server VM 24.161-b13, 32, mixed mode, Oracle Corporation You can use the -d64 argument to the command line to start the JVM in 64 bit mode.

Issue History
Date Modified	Username	Field	Change
02-16-18 06:15	punithan	New Issue
02-16-18 06:16	punithan	Issue Monitored: punithan
03-05-18 15:08	ferg	Note Added: 0006827
03-05-18 15:08	ferg	Assigned To	=> ferg
03-05-18 15:08	ferg	Status	new => closed
03-05-18 15:08	ferg	Resolution	open => no change required

Mantis 1.0.0rc3[^]

30 total queries executed.
26 unique queries executed.