Discussion:
Shutdown Hooks not firing when tomcat is shutdown from within a webapp
Andrew Kujtan
2012-02-07 18:19:42 UTC
Permalink
OS: Windows 7 32bit

Ver.: Apache Tomcat/7.0.25

Jvm: 1.6.0_14-b08



I am attaching a shutdown hook from within my webapp to log some info
after tomcat shuts down,
Runtime.getRuntime().addShutdownHook(mythread, name +
":shutdownHook"));



When I run shutdown.bat from the bin folder the hook runs fine and I get
my output.



In some cases however I am forced to shut down tomcat from within the
webapp, in this case it calls,
org.apache.catalina.startup.Bootstrap.main(new String[] { "stop"
});



Which shuts down tomcat, but in this case the shutdown hook does not get
fired.



Can anyone tell me how to programmatically shut down tomcat such that
the hooks will still fire?

What is the difference between running shutdown.bat and calling stop
programmatically?



Regards,

Andrew Kujtan
Caldarale, Charles R
2012-02-07 18:38:38 UTC
Permalink
Subject: Shutdown Hooks not firing when tomcat is shutdown from within a webapp
OS: Windows 7 32bit
Ver.: Apache Tomcat/7.0.25
Jvm: 1.6.0_14-b08
Thanks for that.
I am attaching a shutdown hook from within my webapp to log
some info after tomcat shuts down,
That really doesn't sound like a good idea at all. I would think you should be using a ServletContextListener.
Can anyone tell me how to programmatically shut down tomcat
such that the hooks will still fire?
There's no real provision for doing so, since that's not a decision a webapp should be making. There's always System.exit()...

- Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Andrew Kujtan
2012-02-07 19:40:31 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 07, 2012 1:39 PM
To: Tomcat Users List
Subject: RE: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
Subject: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
OS: Windows 7 32bit
Ver.: Apache Tomcat/7.0.25
Jvm: 1.6.0_14-b08
Thanks for that.
I am attaching a shutdown hook from within my webapp to log
some info after tomcat shuts down,
That really doesn't sound like a good idea at all. I would think you
should be using a ServletContextListener.
Unfortunately the actions need to happen after tomcat has shutdown, not
just my webapp.
Can anyone tell me how to programmatically shut down tomcat
such that the hooks will still fire?
There's no real provision for doing so, since that's not a decision a
webapp should be making.
I agree, but I don't have a choice, boss says it must kill itself if
some scenario happens.
There's always System.exit()...
When I call System.exit() tomcat doesn't actually shutdown, it looks
like it just is deadlocking or something as I am getting a timeout on
the call that runs it. On the bright side System.exit() does trigger the
shutdown hook to be called, even though the server doesn't shut down, it
just becomes unresponsive but the process remains and is still bound to
all of its ports.

If I call both,

org.apache.catalina.startup.Bootstrap.main(new String[] { "stop"});
System.exit(0);

Tomcat shuts down, and my hook gets called, but I don't know how
ridiculous I want to take this already hacky thing...

Probably stupid question, but is calling System.exit() from a webapp
supposed to be able to kill tomcat?
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE
PROPRIETARY MATERIAL and is thus for use only by the intended
recipient. If you received this in error, please contact the sender
and
delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
Christopher Schultz
2012-02-07 20:08:33 UTC
Permalink
Andrew,
Post by Andrew Kujtan
When I call System.exit() tomcat doesn't actually shutdown
That's weird.
Post by Andrew Kujtan
it looks like it just is deadlocking or something as I am getting
a timeout on the call that runs it.
What does a thread dump show you?
Post by Andrew Kujtan
On the bright side System.exit() does trigger the shutdown hook to
be called, even though the server doesn't shut down, it just
becomes unresponsive but the process remains and is still bound to
all of its ports.
Again, that seems weird. Do you have some kind of deadlock occurring
in your shutdown code? It sounds like:

System.exit: calls your shutdown hook, JVM does not stop
Bootstrap.main({"stop"}): avoids your shutdown hook, JVM stops

Maybe you should check your shutdown hook. :)
Post by Andrew Kujtan
If I call both,
org.apache.catalina.startup.Bootstrap.main(new String[] {
"stop"}); System.exit(0);
Tomcat shuts down, and my hook gets called, but I don't know how
ridiculous I want to take this already hacky thing...
It's a pretty ridiculous hack in any case, right?
Post by Andrew Kujtan
Probably stupid question, but is calling System.exit() from a
webapp supposed to be able to kill tomcat?
Yes. It should kill the JVM, which should shut down everything. If you
want to protect your JVM from webapps that might call System.exit(),
then you need to run under a SecurityManager.

- -chris
Andrew Kujtan
2012-02-07 21:19:24 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 07, 2012 3:09 PM
To: Tomcat Users List
Subject: Re: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Andrew,
Post by Andrew Kujtan
When I call System.exit() tomcat doesn't actually shutdown
That's weird.
Post by Andrew Kujtan
it looks like it just is deadlocking or something as I am getting
a timeout on the call that runs it.
What does a thread dump show you?
Yikes, It is showing it is waiting on the Catalina shutdown hook. What is that hook doing?

here is the relevant trace,

"Asynchronous Notification 'interface com.evertz.registry.ServerRegistryListener: masterChanged'" daemon prio=6 tid=0x28c77000 nid=0x16d8 in Object.wait() [0x2899f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08bc3ac0> (a org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
at java.lang.Thread.join(Thread.java:1143)
- locked <0x08bc3ac0> (a org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
at java.lang.Thread.join(Thread.java:1196)
at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
at java.lang.Shutdown.runHooks(Shutdown.java:79)
at java.lang.Shutdown.sequence(Shutdown.java:123)
at java.lang.Shutdown.exit(Shutdown.java:168)
- locked <0x23c8bf68> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Runtime.exit(Runtime.java:90)
at java.lang.System.exit(System.java:904)
at com.evertz.web.worker.ShutdownManager.shutDown(ShutdownManager.java:27)
at com.evertz.WebShutdownWrapper.stop(WebShutdownWrapper.java:40)
at com.evertz.WebShutdownWrapper.run(WebShutdownWrapper.java:30)
at com.evertz.util.boot.ShutdownAction.run(ShutdownAction.java:57)
at com.evertz.redirect.ClientRedirectionManager.handleNewMaster(ClientRedirectionManager.java:71)
at com.evertz.redirect.ClientRedirectionManager.masterChanged(ClientRedirectionManager.java:94)
at com.evertz.redirect.ClientRedirectionManager.access$300(ClientRedirectionManager.java:20)
at com.evertz.redirect.ClientRedirectionManager$MasterChangeMonitor.masterChanged(ClientRedirectionManager.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.evertz.prod.util.notifier.NotifierInvocationHandler.issueNotification(NotifierInvocationHandler.java:119)
at com.evertz.prod.util.notifier.async.AsynchronousInvocationHandler.access$000(AsynchronousInvocationHandler.java:16)
at com.evertz.prod.util.notifier.async.AsynchronousInvocationHandler$1.run(AsynchronousInvocationHandler.java:38)
at java.lang.Thread.run(Thread.java:619)

Locked ownable synchronizers:
- None
Post by Andrew Kujtan
On the bright side System.exit() does trigger the shutdown hook to
be called, even though the server doesn't shut down, it just
becomes unresponsive but the process remains and is still bound to
all of its ports.
Again, that seems weird. Do you have some kind of deadlock occurring
System.exit: calls your shutdown hook, JVM does not stop
Bootstrap.main({"stop"}): avoids your shutdown hook, JVM stops
Maybe you should check your shutdown hook. :)
The shutdown hook is just a sysout now. And that runs successfully.
Post by Andrew Kujtan
If I call both,
org.apache.catalina.startup.Bootstrap.main(new String[] {
"stop"}); System.exit(0);
Tomcat shuts down, and my hook gets called, but I don't know how
ridiculous I want to take this already hacky thing...
It's a pretty ridiculous hack in any case, right?
Indeed :(
Post by Andrew Kujtan
Probably stupid question, but is calling System.exit() from a
webapp supposed to be able to kill tomcat?
Yes. It should kill the JVM, which should shut down everything. If you
want to protect your JVM from webapps that might call System.exit(),
then you need to run under a SecurityManager.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8xhMEACgkQ9CaO5/Lv0PBXCgCgmM81JJasiDH1rD3lx5yHJTa6
8dsAnii8qNVXOpPbSVNJCfRGnZmPoyV/
=Xsi4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
For additional commands, e-mail: use
Chema
2012-02-07 21:50:50 UTC
Permalink
Post by Andrew Kujtan
"Asynchronous Notification 'interface com.evertz.registry.ServerRegistryListener: masterChanged'" daemon prio=6 tid=0x28c77000 nid=0x16d8 in Object.wait() [0x2899f000]
  java.lang.Thread.State: WAITING (on object monitor)
       at java.lang.Object.wait(Native Method)
       - waiting on <0x08bc3ac0> (a org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
       at java.lang.Thread.join(Thread.java:1143)
       - locked <0x08bc3ac0> (a org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
       at java.lang.Thread.join(Thread.java:1196)
       at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
       at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
       at java.lang.Shutdown.runHooks(Shutdown.java:79)
       at java.lang.Shutdown.sequence(Shutdown.java:123)
       at java.lang.Shutdown.exit(Shutdown.java:168)
       - locked <0x23c8bf68> (a java.lang.Class for java.lang.Shutdown)
       at java.lang.Runtime.exit(Runtime.java:90)
       at java.lang.System.exit(System.java:904)
Can I see ApplicationShutdownHooks source code ?
What 's com.evertz.registry.ServerRegistryListener ?
Andrew Kujtan
2012-02-07 22:01:30 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 07, 2012 4:51 PM
To: Tomcat Users List
Subject: Re: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
Post by Andrew Kujtan
"Asynchronous Notification 'interface
com.evertz.registry.ServerRegistryListener: masterChanged'" daemon
prio=6 tid=0x28c77000 nid=0x16d8 in Object.wait() [0x2899f000]
Post by Andrew Kujtan
  java.lang.Thread.State: WAITING (on object monitor)
       at java.lang.Object.wait(Native Method)
       - waiting on <0x08bc3ac0> (a
org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
Post by Andrew Kujtan
       at java.lang.Thread.join(Thread.java:1143)
       - locked <0x08bc3ac0> (a
org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
Post by Andrew Kujtan
       at java.lang.Thread.join(Thread.java:1196)
       at
java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.ja
va:79)
Post by Andrew Kujtan
       at
24)
Post by Andrew Kujtan
       at java.lang.Shutdown.runHooks(Shutdown.java:79)
       at java.lang.Shutdown.sequence(Shutdown.java:123)
       at java.lang.Shutdown.exit(Shutdown.java:168)
       - locked <0x23c8bf68> (a java.lang.Class for
java.lang.Shutdown)
Post by Andrew Kujtan
       at java.lang.Runtime.exit(Runtime.java:90)
       at java.lang.System.exit(System.java:904)
Can I see ApplicationShutdownHooks source code ?
That is located in java.lang, you can see the source online...
What 's com.evertz.registry.ServerRegistryListener ?
This is just the listener that triggers the shutdown call.
That the trace shows it that I call System.exit(0), which then runs the shutdown hooks, but then gets stuck running the CatalinaShutdownHook. It looks like all CatalinaShutdownHook does is call catalina.stop, I don't really know how to figure out what it is waiting on though.
---------------------------------------------------------------------
Chema
2012-02-07 22:12:12 UTC
Permalink
Post by Andrew Kujtan
Post by Chema
Can I see ApplicationShutdownHooks source code ?
That is located in java.lang, you can see the source online...
Sorry, I meant about *your* app's shutdown hook.
Post by Andrew Kujtan
Post by Chema
What 's com.evertz.registry.ServerRegistryListener ?
Right
Post by Andrew Kujtan
This is just the listener that triggers the shutdown call.
That the trace shows it that I call System.exit(0), which then runs the shutdown hooks, but then gets stuck running the CatalinaShutdownHook. It looks like all CatalinaShutdownHook does is call catalina.stop, I don't really know how to figure out what it is waiting on though.
Post by Chema
---------------------------------------------------------------------
---------------------------------------------------------------------
Andrew Kujtan
2012-02-07 22:20:21 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 07, 2012 5:12 PM
To: Tomcat Users List
Subject: Re: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
Post by Andrew Kujtan
Post by Chema
Can I see ApplicationShutdownHooks source code ?
That is located in java.lang, you can see the source online...
Sorry, I meant about *your* app's shutdown hook.
For debugging I'm only using this, and it does run successfully. The
shutdown hook is not blocking anything,

Runtime.getRuntime().addShutdownHook(
new Thread(new Runnable(){
@Override
public void run() {
System.out.println("HARI-KIRI SUCCESSFUL!");
}
}, "shutdownHook"));
Post by Andrew Kujtan
Post by Chema
What 's com.evertz.registry.ServerRegistryListener ?
Right
Post by Andrew Kujtan
This is just the listener that triggers the shutdown call.
That the trace shows it that I call System.exit(0), which then runs
the shutdown hooks, but then gets stuck running the
CatalinaShutdownHook. It looks like all CatalinaShutdownHook does is
call catalina.stop, I don't really know how to figure out what it is
waiting on though.
Pid
2012-02-08 08:06:25 UTC
Permalink
Post by Andrew Kujtan
-----Original Message-----
Sent: Tuesday, February 07, 2012 5:12 PM
To: Tomcat Users List
Subject: Re: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
Post by Andrew Kujtan
Post by Chema
Can I see ApplicationShutdownHooks source code ?
That is located in java.lang, you can see the source online...
Sorry, I meant about *your* app's shutdown hook.
For debugging I'm only using this, and it does run successfully. The
shutdown hook is not blocking anything,
Runtime.getRuntime().addShutdownHook(
new Thread(new Runnable(){
@Override
public void run() {
System.out.println("HARI-KIRI SUCCESSFUL!");
}
}, "shutdownHook"));
That's because there's nothing here that is loaded by the webapp
classloader. You are probably causing the app's classloader to hang
around when you add the shutdown hook from within the app.

If you really need something to run during shutdown, use a
Tomcat-specific LifecycleListener.


p
Post by Andrew Kujtan
Post by Andrew Kujtan
Post by Chema
What 's com.evertz.registry.ServerRegistryListener ?
Right
Post by Andrew Kujtan
This is just the listener that triggers the shutdown call.
That the trace shows it that I call System.exit(0), which then runs
the shutdown hooks, but then gets stuck running the
CatalinaShutdownHook. It looks like all CatalinaShutdownHook does is
call catalina.stop, I don't really know how to figure out what it is
waiting on though.
---------------------------------------------------------------------
--
[key:62590808]
Christopher Schultz
2012-02-08 17:17:46 UTC
Permalink
Andrew,
Runtime.getRuntime().addShutdownHook( new Thread(new Runnable(){
@Override public void run() { System.out.println("HARI-KIRI
SUCCESSFUL!"); } }, "shutdownHook"));
Note that if this code is running in the webapp, you are getting into
WebappClassLoader issues. Not sure if those are relevant, but if the
classes you expect to use in the shutdown hook are located in lib/ but
the above code is in a class in the webapp, then the anonymous inner
class you are creating (an anonymous Runnable) will itself be loaded
by the WebappClassLoader.

- -chris

Konstantin Kolinko
2012-02-08 00:31:07 UTC
Permalink
OS: Windows 7  32bit
Ver.: Apache Tomcat/7.0.25
Jvm: 1.6.0_14-b08
I am attaching a shutdown hook from within my webapp to log some info
after tomcat shuts down,
   > Runtime.getRuntime().addShutdownHook(mythread, name +
":shutdownHook"));
1. Webapp classloader will be stopped when webapp stops. It happens
long before the hook runs. It would not be able to load any classes.
(and it will keep your web application classes in memory, which is
known as PermGen memory leak).

2. The contextClassLoader of your thread will reference Webapp
classloader, unless you explicitly set it to null.

Tomcat has some logic to detect and warn about some PermGen memory
leak patterns. You might have seen those warnings. It is better to
honour them seriously.
From all of the above adding a shutdown hook from within a web
application is a bad idea. The hook classes must be in
${catalina.base}/lib folder and you can add <Listener> to
conf/server.xml to manage the hook.

(Or better just replace the hook with a Listener).
When I run shutdown.bat from the bin folder the hook runs fine and I get
my output.
In some cases however I am forced to shut down tomcat from within the
webapp, in this case it calls,
   > org.apache.catalina.startup.Bootstrap.main(new String[] { "stop"
});
or you can get a reference to Context, than walk up its parents chain,
and call Engine.getService().getServer().stop(). I think its
destroy() method will be called automatically when await thread stops.

Anyway calling Bootstrap looks better than calling System.exit() that
someone suggested here. When Tomcat is run as a service in Windows
you really must not use System.exit() as it will cause problems for
the service launcher.

You can also call stop() method through JMX.
Which shuts down tomcat, but in this case the shutdown hook does not get
fired.
Can anyone tell me how to programmatically shut down tomcat such that
the hooks will still fire?
As Javadoc for shutdown hooks says, when there are several hooks, they
all will be started at the same time and then run concurrently. The
order between them is random.

Note that the standard hook set by Tomcat shuts down the logging
framework. So if you try to log anything it is likely to be wasted.
What is the difference between running shutdown.bat and calling stop
programmatically?
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08bc3ac0> (a org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
at java.lang.Thread.join(Thread.java:1143)
Thread.join().

It looks like the calling thread (the one that called System.exit())
just waits until the target thread (CatalinaShutdownHook) finishes.

You should look in the logs to see what is printed there with regards
to the server shutdown sequence. Maybe it waits while some server
component shuts down?

Best regards,
Konstantin Kolinko
Andrew Kujtan
2012-02-08 17:09:13 UTC
Permalink
-----Original Message-----
Sent: Tuesday, February 07, 2012 7:31 PM
To: Tomcat Users List
Subject: Re: Shutdown Hooks not firing when tomcat is shutdown from
within a webapp
OS: Windows 7  32bit
Ver.: Apache Tomcat/7.0.25
Jvm: 1.6.0_14-b08
I am attaching a shutdown hook from within my webapp to log some info
after tomcat shuts down,
   > Runtime.getRuntime().addShutdownHook(mythread, name +
":shutdownHook"));
1. Webapp classloader will be stopped when webapp stops. It happens
long before the hook runs. It would not be able to load any classes.
(and it will keep your web application classes in memory, which is
known as PermGen memory leak).
2. The contextClassLoader of your thread will reference Webapp
classloader, unless you explicitly set it to null.
Tomcat has some logic to detect and warn about some PermGen memory
leak patterns. You might have seen those warnings. It is better to
honour them seriously.
From all of the above adding a shutdown hook from within a web
application is a bad idea. The hook classes must be in
${catalina.base}/lib folder and you can add <Listener> to
conf/server.xml to manage the hook.
(Or better just replace the hook with a Listener).
The hook classes are indeed in the lib folder. Could you elaborate on how I would add a listener to manage the hook? I can't find anything in the documentation about it...

I put this into server.xml,
<server>
...
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener"/>

Then what?
When I run shutdown.bat from the bin folder the hook runs fine and I
get
my output.
In some cases however I am forced to shut down tomcat from within the
webapp, in this case it calls,
   > org.apache.catalina.startup.Bootstrap.main(new String[] { "stop"
});
or you can get a reference to Context, than walk up its parents chain,
and call Engine.getService().getServer().stop(). I think its
destroy() method will be called automatically when await thread stops.
Anyway calling Bootstrap looks better than calling System.exit() that
someone suggested here. When Tomcat is run as a service in Windows
you really must not use System.exit() as it will cause problems for
the service launcher.
You can also call stop() method through JMX.
Which shuts down tomcat, but in this case the shutdown hook does not
get
fired.
Can anyone tell me how to programmatically shut down tomcat such that
the hooks will still fire?
As Javadoc for shutdown hooks says, when there are several hooks, they
all will be started at the same time and then run concurrently. The
order between them is random.
Implying that none of the shutdown hooks are running when I am calling Bootstrap.stop?
Note that the standard hook set by Tomcat shuts down the logging
framework. So if you try to log anything it is likely to be wasted.
I changed my hook and make it start a new process (calling calc.exe) and calc only gets called when I run shutdown.bat, not when I stop from bootstrap. So I know it is not running.
What is the difference between running shutdown.bat and calling stop
programmatically?
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x08bc3ac0> (a
org.apache.catalina.startup.Catalina$CatalinaShutdownHook)
at java.lang.Thread.join(Thread.java:1143)
Thread.join().
It looks like the calling thread (the one that called System.exit())
just waits until the target thread (CatalinaShutdownHook) finishes.
You should look in the logs to see what is printed there with regards
to the server shutdown sequence. Maybe it waits while some server
component shuts down?
Any place specifically?

When I set org.apache.catalina.level = FINE, all I get is,

Feb 8, 2012 12:01:03 PM com.evertz.web.worker.ShutdownManager shutDown
INFO: Calling the tomcat shutdown action.
Feb 8, 2012 12:01:03 PM org.apache.catalina.core.StandardService stopInternal
INFO: Stopping service Catalina
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
Loading...