Discussion:
Skip resource path in TLD scanner?
Matt Cosentino
2017-04-27 20:17:50 UTC
Permalink
I need to skip some of the resource paths within WEB-INF. I know there's a property for skipping jar files, but I couldn't find one for resource paths. I reported this as a bug and was told that the property exists. Where is it?

- Matt
Mark Thomas
2017-04-27 22:05:29 UTC
Permalink
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one
for resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Matt Cosentino
2017-04-27 22:39:10 UTC
Permalink
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html

There is one for skipping jar files:

tomcat.util.scan.StandardJarScanFilter.jarsToSkip

I then looked at the source for TldScanner, here is the scanResourcePaths function:

/**
* Scan web application resources for TLDs, recursively.
*
* @param startPath the directory resource to scan
* @throws IOException if there was a problem scanning for or loading a TLD
* @throws SAXException if there was a problem parsing a TLD
*/
protected void scanResourcePaths(String startPath)
throws IOException, SAXException {

Set<String> dirList = context.getResourcePaths(startPath);
if (dirList != null) {
for (String path : dirList) {
if (path.startsWith("/WEB-INF/classes/")) {
// Skip: JSP.7.3.1
} else if (path.startsWith("/WEB-INF/lib/")) {
// Skip: JSP.7.3.1
} else if (path.endsWith("/")) {
scanResourcePaths(path);
} else if (path.startsWith("/WEB-INF/tags/")) {
// JSP 7.3.1: in /WEB-INF/tags only consider implicit.tld
if (path.endsWith("/implicit.tld")) {
parseTld(path);
}
} else if (path.endsWith(TLD_EXT)) {
parseTld(path);
}
}
}
}

It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check any property to skip user defined paths.

- Matt


-----Original Message-----
From: Mark Thomas [mailto:***@apache.org]
Sent: Thursday, April 27, 2017 5:05 PM
To: Tomcat Users List <***@tomcat.apache.org>
Subject: Re: Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one for
resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Mark Thomas
2017-04-28 12:28:26 UTC
Permalink
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?

When I read "skipping resource paths" I was thinking of skipping the
various places where Tomcat treat directories as JARs that then get
scanned for TLDs (which can be configured via the JarScanner). But it
sounds like skipping those won't help you.

How sure are you that it is checking the directories below WEB-INF that
is the cause of the delay? That isn't a typical source of start-up delay
although it is certainly possible.

Finally, what sort of delay are we talking out here? Seconds? Minutes?

Mark
Post by Matt Cosentino
-----Original Message-----
Sent: Thursday, April 27, 2017 5:05 PM
Subject: Re: Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one for
resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Matt Cosentino
2017-04-28 16:00:59 UTC
Permalink
Yes, it's other folders within WEB-INF. I turned on the TldScanner logging and it is definitely what is causing the delay. My situation probably isn't very typical. The delay varies in my various web applications, the worst being about 20 seconds. It all adds up though, and every second counts when our sites are down.

- Matt


-----Original Message-----
From: Mark Thomas [mailto:***@apache.org]
Sent: Friday, April 28, 2017 7:28 AM
To: Tomcat Users List <***@tomcat.apache.org>
Subject: Re: Skip resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?

When I read "skipping resource paths" I was thinking of skipping the various places where Tomcat treat directories as JARs that then get scanned for TLDs (which can be configured via the JarScanner). But it sounds like skipping those won't help you.

How sure are you that it is checking the directories below WEB-INF that is the cause of the delay? That isn't a typical source of start-up delay although it is certainly possible.

Finally, what sort of delay are we talking out here? Seconds? Minutes?

Mark
Post by Matt Cosentino
-----Original Message-----
Sent: Thursday, April 27, 2017 5:05 PM
Subject: Re: Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one
for resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Mark Thomas
2017-04-29 10:01:43 UTC
Permalink
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the TldScanner
logging and it is definitely what is causing the delay. My situation
probably isn't very typical. The delay varies in my various web
applications, the worst being about 20 seconds. It all adds up
though, and every second counts when our sites are down.
There is a solution available but it is intended more for the embedded
use case rather than a standard Tomcat install. Using it in a standard
install would require (effectively) patching Tomcat.

The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code changes.
Doing this with a standard Tomcat install requires changes to the
JasperInitializer (hence the patch). I don't think there is a pure
config way around that but I'll look into it.

A better solution would probably be to make it easier to plugin in a
custom TLDScanner - i.e. purely with config. If you'd like us to explore
this option we should re-open 61052 and adjust accordingly. I don't
think there is enough demand for filtering resource paths to make that
worth implementing.

One final thought. Are you running the web application from a WAR or an
expanded directory? (The latter would be faster).

Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check
any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping the
various places where Tomcat treat directories as JARs that then get
scanned for TLDs (which can be configured via the JarScanner). But it
sounds like skipping those won't help you.
How sure are you that it is checking the directories below WEB-INF
that is the cause of the delay? That isn't a typical source of
start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find
one for resource paths. I reported this as a bug and was told
that the property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Matt Cosentino
2017-11-16 17:11:08 UTC
Permalink
This keeps getting worse, my site was down for over a minute while the TLD scanner ran. There must be something I can do.

- Matt

-----Original Message-----
From: Mark Thomas [mailto:***@apache.org]
Sent: Saturday, April 29, 2017 5:02 AM
To: Tomcat Users List <***@tomcat.apache.org>
Subject: Re: Skip resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the TldScanner
logging and it is definitely what is causing the delay. My situation
probably isn't very typical. The delay varies in my various web
applications, the worst being about 20 seconds. It all adds up though,
and every second counts when our sites are down.
There is a solution available but it is intended more for the embedded use case rather than a standard Tomcat install. Using it in a standard install would require (effectively) patching Tomcat.

The general idea would be to use the TldPreScanned class. That does require all the TLDs to be listed in advance. On the plus side, no scanning delay. On the down side, adding TLDs requires code changes.
Doing this with a standard Tomcat install requires changes to the JasperInitializer (hence the patch). I don't think there is a pure config way around that but I'll look into it.

A better solution would probably be to make it easier to plugin in a custom TLDScanner - i.e. purely with config. If you'd like us to explore this option we should re-open 61052 and adjust accordingly. I don't think there is enough demand for filtering resource paths to make that worth implementing.

One final thought. Are you running the web application from a WAR or an expanded directory? (The latter would be faster).

Mark
Post by Matt Cosentino
- Matt
path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check
any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping the
various places where Tomcat treat directories as JARs that then get
scanned for TLDs (which can be configured via the JarScanner). But it
sounds like skipping those won't help you.
How sure are you that it is checking the directories below WEB-INF
that is the cause of the delay? That isn't a typical source of
start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one
for resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Christopher Schultz
2017-11-17 18:26:00 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Matt,
Post by Matt Cosentino
This keeps getting worse, my site was down for over a minute while
the TLD scanner ran. There must be something I can do.
If you need taglibs, you'll need to do TLS scanning. There really is
no way to avoid that that I know of. One of many reasons I don't like
JSP as a technology.

If you are having a problem with downtime, you have other options that
might help in other areas. For example, you could set up a second
server and load-balance between the two of them. This protects you
from a number of downtime-causing issues such as power failures, JVM
crashes, and -- as in your example here -- planned maintenance.

Running a single instance of your application in a production
environment where uptime matters is really not an appropriate
solution. If you had two servers, your application's restart time
would not have been an issue.

- -chris
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the
TldScanner logging and it is definitely what is causing the
delay. My situation probably isn't very typical. The delay varies
in my various web applications, the worst being about 20 seconds.
It all adds up though, and every second counts when our sites are
down.
There is a solution available but it is intended more for the
embedded use case rather than a standard Tomcat install. Using it
in a standard install would require (effectively) patching Tomcat.
The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code
changes. Doing this with a standard Tomcat install requires changes
to the JasperInitializer (hence the patch). I don't think there is
a pure config way around that but I'll look into it.
A better solution would probably be to make it easier to plugin in
a custom TLDScanner - i.e. purely with config. If you'd like us to
explore this option we should re-open 61052 and adjust accordingly.
I don't think there is enough demand for filtering resource paths
to make that worth implementing.
One final thought. Are you running the web application from a WAR
or an expanded directory? (The latter would be faster).
Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not
check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping
the various places where Tomcat treat directories as JARs that
then get scanned for TLDs (which can be configured via the
JarScanner). But it sounds like skipping those won't help you.
How sure are you that it is checking the directories below
WEB-INF that is the cause of the delay? That isn't a typical
source of start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I
know there's a property for skipping jar files, but I
couldn't find one for resource paths. I reported this as a
bug and was told that the property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloPKbcdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFhLjRAAwjt0UGQ3Or2TTq3P
JmrcEyFnVwXB495dt0nmAVB/ldd12KaCSTBpkf7uhqJWIaB8EuljpH6CWKuhKjYI
FgHTiiGQEXOr3pCYV30ng2BzYr0teV8ZGjCoYF5I8prkt2ie8QVWiD1p3Vsre7qm
j14koqlBDa2K3zSvK9iXtc8t0DPvC9FxxOzroafTKJshxlFwLJVBbNnkfkSkGsCR
l7GPnG8iP43yMXd1/IqWKJMscTKOX++wcX5wpIzPDiCjCa3zGuFnBNreV8YOoTz2
+T9pkEvf5Z2KPZRWUCHHUlpWtu2owL/zaARvR1eD626PWTgDHa4gcFm5NueSe2bj
io88EK5jpo4uiDzmLPweKrXRTMDneOxqu6VAgmnEQVd/eSYBjq4+gT2n2fKQKg72
2w344c+vCBaCUAevNCUU3vBCAMjZ6OlayPafh9nghV+cK30t6qmUN6ECP7q14/zI
pgKhNjZjj4tCSnl1gWZBUB91FihpPyR4ShbvkEYHzfE74kiuTNwUoT7LHmMX0T11
U9xOqkyhGk5xyujp9ZFNqPlzThJ5Es+8dJR2+9gDUI0c6cSr/PUyIEwHvdEq/G+9
CoqtvDGh2kPqwNmNAIOxqoSzykHVNK/3PxKqROPS2EB7xLiPPNbpGR5qG95Rcyef
di1+iVj6Z488CI15bLFoYfLpFwg=
=GCDF
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Ray Holme
2017-11-17 19:39:27 UTC
Permalink
I use JSP and there are taglibs used.But you can cut the scan time way down by not scanning any libraries that don't have TLDs (nested files with name => .tld)You can find this by using "jar t" or "unzip -l" on each library (thanks Chris).
If you are a Unix (Linux or MAC) user you may cheat a little by using the script attached.You may put any all .jar files in .../apache/lib or .../apache/webapps/*/WEB-INF/lib into the
  .../apache/conf/catalina.properties file if they do not have taglibs.So 90% of what I need does not have taglibs (just jstl-impl-...jar DOES and must be left out of the DO NO SCAN line.This took my startup time from a couple minutes to under a second.
Best of luck.


On Friday, November 17, 2017 1:26 PM, Christopher Schultz <***@christopherschultz.net> wrote:


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Matt,
Post by Matt Cosentino
This keeps getting worse, my site was down for over a minute while
the TLD scanner ran. There must be something I can do.
If you need taglibs, you'll need to do TLS scanning. There really is
no way to avoid that that I know of. One of many reasons I don't like
JSP as a technology.

If you are having a problem with downtime, you have other options that
might help in other areas. For example, you could set up a second
server and load-balance between the two of them. This protects you
from a number of downtime-causing issues such as power failures, JVM
crashes, and -- as in your example here -- planned maintenance.

Running a single instance of your application in a production
environment where uptime matters is really not an appropriate
solution. If you had two servers, your application's restart time
would not have been an issue.

- -chris
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the
TldScanner logging and it is definitely what is causing the
delay. My situation probably isn't very typical. The delay varies
in my various web applications, the worst being about 20 seconds.
It all adds up though, and every second counts when our sites are
down.
There is a solution available but it is intended more for the
embedded use case rather than a standard Tomcat install. Using it
in a standard install would require (effectively) patching Tomcat.
The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code
changes. Doing this with a standard Tomcat install requires changes
to the JasperInitializer (hence the patch). I don't think there is
a pure config way around that but I'll look into it.
A better solution would probably be to make it easier to plugin in
a custom TLDScanner - i.e. purely with config. If you'd like us to
explore this option we should re-open 61052 and adjust accordingly.
I don't think there is enough demand for filtering resource paths
to make that worth implementing.
One final thought. Are you running the web application from a WAR
or an expanded directory? (The latter would be faster).
Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not
check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping
the various places where Tomcat treat directories as JARs that
then get scanned for TLDs (which can be configured via the
JarScanner). But it sounds like skipping those won't help you.
How sure are you that it is checking the directories below
WEB-INF that is the cause of the delay? That isn't a typical
source of start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I
know there's a property for skipping jar files, but I
couldn't find one for resource paths. I reported this as a
bug and was told that the property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloPKbcdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFhLjRAAwjt0UGQ3Or2TTq3P
JmrcEyFnVwXB495dt0nmAVB/ldd12KaCSTBpkf7uhqJWIaB8EuljpH6CWKuhKjYI
FgHTiiGQEXOr3pCYV30ng2BzYr0teV8ZGjCoYF5I8prkt2ie8QVWiD1p3Vsre7qm
j14koqlBDa2K3zSvK9iXtc8t0DPvC9FxxOzroafTKJshxlFwLJVBbNnkfkSkGsCR
l7GPnG8iP43yMXd1/IqWKJMscTKOX++wcX5wpIzPDiCjCa3zGuFnBNreV8YOoTz2
+T9pkEvf5Z2KPZRWUCHHUlpWtu2owL/zaARvR1eD626PWTgDHa4gcFm5NueSe2bj
io88EK5jpo4uiDzmLPweKrXRTMDneOxqu6VAgmnEQVd/eSYBjq4+gT2n2fKQKg72
2w344c+vCBaCUAevNCUU3vBCAMjZ6OlayPafh9nghV+cK30t6qmUN6ECP7q14/zI
pgKhNjZjj4tCSnl1gWZBUB91FihpPyR4ShbvkEYHzfE74kiuTNwUoT7LHmMX0T11
U9xOqkyhGk5xyujp9ZFNqPlzThJ5Es+8dJR2+9gDUI0c6cSr/PUyIEwHvdEq/G+9
CoqtvDGh2kPqwNmNAIOxqoSzykHVNK/3PxKqROPS2EB7xLiPPNbpGR5qG95Rcyef
di1+iVj6Z488CI15bLFoYfLpFwg=
=GCDF
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Alex O'Ree
2017-11-18 13:30:28 UTC
Permalink
I'm having similar issues after updating from tomcat7 to tomcat8.5. The
build script for my app adds some sql drivers to tomcat's lib folder,
specifically the derby driver. On bootup tomcat logs a ton of error
messages saying that it couldn't find (what looks to be) internationalized
resource files for derby, which aren't present.
Post by Ray Holme
I use JSP and there are taglibs used.
But you can cut the scan time way down by not scanning any libraries that
don't have TLDs
(nested files with name => .tld)
You can find this by using "jar t" or "unzip -l" on each library (thanks
Chris).
If you are a Unix (Linux or MAC) user you may cheat a little by using the
script attached.
You may put any all .jar files in .../apache/lib or
.../apache/webapps/*/WEB-INF/lib into the
.../apache/conf/catalina.properties file if they do not have taglibs.
So 90% of what I need does not have taglibs (just jstl-impl-...jar DOES
and must be left out of the DO NO SCAN line.
This took my startup time from a couple minutes to under a second.
Best of luck.
On Friday, November 17, 2017 1:26 PM, Christopher Schultz <
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Matt,
Post by Matt Cosentino
This keeps getting worse, my site was down for over a minute while
the TLD scanner ran. There must be something I can do.
If you need taglibs, you'll need to do TLS scanning. There really is
no way to avoid that that I know of. One of many reasons I don't like
JSP as a technology.
If you are having a problem with downtime, you have other options that
might help in other areas. For example, you could set up a second
server and load-balance between the two of them. This protects you
from a number of downtime-causing issues such as power failures, JVM
crashes, and -- as in your example here -- planned maintenance.
Running a single instance of your application in a production
environment where uptime matters is really not an appropriate
solution. If you had two servers, your application's restart time
would not have been an issue.
- -chris
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the
TldScanner logging and it is definitely what is causing the
delay. My situation probably isn't very typical. The delay varies
in my various web applications, the worst being about 20 seconds.
It all adds up though, and every second counts when our sites are
down.
There is a solution available but it is intended more for the
embedded use case rather than a standard Tomcat install. Using it
in a standard install would require (effectively) patching Tomcat.
The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code
changes. Doing this with a standard Tomcat install requires changes
to the JasperInitializer (hence the patch). I don't think there is
a pure config way around that but I'll look into it.
A better solution would probably be to make it easier to plugin in
a custom TLDScanner - i.e. purely with config. If you'd like us to
explore this option we should re-open 61052 and adjust accordingly.
I don't think there is enough demand for filtering resource paths
to make that worth implementing.
One final thought. Are you running the web application from a WAR
or an expanded directory? (The latter would be faster).
Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not
check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping
the various places where Tomcat treat directories as JARs that
then get scanned for TLDs (which can be configured via the
JarScanner). But it sounds like skipping those won't help you.
How sure are you that it is checking the directories below
WEB-INF that is the cause of the delay? That isn't a typical
source of start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds? Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I
know there's a property for skipping jar files, but I
couldn't find one for resource paths. I reported this as a
bug and was told that the property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloPKbcdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFhLjRAAwjt0UGQ3Or2TTq3P
JmrcEyFnVwXB495dt0nmAVB/ldd12KaCSTBpkf7uhqJWIaB8EuljpH6CWKuhKjYI
FgHTiiGQEXOr3pCYV30ng2BzYr0teV8ZGjCoYF5I8prkt2ie8QVWiD1p3Vsre7qm
j14koqlBDa2K3zSvK9iXtc8t0DPvC9FxxOzroafTKJshxlFwLJVBbNnkfkSkGsCR
l7GPnG8iP43yMXd1/IqWKJMscTKOX++wcX5wpIzPDiCjCa3zGuFnBNreV8YOoTz2
+T9pkEvf5Z2KPZRWUCHHUlpWtu2owL/zaARvR1eD626PWTgDHa4gcFm5NueSe2bj
io88EK5jpo4uiDzmLPweKrXRTMDneOxqu6VAgmnEQVd/eSYBjq4+gT2n2fKQKg72
2w344c+vCBaCUAevNCUU3vBCAMjZ6OlayPafh9nghV+cK30t6qmUN6ECP7q14/zI
pgKhNjZjj4tCSnl1gWZBUB91FihpPyR4ShbvkEYHzfE74kiuTNwUoT7LHmMX0T11
U9xOqkyhGk5xyujp9ZFNqPlzThJ5Es+8dJR2+9gDUI0c6cSr/PUyIEwHvdEq/G+9
CoqtvDGh2kPqwNmNAIOxqoSzykHVNK/3PxKqROPS2EB7xLiPPNbpGR5qG95Rcyef
di1+iVj6Z488CI15bLFoYfLpFwg=
=GCDF
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
---------------------------------------------------------------------
Christopher Schultz
2017-11-20 14:22:51 UTC
Permalink
Matt Cosentino
2017-11-20 17:06:02 UTC
Permalink
Thanks, but I'm talking about resource paths and not jar files.

- Matt

From: Ray Holme [mailto:***@yahoo.com.INVALID]
Sent: Friday, November 17, 2017 1:39 PM
To: Tomcat Users List <***@tomcat.apache.org>
Subject: Re: Skip resource path in TLD scanner?

I use JSP and there are taglibs used.
But you can cut the scan time way down by not scanning any libraries that don't have TLDs
(nested files with name => .tld)
You can find this by using "jar t" or "unzip -l" on each library (thanks Chris).

If you are a Unix (Linux or MAC) user you may cheat a little by using the script attached.
You may put any all .jar files in .../apache/lib or .../apache/webapps/*/WEB-INF/lib into the
.../apache/conf/catalina.properties file if they do not have taglibs.
So 90% of what I need does not have taglibs (just jstl-impl-...jar DOES and must be left out of the DO NO SCAN line.
This took my startup time from a couple minutes to under a second.

Best of luck.

On Friday, November 17, 2017 1:26 PM, Christopher Schultz <***@christopherschultz.net<mailto:***@christopherschultz.net>> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Matt,
Post by Matt Cosentino
This keeps getting worse, my site was down for over a minute while
the TLD scanner ran. There must be something I can do.
If you need taglibs, you'll need to do TLS scanning. There really is
no way to avoid that that I know of. One of many reasons I don't like
JSP as a technology.

If you are having a problem with downtime, you have other options that
might help in other areas. For example, you could set up a second
server and load-balance between the two of them. This protects you
from a number of downtime-causing issues such as power failures, JVM
crashes, and -- as in your example here -- planned maintenance.

Running a single instance of your application in a production
environment where uptime matters is really not an appropriate
solution. If you had two servers, your application's restart time
would not have been an issue.

- -chris
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the
TldScanner logging and it is definitely what is causing the
delay. My situation probably isn't very typical. The delay varies
in my various web applications, the worst being about 20 seconds.
It all adds up though, and every second counts when our sites are
down.
There is a solution available but it is intended more for the
embedded use case rather than a standard Tomcat install. Using it
in a standard install would require (effectively) patching Tomcat.
The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code
changes. Doing this with a standard Tomcat install requires changes
to the JasperInitializer (hence the patch). I don't think there is
a pure config way around that but I'll look into it.
A better solution would probably be to make it easier to plugin in
a custom TLDScanner - i.e. purely with config. If you'd like us to
explore this option we should re-open 61052 and adjust accordingly.
I don't think there is enough demand for filtering resource paths
to make that worth implementing.
One final thought. Are you running the web application from a WAR
or an expanded directory? (The latter would be faster).
Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not
check any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping
the various places where Tomcat treat directories as JARs that
then get scanned for TLDs (which can be configured via the
JarScanner). But it sounds like skipping those won't help you.
How sure are you that it is checking the directories below
WEB-INF that is the cause of the delay? That isn't a typical
source of start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I
know there's a property for skipping jar files, but I
couldn't find one for resource paths. I reported this as a
bug and was told that the property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org<http://gpgtools.org/>
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloPKbcdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFhLjRAAwjt0UGQ3Or2TTq3P
JmrcEyFnVwXB495dt0nmAVB/ldd12KaCSTBpkf7uhqJWIaB8EuljpH6CWKuhKjYI
FgHTiiGQEXOr3pCYV30ng2BzYr0teV8ZGjCoYF5I8prkt2ie8QVWiD1p3Vsre7qm
j14koqlBDa2K3zSvK9iXtc8t0DPvC9FxxOzroafTKJshxlFwLJVBbNnkfkSkGsCR
l7GPnG8iP43yMXd1/IqWKJMscTKOX++wcX5wpIzPDiCjCa3zGuFnBNreV8YOoTz2
+T9pkEvf5Z2KPZRWUCHHUlpWtu2owL/zaARvR1eD626PWTgDHa4gcFm5NueSe2bj
io88EK5jpo4uiDzmLPweKrXRTMDneOxqu6VAgmnEQVd/eSYBjq4+gT2n2fKQKg72
2w344c+vCBaCUAevNCUU3vBCAMjZ6OlayPafh9nghV+cK30t6qmUN6ECP7q14/zI
pgKhNjZjj4tCSnl1gWZBUB91FihpPyR4ShbvkEYHzfE74kiuTNwUoT7LHmMX0T11
U9xOqkyhGk5xyujp9ZFNqPlzThJ5Es+8dJR2+9gDUI0c6cSr/PUyIEwHvdEq/G+9
CoqtvDGh2kPqwNmNAIOxqoSzykHVNK/3PxKqROPS2EB7xLiPPNbpGR5qG95Rcyef
di1+iVj6Z488CI15bLFoYfLpFwg=
=GCDF
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org<mailto:users-***@tomcat.apache.org>
For additional commands, e-mail: users-***@tomcat.apache.org<mailto:users-***@tomcat.apache.org>
Matt Cosentino
2017-11-20 17:04:38 UTC
Permalink
While that is good advice, it's not necessarily an appropriate solution for this. It's not that I don't want any TLD scanning, it's that it is scanning folders that I know don't have TLDs and there is no reason to scan them. The scanner already decides to skip the classes and lib paths, so it could definitely skip a user defined list of paths. I guess for now I could see if I can move the files in these paths out of WEB-INF to keep them from being scanned.

- Matt

-----Original Message-----
From: Christopher Schultz [mailto:***@christopherschultz.net]
Sent: Friday, November 17, 2017 12:26 PM
To: ***@tomcat.apache.org
Subject: Re: Skip resource path in TLD scanner?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Matt,
Post by Matt Cosentino
This keeps getting worse, my site was down for over a minute while the
TLD scanner ran. There must be something I can do.
If you need taglibs, you'll need to do TLS scanning. There really is no way to avoid that that I know of. One of many reasons I don't like JSP as a technology.

If you are having a problem with downtime, you have other options that might help in other areas. For example, you could set up a second server and load-balance between the two of them. This protects you from a number of downtime-causing issues such as power failures, JVM crashes, and -- as in your example here -- planned maintenance.

Running a single instance of your application in a production environment where uptime matters is really not an appropriate solution. If you had two servers, your application's restart time would not have been an issue.

- -chris
Post by Matt Cosentino
Sent: Saturday, April 29, 2017 5:02 AM
resource path in TLD scanner?
Post by Matt Cosentino
Yes, it's other folders within WEB-INF. I turned on the TldScanner
logging and it is definitely what is causing the delay. My situation
probably isn't very typical. The delay varies in my various web
applications, the worst being about 20 seconds.
It all adds up though, and every second counts when our sites are
down.
There is a solution available but it is intended more for the embedded
use case rather than a standard Tomcat install. Using it in a standard
install would require (effectively) patching Tomcat.
The general idea would be to use the TldPreScanned class. That does
require all the TLDs to be listed in advance. On the plus side, no
scanning delay. On the down side, adding TLDs requires code changes.
Doing this with a standard Tomcat install requires changes to the
JasperInitializer (hence the patch). I don't think there is a pure
config way around that but I'll look into it.
A better solution would probably be to make it easier to plugin in a
custom TLDScanner - i.e. purely with config. If you'd like us to
explore this option we should re-open 61052 and adjust accordingly.
I don't think there is enough demand for filtering resource paths to
make that worth implementing.
One final thought. Are you running the web application from a WAR or
an expanded directory? (The latter would be faster).
Mark
Post by Matt Cosentino
- Matt
-----Original Message----- From: Mark Thomas
resource path in TLD scanner?
Post by Matt Cosentino
https://tomcat.apache.org/tomcat-8.0-doc/config/systemprops.html
tomcat.util.scan.StandardJarScanFilter.jarsToSkip
<snip/>
Post by Matt Cosentino
It skips /WEB-INF/classes/ and /WEB-INF/lib/, but it does not check
any property to skip user defined paths.
Is it other paths within WEB-INF you need to skip?
When I read "skipping resource paths" I was thinking of skipping the
various places where Tomcat treat directories as JARs that then get
scanned for TLDs (which can be configured via the JarScanner). But it
sounds like skipping those won't help you.
How sure are you that it is checking the directories below WEB-INF
that is the cause of the delay? That isn't a typical source of
start-up delay although it is certainly possible.
Finally, what sort of delay are we talking out here? Seconds?
Minutes?
Mark
Post by Matt Cosentino
-----Original Message----- From: Mark Thomas
Skip resource path in TLD scanner?
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know
there's a property for skipping jar files, but I couldn't find one
for resource paths. I reported this as a bug and was told that the
property exists. Where is it?
Where have you looked?
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloPKbcdHGNocmlzQGNo
cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFhLjRAAwjt0UGQ3Or2TTq3P
JmrcEyFnVwXB495dt0nmAVB/ldd12KaCSTBpkf7uhqJWIaB8EuljpH6CWKuhKjYI
FgHTiiGQEXOr3pCYV30ng2BzYr0teV8ZGjCoYF5I8prkt2ie8QVWiD1p3Vsre7qm
j14koqlBDa2K3zSvK9iXtc8t0DPvC9FxxOzroafTKJshxlFwLJVBbNnkfkSkGsCR
l7GPnG8iP43yMXd1/IqWKJMscTKOX++wcX5wpIzPDiCjCa3zGuFnBNreV8YOoTz2
+T9pkEvf5Z2KPZRWUCHHUlpWtu2owL/zaARvR1eD626PWTgDHa4gcFm5NueSe2bj
io88EK5jpo4uiDzmLPweKrXRTMDneOxqu6VAgmnEQVd/eSYBjq4+gT2n2fKQKg72
2w344c+vCBaCUAevNCUU3vBCAMjZ6OlayPafh9nghV+cK30t6qmUN6ECP7q14/zI
pgKhNjZjj4tCSnl1gWZBUB91FihpPyR4ShbvkEYHzfE74kiuTNwUoT7LHmMX0T11
U9xOqkyhGk5xyujp9ZFNqPlzThJ5Es+8dJR2+9gDUI0c6cSr/PUyIEwHvdEq/G+9
CoqtvDGh2kPqwNmNAIOxqoSzykHVNK/3PxKqROPS2EB7xLiPPNbpGR5qG95Rcyef
di1+iVj6Z488CI15bLFoYfLpFwg=
=GCDF
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

Т���������������������������������������������������������������������ХF�V�7V'67&�&R�R���âW6W'2�V�7V'67&�&TF��6B�6�R��&pФf�"FF�F����6����G2�R�
Violeta Georgieva
2017-04-28 06:29:38 UTC
Permalink
Hi,
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know there's
a property for skipping jar files, but I couldn't find one for resource
paths. I reported this as a bug and was told that the property exists.
Where is it?

Check this wiki https://wiki.apache.org/tomcat/HowTo/FasterStartUp

Regards,
Violeta
Post by Matt Cosentino
- Matt
Matt Cosentino
2017-04-28 15:50:40 UTC
Permalink
I already did, that only mentions skipping jar files.

- Matt


-----Original Message-----
From: Violeta Georgieva [mailto:***@apache.org]
Sent: Friday, April 28, 2017 1:30 AM
To: Tomcat Users List <***@tomcat.apache.org>
Subject: Re: Skip resource path in TLD scanner?

Hi,
Post by Matt Cosentino
I need to skip some of the resource paths within WEB-INF. I know there's
a property for skipping jar files, but I couldn't find one for resource paths. I reported this as a bug and was told that the property exists.
Where is it?

Check this wiki https://wiki.apache.org/tomcat/HowTo/FasterStartUp

Regards,
Violeta
Post by Matt Cosentino
- Matt
B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB��[��X��ܚX�KK[XZ[�\�\��][��X��ܚX�P�X�] �\X�K�ܙ�B��܈Y][ۘ[��[X[��K[
Loading...