What is the correct place to specifiy SPI service files for Java?

[Thorsten Schöning]

Hi all,

I would like to get some attention on some older SO-question[1] about
where to place SPI-service files of Java.

Following the docs, those should be placed in META-INF at the top
level of some JAR and webapps designed to be used with Tomcat provide
such a folder. But it seems to be ignored during search for
SPI-service files, instead those seem to be assumed in
WEB-INF/classes/META-INF/services. The can easily be seen e.g. using
Process Monitor in Windows.

As the SO-question lacks some official Tomcat-documentation or such, I
am asking here: What is the correct place for such service files? Can
this be configured somehow? Why is META-INF at the top level ignored?
Is it used at all for anything or only WEB-INF/classes/META-INF for
anything which is normally assumed to be in META-INF itself?

Thanks!

[1]: https://stackoverflow.com/questions/7692497/tomcat-wont-load-my-meta-inf-services-javax-servlet-servletcontainerinitializ

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Mark Thomas]

Post by Thorsten Schöning
Hi all,
I would like to get some attention on some older SO-question[1] about
where to place SPI-service files of Java.
Following the docs, those should be placed in META-INF at the top
level of some JAR and webapps designed to be used with Tomcat provide
such a folder.

Correct. With the caveat that if the web application defines an
<absolute-ordering> in web.xml, only those JARs are searched.

See
https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/startup/WebappServiceLoader.java
for details.

Mark

Post by Thorsten Schöning
But it seems to be ignored during search for
SPI-service files, instead those seem to be assumed in
WEB-INF/classes/META-INF/services. The can easily be seen e.g. using
Process Monitor in Windows.
As the SO-question lacks some official Tomcat-documentation or such, I
am asking here: What is the correct place for such service files? Can
this be configured somehow? Why is META-INF at the top level ignored?
Is it used at all for anything or only WEB-INF/classes/META-INF for
anything which is normally assumed to be in META-INF itself?
Thanks!
[1]: https://stackoverflow.com/questions/7692497/tomcat-wont-load-my-meta-inf-services-javax-servlet-servletcontainerinitializ
Mit freundlichen Grüßen,
Thorsten Schöning

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Thorsten Schöning]

Guten Tag Mark Thomas,

Correct.[...]

But as could have been read in the following paragraph of my mail and
the SO-link, it doesn't work that way at least in Tomcat 7.0.90 even
without absolute ordering in web.xml.

Enumeration<URL> resources;
if (loader == null) {
resources = ClassLoader.getSystemResources(configFile);
} else {
resources = loader.getResources(configFile);
}

https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/startup/WebappServiceLoader.java#L132

That code is NOT using "META-INF/services" at the top level of the web
project, but "WEB-INF/classes/META-INF/services" instead. That can be
clearly seen e.g. using Process Monitor.

So is this a bug or needs some configuration or has changed in newer
versions of Tomcat or whatever?

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Mark Thomas]

Post by Thorsten Schöning
Guten Tag Mark Thomas,

Correct.[...]

But as could have been read in the following paragraph of my mail and
the SO-link, it doesn't work that way at least in Tomcat 7.0.90 even
without absolute ordering in web.xml.

Enumeration<URL> resources;
if (loader == null) {
resources = ClassLoader.getSystemResources(configFile);
} else {
resources = loader.getResources(configFile);
}

https://github.com/apache/tomcat/blob/trunk/java/org/apache/catalina/startup/WebappServiceLoader.java#L132
That code is NOT using "META-INF/services" at the top level of the web> project,

Nor should it. foo.war!/META-INF/services is not a valid location for an
SPI file.

Post by Thorsten Schöning
but "WEB-INF/classes/META-INF/services" instead. That can be
clearly seen e.g. using Process Monitor.

The correct locations are:
foo.war!/WEB-INF/lib/*.jar!/META-INF/services

which is where Tomcat looks.

Tomcat uses ClassLoader.getResources() which will also pick up
foo.war!/WEB-INF/classes/META-INF/services since the class loader
doesn't differentiate between a resource located at the root of a JAR or
a resource located in at the root of /WEB-INF/classes

There are benefits to this as IDE / Tomcat integrations have a habit of
placing classes in /WEB-INF/classes rather than in JAR file in /WEB-INF/lib

Post by Thorsten Schöning
So is this a bug or needs some configuration or has changed in newer
versions of Tomcat or whatever?

No bug. No configuration required. No change in newer Tomcat versions.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Thorsten Schöning]

Guten Tag Mark Thomas,

Post by Mark Thomas
Nor should it. foo.war!/META-INF/services is not a valid location for an
SPI file.

[...]

Post by Mark Thomas
foo.war!/WEB-INF/lib/*.jar!/META-INF/services

So your argument is that a WAR is not a JAR and only JARs can contain
META-INF/services? Is there a reason for such decision, is it
something Java demands? Regarding the SO-question, JBoss seems to work

Post by Mark Thomas
What I do find odd however is that JBoss does seem to work with my
setup and can discover services inside the Services folder even if
you don't have them wrapped in a Jar file...

https://stackoverflow.com/questions/7692497/tomcat-wont-load-my-meta-inf-services-javax-servlet-servletcontainerinitializ#comment9883761_8057393

Which makes sense to me, reusing META-INF of the WAR is the first
thing one most likely considers. The docs for ServiceLoader seem to at

Post by Mark Thomas
Service providers can be installed in an implementation of the Java
platform in the form of extensions, that is, jar files placed into
any of the usual extension directories. Providers can also be made
available by adding them to the application's class path or by some
other platform-specific means.

https://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html

So, Tomcat prefers Jars because only those are explicitly mentioned or
what is the reason?

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Mark Thomas]

Post by Thorsten Schöning
Guten Tag Mark Thomas,

Post by Mark Thomas
Nor should it. foo.war!/META-INF/services is not a valid location for an
SPI file.

[...]

Post by Mark Thomas
foo.war!/WEB-INF/lib/*.jar!/META-INF/services

So your argument is that a WAR is not a JAR and only JARs can contain
META-INF/services? Is there a reason for such decision, is it
something Java demands?

Service files are loaded by class loaders from the META-INF/services
directory.

*.jar!/META-INF/services
and
*.war/WEB-INF/classes/META-INF/services
are visible to class loaders

*.war!/META-INF/services
is not.

The servlet expert group recently discussed WAR vs JAR in the context of
Java 9 and mutli-version JARs. The conclusion was (I'm paraphrasing)
that WARs are not a specialised form of JAR and while they share a
common format a feature that is available to a JAR is NOT automatically
available to a WAR unless the Servlet spec (of Java EE spec) explicitly
states otherwise.

Containers are free to add container specific extensions if they wish
but they come with the usual (lack of) interoperability warnings.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Thorsten Schöning]

Guten Tag Mark Thomas,

The servlet expert group recently discussed WAR vs JAR[...]

Thanks for the explanation, make things more clear tor me. I've added
your answer to the SO-question, because it provides the missing
background I wanted to read about, but if you want the credits, feel
free to add it yourself and I will delete mine.

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

[Thorsten Schöning]

Guten Tag Mark Thomas,

Post by Mark Thomas
Service files are loaded by class loaders from the META-INF/services
directory.
*.jar!/META-INF/services
and
*.war/WEB-INF/classes/META-INF/services
are visible to class loaders
*.war!/META-INF/services
is not.

I just came across another issue with using service files in my
environment: "Something" works in Ubuntu 16.04 with Tomcat 7 and Java
8, while it doesn't in Windows 10 with the same version of Tomcat and
Java.

The important thing to note is that I'm using Axis 2 in this scenario
and the service file is part of a service I'm hosting within Axis 2.

Post by Mark Thomas
[...]\webapps\axis2\WEB-INF\services\de.am_soft.sm_mtg.backend\META-INF\services\SOME_FILE

Using Process Monitor I can see that only the following directories

Post by Mark Thomas
[...]\webapps\axis2\WEB-INF\classes\META-INF\services\SOME_FILE
[...]\lib\META-INF\services\SOME_FILE

Querying the above two dirs looks like what you have written before
and that explains why it fails on Windows. But it doesn't on Ubuntu,
while it does fail if I remove the "services"-dir where it is
currently. So querying this dir seems non-standard.

But who is querying it most likely, Tomcat or Axis 2? I came across
different classloaders in Axis 2 for different OS in the past already,
so I guess it has to do with Axis 2. What do you think?

Post by Mark Thomas
ClassLoader cl = MdRecOmsEnc.class.getClassLoader();
ServiceLoader<MdRecOmsEnc> sl = ServiceLoader.load(MdRecOmsEnc.class, cl);
Iterator<MdRecOmsEnc> it = sl.iterator();

Providing the classloader is needed for other issues in very specific
environments, but in my opinion shouldn't be the root cause, as that
is provided with Ubuntu as well.

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org