Hostnames with underscores

[Amit Pande]

Hello all,

Recent Tomcat versions (8.5.32 I think) has made a stricter validation for hostnames with underscores in it. (https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

This is understandably for addressing security issues (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and enforcing RFC compliance, in some way.

Our recent upgrade to Tomcat (8.5.34), we observed:

Note: further occurrences of request parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a domain name.
at org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
at org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
at org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
at org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)


The main issue for us now is that, since we ship Tomcat with our product (IOW Tomcat runs in customer environments), this will break our product functionality if customers have hostnames with underscore. Ideally, they should correct the host names (to be RFC compliant) but customers would be really averse to change hostname as it might a widespread change in their environments.

With Spring also, we ran into same issue but in later releases of Spring this was fixed. I

Would it be a good idea to make this strict check configurable so that we can continue to cater to our customers without breaking the functionality?

Are there any other alternates to help solve this issue?

Thanks,
Amit

[M. Manna]

Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?

Post by Amit Pande
Hello all,
Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
hostnames with underscores in it. (
https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)
This is understandably for addressing security issues (
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
enforcing RFC compliance, in some way.
Note: further occurrences of request parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a domain name.
at
org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
at
org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
at
org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
at
org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net
.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net
.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
The main issue for us now is that, since we ship Tomcat with our product
(IOW Tomcat runs in customer environments), this will break our product
functionality if customers have hostnames with underscore. Ideally, they
should correct the host names (to be RFC compliant) but customers would be
really averse to change hostname as it might a widespread change in their
environments.
With Spring also, we ran into same issue but in later releases of Spring this was fixed. I
Would it be a good idea to make this strict check configurable so that we
can continue to cater to our customers without breaking the functionality?
Are there any other alternates to help solve this issue?
Thanks,
Amit

[Amit Pande]

Yes, I did check the description, but did not find reference to underscore.

From the documentation:
“The value may be any combination of the following characters: " < > [ \ ] ^ ` { | } . “

But I admit that I did not actually verify it and will see if these attributes work for underscore too.

Thanks,
Amit

On Oct 26, 2018, at 12:02 PM, M. Manna <***@gmail.com<mailto:***@gmail.com>> wrote:

Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?



On Fri, 26 Oct 2018 at 18:00, Amit Pande <***@veritas.com<mailto:***@veritas.com>> wrote:

Hello all,

Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
hostnames with underscores in it. (
https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

This is understandably for addressing security issues (
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
enforcing RFC compliance, in some way.

Our recent upgrade to Tomcat (8.5.34), we observed:

Note: further occurrences of request parsing errors will be logged at
DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a
domain name.
at
org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
at
org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
at
org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
at
org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)


The main issue for us now is that, since we ship Tomcat with our product
(IOW Tomcat runs in customer environments), this will break our product
functionality if customers have hostnames with underscore. Ideally, they
should correct the host names (to be RFC compliant) but customers would be
really averse to change hostname as it might a widespread change in their
environments.

With Spring also, we ran into same issue but in later releases of Spring
this was fixed. I

Would it be a good idea to make this strict check configurable so that we
can continue to cater to our customers without breaking the functionality?

Are there any other alternates to help solve this issue?

Thanks,
Amit

[M. Manna]

Read the description on RFC 7230 and 3986 - just to be sure. You might be
right after all.

Post by Amit Pande
Yes, I did check the description, but did not find reference to underscore.
“The value may be any combination of the following characters: " < > [ \ ]
^ ` { | } . “
But I admit that I did not actually verify it and will see if these
attributes work for underscore too.
Thanks,
Amit
Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?
Hello all,
Recent Tomcat versions (8.5.32 I think) has made a stricter validation for
hostnames with underscores in it. (
https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)
This is understandably for addressing security issues (
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6816) and
enforcing RFC compliance, in some way.
Note: further occurrences of request parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: The character [_] is never valid in a domain name.
at
org.apache.tomcat.util.http.parser.HttpParser$DomainParseState.next(HttpParser.java:946)
at
org.apache.tomcat.util.http.parser.HttpParser.readHostDomainName(HttpParser.java:842)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:66)
at org.apache.tomcat.util.http.parser.Host.parse(Host.java:40)
at
org.apache.coyote.AbstractProcessor.parseHost(AbstractProcessor.java:286)
at
org.apache.coyote.http11.Http11Processor.prepareRequest(Http11Processor.java:1203)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:776)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net<http://org.apache.tomcat.util.net>
.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
The main issue for us now is that, since we ship Tomcat with our product
(IOW Tomcat runs in customer environments), this will break our product
functionality if customers have hostnames with underscore. Ideally, they
should correct the host names (to be RFC compliant) but customers would be
really averse to change hostname as it might a widespread change in their
environments.
With Spring also, we ran into same issue but in later releases of Spring this was fixed. I
Would it be a good idea to make this strict check configurable so that we
can continue to cater to our customers without breaking the functionality?
Are there any other alternates to help solve this issue?
Thanks,
Amit

[Mark Thomas]

Post by M. Manna
Have you checked the connector config doc for relaxedPathChars and
relaxedQueryChars?

Those options have no impact on the parsing of the hist name.

Post by M. Manna

Post by Amit Pande
Hello all,
Recent Tomcat versions (8.5.32 I think) has made a stricter

validation for

Post by Amit Pande
hostnames with underscores in it. (
https://bz.apache.org/bugzilla/show_bug.cgi?id=62371)

<snip/>

Post by M. Manna

Post by Amit Pande
The main issue for us now is that, since we ship Tomcat with our

product

Post by Amit Pande
(IOW Tomcat runs in customer environments), this will break our

product

Post by Amit Pande
functionality if customers have hostnames with underscore. Ideally,

they

Post by Amit Pande
should correct the host names (to be RFC compliant) but customers

would be

Post by Amit Pande
really averse to change hostname as it might a widespread change in

their

Post by Amit Pande
environments.
With Spring also, we ran into same issue but in later releases of

Spring

Post by Amit Pande
this was fixed. I
Would it be a good idea to make this strict check configurable so

that we

Post by Amit Pande
can continue to cater to our customers without breaking the

functionality?

The host names are invalid and should be fixed. Generally, Tomcat does not provide options to break spec compliance unless there is a compelling case to do so. I haven't seen such a case made for relaxing the host name parsing.

Post by M. Manna

Post by Amit Pande
Are there any other alternates to help solve this issue?

Short of patching Tomcat yourself (not a course of action I recommend), no.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org