Discussion:
Request header too large..
Amit Pande
2018-11-02 21:16:45 UTC
Permalink
As per current implementation (below snippet is from 8.5.28), if the request header is too large (by default >8K, the default maxHttpHeaderSize), is thrown below error message is seen.


For request --
02-Nov-2018 15:15:47.649 INFO [catalina-exec-40] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Request header is too large
at org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:575)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:703)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)

For response—

02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40] org.apache.coyote.http11.Http11Processor.endRequest Error finishing response
org.apache.coyote.http11.HeadersTooLargeException: An attempt was made to write more data to the response headers than there was room available in the buffer. Increase maxHttpHeaderSize on the connector or write less data into the response headers.
at org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Http11OutputBuffer.java:544)
at org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.java:481)
at org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuffer.java:361)
at org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor.java:1339)
at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
at org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java:1523)


However, this doesn’t tell us the name of the offending header which I think is valuable in debugging the issues.

Do you feel it’s worth the enhancement request?

Also, I noticed the comments

“
// (1) Headers plus request line size does not exceed its limit



// (2) There are enough bytes to avoid expanding the buffer when


// reading body
“

It says request line size ..does that mean maxHttpHeaderSize comes in play for request line (which includes query params) as well?

Thanks,
Amit
Christopher Schultz
2018-11-02 22:25:24 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Amit,
Post by Amit Pande
As per current implementation (below snippet is from 8.5.28), if
the request header is too large (by default >8K, the default
maxHttpHeaderSize), is thrown below error message is seen.
For request -- 02-Nov-2018 15:15:47.649 INFO [catalina-exec-40]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
request header Note: further occurrences of HTTP header parsing
errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Request header is too large at
org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuf
fer.java:575)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:70
3)
Post by Amit Pande
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh
t.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPro
tocol.java:790)
Post by Amit Pande
For response—
02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40]
org.apache.coyote.http11.Http11Processor.endRequest Error finishing
response org.apache.coyote.http11.HeadersTooLargeException: An
attempt was made to write more data to the response headers than
there was room available in the buffer. Increase maxHttpHeaderSize
on the connector or write less data into the response headers. at
org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Htt
p11OutputBuffer.java:544)
at
org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.jav
a:481)
Post by Amit Pande
at
org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuf
fer.java:361)
at
org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor
.java:1339)
Post by Amit Pande
at
org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
at
org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java
:1523)
Post by Amit Pande
However, this doesn’t tell us the name of the offending header
which I think is valuable in debugging the issues.
Do you feel it’s worth the enhancement request?
Also, I noticed the comments
“ // (1) Headers plus request line size does not exceed its limit
// (2) There are enough bytes to avoid expanding the buffer when
// reading body “
It says request line size ..does that mean maxHttpHeaderSize comes
in play for request line (which includes query params) as well?
Yes.

The request headers are read before selecting which context will
service the request. The request line, which not formally a part of
the "headers" per the HTTP spec, are considered to be "part of the
headers" as far as Tomcat is concerned, because they all share the
same buffer.

To locate the offending header, look at the end of the request, then
re-wind to the previous "\r\nheader:". If you don't find
"\r\nheader:", then it was the request line. But it could be request +
partial-header-name, so you'd have to be careful.

I'm not sure it's worth it to add a debugging enhancement (you
probably already know what your huge headers are), but I am by no
means the King of Tomcat: feel free to request that enhancement in BZ
and see if you can get some support.

If you write a patch/PR for it, it will almost certainly be added as
long as its not too intrusive :)

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvcztQACgkQHPApP6U8
pFhj+Q//QIMUy44I2SDH5kM7UeMk2SCB/5J5hW79RGfIGITFpmEMttBONF8wilg7
U7ldyjDuu2Msn+ynRPVg5lxf9pq3OQJKySv7xFc3AG4C44em4xRrJI8aHGsLxH5N
dQyyJY/oP61HzeiiiyDLYxQY3sidC+4gieOEJ2oG5ZsXJwFpvfc0Q1LyiIzN8fXa
o2PkH0B2kPsksgXD8WZLPXAFMgAUdmUu0wVzM0lt/kBvlG9LGW/vEd+fVeLrG93s
UEs3IrWT2aTWnOVyZl2LAPhL2GynfrDUmsmsc8EF6kljXiX3IWphaH9z1nbCpLFM
zDsUoIRpqm1sgXObV52zW8YbnaUzIEAZaI7lIlrOctRvc9ACeCvKkIc+aVylgnmR
zpEhFz/gOchl1T6x2+MXLHavch653F8OgYY52/TCqxQheVKSHjc2jJu9UTcb+7z0
EBbfeWvUeEdNd7edGORhKx6tJLotbRhF1Yr2sjxxkY8I31RGHv+ZlNStOuxr5uoI
0c68L2XGbe9gBaLbjopYOfb+Y8y73iX3wfJPKIlvXl8LkupA1wyg0NAwOn+ZDL2E
Rl865F3+DaaVaFoSRkpgp7pUrKfh9wi/N2CdCJ13hH6WY32/JD8+Z2X+YuRfMu6+
6dh+NiVvrQc998/gutkx+OnH+RTC0STJp6RTzYNZbnL5hi5z1Ms=
=bjZe
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Amit Pande
2018-11-03 01:35:14 UTC
Permalink
Thanks Chris. Yes, I will soon send out the patch for review.

Thanks,
Amit

On 11/2/18, 5:25 PM, "Christopher Schultz" <***@christopherschultz.net> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Amit,
Post by Amit Pande
As per current implementation (below snippet is from 8.5.28), if
the request header is too large (by default >8K, the default
maxHttpHeaderSize), is thrown below error message is seen.
For request -- 02-Nov-2018 15:15:47.649 INFO [catalina-exec-40]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
request header Note: further occurrences of HTTP header parsing
errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Request header is too large at
org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuf
fer.java:575)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:70
3)
Post by Amit Pande
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh
t.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractPro
tocol.java:790)
Post by Amit Pande
For response—
02-Nov-2018 15:15:47.867 SEVERE [catalina-exec-40]
org.apache.coyote.http11.Http11Processor.endRequest Error finishing
response org.apache.coyote.http11.HeadersTooLargeException: An
attempt was made to write more data to the response headers than
there was room available in the buffer. Increase maxHttpHeaderSize
on the connector or write less data into the response headers. at
org.apache.coyote.http11.Http11OutputBuffer.checkLengthBeforeWrite(Htt
p11OutputBuffer.java:544)
at
org.apache.coyote.http11.Http11OutputBuffer.write(Http11OutputBuffer.jav
a:481)
Post by Amit Pande
at
org.apache.coyote.http11.Http11OutputBuffer.sendStatus(Http11OutputBuf
fer.java:361)
at
org.apache.coyote.http11.Http11Processor.prepareResponse(Http11Processor
.java:1339)
Post by Amit Pande
at
org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:260)
at
org.apache.coyote.http11.Http11Processor.endRequest(Http11Processor.java
:1523)
Post by Amit Pande
However, this doesn’t tell us the name of the offending header
which I think is valuable in debugging the issues.
Do you feel it’s worth the enhancement request?
Also, I noticed the comments
“ // (1) Headers plus request line size does not exceed its limit
// (2) There are enough bytes to avoid expanding the buffer when
// reading body “
It says request line size ..does that mean maxHttpHeaderSize comes
in play for request line (which includes query params) as well?
Yes.

The request headers are read before selecting which context will
service the request. The request line, which not formally a part of
the "headers" per the HTTP spec, are considered to be "part of the
headers" as far as Tomcat is concerned, because they all share the
same buffer.

To locate the offending header, look at the end of the request, then
re-wind to the previous "\r\nheader:". If you don't find
"\r\nheader:", then it was the request line. But it could be request +
partial-header-name, so you'd have to be careful.

I'm not sure it's worth it to add a debugging enhancement (you
probably already know what your huge headers are), but I am by no
means the King of Tomcat: feel free to request that enhancement in BZ
and see if you can get some support.

If you write a patch/PR for it, it will almost certainly be added as
long as its not too intrusive :)

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvcztQACgkQHPApP6U8
pFhj+Q//QIMUy44I2SDH5kM7UeMk2SCB/5J5hW79RGfIGITFpmEMttBONF8wilg7
U7ldyjDuu2Msn+ynRPVg5lxf9pq3OQJKySv7xFc3AG4C44em4xRrJI8aHGsLxH5N
dQyyJY/oP61HzeiiiyDLYxQY3sidC+4gieOEJ2oG5ZsXJwFpvfc0Q1LyiIzN8fXa
o2PkH0B2kPsksgXD8WZLPXAFMgAUdmUu0wVzM0lt/kBvlG9LGW/vEd+fVeLrG93s
UEs3IrWT2aTWnOVyZl2LAPhL2GynfrDUmsmsc8EF6kljXiX3IWphaH9z1nbCpLFM
zDsUoIRpqm1sgXObV52zW8YbnaUzIEAZaI7lIlrOctRvc9ACeCvKkIc+aVylgnmR
zpEhFz/gOchl1T6x2+MXLHavch653F8OgYY52/TCqxQheVKSHjc2jJu9UTcb+7z0
EBbfeWvUeEdNd7edGORhKx6tJLotbRhF1Yr2sjxxkY8I31RGHv+ZlNStOuxr5uoI
0c68L2XGbe9gBaLbjopYOfb+Y8y73iX3wfJPKIlvXl8LkupA1wyg0NAwOn+ZDL2E
Rl865F3+DaaVaFoSRkpgp7pUrKfh9wi/N2CdCJ13hH6WY32/JD8+Z2X+YuRfMu6+
6dh+NiVvrQc998/gutkx+OnH+RTC0STJp6RTzYNZbnL5hi5z1Ms=
=bjZe
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org



B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB��[��X��ܚX�KK[XZ[�\�\��][��X��ܚX�P�X�] �\X�K�ܙ�B��܈Y][ۘ[��[X[��K[
Mark Thomas
2018-11-03 09:39:08 UTC
Permalink
Post by Amit Pande
Thanks Chris. Yes, I will soon send out the patch for review.
Logically, I can see some hurdles for such a patch that are going to be
difficult to overcome.

Tomcat will process all of the received data up to the limit. It is on
the subsequent read to get more data that the exception is thrown.

The issue is how to determine which header is actually the problem as it
may not be the one currently being read.

Consider the following:

ReqLine - 1024 bytes
Headers - 5 * 128 bytes
Header - 1 * 6*1024 bytes
Headers - 5 * 128 bytes

Is the problem the 6k header? The 1k request line? Has the client
duplicated the block of 5 128 byte headers? Or is the request valid and
Tomcat's limit too low?

There is no way for Tomcat to determine where the root cause lies. The
only reliable way to figure out what went wrong is for a knowledgeable
system admin to look at the complete set of headers.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org
Amit Pande
2018-11-06 00:02:29 UTC
Permalink
Thanks Mark for the update. Seems like there is a potential bug in our client components which happened to keep on adding to existing header list.

https://tomcat.apache.org/tomcat-8.5-doc/config/http.html

From the documentation:

maxHttpHeaderSize :The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB).

Although some online searches (e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) suggest that this maxHttpHeaderSize applies to request line as well, could the documentation be made more elaborate?

Can we mention that this attribute puts the upper bound on combined size of all request headers and the request line?

Thanks,
Amit
Post by Amit Pande
Thanks Chris. Yes, I will soon send out the patch for review.
Logically, I can see some hurdles for such a patch that are going to be
difficult to overcome.

Tomcat will process all of the received data up to the limit. It is on
the subsequent read to get more data that the exception is thrown.

The issue is how to determine which header is actually the problem as it
may not be the one currently being read.

Consider the following:

ReqLine - 1024 bytes
Headers - 5 * 128 bytes
Header - 1 * 6*1024 bytes
Headers - 5 * 128 bytes

Is the problem the 6k header? The 1k request line? Has the client
duplicated the block of 5 128 byte headers? Or is the request valid and
Tomcat's limit too low?

There is no way for Tomcat to determine where the root cause lies. The
only reliable way to figure out what went wrong is for a knowledgeable
system admin to look at the complete set of headers.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org



B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB��[��X��ܚX�KK[XZ[�\�\��][��X��ܚX�P�X�] �\X�K�ܙ�B��܈Y][ۘ[��[X[��K[
Mark Thomas
2018-11-06 10:22:32 UTC
Permalink
Post by Amit Pande
Thanks Mark for the update. Seems like there is a potential bug in our client components which happened to keep on adding to existing header list.
https://tomcat.apache.org/tomcat-8.5-doc/config/http.html
maxHttpHeaderSize :The maximum size of the request and response HTTP header, specified in bytes. If not specified, this attribute is set to 8192 (8 KB).
Although some online searches (e.g. https://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat) suggest that this maxHttpHeaderSize applies to request line as well, could the documentation be made more elaborate?
Can we mention that this attribute puts the upper bound on combined size of all request headers and the request line?
Sure. Patch / pull request welcome.

Mark
Post by Amit Pande
Thanks,
Amit
Post by Amit Pande
Thanks Chris. Yes, I will soon send out the patch for review.
Logically, I can see some hurdles for such a patch that are going to be
difficult to overcome.
Tomcat will process all of the received data up to the limit. It is on
the subsequent read to get more data that the exception is thrown.
The issue is how to determine which header is actually the problem as it
may not be the one currently being read.
ReqLine - 1024 bytes
Headers - 5 * 128 bytes
Header - 1 * 6*1024 bytes
Headers - 5 * 128 bytes
Is the problem the 6k header? The 1k request line? Has the client
duplicated the block of 5 128 byte headers? Or is the request valid and
Tomcat's limit too low?
There is no way for Tomcat to determine where the root cause lies. The
only reliable way to figure out what went wrong is for a knowledgeable
system admin to look at the complete set of headers.
Mark
---------------------------------------------------------------------
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@tomcat.apache.org
For additional commands, e-mail: users-***@tomcat.apache.org

Loading...