Content length when HTTP Transfer-encoding is chunked

[Mahesh Seshan]

Hello,

I can obtain the content-length of a HTTP Servlet Request when the
Content-Length header is added by the client. However, when the
Transfer-encoding is chunked, how do I programatically obtain the
content length ?

The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.

Any help or pointers appreciated.

Thank you,

-mahesh

[Rainer Jung]

Post by Mahesh Seshan
Hello,
I can obtain the content-length of a HTTP Servlet Request when the
Content-Length header is added by the client. However, when the
Transfer-encoding is chunked, how do I programatically obtain the
content length ?

You can't without consuming the body.

Post by Mahesh Seshan
The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.

Exactly that's the reason, why there is no easy way to get the length.
Whatever component that could provide you with that information would
need to read all of the body (and buffer it for your later use) in order
to measure its length.

Chunked encoding works with chunks, individual chunks usually are small
and the protocol tells us, how big the next chunk is, but we never know
how big the size of all chunks together is.

Regards,

Rainer

[Mahesh Seshan]

Rainer,

Thank you very much for the prompt response.

To be more specific, Apache CXF is the consumer of HTTP request. Are
you indicating that Apache CXF is the one reading from Socket input
stream ? And that Tomcat HTTP Connector simply hands over the
Inputstream to CXF to consume the chunked contents ?

-mahesh

Post by Rainer Jung

Post by Mahesh Seshan
Hello,
I can obtain the content-length of a HTTP Servlet Request when the
Content-Length header is added by the client. However, when the
Transfer-encoding is chunked, how do I programatically obtain the
content length ?

You can't without consuming the body.

Post by Mahesh Seshan
The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.

Exactly that's the reason, why there is no easy way to get the length.
Whatever component that could provide you with that information would need
to read all of the body (and buffer it for your later use) in order to
measure its length.
Chunked encoding works with chunks, individual chunks usually are small and
the protocol tells us, how big the next chunk is, but we never know how big
the size of all chunks together is.
Regards,
Rainer
---------------------------------------------------------------------

[Rainer Jung]

Post by Mahesh Seshan
Rainer,
Thank you very much for the prompt response.
To be more specific, Apache CXF is the consumer of HTTP request. Are
you indicating that Apache CXF is the one reading from Socket input
stream ? And that Tomcat HTTP Connector simply hands over the
Inputstream to CXF to consume the chunked contents ?

Tomcat does the chunk decoding when the application consumes the data
via reading from the input stream. So the application will not be able
to see the real chunks, but Tomcat doesn't buffer the full POST body
before the application consumes it. It is important for chunked encoding
to support a streaming architecture.

Think about uploading a DVD ISO image, and the web server buffering the
whole image im memory before handing it over to an aplication. That
wouldn't be nice.

Regards,

Rainer

Post by Mahesh Seshan
-mahesh

Post by Rainer Jung

Post by Mahesh Seshan
Hello,
I can obtain the content-length of a HTTP Servlet Request when the
Content-Length header is added by the client. However, when the
Transfer-encoding is chunked, how do I programatically obtain the
content length ?

You can't without consuming the body.

Post by Mahesh Seshan
The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.

Exactly that's the reason, why there is no easy way to get the length.
Whatever component that could provide you with that information would need
to read all of the body (and buffer it for your later use) in order to
measure its length.
Chunked encoding works with chunks, individual chunks usually are small and
the protocol tells us, how big the next chunk is, but we never know how big
the size of all chunks together is.
Regards,
Rainer

[Christopher Schultz]

Mahesh,

Post by Mahesh Seshan
The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.

Given Rainer's comments, it seems like a reasonable course of action is
to read bytes from the request until you hit a maximum that you set
(say, 1MB). At that point, you can stop reading and return an error code
to the client.

I don't know about chunked encoding, but I seem to recall that Tomcat
can't recycle a request connection until all the data has been read from
the client (even if it is discarded). Hopefully, someone will correct me
and that won't be the case.

- -chris

[clam715]

I have the same problem. Can anyone tell me how to fix the problem? I try
to use filter to remove content-length, however it only work for
content-length that is greater than 8k. for page with less than 8k,
content-length header will always show up. Thank You

Post by Mahesh Seshan
Hello,
I can obtain the content-length of a HTTP Servlet Request when the
Content-Length header is added by the client. However, when the
Transfer-encoding is chunked, how do I programatically obtain the
content length ?
The task at hand for me is to prevent a HTTP client from issuing a
request that has a huge payload that could pose a resource risk on the
HTTP server.
Any help or pointers appreciated.
Thank you,
-mahesh
---------------------------------------------------------------------