WebSocket message size limits

Post by Igor Urisman
Dear All,
I am looking for help in understanding why the size of the inbound
WebSocket message is limited to 125 bytes.

It isn't, at least not by Tomcat.

Post by Igor Urisman
I realize that this may not
even be the right place for my question, but am still hoping for a clue.
From looking at the RFC 6455, Sec. 5.2 Base Framing Protocol, I am making
1. There's nothing in it to suggest a payload length asymmetry between
inbound and outbound messages. Yet, although I am able to send very large
messages to the browser, an attempt to send anything over 125 bytes results
in error and a connection shutdown. (I tried FF and Chrome on a Mac).
2. It's easy to see from the wire protocol why 125 is the simplest payload
length but other sizes up to unsigned 64 bit int are supported. So,
browser's failure to transmit more than 125 bits indicates both, the most
restrictive payload size AND lack of support for fragmented messages.
The error that FF gives reads "The decoded text message was too big for the
output buffer and the endpoint does not support partial messages" which to
me reads like they are saying that Tomcat did not indicate during handshake
that it accepts multi-part messages. True?

False. There is nothing in the handshake that allows one end not to
support multi-part messages nor to limit the maximum message length.

That looks like a client side issue to me. Maybe you need to make the
client side output buffer bigger.

Tomcat's web socket support (client and server) has been tested with the
Autobahn testsuite that includes tests with significantly larger
messages than 125 bytes both as single frames and as multiple frames.

Post by Igor Urisman
I can't speak for others, but for my project 125 bytes is unacceptably
small. So, fundamentally what I need to know is this: do I need to
implement my own fragmenting or am I missing something?

You are missing something. No idea what since you haven't provided any
details of your client side implementation.

Mark

Igor Urisman

2013-09-23 04:49:43 UTC

Mark,

Sorry for the delay in responding. I am working on this project in my spare
time.
That I was missing something wasn't the question. The question was what.

I think I figured it out, but wanted to follow up for two reasons. One is
the benefit of
others in my situation (this stuff is still pretty sparsely documented) and
two, there are
some further points that I am still not sure about and hope someone could
comment on.

Because of security concerns, the transmission of large messages over a
WebSocket
has different semantics for in- (from client to server) and out-bound (from
server to
client) messages. Since it is up to the client to establish a WebSocket
session, the
server is free to send a message of any size by simply calling

javax.websocket.RemoteEndpoint.sendText(String message)

I imagine client implementations may differ in how they deal with
particularly large
payloads, but anything reasonable will be transmitted and received just
fine and at
lest as defined by the JSR 356, sizes of up to unsigned long are supported.

The same is not the case for the inbound messages. And the way Tomcat
handles
large messages is neither standard, nor at this point (well) documented.
Here's what
I have discovered:

The websocket.send(String) method takes a payload of any size. (There may
be further nuances in cases when client generates inbound traffic faster
than the
network is able to transmit it, but at least when bufferedAmount == 0, this
statement is
true.)

However, the server implementation is free to pick the maximum size of a
payload
that it is willing to receive as a whole. Tomcat designers chose that size
to be 125
bytes. Reasonable number given the particulars of the wire level protocol,
but not
one defined in the standard.

In other words, if I want to receive inbound message that can be longer
than 125
bytes, I need to configure the server side session with a message handler
that
implements
javax.websocket.MessageHandler.Partial

The implementation will partition the message into fragments of up to 8192
characters
and make them available to the app code via the implementation of
onMessage(String part, boolean last) method.

It appears that there's also a talk of exposing the inbound message via a
Reader <http://www.oracle.com/technetwork/articles/java/jsr356-1937161.html>but
that's not in Java EE 7.

So, just as a specific example, my implementation looks like the following:

public class FermiMessageHandler implements MessageHandler.Partial<String> {

private Session session;
private int maxMessageSize = 20000;
private StringBuilder messageBuffer = new StringBuilder(maxMessageSize);

FermiMessageHandler(Session session) {
this.session = session;
}

@Override
public void onMessage(String msgPart, boolean last) {
if( messageBuffer.length() + msgPart.length() > maxMessageSize) {
session.close(new
CloseReason(CloseReason.CloseCodes.CLOSED_ABNORMALLY, "Message is too
long");
}
else {
messageBuffer.append(msgPart);
if (last) {
String message = messageBuffer.toString();
// We have a complete message. Do something with it.
messageBuffer.setLength(0);
}
}
}
}

Thanks,
-Igor.

Post by Igor Urisman
Dear All,
I am looking for help in understanding why the size of the inbound
WebSocket message is limited to 125 bytes.

It isn't, at least not by Tomcat.

large

Post by Igor Urisman
messages to the browser, an attempt to send anything over 125 bytes

results

Post by Igor Urisman
in error and a connection shutdown. (I tried FF and Chrome on a Mac).
2. It's easy to see from the wire protocol why 125 is the simplest

payload

Post by Igor Urisman
length but other sizes up to unsigned 64 bit int are supported. So,
browser's failure to transmit more than 125 bits indicates both, the most
restrictive payload size AND lack of support for fragmented messages.
The error that FF gives reads "The decoded text message was too big for

the

Post by Igor Urisman
output buffer and the endpoint does not support partial messages" which

Post by Igor Urisman
me reads like they are saying that Tomcat did not indicate during

handshake

Post by Igor Urisman
that it accepts multi-part messages. True?

False. There is nothing in the handshake that allows one end not to
support multi-part messages nor to limit the maximum message length.
That looks like a client side issue to me. Maybe you need to make the
client side output buffer bigger.
Tomcat's web socket support (client and server) has been tested with the
Autobahn testsuite that includes tests with significantly larger
messages than 125 bytes both as single frames and as multiple frames.

You are missing something. No idea what since you haven't provided any
details of your client side implementation.
Mark
---------------------------------------------------------------------

Mark Thomas

2013-09-23 10:39:30 UTC

This post might be inappropriate. Click to display it.

Igor Urisman

2013-09-23 15:44:06 UTC

Thanks for the speedy reply, Mark.

I have thought about that code for a minute. You're right; what it does is
construct the entire
message in memory. My use case has no use for partials and the message
sizes are
tens to hundreds of Kb. Didn't mean to defeat anything there, just the use
case.

If there is a way to change the default min size of a whole message, that
would certainly be
the way to go,---another right for you. There's only one problem: I don't
know how.
Do you?

-Igor.

Post by Igor Urisman
However, the server implementation is free to pick the maximum size of a
payload
that it is willing to receive as a whole. Tomcat designers chose that

size

Post by Igor Urisman
to be 125
bytes. Reasonable number given the particulars of the wire level

protocol,

Post by Igor Urisman
but not
one defined in the standard.

That statement not correct. The default incoming buffer size for text
and binary messages is 8k bytes. This applies to both the client and the
server.
There is a specification (RFC6455) mandated limit on control message
payloads of 125 bytes.

Post by Igor Urisman
It appears that there's also a talk of exposing the inbound message via a
Reader <

http://www.oracle.com/technetwork/articles/java/jsr356-1937161.html>but

Post by Igor Urisman
that's not in Java EE 7.

Another incorrect statement. MessageHandler.Whole<Reader> is supported
in JSR-356. Note that Readers only work with whole messages, not partial
ones.

Post by Igor Urisman
So, just as a specific example, my implementation looks like the
public class FermiMessageHandler implements

MessageHandler.Partial<String> {

Post by Igor Urisman
private Session session;
private int maxMessageSize = 20000;
private StringBuilder messageBuffer = new

StringBuilder(maxMessageSize);

Post by Igor Urisman
FermiMessageHandler(Session session) {
this.session = session;
}
@Override
public void onMessage(String msgPart, boolean last) {
if( messageBuffer.length() + msgPart.length() > maxMessageSize) {
session.close(new
CloseReason(CloseReason.CloseCodes.CLOSED_ABNORMALLY, "Message is too
long");
}
else {
messageBuffer.append(msgPart);
if (last) {
String message = messageBuffer.toString();
// We have a complete message. Do something with it.
messageBuffer.setLength(0);
}
}
}
}

Just think about what you have done for a minute. The whole point of
partial messages is so that you don't have to buffer the entire message
in memory before processing it. Your code defeats the point of that
entirely. You'd be better off increasing the maximum message size
supported by the implementation and using MessageHandler.Whole<String>.
Better still would be to re-write your handler so it actually processed
partial messages.
Mark
---------------------------------------------------------------------

Mark Thomas

2013-09-23 16:07:32 UTC

Post by Igor Urisman
Thanks for the speedy reply, Mark.
I have thought about that code for a minute. You're right; what it does is
construct the entire
message in memory. My use case has no use for partials and the message
sizes are
tens to hundreds of Kb. Didn't mean to defeat anything there, just the use
case.
If there is a way to change the default min size of a whole message, that
would certainly be
the way to go,---another right for you. There's only one problem: I don't
know how.
Do you?

See the Tomcat 8 WebSocket docs for details:
http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html

Mark

Igor Urisman

2013-09-24 05:15:29 UTC

Thanks again, Mark, for the quick turnaround.
Which of the 5 parameters on this page would be responsible for changing
the 125-byte max whole text message size?
-Igor.

Post by Igor Urisman
Thanks for the speedy reply, Mark.
I have thought about that code for a minute. You're right; what it does

Post by Igor Urisman
construct the entire
message in memory. My use case has no use for partials and the message
sizes are
tens to hundreds of Kb. Didn't mean to defeat anything there, just the

use

Post by Igor Urisman
case.
If there is a way to change the default min size of a whole message, that
would certainly be
the way to go,---another right for you. There's only one problem: I

don't

Post by Igor Urisman
know how.
Do you?

http://tomcat.apache.org/tomcat-8.0-doc/web-socket-howto.html
Mark
---------------------------------------------------------------------

Niki Dokovski

2013-09-24 06:56:54 UTC

This post might be inappropriate. Click to display it.

Igor Urisman

2013-09-25 16:04:05 UTC

You're right, Nikki,-- he has. There was a point to all this in the
beginning but I think it's all good now.
All seems to work as advertized. I went back to some earlier test cases
and what I thought was failing
is in fact working fine.
-Igor.

Post by Niki Dokovski

Post by Igor Urisman
Thanks again, Mark, for the quick turnaround.
Which of the 5 parameters on this page would be responsible for changing
the 125-byte max whole text message size?

Mark did a great job describing the properties and if it's still unclear
looking at the origins is always an option.
125 sizing is for the control frames [1]. The internal buffers for text and
binary frames are 8k. Respective sizes are subject to configuration by
org.apache.tomcat.websocket.textBufferSize and
org.apache.tomcat.websocket.binaryBufferSize
Cheers
Niki
[1] http://tools.ietf.org/html/rfc6455#section-5.5
-Igor.

Post by Igor Urisman
Thanks for the speedy reply, Mark.
I have thought about that code for a minute. You're right; what it

does

Post by Mark Thomas
is

Post by Igor Urisman
construct the entire
message in memory. My use case has no use for partials and the

message

Post by Igor Urisman
sizes are
tens to hundreds of Kb. Didn't mean to defeat anything there, just

the

Post by Mark Thomas
use

Post by Igor Urisman
case.
If there is a way to change the default min size of a whole message,

that