[jboss-dev-forums] [Design of JBossCache] - String size limitations with TreeCacheMarshaller

[manik.surtani@jboss.com]

See http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3968346 and JBCACHE-756.

The story is that Strings are written using writeUTF() which has a 65535 byte limit.  See:

http://java.sun.com/j2se/1.5.0/docs/api/java/io/DataOutput.html#writeUTF(java.lang.String)

While I initially thought that fixing this would:

1)  Involve having to write String objects directly (inefficient) as any other form of writing a byte stream will result in hitting some limit or the other.
2)  Break binary stream compat with the 1.4 series.

I've expressed these in the JIRA task.

Thinking about this a bit more,

1)  Perhaps we can chunk Strings into several writeUTF() calls.  2 bytes can be written first, to identify how many chunks to expect.  This will give us a limit of 2^16 x 2^16 bytes, or 4GB of String data (!).

2)  We can make this backward-compatible by using the current format for Strings under 64kb, and a new magic number (MAGIC_NUMBER_LARGE_STRING ?) for a 'chunked' version, up to 4GB Strings.

Thoughts? 

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3968359#3968359

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=3968359