[jbosscache-dev] Critical CacheMarshaller issue

[Manik Surtani]

A nasty bug, spotted by someone in the user forum (initially as a CCE)

	http://jira.jboss.org/jira/browse/JBCACHE-1211

Copying from the JIRA:

"This is a nasty. What started life as an optimisation for certain  
types of objects in a marshalled stream (Fqn, GlobalTransactio, String  
and Serializable) has become a major limitation in that a single  
stream can only hold up to 32767 different (not equal()) instances of  
such objects.

Basically the optimisation was, for example, instead of writing  
"hello" to a stream twice, just write it once and use a reference for  
all subsequent times. Unfortunately this reference was encoded as a  
short, hence the limitation of 32767.

Fixing this will definitely break wire compatibility with JBoss Cache  
2.0.0, although JBC does allow backward compatibility by specifying  
replication version in your configuration, thanks to the  
VersionAwareMarshaller. "

So I guess this mandates the need for a CacheMarshaller210.  The  
question is how do we fix this.  The obvious thing is to replace the  
short references with integers.  The 2 ^ 31 - 1 number of references  
this would allow should be plenty!  The drawback though, is larger  
streams.  4-byte refs instead of 2-byte refs can be an unnecessary  
overhead especially if objects aren't repeated much.

One thing we could do is just limit ref counting to Fqn and  
GlobalTransaction types, of which we know can often be repeated in a  
modification list, and where the optimisation can benefit, and assume  
it isn't worth optimising in the same way for user data (Strings and  
Serializables).

Or maybe we should drop the ref counting altogether, save a few more  
bytes' worth of payload.

Thoughts?
--
Manik Surtani
Lead, JBoss Cache
manik at jboss.org