[infinispan-dev] Cache Store Marshalling

[Dan Berindei]

+100 to use a separate marshaller for user types, but I think the
persistence configuration is the wrong place for it. Users shouldn't
have to implement an AdvancedExternalizer for efficient replication
between nodes if they already have a marshaller that's efficient
enough for use in a store.

I would have liked the default user type marshaller to use
jboss-marshalling with the all the 8.2.x internal externalizers, in
order to have backwards compatibility for the stores without needing
an upgrade tool. But it seems like the stores change their layout all
the time anyway, so I would settle for some version metadata in the
store and better error messages instead.

I'm not sure about the enum, don't like the feeling that marshallers
not bundled with the core are somehow second-class. I'd rather have a
single marshaller name string attribute, have a NAME constant in each
marshaller class, and register the names on startup, either via
ServiceLoader or via ModuleLifecycle.

There's also the question of whether we should support passing in a
marshaller instance instead of a class name (I think we should, at
least for testing) and whether we should allow custom configuration
attributes/elements for the marshaller in the XML configuration (no
idea).

Cheers
Dan



On Tue, Mar 14, 2017 at 1:26 PM, Ryan Emerson <remerson at redhat.com> wrote:
> Hi All,
>
> Currently the CacheWriterInterceptor utilises the internal marshaller for marshalling entries before they are sent to the configured cache stores. This causes several problems [1], most notably that changes to the internal marshaller make stored data incompatible across Infinispan versions.
>
> I propose that we decouple the internal and store marshaller. To allow 9.x versions to remain compatible, we should default to the internal marshaller (until 10.x at least), but optionally allow users to specify a custom StreamingMarshaller implementation as part of their PersistenceConfiguration. As we already have the protostuff and kryo bridges, users would specify an enum for the marshaller they want to use as well as an optional class string if a custom implementation is required. So for example:
>
> enum StoreMarshaller {
>    CUSTOM,
>    INTERNAL,
>    KRYO
>    PROTOSTUFF;
> }
>
> new ConfigurationBuilder()
>     .persistence()
>         .marshaller(StoreMarshaller.CUSTOM)
>         .marshallerClass("org.example.Marshaller")
>          ...
>
> Finally, Gustavo brought flatbuffers[2] to my attention which could be a good option to provide for users as one of the default StoreMarshaller implementations.
>
> WDYT?
>
> Cheers
> Ryan
>
> [1] https://docs.google.com/document/d/1PR0eYgjhqXUR5w03npS7TdOs2KDZjfdMh0au8XAsyJY
> [2] https://google.github.io/flatbuffers/
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev