[infinispan-dev] Executing server tasks that contain POJOs

[Gustavo Fernandes]

On Thu, Mar 30, 2017 at 5:01 PM, Gustavo Fernandes <gustavo at infinispan.org>
wrote:

> On Thu, Mar 30, 2017 at 4:49 PM, Galder Zamarreño <galder at redhat.com>
> wrote:
>
>>
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>> > On 30 Mar 2017, at 17:15, Gustavo Fernandes <gustavo at infinispan.org>
>> wrote:
>> >
>> >
>> >
>> > On Thu, Mar 30, 2017 at 1:51 PM, Galder Zamarreño <galder at redhat.com>
>> wrote:
>> > Hi all,
>> >
>> > For a demo I'm giving next week, I'd like to show how to use
>> distributed streams via a remote server task. All server tasks that we have
>> in testsuite rely on primitives but in my case I wanted to use POJOs.
>> >
>> > To do that, I needed to get compatibility mode working in such way that
>> those POJOs could be unmarshalled for the server task. Since in another
>> demo I'm showing Protostream based POJOs, I thought I'd try to use that as
>> mechanism to unmarshall POJOs server side.
>> >
>> > We have a test for such scenario [1], but the reality (running on a
>> proper server) is anything that simple. Here's a list of things I've found
>> out while creating a WordCount example that relies on a POJO:
>> >
>> > 1. Out of the box, it's impossible to set compatibility marshaller to
>> org.infinispan.query.remote.CompatibilityProtoStreamMarshaller [1]
>> because "org.infinispan.main" classloader can't access that class. I worked
>> around that by tweaking the module.xml to have an optional dependency to
>> "org.infinispan.remote-query.server" module.
>> >
>> > 2. After doing that, I had to register the protofile and associated
>> classes remotely in the server. Again, there's no out of the box mechanism
>> for that, so I created a remote server task that would do that [3].
>> >
>> >
>> > AFAICT, you should be able to do that doing a PUT in the
>> Protobuf_Metadata cache, which entails having auth enabled. This cache
>> should be REPL_SYNC, so no need to run a server task.
>>
>> Good point but not so sure it completely removes the need for the task.
>> The task does two things:
>>
>> 1. Call ProtobufMetadataManager.registerProtofile, which as you say
>> could be swapped with a cache.put on the metadata cache.
>>
>> 2. Call ProtobufMetadataManager.registerMarshaller. This goes deep into
>> updating SerializationContextImpl, which seems independent of any
>> replicated cache.
>>
>
>
> AFAICT (again), there is an internal listener or interceptor that upon
> metadata cache change, will update the SerCtx on all nodes.
>
> Gustavo
>


Ops, misread that, you are registering marshallers *in the server*, so yes,
you probably need a task for that :)



>
>
>>
>> In fact, I had originally set up the task to execute in only in one node,
>> but when I did that I found that marshallers were not registered in all
>> nodes, so I had to execute the task in all nodes.
>>
>> I guess the task could be just limited to only executing 2.) in all nodes
>> (and store the protofile contents by accessing the cache remotely), but I
>> can't see how I can avoid the task altogether right now.
>>
>> >
>> >
>> >
>> > 3. Finally, with all that in place, I was able to complete the
>> WordCount test [4] with a final caveat: the return of the word count, and
>> words protofile registration, tasks return objects that are not marshalled
>> by the compatibility marshaller, so I had to make sure that the remote
>> cache manager used for those tasks uses the default marshaller.
>> >
>> > Clearly we need to improve on this, and we have plans to address these
>> issues (with new upcoming transcoding capabilities), but I thought it'd be
>> worth mentioning the problems found in case anyone else encounters them
>> before transcoding is in place.
>> >
>> > Cheers,
>> >
>> > [1] https://github.com/galderz/datagrid-patterns/blob/master/ser
>> ver-config/domain/domain.xml#L139
>> > [2] https://github.com/galderz/datagrid-patterns/blob/master/ser
>> ver-config/org.infinispan.main_module.xml#L18
>> > [3] https://github.com/galderz/datagrid-patterns/blob/master/ana
>> lytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java
>> > [4] https://github.com/galderz/datagrid-patterns/blob/master/ana
>> lytics-stream/tasks-client/src/test/java/test/WordCountTest.java
>> > --
>> > Galder Zamarreño
>> > Infinispan, Red Hat
>> >
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> >
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20170330/cef1cee5/attachment-0001.html