On Thu, Mar 30, 2017 at 4:49 PM, Galder Zamarreño <galder(a)redhat.com> wrote:
--
Galder Zamarreño
Infinispan, Red Hat
> On 30 Mar 2017, at 17:15, Gustavo Fernandes <gustavo(a)infinispan.org>
wrote:
>
>
>
> On Thu, Mar 30, 2017 at 1:51 PM, Galder Zamarreño <galder(a)redhat.com>
wrote:
> Hi all,
>
> For a demo I'm giving next week, I'd like to show how to use distributed
streams via a remote server task. All server tasks that we have in
testsuite rely on primitives but in my case I wanted to use POJOs.
>
> To do that, I needed to get compatibility mode working in such way that
those POJOs could be unmarshalled for the server task. Since in another
demo I'm showing Protostream based POJOs, I thought I'd try to use that as
mechanism to unmarshall POJOs server side.
>
> We have a test for such scenario [1], but the reality (running on a
proper server) is anything that simple. Here's a list of things I've found
out while creating a WordCount example that relies on a POJO:
>
> 1. Out of the box, it's impossible to set compatibility marshaller to
org.infinispan.query.remote.CompatibilityProtoStreamMarshaller [1]
because "org.infinispan.main" classloader can't access that class. I
worked
around that by tweaking the module.xml to have an optional dependency to
"org.infinispan.remote-query.server" module.
>
> 2. After doing that, I had to register the protofile and associated
classes remotely in the server. Again, there's no out of the box mechanism
for that, so I created a remote server task that would do that [3].
>
>
> AFAICT, you should be able to do that doing a PUT in the
Protobuf_Metadata cache, which entails having auth enabled. This cache
should be REPL_SYNC, so no need to run a server task.
Good point but not so sure it completely removes the need for the task.
The task does two things:
1. Call ProtobufMetadataManager.registerProtofile, which as you say could
be swapped with a cache.put on the metadata cache.
2. Call ProtobufMetadataManager.registerMarshaller. This goes deep into
updating SerializationContextImpl, which seems independent of any
replicated cache.
AFAICT (again), there is an internal listener or interceptor that upon
metadata cache change, will update the SerCtx on all nodes.
Gustavo
In fact, I had originally set up the task to execute in only in one node,
but when I did that I found that marshallers were not registered in all
nodes, so I had to execute the task in all nodes.
I guess the task could be just limited to only executing 2.) in all nodes
(and store the protofile contents by accessing the cache remotely), but I
can't see how I can avoid the task altogether right now.
>
>
>
> 3. Finally, with all that in place, I was able to complete the WordCount
test [4] with a final caveat: the return of the word count, and words
protofile registration, tasks return objects that are not marshalled by the
compatibility marshaller, so I had to make sure that the remote cache
manager used for those tasks uses the default marshaller.
>
> Clearly we need to improve on this, and we have plans to address these
issues (with new upcoming transcoding capabilities), but I thought it'd be
worth mentioning the problems found in case anyone else encounters them
before transcoding is in place.
>
> Cheers,
>
> [1]
https://github.com/galderz/datagrid-patterns/blob/master/
server-config/domain/domain.xml#L139
> [2]
https://github.com/galderz/datagrid-patterns/blob/master/
server-config/org.infinispan.main_module.xml#L18
> [3]
https://github.com/galderz/datagrid-patterns/blob/master/
analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java
> [4]
https://github.com/galderz/datagrid-patterns/blob/master/
analytics-stream/tasks-client/src/test/java/test/WordCountTest.java
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev