<div dir="ltr">On Thu, Mar 30, 2017 at 4:49 PM, Galder Zamarreño <span dir="ltr"><<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
--<br>
Galder Zamarreño<br>
Infinispan, Red Hat<br>
<br>
</span><span class="">> On 30 Mar 2017, at 17:15, Gustavo Fernandes <<a href="mailto:gustavo@infinispan.org">gustavo@infinispan.org</a>> wrote:<br>
><br>
><br>
><br>
> On Thu, Mar 30, 2017 at 1:51 PM, Galder Zamarreño <<a href="mailto:galder@redhat.com">galder@redhat.com</a>> wrote:<br>
> Hi all,<br>
><br>
> For a demo I'm giving next week, I'd like to show how to use distributed streams via a remote server task. All server tasks that we have in testsuite rely on primitives but in my case I wanted to use POJOs.<br>
><br>
> To do that, I needed to get compatibility mode working in such way that those POJOs could be unmarshalled for the server task. Since in another demo I'm showing Protostream based POJOs, I thought I'd try to use that as mechanism to unmarshall POJOs server side.<br>
><br>
> We have a test for such scenario [1], but the reality (running on a proper server) is anything that simple. Here's a list of things I've found out while creating a WordCount example that relies on a POJO:<br>
><br>
> 1. Out of the box, it's impossible to set compatibility marshaller to org.infinispan.query.remote.<wbr>CompatibilityProtoStreamMarsha<wbr>ller [1] because "org.infinispan.main" classloader can't access that class. I worked around that by tweaking the module.xml to have an optional dependency to "org.infinispan.remote-query.<wbr>server" module.<br>
><br>
> 2. After doing that, I had to register the protofile and associated classes remotely in the server. Again, there's no out of the box mechanism for that, so I created a remote server task that would do that [3].<br>
><br>
><br>
> AFAICT, you should be able to do that doing a PUT in the Protobuf_Metadata cache, which entails having auth enabled. This cache should be REPL_SYNC, so no need to run a server task.<br>
<br>
</span>Good point but not so sure it completely removes the need for the task. The task does two things:<br>
<br>
1. Call ProtobufMetadataManager.<wbr>registerProtofile, which as you say could be swapped with a cache.put on the metadata cache.<br>
<br>
2. Call ProtobufMetadataManager.<wbr>registerMarshaller. This goes deep into updating SerializationContextImpl, which seems independent of any replicated cache.<br></blockquote><div><br><br></div><div>AFAICT (again), there is an internal listener or interceptor that upon metadata cache change, will update the SerCtx on all nodes.<br><br></div><div>Gustavo<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
In fact, I had originally set up the task to execute in only in one node, but when I did that I found that marshallers were not registered in all nodes, so I had to execute the task in all nodes.<br>
<br>
I guess the task could be just limited to only executing 2.) in all nodes (and store the protofile contents by accessing the cache remotely), but I can't see how I can avoid the task altogether right now.<br>
<div class="HOEnZb"><div class="h5"><br>
><br>
><br>
><br>
> 3. Finally, with all that in place, I was able to complete the WordCount test [4] with a final caveat: the return of the word count, and words protofile registration, tasks return objects that are not marshalled by the compatibility marshaller, so I had to make sure that the remote cache manager used for those tasks uses the default marshaller.<br>
><br>
> Clearly we need to improve on this, and we have plans to address these issues (with new upcoming transcoding capabilities), but I thought it'd be worth mentioning the problems found in case anyone else encounters them before transcoding is in place.<br>
><br>
> Cheers,<br>
><br>
> [1] <a href="https://github.com/galderz/datagrid-patterns/blob/master/server-config/domain/domain.xml#L139" rel="noreferrer" target="_blank">https://github.com/galderz/<wbr>datagrid-patterns/blob/master/<wbr>server-config/domain/domain.<wbr>xml#L139</a><br>
> [2] <a href="https://github.com/galderz/datagrid-patterns/blob/master/server-config/org.infinispan.main_module.xml#L18" rel="noreferrer" target="_blank">https://github.com/galderz/<wbr>datagrid-patterns/blob/master/<wbr>server-config/org.infinispan.<wbr>main_module.xml#L18</a><br>
> [3] <a href="https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java" rel="noreferrer" target="_blank">https://github.com/galderz/<wbr>datagrid-patterns/blob/master/<wbr>analytics-stream/tasks-server/<wbr>src/main/java/test/<wbr>WordsProtoTask.java</a><br>
> [4] <a href="https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-client/src/test/java/test/WordCountTest.java" rel="noreferrer" target="_blank">https://github.com/galderz/<wbr>datagrid-patterns/blob/master/<wbr>analytics-stream/tasks-client/<wbr>src/test/java/test/<wbr>WordCountTest.java</a><br>
> --<br>
> Galder Zamarreño<br>
> Infinispan, Red Hat<br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/infinispan-<wbr>dev</a><br>
><br>
> ______________________________<wbr>_________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/infinispan-<wbr>dev</a><br>
<br>
<br>
______________________________<wbr>_________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/<wbr>mailman/listinfo/infinispan-<wbr>dev</a></div></div></blockquote></div><br></div></div>