<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 30, 2017 at 5:01 PM, Gustavo Fernandes <span dir="ltr"><<a href="mailto:gustavo@infinispan.org" target="_blank">gustavo@infinispan.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span class="">On Thu, Mar 30, 2017 at 4:49 PM, Galder Zamarreño <span dir="ltr"><<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>></span> wrote:<br></span><div class="gmail_extra"><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span><br>
--<br>
Galder Zamarreño<br>
Infinispan, Red Hat<br>
<br>
</span><span>> On 30 Mar 2017, at 17:15, Gustavo Fernandes <<a href="mailto:gustavo@infinispan.org" target="_blank">gustavo@infinispan.org</a>> wrote:<br>
><br>
><br>
><br>
> On Thu, Mar 30, 2017 at 1:51 PM, Galder Zamarreño <<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>> wrote:<br>
> Hi all,<br>
><br>
> For a demo I'm giving next week, I'd like to show how to use distributed streams via a remote server task. All server tasks that we have in testsuite rely on primitives but in my case I wanted to use POJOs.<br>
><br>
> To do that, I needed to get compatibility mode working in such way that those POJOs could be unmarshalled for the server task. Since in another demo I'm showing Protostream based POJOs, I thought I'd try to use that as mechanism to unmarshall POJOs server side.<br>
><br>
> We have a test for such scenario [1], but the reality (running on a proper server) is anything that simple. Here's a list of things I've found out while creating a WordCount example that relies on a POJO:<br>
><br>
> 1. Out of the box, it's impossible to set compatibility marshaller to <a href="http://org.infinispan.query.remote.Co">org.infinispan.query.remote.Co</a><wbr>mpatibilityProtoStreamMarshall<wbr>er [1] because "org.infinispan.main" classloader can't access that class. I worked around that by tweaking the module.xml to have an optional dependency to "org.infinispan.remote-query.s<wbr>erver" module.<br>
><br>
> 2. After doing that, I had to register the protofile and associated classes remotely in the server. Again, there's no out of the box mechanism for that, so I created a remote server task that would do that [3].<br>
><br>
><br>
> AFAICT, you should be able to do that doing a PUT in the Protobuf_Metadata cache, which entails having auth enabled. This cache should be REPL_SYNC, so no need to run a server task.<br>
<br>
</span>Good point but not so sure it completely removes the need for the task. The task does two things:<br>
<br>
1. Call ProtobufMetadataManager.regist<wbr>erProtofile, which as you say could be swapped with a cache.put on the metadata cache.<br>
<br>
2. Call ProtobufMetadataManager.regist<wbr>erMarshaller. This goes deep into updating SerializationContextImpl, which seems independent of any replicated cache.<br></blockquote><div><br><br></div></span><div>AFAICT (again), there is an internal listener or interceptor that upon metadata cache change, will update the SerCtx on all nodes.<span class="HOEnZb"><font color="#888888"><br><br></font></span></div><span class="HOEnZb"><font color="#888888"><div>Gustavo<br></div></font></span></div></div></div></blockquote><div><br><br></div><div>Ops, misread that, you are registering marshallers *in the server*, so yes, you probably need a task for that :)<br></div><div><br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="HOEnZb"><font color="#888888"><div></div></font></span><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
In fact, I had originally set up the task to execute in only in one node, but when I did that I found that marshallers were not registered in all nodes, so I had to execute the task in all nodes.<br>
<br>
I guess the task could be just limited to only executing 2.) in all nodes (and store the protofile contents by accessing the cache remotely), but I can't see how I can avoid the task altogether right now.<br>
<div class="m_-7179315547095768238HOEnZb"><div class="m_-7179315547095768238h5"><br>
><br>
><br>
><br>
> 3. Finally, with all that in place, I was able to complete the WordCount test [4] with a final caveat: the return of the word count, and words protofile registration, tasks return objects that are not marshalled by the compatibility marshaller, so I had to make sure that the remote cache manager used for those tasks uses the default marshaller.<br>
><br>
> Clearly we need to improve on this, and we have plans to address these issues (with new upcoming transcoding capabilities), but I thought it'd be worth mentioning the problems found in case anyone else encounters them before transcoding is in place.<br>
><br>
> Cheers,<br>
><br>
> [1] <a href="https://github.com/galderz/datagrid-patterns/blob/master/server-config/domain/domain.xml#L139" rel="noreferrer" target="_blank">https://github.com/galderz/dat<wbr>agrid-patterns/blob/master/ser<wbr>ver-config/domain/domain.xml#<wbr>L139</a><br>
> [2] <a href="https://github.com/galderz/datagrid-patterns/blob/master/server-config/org.infinispan.main_module.xml#L18" rel="noreferrer" target="_blank">https://github.com/galderz/dat<wbr>agrid-patterns/blob/master/ser<wbr>ver-config/org.infinispan.main<wbr>_module.xml#L18</a><br>
> [3] <a href="https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java" rel="noreferrer" target="_blank">https://github.com/galderz/dat<wbr>agrid-patterns/blob/master/ana<wbr>lytics-stream/tasks-server/src<wbr>/main/java/test/WordsProtoTask<wbr>.java</a><br>
> [4] <a href="https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-client/src/test/java/test/WordCountTest.java" rel="noreferrer" target="_blank">https://github.com/galderz/dat<wbr>agrid-patterns/blob/master/ana<wbr>lytics-stream/tasks-client/src<wbr>/test/java/test/WordCountTest.<wbr>java</a><br>
> --<br>
> Galder Zamarreño<br>
> Infinispan, Red Hat<br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailma<wbr>n/listinfo/infinispan-dev</a><br>
><br>
> ______________________________<wbr>_________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailma<wbr>n/listinfo/infinispan-dev</a><br>
<br>
<br>
______________________________<wbr>_________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailma<wbr>n/listinfo/infinispan-dev</a></div></div></blockquote></span></div><br></div></div>
</blockquote></div><br></div></div>