[infinispan-dev] Executing server tasks that contain POJOs

[Galder Zamarreño]

Hey Ramesh,

I don't know your use case very well, so allow me to ask you some qs:

1. When does your marshaller come into play? At the compatibility layer? Or is it used as a client marshaller?

1a. If it's at the compatibility layer, why can't you use CompatibilityProtoStreamMarshaller?

Your description below makes it sound like your marshaller does similar work to CompatibilityProtoStreamMarshaller, hence the questions and see whether it could fit your use case.

Cheers,
--
Galder Zamarreño
Infinispan, Red Hat

> On 30 Mar 2017, at 16:30, Ramesh Reddy <rareddy at redhat.com> wrote:
> 
> Glader,
> 
> FWIW, I am rewriting Teiid translator to Infinispan, where I needed a portable marshaller that is simply based on .proto file as you describe issues with your step #2. The use of predefined custom java marshaller is not viable in my usecase, as I want to dynamically convert Relational Tables defined in the Teiid into POJO's in Infinispan, and ability to query them. I have written a such marshaller for my usecase you can see testcase at [1]. Basically I capture the metadata from .proto file and use that information in encoding/decoding the protostream, *most* of the needed code is already there in the ProtoStream libraries. 
> 
> BTW, with your Task example you gave me another idea how I can even further enhance the this integration layer in terms of updating multiple POJOs in single call :)
> 
> Ramesh..
> 
> [1] https://github.com/rareddy/infinispan/blob/master/translator-infinispan-hotrod/src/test/java/org/teiid/translator/infinispan/hotrod/TestTeiidTableMarsheller.java
> 
> 
> ----- Original Message -----
>> Hi all,
>> 
>> For a demo I'm giving next week, I'd like to show how to use distributed
>> streams via a remote server task. All server tasks that we have in testsuite
>> rely on primitives but in my case I wanted to use POJOs.
>> 
>> To do that, I needed to get compatibility mode working in such way that those
>> POJOs could be unmarshalled for the server task. Since in another demo I'm
>> showing Protostream based POJOs, I thought I'd try to use that as mechanism
>> to unmarshall POJOs server side.
>> 
>> We have a test for such scenario [1], but the reality (running on a proper
>> server) is anything that simple. Here's a list of things I've found out
>> while creating a WordCount example that relies on a POJO:
>> 
>> 1. Out of the box, it's impossible to set compatibility marshaller to
>> org.infinispan.query.remote.CompatibilityProtoStreamMarshaller [1] because
>> "org.infinispan.main" classloader can't access that class. I worked around
>> that by tweaking the module.xml to have an optional dependency to
>> "org.infinispan.remote-query.server" module.
>> 
>> 2. After doing that, I had to register the protofile and associated classes
>> remotely in the server. Again, there's no out of the box mechanism for that,
>> so I created a remote server task that would do that [3].
>> 
>> 3. Finally, with all that in place, I was able to complete the WordCount test
>> [4] with a final caveat: the return of the word count, and words protofile
>> registration, tasks return objects that are not marshalled by the
>> compatibility marshaller, so I had to make sure that the remote cache
>> manager used for those tasks uses the default marshaller.
>> 
>> Clearly we need to improve on this, and we have plans to address these issues
>> (with new upcoming transcoding capabilities), but I thought it'd be worth
>> mentioning the problems found in case anyone else encounters them before
>> transcoding is in place.
>> 
>> Cheers,
>> 
>> [1]
>> https://github.com/galderz/datagrid-patterns/blob/master/server-config/domain/domain.xml#L139
>> [2]
>> https://github.com/galderz/datagrid-patterns/blob/master/server-config/org.infinispan.main_module.xml#L18
>> [3]
>> https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-server/src/main/java/test/WordsProtoTask.java
>> [4]
>> https://github.com/galderz/datagrid-patterns/blob/master/analytics-stream/tasks-client/src/test/java/test/WordCountTest.java
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev