[infinispan-dev] ProtoStream and ease of use
Adrian Nistor
anistor at redhat.com
Tue Jul 16 12:24:50 EDT 2013
By 'that' you mean adding @PSType to each an every field? That works but
is verbose and generally a PITA.
On 07/16/2013 07:06 PM, Mircea Markus wrote:
> On 15 Jul 2013, at 18:29, Adrian Nistor <anistor at redhat.com> wrote:
>
>> BaseMessage is not mandatory. It's there to 'help' you implement Message interface, which is optional anyway and trivially simple to implement; it's only a holder for the bag of unknown fields. Ignore that and you loose support for maintaining unknown fields but otherwise everything works fine.
>>
>> The class field ordering issue is problematic. Besides that, schema evolution might lead to removed fields so we must be able to skip a range of ids easily. I think the sane (but verbose) way to solve this is to make @PSType mandatory for all fields to be marshalled as you suggested, and PSType should have and 'index' or 'id' parameter. But that's mostly goodbye convention then.
> why wouldn't that work?
>
>> Let's also note that were' striving to be cross language here and on the server side we won't have the user's domain classes available in CP (because they might not even be written in java). So the annotations are not going to help there. We would still need the protobuf file, which could be generated based on the annotated java code as we discussed earlier today, but ... generators kill kittens :( so would try to avoid that.
>>
>> Since we want to be cross-language I think we should go schema-first. So maybe we could have the protobuf hand-written and the java classes (pojos) minimally annotated with just the full name of the corresponding protobuf message type. From here everything could work automagically provided that class field names match the field name in the proto file and also field types match so no value convertor needs to be specified. If they don't, then annotate.
>>
>> Does this seem a little better?
>>
>> On 07/15/2013 06:57 PM, Emmanuel Bernard wrote:
>>
>>> ProtoStream is like this
>>>
>>> https://github.com/infinispan/protostream/blob/master/core/src/test/java/org/infinispan/protostream/domain/Account.java
>>> https://github.com/infinispan/protostream/blob/master/core/src/test/java/org/infinispan/protostream/domain/marshallers/AccountMarshaller.java
>>> (I believe the BaseMessage superclass of Account is optional, not sure).
>>>
>>> A convention based approach would be like this
>>>
>>> package org.infinispan.protostream.domain;
>>>
>>> import org.infinispan.protostream.BaseMessage;
>>>
>>> /**
>>> * @author ebernard at redhat.com
>>> */
>>> public class Account {
>>>
>>> private int id;
>>> private String description;
>>>
>>> public int getId() {
>>> return id;
>>> }
>>>
>>> public void setId(int id) {
>>> this.id = id;
>>> }
>>>
>>> public String getDescription() {
>>> return description;
>>> }
>>>
>>> public void setDescription(String description) {
>>> this.description = description;
>>> }
>>>
>>> @Override
>>> public String toString() {
>>> return "Account{" +
>>> "id=" + id +
>>> ", description='" + description + '\'' +
>>> ", unknownFieldSet='" + unknownFieldSet + '\'' +
>>> '}';
>>> }
>>> }
>>>
>>> Or let's imagine that we need to make id use a specific protobuf type
>>>
>>> package org.infinispan.protostream.domain;
>>>
>>> import org.infinispan.protostream.BaseMessage;
>>>
>>> /**
>>> * @author ebernard at redhat.com
>>> */
>>> public class Account {
>>>
>>> @PSType(UINT32)
>>> private int id;
>>> private String description;
>>>
>>> public int getId() {
>>> return id;
>>> }
>>>
>>> public void setId(int id) {
>>> this.id = id;
>>> }
>>>
>>> public String getDescription() {
>>> return description;
>>> }
>>>
>>> public void setDescription(String description) {
>>> this.description = description;
>>> }
>>>
>>> @Override
>>> public String toString() {
>>> return "Account{" +
>>> "id=" + id +
>>> ", description='" + description + '\'' +
>>> ", unknownFieldSet='" + unknownFieldSet + '\'' +
>>> '}';
>>> }
>>> }
>>>
>>> Note that a concern is that field ordering (in the bytecode) is not guaranteed across VMs and compilation and I believe that is an important factor of ProtoBuf. So somehow we would need a way to express field indexes wich would amke the annotation approach more verbose.
>>>
>>> On Mon 2013-07-15 16:04, Manik Surtani wrote:
>>>> I'm sorry I missed this. Is there an example of each API somewhere?
>>>>
>>>> On 15 Jul 2013, at 14:01, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>>>>
>>>>> Mircea, Adrian and I had an IRC chat on ProtoStream and ProtoStuff.
>>>>>
>>>>> check out
>>>>> http://transcripts.jboss.org/channel/irc.freenode.org/%23infinispan/2013/%23infinispan.2013-07-15.log.html
>>>>> starting at 11:00 and finishing at 12:30
>>>>>
>>>>> A short summary of what has been discussed:
>>>>>
>>>>> - ProtoStream is a good cross-platform solution but
>>>>> - complicated for the simple pure Java case
>>>>> - encourages a technical superclass (EJB 2 !!!!!)
>>>>> - ProtoStuff convention + annotation based approach
>>>>> https://code.google.com/p/protostuff/wiki/ProtostuffRuntime is nice
>>>>> for the pure Java case
>>>>> - ProtoStuff is many things and has a non ProtoBuf compliant format for
>>>>> cycle ref and polymorphism
>>>>> - ProtoStream supports unknown fields (future version of a schema),
>>>>> ProtoBuf does not
>>>>> - we could build a convention based solution atop ProtoStream
>>>>> - assuming UnknownFieldSet and BaseMessage are optional
>>>>> - using (cross platform) conventions
>>>>> - with metadata to go beyond conventions (annotation, programmatic
>>>>> API, XML...)
>>>>>
>>>>> public long size; //uses fixed64 by default
>>>>> @PSType(UINT64) long size; //override protobuf type
>>>>>
>>>>> - Infinispan will/could(?) have a repo of schema that can be queries
>>>>> - we are talking about how the schema is resolved / generated
>>>>> - what we send through the wire is independent of the Proto*
>>>>>
>>>>> A ProtoBuf vs ProtoStream comparison points
>>>>> https://gist.github.com/mmarkus/5999646
>>>>>
>>>>> Emmanuel
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> twitter.com/maniksurtani
>>>>
>>>> Platform Architect, JBoss Data Grid
>>>> http://red.ht/data-grid
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> Cheers,
More information about the infinispan-dev
mailing list