Re: [aerogear-dev] Sync Notes / Early issues

On Jan 26, 2015, at 2:36 PM, Lucas Holmquist <lholmqui(a)redhat.com&gt; wrote: > > On Jan 26, 2015, at 11:49 AM, Summers Pittman <supittma(a)redhat.com <mailto:supittma@redhat.com>> wrote: > > On 01/26/2015 11:37 AM, Lucas Holmquist wrote: >> Perhaps we have already brought this up, but what are we(our sync stuff) trying to be. >> >> Are we trying to be Dropbox/google docs/etc….. or something else(what that is i’m not sure). >> >> What are our real use cases here? > Here are some simple ones that I want the sync server for > * A Conference app user synchronizes a personalized schedule among his devices i sort of like this one. it is somewhat a real-time thing. I guess the real question around what our Sync Stuff wants to be when it gets older, is does it want to be Real-Time? which i think we’ve probably agreed on more than once > * A Users preferences are restored / synchronized across his devices (filtered by some kind of profile. IE phone preference, tablet preferences, desktop preferences) > * A list of content is continually updated (think Facebook feed) > > So this gives us two usage patterns > > 1. A single user-document synchronized across devices > 2. A single document synchronized across all user*devices. >

> I don't think anyone has thought of some truly collaborative things like Google docs where you have multiple users on multiple devices editing multiple documents all with different permission tuples. > > Many things can be fit in these two scenarios however. In the Dropbox use case you mention the "document" is the users files, metadata, and their location. The actual data will probably be hosted some place else. > >> >> >>> On Jan 26, 2015, at 11:31 AM, Summers Pittman <supittma(a)redhat.com <mailto:supittma@redhat.com>> wrote: >>> >>> On 01/26/2015 11:26 AM, Randall Hauch wrote: >>>> From this and previous threads, it sounds like you’re targeting how multiple users could collaboratively edit the same document. Essentially, this is like Google Docs, though perhaps a bit lighter weight. Am I misunderstanding the scenario? >>> Document is a generic term. It is a wrapper around a JSON serializable >>> object. >>>> >>>> For example, would this kind of data sync be useful when there are millions of shared documents/entities? For example, a Yelp-like app would probably want to have an entity/document for each restaurant or business, where that document contained an aggregation of information about that business. In my mind, these kinds of documents are more akin to JPA entities — or rather akin to aggregates of related JPA entity objects. >>> Yes ish. Right now the sync server is very young, but we can model the >>> data as a single document and have the apps display views into it and >>> make edits appropriately. The trick will be making sure that the >>> serialization is always the same (IE two clients aren't constantly >>> updating because they serialize data in different orders) >>>> >>>> Best regards, >>>> >>>> Randall >>>> >>>> >>>>> On Jan 26, 2015, at 9:38 AM, Summers Pittman <supittma(a)redhat.com <mailto:supittma@redhat.com>> wrote: >>>>> >>>>> >>>>> Summary (From my POV on Android): >>>>> >>>>> Sync Server doesn't use any authentication or ownership tracking. >>>>> GCM-XMPP bridge needs a lot of love >>>>> We need to define a different connection lifecycle for GCM. >>>>> The in memory data store is problematic because clients and servers >>>>> must be stopped and started atomically >>>>> We might want to show off syncing different types of documents (i.e. >>>>> a todo list in addition to Luke's hobbies) >>>>> Fixing the GCM bridge is probably a couple weeks of work to get it >>>>> "solid". That will be a good alpha.1/preview to show off. >>>>> >>>>> So Last week I put together a demo to try and stretch the legs of the >>>>> Android Sync Client APIs. >>>>> >>>>> It crashes, a lot. Which is a bit to be expected as the code hasn't >>>>> really be used for, well, anything until now. We will get to that though. >>>>> >>>>> Here is the alpha.1 workflow. You log in and you see your docs. You can >>>>> edit your docs or you can create new ones. In the future I would like >>>>> to add sharing and collaboration but that's the future. Here's a flow >>>>> chart for the visual thinkers out there (with real screen caps from the >>>>> real working app) >>>>> >>>>> * >>>>> https://docs.google.com/drawings/d/145XuutxR1yY0k81w2nIIy980itDIwxzH_gcDE... <https://docs.google.com/drawings/d/145XuutxR1yY0k81w2nIIy980itDIwxzH_gcDE... >>>>> >>>>> To make all of this work I use a RESTful server which tracks a user's >>>>> username and the documents they "own". The sync server just syncs and >>>>> serves the document you ask for. It has no authentication and any doc >>>>> you ask for you get to be an editor on. >>>>> >>>>> The client uses the GCM XMPP¹ bridge I wrote while drunk on the side of >>>>> a mountain and it shows. The biggest issue is that shadows for >>>>> documents aren't getting created right sometimes because either 1) the >>>>> client or server bounced and the data stores are no long synced or 2) >>>>> the server thinks there are more clients than actually are connected. >>>>> GCM-XMPP doesn't supply connection/disconnection information like >>>>> WebSockets will. Instead we just know that some messages we sent a >>>>> while ago weren't delivered. We need to figure out how to turn this >>>>> into connection and disconnection information in a way that lets the >>>>> shadows exist correctly. >>>>> >>>>> Another issue that needs to be addressed is using something other than >>>>> the Luke Skywalker hobbies document. (Or maybe showing off multiple >>>>> document types in the demo). I'm up for suggestions. >>>>> >>>>> Anyway, the principles (diff sync with a restful documentId broker for >>>>> security) are sound. I think we can buff out some implementation >>>>> details and have a good alpha/preview/poc release in the few weeks -> 1 >>>>> month time frame for Android and the sync server. >>>>> >>>>> -- >>>>> Summers Pittman >>>>>>> Phone:404 941 4698 >>>>>>> Java is my crack. >>>>> >>>>> >>>>> Foot Notes: >>>>> 1. The sync server has two client connection technologies : WebSockets >>>>> and GCM-XMPP. Android uses the GCM-XMPP because it takes all of the >>>>> nasty connection handling code and gives let's Android deal with it. >>>>> More info here : https://developer.android.com/google/gcm/ccs.html <https://developer.android.com/google/gcm/ccs.html> >>>>> >>>>> _______________________________________________ >>>>> aerogear-dev mailing list >>>>> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> >>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev <https://lists.jboss.org/mailman/listinfo/aerogear-dev> >>>> >>>> _______________________________________________ >>>> aerogear-dev mailing list >>>> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> >>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev <https://lists.jboss.org/mailman/listinfo/aerogear-dev> >>> >>> >>> -- >>> Summers Pittman >>>>> Phone:404 941 4698 >>>>> Java is my crack. >>> >>> _______________________________________________ >>> aerogear-dev mailing list >>> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> >>> https://lists.jboss.org/mailman/listinfo/aerogear-dev <https://lists.jboss.org/mailman/listinfo/aerogear-dev> >> >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/aerogear-dev <https://lists.jboss.org/mailman/listinfo/aerogear-dev> > > -- > Summers Pittman > >>Phone:404 941 4698 > >>Java is my crack. > _______________________________________________ > aerogear-dev mailing list > aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/aerogear-dev _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/aerogear-dev <https://lists.jboss.org/mailman/listinfo/aerogear-dev>