[aerogear-dev] Prepping for Data Sync in AeroGear.js

[Kris Borchers]

On Oct 10, 2012, at 3:22 PM, Burr Sutter <bsutter at redhat.com> wrote:

> Ideally our framework would be highly configurable:
> - collision detection and resolution at the row/tuple/object-level
> - or at the column/property level
> - sync-synchronization - block the user thread
> - async-synchronization - perform on a "background" thread
> - utilization push notifications to cause the client-app to walk up and request the changes
> 
> It is relatively rare to have a collision (two users or two batch processes) updating the same record simultaneously but it is a use case that should be addressed.   There was one technology I worked with in the past that took a hash of the before "image" of the record and submit that back with the deltas - relatively small over-the-wire packet and allowed easy detection to see if another data element changed.

That's actually a good idea. Easy enough to stringify and hash a chunk of data and send that along with the changed record. Then the server does the same hash before an update and can report collisions. I like that a lot actually.
> 
> On Oct 10, 2012, at 12:53 PM, Sébastien Blanc wrote:
> 
>> I think we should also take a look at the "slowly changing dimension " theories : 
>> http://en.m.wikipedia.org/wiki/Slowly_changing_dimension#
>> In particularly type VI
>> 
>> Envoyé de mon iPhone
>> 
>> Le Oct 10, 2012 à 18:38, Douglas Campos <qmx at qmx.me> a écrit :
>> 
>>> 
>>> On Oct 10, 2012, at 1:07 PM, Sébastien Blanc wrote:
>>> 
>>>> Hi,
>>>> Really interesting. I'm also thinking a lot about the data synch process and I'm glad to see that your ideas are pretty much the same as me.
>>>> But great challenges to come like dealing multiple edits of the same record by different clients :)
>>> One strategy that I've seen in the wild is having smaller records, (a contact entity could potentially have a timestamp for each field, to aid during the merge)
>>> 
>>> I know it's pretty aggressive and wasteful - just food for thought
>>> 
>>>> About triggering the synch process we could start a POC with the "online" event on the client side, that could spawn a webwork script taking care of the synch process. 
>>>> On the server side, we have to manage a sort of queue maintaining the state of the "dirty" records on which the clients can subscribe (and being noticed by push events , web sockets etc ...) , very exciting stuff to come.
>>>> 
>>>> Seb
>>>> Envoyé de mon iPhone
>>>> 
>>>> Le Oct 10, 2012 à 17:31, Kris Borchers <kris at redhat.com> a écrit :
>>>> 
>>>>> Hey all,
>>>>> 
>>>>> Just wanted to let you know what I am working on right now. It is very early but Matthias loves e-mail so I thought I would throw this out there.
>>>>> 
>>>>> Basically, right now all I am doing is working on keeping track of the status of data in DataManager. Assuming a new dataSync setting is set to true, these are the things I am planning.
>>>>> 
>>>>>  • New record is added to store, status is set to NEW and a UUID is generated
>>>>>  • Record is updated, status is set to MODIFIED
>>>>>  • Record is removed, status is set to REMOVED (data is not actually removed, future will need to keep storage limits in mind)
>>>>>  • Add a new sync method that will run through the data and sync with the server. Assumes client is data of record for now during development until we can determine a strategy for informing the client how the server tracks data status and how it should be informed that this data is being synced
>>>>>      • This brings us back to the discussion about sending metadata to the client on first app load. That metadata could inform the client of the sync strategy
>>>>> 
>>>>> There's probably more to this but these are my original thoughts right now.
>>>>> 
>>>>> Kris
>>>>> _______________________________________________
>>>>> aerogear-dev mailing list
>>>>> aerogear-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>> _______________________________________________
>>>> aerogear-dev mailing list
>>>> aerogear-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>> 
>>> -- qmx
>>> 
>>> 
>>> _______________________________________________
>>> aerogear-dev mailing list
>>> aerogear-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>> 
>> _______________________________________________
>> aerogear-dev mailing list
>> aerogear-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
> 
> 
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev