[aerogear-dev] Prepping for Data Sync in AeroGear.js

[Kris Borchers]

OK so the first pass at just tracking data status is here

https://github.com/aerogear/aerogear-js/commit/b110eafa93ce9ea39febbcea72fdc04c28ee50c3

Please take a look and let me know what you think. Not much but a start. :)

On Oct 10, 2012, at 3:25 PM, Kris Borchers <kris at redhat.com> wrote:

> 
> On Oct 10, 2012, at 3:22 PM, Burr Sutter <bsutter at redhat.com> wrote:
> 
>> Ideally our framework would be highly configurable:
>> - collision detection and resolution at the row/tuple/object-level
>> - or at the column/property level
>> - sync-synchronization - block the user thread
>> - async-synchronization - perform on a "background" thread
>> - utilization push notifications to cause the client-app to walk up and request the changes
>> 
>> It is relatively rare to have a collision (two users or two batch processes) updating the same record simultaneously but it is a use case that should be addressed.   There was one technology I worked with in the past that took a hash of the before "image" of the record and submit that back with the deltas - relatively small over-the-wire packet and allowed easy detection to see if another data element changed.
> 
> That's actually a good idea. Easy enough to stringify and hash a chunk of data and send that along with the changed record. Then the server does the same hash before an update and can report collisions. I like that a lot actually.
>> 
>> On Oct 10, 2012, at 12:53 PM, Sébastien Blanc wrote:
>> 
>>> I think we should also take a look at the "slowly changing dimension " theories : 
>>> http://en.m.wikipedia.org/wiki/Slowly_changing_dimension#
>>> In particularly type VI
>>> 
>>> Envoyé de mon iPhone
>>> 
>>> Le Oct 10, 2012 à 18:38, Douglas Campos <qmx at qmx.me> a écrit :
>>> 
>>>> 
>>>> On Oct 10, 2012, at 1:07 PM, Sébastien Blanc wrote:
>>>> 
>>>>> Hi,
>>>>> Really interesting. I'm also thinking a lot about the data synch process and I'm glad to see that your ideas are pretty much the same as me.
>>>>> But great challenges to come like dealing multiple edits of the same record by different clients :)
>>>> One strategy that I've seen in the wild is having smaller records, (a contact entity could potentially have a timestamp for each field, to aid during the merge)
>>>> 
>>>> I know it's pretty aggressive and wasteful - just food for thought
>>>> 
>>>>> About triggering the synch process we could start a POC with the "online" event on the client side, that could spawn a webwork script taking care of the synch process. 
>>>>> On the server side, we have to manage a sort of queue maintaining the state of the "dirty" records on which the clients can subscribe (and being noticed by push events , web sockets etc ...) , very exciting stuff to come.
>>>>> 
>>>>> Seb
>>>>> Envoyé de mon iPhone
>>>>> 
>>>>> Le Oct 10, 2012 à 17:31, Kris Borchers <kris at redhat.com> a écrit :
>>>>> 
>>>>>> Hey all,
>>>>>> 
>>>>>> Just wanted to let you know what I am working on right now. It is very early but Matthias loves e-mail so I thought I would throw this out there.
>>>>>> 
>>>>>> Basically, right now all I am doing is working on keeping track of the status of data in DataManager. Assuming a new dataSync setting is set to true, these are the things I am planning.
>>>>>> 
>>>>>> • New record is added to store, status is set to NEW and a UUID is generated
>>>>>> • Record is updated, status is set to MODIFIED
>>>>>> • Record is removed, status is set to REMOVED (data is not actually removed, future will need to keep storage limits in mind)
>>>>>> • Add a new sync method that will run through the data and sync with the server. Assumes client is data of record for now during development until we can determine a strategy for informing the client how the server tracks data status and how it should be informed that this data is being synced
>>>>>>     • This brings us back to the discussion about sending metadata to the client on first app load. That metadata could inform the client of the sync strategy
>>>>>> 
>>>>>> There's probably more to this but these are my original thoughts right now.
>>>>>> 
>>>>>> Kris
>>>>>> _______________________________________________
>>>>>> aerogear-dev mailing list
>>>>>> aerogear-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>>> _______________________________________________
>>>>> aerogear-dev mailing list
>>>>> aerogear-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>> 
>>>> -- qmx
>>>> 
>>>> 
>>>> _______________________________________________
>>>> aerogear-dev mailing list
>>>> aerogear-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>> 
>>> _______________________________________________
>>> aerogear-dev mailing list
>>> aerogear-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>> 
>> 
>> _______________________________________________
>> aerogear-dev mailing list
>> aerogear-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
> 
> 
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev