10:09 a.m.
Just to get some discussions going and have something to talk about I’ve drafted some API,
so what do you think?
// [option 1 fully automatic we create a pipe and add the posibilty to add a store for
failover and sync just happens at on- and offline events]
// and because merging can fail users can add a conflict handlers
Builder builder = Builder.createPipe(pipeConfig).addFailoverStore(storeConfig);
Pipe<Car> pipe = builder.pipe(Car.class);
pipe.addConfictHandler(new ConflictHandler() {
public void conflict(Field originalField, Field newField) {
// user interaction
}
});
// [option 2 explicit let the user specify when to sync and what to sync]
SyncedPipe pipe = SyncPipeBuilder.build(options); // SyncPipe Store and Pipe togheter
// or we only use a store to sync and tell the sync manager where to sync to
SyncManager syncManager = new SyncManger();
syncManger.filter(readFilter); // maybe we don't want to sync all data but just some
part
syncManager.addConnectionHandler(new ConnectionHandler() {
public void onConnection() {
syncManger.sync(pipe);
}
});
syncManger.addConfictHandler(new ConflictHandler() {
public void conflict(Field original, Field new) {
// user interaction
}
});
Cheers,
Erik Jan
6:16 p.m.
One of the questions I have about this API is how are push events
handled with sync or is this just an offline cache?
IE how is data sent down to the client when things change (or is that
not for this use case)
On Wed 04 Dec 2013 04:09:30 AM EST, Erik Jan de Wit wrote:
Just to get some discussions going and have something to talk about
I’ve drafted some API, so what do you think?
// [option 1 fully automatic we create a pipe and add the posibilty to
add a store for failover and sync just happens at on- and offline events]
// and because merging can fail users can add a conflict handlers
Builder builder =
Builder.createPipe(pipeConfig).addFailoverStore(storeConfig);
Pipe<Car> pipe = builder.pipe(Car.class);
pipe.addConfictHandler(new ConflictHandler() {
public void conflict(Field originalField, Field newField) {
// user interaction
}
});
// [option 2 explicit let the user specify when to sync and what to sync]
SyncedPipe pipe = SyncPipeBuilder.build(options); // SyncPipe Store
and Pipe togheter
// or we only use a store to sync and tell the sync manager where to
sync to
SyncManager syncManager = new SyncManger();
syncManger.filter(readFilter); // maybe we don't want to sync all data
but just some part
syncManager.addConnectionHandler(new ConnectionHandler() {
public void onConnection() {
syncManger.sync(pipe);
}
});
syncManger.addConfictHandler(new ConflictHandler() {
public void conflict(Field original, Field new) {
// user interaction
}
});
Cheers,
Erik Jan
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
8:44 a.m.
My idea was that when the client calls sync the server side state and the client side
state get merged and the delta is send
On 6 Dec,2013, at 18:16 , Summers Pittman <supittma(a)redhat.com> wrote:
One of the questions I have about this API is how are push events
handled with sync or is this just an offline cache?
IE how is data sent down to the client when things change (or is that not for this use
case)
2:27 p.m.
On Mon 09 Dec 2013 02:44:59 AM EST, Erik Jan de Wit wrote:
My idea was that when the client calls sync the server side state and
the client side state get merged and the delta is send
So this would also require a
server definition?
On 6 Dec,2013, at 18:16 , Summers Pittman <supittma(a)redhat.com> wrote:
> One of the questions I have about this API is how are push events handled with sync
or is this just an offline cache?
>
> IE how is data sent down to the client when things change (or is that not for this
use case)
>
2:30 p.m.
On Mon, Dec 9, 2013 at 2:27 PM, Summers Pittman <supittma(a)redhat.com> wrote:
On Mon 09 Dec 2013 02:44:59 AM EST, Erik Jan de Wit wrote:
> My idea was that when the client calls sync the server side state and
the client side state get merged and the delta is send
So this would also require a server definition?
I have the same interrogations, what do we want do deliver :
- an Unified Synch Server ( USS )
- Client Libs (ios, android, web, cordova plugin)
> On 6 Dec,2013, at 18:16 , Summers Pittman <supittma(a)redhat.com> wrote:
>
>> One of the questions I have about this API is how are push events
handled with sync or is this just an offline cache?
>>
>> IE how is data sent down to the client when things change (or is that
not for this use case)
>>
>
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
3:56 p.m.
This proposal is just java based client API, if we nail down how a user could build an app
that has sync functionality, all other things will fall into place. I suggest we keep the
protocol and the server implementation as simple as possible and like qmx also suggested.
JSON based with a hash as revision number for merging.
On 9 Dec,2013, at 14:30 , Sebastien Blanc <scm.blanc(a)gmail.com> wrote:
On Mon, Dec 9, 2013 at 2:27 PM, Summers Pittman <supittma(a)redhat.com> wrote:
On Mon 09 Dec 2013 02:44:59 AM EST, Erik Jan de Wit wrote:
> My idea was that when the client calls sync the server side state and the client
side state get merged and the delta is send
So this would also require a server definition?
I have the same interrogations, what do we want do deliver :
- an Unified Synch Server ( USS )
- Client Libs (ios, android, web, cordova plugin)
>
> On 6 Dec,2013, at 18:16 , Summers Pittman <supittma(a)redhat.com> wrote:
>
>> One of the questions I have about this API is how are push events handled with
sync or is this just an offline cache?
>>
>> IE how is data sent down to the client when things change (or is that not for
this use case)
>>
>
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:06 p.m.
On 12/09/2013 09:56 AM, Erik Jan de Wit wrote:
This proposal is just java based client API, if we nail down how a
user could build an app that has sync functionality, all other things
will fall into place. I suggest we keep the protocol and the server
implementation as simple as possible and like qmx also suggested. JSON
based with a hash as revision number for merging.
For real time sync this is a great primer:
https://neil.fraser.name/writing/sync/. The concepts can be expanded
beyond text of course.
I guess my question is do we want a really low level but universal
protocol which requires server support, or several separate APIs which
can handle legacy servers, servers with minor changes, or real time
capable servers?
As examples:
Legacy Servers can be periodic pollers.
Minor changes can be a sync on push type thing.
And realtime, is well, real time.
On 9 Dec,2013, at 14:30 , Sebastien Blanc <scm.blanc(a)gmail.com
<mailto:scm.blanc@gmail.com>> wrote:
>
>
>
> On Mon, Dec 9, 2013 at 2:27 PM, Summers Pittman <supittma(a)redhat.com
> <mailto:supittma@redhat.com>> wrote:
>
> On Mon 09 Dec 2013 02:44:59 AM EST, Erik Jan de Wit wrote:
> > My idea was that when the client calls sync the server side
> state and the client side state get merged and the delta is send
> So this would also require a server definition?
>
> I have the same interrogations, what do we want do deliver :
>
> - an Unified Synch Server ( USS )
> - Client Libs (ios, android, web, cordova plugin)
>
> >
> > On 6 Dec,2013, at 18:16 , Summers Pittman <supittma(a)redhat.com
> <mailto:supittma@redhat.com>> wrote:
> >
> >> One of the questions I have about this API is how are push
> events handled with sync or is this just an offline cache?
> >>
> >> IE how is data sent down to the client when things change (or
> is that not for this use case)
> >>
> >
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
8:34 a.m.
On 9 Dec,2013, at 17:06 , Summers Pittman <supittma(a)redhat.com> wrote:
For real time sync this is a great primer:
https://neil.fraser.name/writing/sync/. The concepts can be expanded beyond text of
course.
That is a great article, but all these techniques are only cool for documents. We could
support documents of course, but I was thinking more about Pipes and syncing those when
one has been offline for a while. So for example when you have a Car that has make Toyota
and one changes it to Toiota and another changes it Toyotas merging these to changes to
Toiotas is always wrong. We don’t need merge support we only need conflict resolution.
I guess my question is do we want a really low level but universal protocol which
requires server support, or several separate APIs which can handle legacy servers, servers
with minor changes, or real time capable servers?
Cool idea to connect a legacy backend and let the front-end deal with the sync. But we
can’t support conflict resolution in this scenario. For example client1 and client2 change
the same car object. Client1 changes the property colour from red to green and client2
changes it from red to blue. The change of client1 takes place first then the change of
client2 will be a conflict. Now to detect that conflict we could fetch the entity again
and check if the colour is still the same as our original value, but it could change in
the mean time again. So there is no guarantee that the data will be consistent.
Of course we could periodically fetch data from a legacy server and merge that with the
client state, but I don’t think that this is super useful the power of sync is changing
things to be able to work offline. So I think we need a protocol, can be really simple
just versioned JSON, and a server that will compare the changes and return conflicts.
As examples:
Legacy Servers can be periodic pollers.
Minor changes can be a sync on push type thing.
And realtime, is well, real time.
9:11 a.m.
On Tue, Dec 10, 2013 at 8:34 AM, Erik Jan de Wit <edewit(a)redhat.com> wrote:
On 9 Dec,2013, at 17:06 , Summers Pittman <supittma(a)redhat.com> wrote:
For real time sync this is a great primer:
https://neil.fraser.name/writing/sync/. The concepts can be expanded
beyond text of course.
That is a great article, but all these techniques are only cool for
documents. We could support documents of course, but I was thinking more
about Pipes and syncing those when one has been offline for a while. So for
example when you have a Car that has make Toyota and one changes it to
Toiota and another changes it Toyotas merging these to changes to Toiotas
is always wrong. We don’t need merge support we only need conflict
resolution.
+1
I guess my question is do we want a really low level but universal
protocol which requires server support, or several separate APIs which can
handle legacy servers, servers with minor changes, or real time capable
servers?
Cool idea to connect a legacy backend and let the front-end deal with the
sync. But we can’t support conflict resolution in this scenario. For
example client1 and client2 change the same car object. Client1 changes the
property colour from red to green and client2 changes it from red to blue.
The change of client1 takes place first then the change of client2 will be
a conflict. Now to detect that conflict we could fetch the entity again and
check if the colour is still the same as our original value, but it could
change in the mean time again. So there is no guarantee that the data will
be consistent.
Of course we could periodically fetch data from a legacy server and merge
that with the client state, but I don’t think that this is super useful the
power of sync is changing things to be able to work offline. So I think we
need a protocol, can be really simple just versioned JSON, and a server
that will compare the changes and return conflicts.
Yes, let's keep it simple for now. So, here in this situation the server
just report the conflict, right ? He does not take action on the conflict
itself. Do we state here that the client will always be responsible in
resolving the conflict ?
In your first message, you define a ConflictHandler interface, do we also
want to provide some implementations of the ConflictHandler that covers
most of the use cases ?
As examples:
Legacy Servers can be periodic pollers.
Minor changes can be a sync on push type thing.
And realtime, is well, real time.
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
10:32 a.m.
Yes, let's keep it simple for now. So, here in this situation the server just report
the conflict, right ? He does not take action on the conflict itself. Do we state here
that the client will always be responsible in resolving the conflict ?
+1 for keeping it simple, right the server will report a conflict and the user will in the
most simple scenario decide which change to keep. That is why I introduced this
ConflictHandler so the developer can present these conflicts to the user.
In your first message, you define a ConflictHandler interface, do we
also want to provide some implementations of the ConflictHandler that covers most of the
use cases ?
Sure we could have a default implementation that just takes the newest as the new version,
but we should keep other version as well.
4:13 p.m.
On 12/10/2013 02:34 AM, Erik Jan de Wit wrote:
On 9 Dec,2013, at 17:06 , Summers Pittman <supittma(a)redhat.com
<mailto:supittma@redhat.com>> wrote:
> For real time sync this is a great primer:
> https://neil.fraser.name/writing/sync/. The concepts can be expanded
> beyond text of course.
That is a great article, but all these techniques are only cool for
documents.
This technique is good for synced text. We've hooked our horse to
the
JSON cart pretty hard so we should be able to deal with that for a large
number of cases.
We could support documents of course, but I was thinking more about
Pipes and syncing those when one has been offline for a while.
So Pipes aren't
data per se. They are a one way query mechanism (IE the
pipe doesn't start spewing data all over your living room floor for no
reason.) Pipes can be used by Sync to fetch data once a signal to
refresh has been received however.
So for example when you have a Car that has make Toyota and one
changes it to Toiota and another changes it Toyotas merging these to
changes to Toiotas is always wrong. We don't need merge support we
only need conflict resolution.
'Toiotas' Yes, 'Toiotas' (no cap)
no. That would actually be correct in
patch diff merge.
>
> I guess my question is do we want a really low level but universal
> protocol which requires server support, or several separate APIs
> which can handle legacy servers, servers with minor changes, or real
> time capable servers?
Cool idea to connect a legacy backend and let the front-end deal with
the sync. But we can't support conflict resolution in this scenario.
We can
limit legacy servers to read only data (from a client perspective).
For example client1 and client2 change the same car object. Client1
changes the property colour from red to green and client2 changes it
from red to blue. The change of client1 takes place first then the
change of client2 will be a conflict. Now to detect that conflict we
could fetch the entity again and check if the colour is still the same
as our original value, but it could change in the mean time again. So
there is no guarantee that the data will be consistent.
Of course we could periodically fetch data from a legacy server and
merge that with the client state, but I don't think that this is super
useful the power of sync is changing things to be able to work offline.
Of course, offline is just laggy sync.
But lets take two separate but similar legacy use cases. Downloading the
weather forecast based on your location and synchronizing a todo list.
The weather data probably won't change during the day. So every night
at midnight + jitter our sync can download the forecast from a restful
URL and be done. There is no conflict because the client data can not /
will not change. This is a very simple case to implement.
Syncing a Todo list is more complicated because it can change from
multiple clients. HOWEVER the application has a restriction that todo
list items may only be added and removed but not reordered or edited.
The flow for that could look like
1. User 1 on Android creates a TODO list.
2. Android app creates TODO list copy on server.
3. Android app registers a sync with the server for the todo list
4. User 1 adds item to TODO list.
5. Server syncs its list with Users list. returns OK.
6. User 1 access his list in web app.
7. Web App registers a sync on the server with the todo list.
8. User 1 Edits his list.
9. Web App applies edits and sends deltas to Android app.
10. repeate ad nausiem.
In this case conflicts can only happen if a user deletes the same item
(no practical conflict). So a conflict resolver will be trivial.
So I think we need a protocol, can be really simple just versioned
JSON, and a server that will compare the changes and return conflicts.
Yeah. The
more I poke and play with things the less certain I am that
adding it to the pipe is a good idea. I think we need something separate.
>
> As examples:
> Legacy Servers can be periodic pollers.
> Minor changes can be a sync on push type thing.
> And realtime, is well, real time.
>
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:16 p.m.
On Tue, Dec 10, 2013 at 4:13 PM, Summers Pittman <supittma(a)redhat.com>wrote:
On 12/10/2013 02:34 AM, Erik Jan de Wit wrote:
On 9 Dec,2013, at 17:06 , Summers Pittman <supittma(a)redhat.com> wrote:
For real time sync this is a great primer:
https://neil.fraser.name/writing/sync/. The concepts can be expanded
beyond text of course.
That is a great article, but all these techniques are only cool for
documents.
This technique is good for synced text. We've hooked our horse to the
JSON cart pretty hard so we should be able to deal with that for a large
number of cases.
We could support documents of course, but I was thinking more about
Pipes and syncing those when one has been offline for a while.
So Pipes aren't data per se. They are a one way query mechanism (IE the
pipe doesn't start spewing data all over your living room floor for no
reason.) Pipes can be used by Sync to fetch data once a signal to refresh
has been received however.
So for example when you have a Car that has make Toyota and one changes
it to Toiota and another changes it Toyotas merging these to changes to
Toiotas is always wrong. We don’t need merge support we only need
conflict resolution.
'Toiotas' Yes, 'Toiotas' (no cap) no. That would actually be correct in
patch diff merge.
What's the difference between these 2 ?
Bu the point of Erik was that we don't want this kind of merge, no ?
I guess my question is do we want a really low level but universal
protocol which requires server support, or several separate APIs which can
handle legacy servers, servers with minor changes, or real time capable
servers?
Cool idea to connect a legacy backend and let the front-end deal with
the sync. But we can’t support conflict resolution in this scenario.
We can limit legacy servers to read only data (from a client perspective).
For example client1 and client2 change the same car object. Client1
changes the property colour from red to green and client2 changes it from
red to blue. The change of client1 takes place first then the change of
client2 will be a conflict. Now to detect that conflict we could fetch the
entity again and check if the colour is still the same as our original
value, but it could change in the mean time again. So there is no guarantee
that the data will be consistent.
Of course we could periodically fetch data from a legacy server and
merge that with the client state, but I don’t think that this is super
useful the power of sync is changing things to be able to work offline.
Of course, offline is just laggy sync.
But lets take two separate but similar legacy use cases. Downloading the
weather forecast based on your location and synchronizing a todo list.
The weather data probably won't change during the day. So every night at
midnight + jitter our sync can download the forecast from a restful URL and
be done. There is no conflict because the client data can not / will not
change. This is a very simple case to implement.
Syncing a Todo list is more complicated because it can change from
multiple clients. HOWEVER the application has a restriction that todo list
items may only be added and removed but not reordered or edited.
The flow for that could look like
1. User 1 on Android creates a TODO list.
2. Android app creates TODO list copy on server.
3. Android app registers a sync with the server for the todo list
4. User 1 adds item to TODO list.
5. Server syncs its list with Users list. returns OK.
6. User 1 access his list in web app.
7. Web App registers a sync on the server with the todo list.
8. User 1 Edits his list.
9. Web App applies edits and sends deltas to Android app.
10. repeate ad nausiem.
In this case conflicts can only happen if a user deletes the same item (no
practical conflict). So a conflict resolver will be trivial.
So I think we need a protocol, can be really simple just versioned JSON,
and a server that will compare the changes and return conflicts.
Yeah. The more I poke and play with things the less certain I am that
adding it to the pipe is a good idea. I think we need something separate.
As examples:
Legacy Servers can be periodic pollers.
Minor changes can be a sync on push type thing.
And realtime, is well, real time.
_______________________________________________
aerogear-dev mailing
listaerogear-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
6:59 p.m.
On 12/10/2013 11:16 AM, Sebastien Blanc wrote:
On Tue, Dec 10, 2013 at 4:13 PM, Summers Pittman <supittma(a)redhat.com
<mailto:supittma@redhat.com>> wrote:
On 12/10/2013 02:34 AM, Erik Jan de Wit wrote:
>
> On 9 Dec,2013, at 17:06 , Summers Pittman <supittma(a)redhat.com
> <mailto:supittma@redhat.com>> wrote:
>
>> For real time sync this is a great primer:
>> https://neil.fraser.name/writing/sync/. The concepts can be
>> expanded beyond text of course.
>
> That is a great article, but all these techniques are only cool
> for documents.
This technique is good for synced text. We've hooked our horse to
the JSON cart pretty hard so we should be able to deal with that
for a large number of cases.
> We could support documents of course, but I was thinking more
> about Pipes and syncing those when one has been offline for a while.
So Pipes aren't data per se. They are a one way query mechanism
(IE the pipe doesn't start spewing data all over your living room
floor for no reason.) Pipes can be used by Sync to fetch data
once a signal to refresh has been received however.
> So for example when you have a Car that has make Toyota and one
> changes it to Toiota and another changes it Toyotas merging these
> to changes to Toiotas is always wrong. We don't need merge
> support we only need conflict resolution.
'Toiotas' Yes, 'Toiotas' (no cap) no. That would actually be
correct in patch diff merge.
What's the difference between these 2 ?
Bu the point of Erik was that we don't want this kind of merge, no ?
The
difference is we have different opinions, use cases, and ideas of
"correct". Our users will also have said ideas so our solution should
make it possible for both to be supported (even if we only provide one).
>
>>
>> I guess my question is do we want a really low level but
>> universal protocol which requires server support, or several
>> separate APIs which can handle legacy servers, servers with
>> minor changes, or real time capable servers?
>
> Cool idea to connect a legacy backend and let the front-end deal
> with the sync. But we can't support conflict resolution in this
> scenario.
We can limit legacy servers to read only data (from a client
perspective).
> For example client1 and client2 change the same car object.
> Client1 changes the property colour from red to green and client2
> changes it from red to blue. The change of client1 takes place
> first then the change of client2 will be a conflict. Now to
> detect that conflict we could fetch the entity again and check if
> the colour is still the same as our original value, but it could
> change in the mean time again. So there is no guarantee that the
> data will be consistent.
>
> Of course we could periodically fetch data from a legacy server
> and merge that with the client state, but I don't think that this
> is super useful the power of sync is changing things to be able
> to work offline.
Of course, offline is just laggy sync.
But lets take two separate but similar legacy use cases.
Downloading the weather forecast based on your location and
synchronizing a todo list.
The weather data probably won't change during the day. So every
night at midnight + jitter our sync can download the forecast from
a restful URL and be done. There is no conflict because the client
data can not / will not change. This is a very simple case to
implement.
Syncing a Todo list is more complicated because it can change
from multiple clients. HOWEVER the application has a restriction
that todo list items may only be added and removed but not
reordered or edited.
The flow for that could look like
1. User 1 on Android creates a TODO list.
2. Android app creates TODO list copy on server.
3. Android app registers a sync with the server for the todo list
4. User 1 adds item to TODO list.
5. Server syncs its list with Users list. returns OK.
6. User 1 access his list in web app.
7. Web App registers a sync on the server with the todo list.
8. User 1 Edits his list.
9. Web App applies edits and sends deltas to Android app.
10. repeate ad nausiem.
In this case conflicts can only happen if a user deletes the same
item (no practical conflict). So a conflict resolver will be
trivial.
> So I think we need a protocol, can be really simple just
> versioned JSON, and a server that will compare the changes and
> return conflicts.
Yeah. The more I poke and play with things the less certain I am
that adding it to the pipe is a good idea. I think we need
something separate.
>
>>
>> As examples:
>> Legacy Servers can be periodic pollers.
>> Minor changes can be a sync on push type thing.
>> And realtime, is well, real time.
>>
>
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
8:45 p.m.
Sorry because I got lost into this thread, but do we have any use case scenarios to
understand which problem we are willing to solve?
I think would be nice to start to enumerate at least 3 and choose 1 to stay focused.
Thoughts?
--
abstractj
On December 10, 2013 at 3:59:45 PM, Summers Pittman (supittma(a)redhat.com) wrote:
> The difference is we have different opinions, use cases, and
ideas of "correct". Our users will also have said ideas so our
solution should make it possible for both to be supported (even
if we only provide one).
10:09 p.m.
Agreed, we could use Summer's table as start point and extract maybe more
"concrete" use cases from it ?
https://gist.github.com/secondsun/e6552abfbf51ed915d92#file-table-of-cont...
Use Case Push/PollRealtimeServer APIs UsedPeriodic Read Only Update
PollNAny legacyPipelines,
Authentication, Stores Real Time Read Only UpdatePush Y(ish)Legacy +
Updates for Unified PushUnified Push, Store Simple Settings Sync
PushY(ish)Legacy + Updates for Unified Push + Updates for conflict
mgmtPipelines,
Authentication, UnifiedPush, Store Real Time Text SyncPush YNeeds lots of
custom code, vert.x realtime componentUnifiedPush, Store
On Tue, Dec 10, 2013 at 8:45 PM, Bruno Oliveira <bruno(a)abstractj.org> wrote:
Sorry because I got lost into this thread, but do we have any use
case
scenarios to understand which problem we are willing to solve?
I think would be nice to start to enumerate at least 3 and choose 1 to
stay focused. Thoughts?
--
abstractj
On December 10, 2013 at 3:59:45 PM, Summers Pittman (supittma(a)redhat.com)
wrote:
> > The difference is we have different opinions, use cases, and
> ideas of "correct". Our users will also have said ideas so our
> solution should make it possible for both to be supported (even
> if we only provide one).
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
12:48 a.m.
+1 to include this table at our
documentation https://github.com/aerogear/aerogear.org/blob/6de2250fca915...
Do we have any scenario for detecting synchronization conflicts between client and
server?
--
abstractj
On December 10, 2013 at 7:09:53 PM, Sebastien Blanc (scm.blanc(a)gmail.com) wrote:
Agreed, we could use Summer's table as start point and extract
maybe more
"concrete" use cases from it ?
https://gist.github.com/secondsun/e6552abfbf51ed915d92#file-table-of-cont...
Use Case Push/PollRealtimeServer APIs UsedPeriodic Read Only
Update
PollNAny legacyPipelines,
Authentication, Stores Real Time Read Only UpdatePush Y(ish)Legacy
+
Updates for Unified PushUnified Push, Store Simple Settings
Sync
PushY(ish)Legacy + Updates for Unified Push + Updates for conflict
mgmtPipelines,
Authentication, UnifiedPush, Store Real Time Text SyncPush
YNeeds lots of
custom code, vert.x realtime componentUnifiedPush, Store
On Tue, Dec 10, 2013 at 8:45 PM, Bruno Oliveira
wrote:
> Sorry because I got lost into this thread, but do we have any use
case
> scenarios to understand which problem we are willing to solve?
>
> I think would be nice to start to enumerate at least 3 and choose
1 to
> stay focused. Thoughts?
>
> --
> abstractj
>
> On December 10, 2013 at 3:59:45 PM, Summers Pittman (supittma(a)redhat.com)
> wrote:
> > > The difference is we have different opinions, use cases,
and
> > ideas of "correct". Our users will also have said ideas so our
> > solution should make it possible for both to be supported (even
> > if we only provide one).
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
3:35 p.m.
On 12/10/2013 06:48 PM, Bruno Oliveira wrote:
+1 to include this table at our documentation
https://github.com/aerogear/aerogear.org/blob/6de2250fca9150110ec03eee2af...
Do we have any scenario for detecting synchronization conflicts between client and
server?
We've straw manned a few.
Least complex: The user has a set features on two devices each with
their own settings. These are synchronized over all of the user's
devices. On the first device he disables a feature (and thus all of its
settings), but on the second device he enables settings for the feature
he disabled on the other device.
How do these resolve and how is the resolution transmitted.
More complex: Multiple users are editing a todo list. Two users delete
the first item. How do we ensure that only one item is deleted? (Or how
do we build our APIs and protocol so that the backend server deletes
only one item)
Most complex: N (where N is > 2) edit the same document and create
conflicts. How do we enable resolving all of these and resyncing the
original document.
Even more most complex: The same about but with an Image / binary data/
etc.
--
abstractj
On December 10, 2013 at 7:09:53 PM, Sebastien Blanc (scm.blanc(a)gmail.com) wrote:
>
> Agreed, we could use Summer's table as start point and extract
> maybe more
> "concrete" use cases from it ?
>
> https://gist.github.com/secondsun/e6552abfbf51ed915d92#file-table-of-cont...
>
> Use Case Push/PollRealtimeServer APIs UsedPeriodic Read Only
> Update
> PollNAny legacyPipelines,
> Authentication, Stores Real Time Read Only UpdatePush Y(ish)Legacy
> +
> Updates for Unified PushUnified Push, Store Simple Settings
> Sync
> PushY(ish)Legacy + Updates for Unified Push + Updates for conflict
> mgmtPipelines,
> Authentication, UnifiedPush, Store Real Time Text SyncPush
> YNeeds lots of
> custom code, vert.x realtime componentUnifiedPush, Store
>
>
> On Tue, Dec 10, 2013 at 8:45 PM, Bruno Oliveira
> wrote:
>
>> Sorry because I got lost into this thread, but do we have any use
> case
>> scenarios to understand which problem we are willing to solve?
>>
>> I think would be nice to start to enumerate at least 3 and choose
> 1 to
>> stay focused. Thoughts?
>>
>> --
>> abstractj
>>
>> On December 10, 2013 at 3:59:45 PM, Summers Pittman (supittma(a)redhat.com)
>> wrote:
>>>> The difference is we have different opinions, use cases,
> and
>>> ideas of "correct". Our users will also have said ideas so our
>>> solution should make it possible for both to be supported (even
>>> if we only provide one).
>> _______________________________________________
>> aerogear-dev mailing list
>> aerogear-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:39 p.m.
>
> That is a great article, but all these techniques are only cool for documents.
This technique is good for synced text. We've hooked our horse to the JSON cart
pretty hard so we should be able to deal with that for a large number of cases.
Yes, synced text and yes JSON, is text as well, but it does not make sense to merge JSON
because merging this data will always be wrong! I’ll use the car example one more time:
one user wants to change from Toyotas to Toyota and another user wants to change from
Toyotas to Toiotas when a system merges those changes to Toiota it (the system) made up
it’s own change and really nobody wanted that to happen.
If I would write a book and changed a paragraph and somebody else was writing a paragraph
in the same book merging those would make sense.
> We could support documents of course, but I was thinking more
about Pipes and syncing those when one has been offline for a while.
So Pipes aren't data per se. They are a one way query mechanism (IE the pipe
doesn't start spewing data all over your living room floor for no reason.) Pipes can
be used by Sync to fetch data once a signal to refresh has been received however.
So you see no use case to make Pipes able to sync, because I think it would be a nice
addition
> Cool idea to connect a legacy backend and let the front-end deal
with the sync. But we can’t support conflict resolution in this scenario.
We can limit legacy servers to read only data (from a client perspective).
So my idea of sync is to be able to work offline and then sync the data back to the server
when I’m online again. Just getting data from a server periodically is not really what I
had in mind for syncing, I think that it doesn’t need an API, one can just fetch data
again in an OnlineEvent.
6:58 p.m.
On 12/10/2013 11:39 AM, Erik Jan de Wit wrote:
>> That is a great article, but all these techniques are only
cool for documents.
> This technique is good for synced text. We've hooked our horse to the JSON cart
pretty hard so we should be able to deal with that for a large number of cases.
Yes, synced text and yes JSON, is text as well, but it does not make sense to merge JSON
because merging this data will always be wrong! I’ll use the car example one more time:
one user wants to change from Toyotas to Toyota and another user wants to change from
Toyotas to Toiotas when a system merges those changes to Toiota it (the system) made up
it’s own change and really nobody wanted that to happen.
Hi,
diff merge patch covers your example without generating a conflict.
https://neil.fraser.name/software/diff_match_patch/svn/trunk/demos/demo_p...
If I would write a book and changed a paragraph and somebody else was writing a paragraph
in the same book merging those would make sense.
>> We could support documents of course, but I was thinking more about Pipes and
syncing those when one has been offline for a while.
> So Pipes aren't data per se. They are a one way query mechanism (IE the pipe
doesn't start spewing data all over your living room floor for no reason.) Pipes can
be used by Sync to fetch data once a signal to refresh has been received however.
So you see no use case to make Pipes able to sync, because I think it would be a nice
addition
I don't feel like Pipes are the thing that does syncing.
Does HTTP sync? Does SQL sync? The answer to both cases are no. But an
applicaiton can use both HTTP and SQL to do what we are thinking of when
we say sync.
>> Cool idea to connect a legacy backend and let the front-end deal with the sync.
But we can’t support conflict resolution in this scenario.
> We can limit legacy servers to read only data (from a client perspective).
So my idea of sync is to be able to work offline and then sync the data back to the
server when I’m online again. Just getting data from a server periodically is not really
what I had in mind for syncing, I think that it doesn’t need an API, one can just fetch
data again in an OnlineEvent.
It is still a valid use case for online support and
there is a TON of
work around just that with mobile.
1) adding a jitter to when you sync so the server doesn't get every user
hitting it at the exact same time
2) retry and exponential backoff
3) notifying the external system of data changes
4) Managing and batching sync operations so that we don't kill the battery.
Just because I want to start with a simple use case doesn't mean you can
dismiss the use case.
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
8:30 a.m.
Just because I want to start with a simple use case doesn't mean you can
dismiss the use case.
I don’t want to dismiss anything just wanted to know what you think about my idea to
combine a pipe and store to create a pipe that continues to work when the user goes
offline. Obviously you don’t feel my idea makes any sense and I think that you sync isn’t
a super powerful feature to have (there are to many pre conditions in my opinion), but we
could have it, not dismissing it.
What do others think about this discussion? What use cases should we support first? Can
you think of other ways/things to sync?
10:37 a.m.
First, I appreciate the discussion in this thread as this is all new to me.
What use cases should we support first?
I'd be for looking
into the document use case first, and hold off with the
transactional/Operational Transformation one for now. This mainly because
is seems to be a lot simpler.
qmx suggested using something similar to CouchDB. CouchDB uses a revision
field for each document to handle/detect conflicts.
The following is a concrete example of how this works with CouchDB:
Create a database:
curl -X PUT http://127.0.0.1:5984/agsync
{"ok":true}
Get a uuid:
curl -X GET http://127.0.0.1:5984/_uuids
{"uuids":["f11075199bcea80849204660d7000d53"]}
Create a car object(document) using the above uuid:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53-d
'{"car": {"make":"Toyota",
"color":"red"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b"}
Notice that the server returned a new document revision.
Now clientA updates the car object, notice that we are specifying the
document revision:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53-d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"blue"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"2-588a9da2e252e9a3d2fc0634610a0c25"}
Also, notice how the server returned a new document revision.
Now, clientB still has the old revision and tries to update the car object:
curl -v -X PUT
http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"yellow"}}'
< HTTP/1.1 409 Conflict
{"error":"conflict","reason":"Document update
conflict."}
This conflict would the have to be handled by the client. Could we start
out with something similar and as simple as this and evolve it?
On 11 December 2013 08:30, Erik Jan de Wit <edewit(a)redhat.com> wrote:
>
> Just because I want to start with a simple use case doesn't mean you can
> dismiss the use case.
>
I don’t want to dismiss anything just wanted to know what you think about
my idea to combine a pipe and store to create a pipe that continues to work
when the user goes offline. Obviously you don’t feel my idea makes any
sense and I think that you sync isn’t a super powerful feature to have
(there are to many pre conditions in my opinion), but we could have it, not
dismissing it.
What do others think about this discussion? What use cases should we
support first? Can you think of other ways/things to sync?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
10:44 a.m.
Very very very simple with CounchDB +4242424242
--
abstractj
On December 11, 2013 at 7:37:51 AM, Daniel Bevenius (daniel.bevenius(a)gmail.com) wrote:
> This conflict would the have to be handled by the client. Could
we start out with something similar and as simple as this and evolve
it?
3:47 p.m.
On 12/11/2013 04:44 AM, Bruno Oliveira wrote:
Very very very simple with CounchDB +4242424242
One of my concerns with Couch is it is another
application/server/dependency to package and keep up with.
Of course if we want to build an aerogear-sync-server application
instead of an aerogear-sync-sererver spec that servers can implement it
makes a lot more sense.
Can CouchDB be embedded in a Java EE application?
3:56 p.m.
One of my concerns with Couch is it is another
application/server/dependency
to package and keep up with.
The suggestion is that we do something similar to CouchDB and perhaps lend
some ideas from their protocol and how they handle conflicts. Not that we
base this on CouchDB, sorry if this was confusing.
A server implementation might start off delegating to a CouchDB instance
just to get up and running. This way our client libs would have something
work against.
On 11 December 2013 15:47, Summers Pittman <supittma(a)redhat.com> wrote:
On 12/11/2013 04:44 AM, Bruno Oliveira wrote:
> Very very very simple with CounchDB +4242424242
>
One of my concerns with Couch is it is another
application/server/dependency to package and keep up with.
Of course if we want to build an aerogear-sync-server application
instead of an aerogear-sync-sererver spec that servers can implement it
makes a lot more sense.
Can CouchDB be embedded in a Java EE application?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
4:48 p.m.
On 12/11/2013 09:56 AM, Daniel Bevenius wrote:
>One of my concerns with Couch is it is another
application/server/dependency to package and keep up with.
The suggestion is that we do something similar to CouchDB and perhaps
lend some ideas from their protocol and how they handle conflicts. Not
that we base this on CouchDB, sorry if this was confusing.
A server implementation might start off delegating to a CouchDB
instance just to get up and running. This way our client libs would
have something work against.
Ok, just wanted to make sure.
On 11 December 2013 15:47, Summers Pittman <supittma(a)redhat.com
<mailto:supittma@redhat.com>> wrote:
On 12/11/2013 04:44 AM, Bruno Oliveira wrote:
> Very very very simple with CounchDB +4242424242
>
One of my concerns with Couch is it is another
application/server/dependency to package and keep up with.
Of course if we want to build an aerogear-sync-server application
instead of an aerogear-sync-sererver spec that servers can
implement it
makes a lot more sense.
Can CouchDB be embedded in a Java EE application?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org <mailto:aerogear-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
11:21 a.m.
That is what I suggested, but then at the same time connect that to pipes and stores, but
we can start even simpler and first develop something like this.
On 11 Dec,2013, at 10:37 , Daniel Bevenius <daniel.bevenius(a)gmail.com> wrote:
First, I appreciate the discussion in this thread as this is all new
to me.
> What use cases should we support first?
I'd be for looking into the document use case first, and hold off with the
transactional/Operational Transformation one for now. This mainly because is seems to be a
lot simpler.
qmx suggested using something similar to CouchDB. CouchDB uses a revision field for each
document to handle/detect conflicts.
The following is a concrete example of how this works with CouchDB:
Create a database:
curl -X PUT http://127.0.0.1:5984/agsync
{"ok":true}
Get a uuid:
curl -X GET http://127.0.0.1:5984/_uuids
{"uuids":["f11075199bcea80849204660d7000d53"]}
Create a car object(document) using the above uuid:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"car": {"make":"Toyota",
"color":"red"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b"}
Notice that the server returned a new document revision.
Now clientA updates the car object, notice that we are specifying the document revision:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"blue"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"2-588a9da2e252e9a3d2fc0634610a0c25"}
Also, notice how the server returned a new document revision.
Now, clientB still has the old revision and tries to update the car object:
curl -v -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"yellow"}}'
< HTTP/1.1 409 Conflict
{"error":"conflict","reason":"Document update
conflict."}
This conflict would the have to be handled by the client. Could we start out with
something similar and as simple as this and evolve it?
On 11 December 2013 08:30, Erik Jan de Wit <edewit(a)redhat.com> wrote:
>
> Just because I want to start with a simple use case doesn't mean you can
> dismiss the use case.
>
I don’t want to dismiss anything just wanted to know what you think about my idea to
combine a pipe and store to create a pipe that continues to work when the user goes
offline. Obviously you don’t feel my idea makes any sense and I think that you sync isn’t
a super powerful feature to have (there are to many pre conditions in my opinion), but we
could have it, not dismissing it.
What do others think about this discussion? What use cases should we support first? Can
you think of other ways/things to sync?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
11:21 a.m.
Hi all,
Here's my view:
Different options at different levels. Some options are corellated.
SyncManager: when to trigger:
1. periodically
2. automatic: online/offline detection
3. manually
Server vs peer-to-peer context
1. server context with revision tag, client send his updates, conflict resolution server
side
2. no prior context :
http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p218.pdf
- queing push
- pull
Granularity
1. Document based. revision at object/fields level.
2. OT
Conflicts resolution:
1. reject
2. proposed merge
3. prompt user
Like Dan, I think we could have several implementations of sync depending on the use cases
and let the end user (developer) picks the right one ffor the job.
But we need to start with one of them, simplest one would be:
- manual trigger sync
- server side context with revision number
- offline caching
- client send his updates
- server conflicts resolution with rejection of client changes
++
Corinne
On Dec 11, 2013, at 10:37 AM, Daniel Bevenius <daniel.bevenius(a)gmail.com> wrote:
First, I appreciate the discussion in this thread as this is all new
to me.
> What use cases should we support first?
I'd be for looking into the document use case first, and hold off with the
transactional/Operational Transformation one for now. This mainly because is seems to be a
lot simpler.
qmx suggested using something similar to CouchDB. CouchDB uses a revision field for each
document to handle/detect conflicts.
The following is a concrete example of how this works with CouchDB:
Create a database:
curl -X PUT http://127.0.0.1:5984/agsync
{"ok":true}
Get a uuid:
curl -X GET http://127.0.0.1:5984/_uuids
{"uuids":["f11075199bcea80849204660d7000d53"]}
Create a car object(document) using the above uuid:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"car": {"make":"Toyota",
"color":"red"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b"}
Notice that the server returned a new document revision.
Now clientA updates the car object, notice that we are specifying the document revision:
curl -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"blue"}}'
{"ok":true,"id":"f11075199bcea80849204660d7000d53","rev":"2-588a9da2e252e9a3d2fc0634610a0c25"}
Also, notice how the server returned a new document revision.
Now, clientB still has the old revision and tries to update the car object:
curl -v -X PUT http://127.0.0.1:5984/agsync/f11075199bcea80849204660d7000d53 -d
'{"_rev":"1-dffbd1dbd1b9ecbffc1b02a34cbaea0b", "car":
{"make":"Toyota","color":"yellow"}}'
< HTTP/1.1 409 Conflict
{"error":"conflict","reason":"Document update
conflict."}
This conflict would the have to be handled by the client. Could we start out with
something similar and as simple as this and evolve it?
On 11 December 2013 08:30, Erik Jan de Wit <edewit(a)redhat.com> wrote:
>
> Just because I want to start with a simple use case doesn't mean you can
> dismiss the use case.
>
I don’t want to dismiss anything just wanted to know what you think about my idea to
combine a pipe and store to create a pipe that continues to work when the user goes
offline. Obviously you don’t feel my idea makes any sense and I think that you sync isn’t
a super powerful feature to have (there are to many pre conditions in my opinion), but we
could have it, not dismissing it.
What do others think about this discussion? What use cases should we support first? Can
you think of other ways/things to sync?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
4:22 p.m.
+1 for starting with an implementation similar to CouchDB. This looks
like the Offline Lock pattern where there are version stamps and a
client cannot fulfill a transaction if he has an old current version
stamp.
Regarding the concepts that Summers sent
https://neil.fraser.name/writing/sync/ I feel that are some Open Source
implementations of these concepts out there.
e.g google-diff-match-patch[1]
And something more, there is a security parameter in the data sync
concept. The database might contain data which should be
visible/accessible to all the clients.
[1]: https://code.google.com/p/google-diff-match-patch/
On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
This conflict would the have to be handled by the client. Could we
start out with something similar and as simple as this and evolve it?
4:26 p.m.
apologies, just a correction. The security concern is that the database
might contain data which should be NOT visible/accessible to all the
clients.
On Wed, 2013-12-11 at 16:22 +0100, Apostolos Emmanouilidis wrote:
+1 for starting with an implementation similar to CouchDB. This
looks
like the Offline Lock pattern where there are version stamps and a
client cannot fulfill a transaction if he has an old current version
stamp.
Regarding the concepts that Summers sent
https://neil.fraser.name/writing/sync/ I feel that are some Open Source
implementations of these concepts out there.
e.g google-diff-match-patch[1]
And something more, there is a security parameter in the data sync
concept. The database might contain data which should be
visible/accessible to all the clients.
[1]: https://code.google.com/p/google-diff-match-patch/
On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
> This conflict would the have to be handled by the client. Could we
> start out with something similar and as simple as this and evolve it?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
4:33 p.m.
wanted to dump some thoughts i was having
Client Side Perspective
let me try to dump some of the client side thoughts i have, which will probably be more
javascript centric
i see data sync as keeping DataManager stores in sync with the server side
not sure if it makes sense without it?
I can see using Notifier using sockJS connecting to a netty server(?) to do the
"real-time" updates
Sync should be plugable, so an existing REST application using DataManager can just add it
in or turn it off easily
the sync should have some events that the user can "listen" for, to deal with
conflict/changes and such
there is another type of sync that i'm also thinking off.
Since Pipeline and Datamanager are not tied to each other, do we want to think about the
possiblity of "syncing" them. that is when i do a read currently on a pipe, i
have to manually store the data, should that be automatic, this used to happen in the way
begining of pipeline, but we removed it since we wanted to keep it modular
On Dec 11, 2013, at 10:26 AM, Apostolos Emmanouilidis <aemmanou(a)redhat.com> wrote:
apologies, just a correction. The security concern is that the
database
might contain data which should be NOT visible/accessible to all the
clients.
On Wed, 2013-12-11 at 16:22 +0100, Apostolos Emmanouilidis wrote:
> +1 for starting with an implementation similar to CouchDB. This looks
> like the Offline Lock pattern where there are version stamps and a
> client cannot fulfill a transaction if he has an old current version
> stamp.
>
> Regarding the concepts that Summers sent
> https://neil.fraser.name/writing/sync/ I feel that are some Open Source
> implementations of these concepts out there.
> e.g google-diff-match-patch[1]
>
> And something more, there is a security parameter in the data sync
> concept. The database might contain data which should be
> visible/accessible to all the clients.
>
> [1]: https://code.google.com/p/google-diff-match-patch/
>
>
> On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
>
>
>> This conflict would the have to be handled by the client. Could we
>> start out with something similar and as simple as this and evolve it?
>
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:13 p.m.
On Wed, Dec 11, 2013 at 4:33 PM, Lucas Holmquist <lholmqui(a)redhat.com>wrote:
wanted to dump some thoughts i was having
Client Side Perspective
let me try to dump some of the client side thoughts i have, which will
probably be more javascript centric
-
i see data sync as keeping DataManager stores in sync with the server
side
-
not sure if it makes sense without it?
-
I can see using Notifier using sockJS connecting to a netty server(?)
to do the "real-time" updates
-
Sync should be plugable, so an existing REST application using
DataManager can just add it in or turn it off easily
-
the sync should have some events that the user can "listen" for, to
deal with conflict/changes and such
-
there is another type of sync that i'm also thinking off.
Since Pipeline and Datamanager are not tied to each other, do we want
to think about the possiblity of "syncing" them. that is when i do a read
currently on a pipe, i have to manually store the data, should that be
automatic, this used to happen in the way begining of pipeline, but we
removed it since we wanted to keep it modular
Yes, I think that makes totally sense. If you it should be automatic,
well, we
could configure it to be or not. In the same spirit, I was
thinking of some kind of storeFallback option we can pass to a pipe. If we
call a read on a pipe and there is no connection, we fallback automatically
on the local store to retrieve the data (with some metadata/flag saying
it's local data).
On Dec 11, 2013, at 10:26 AM, Apostolos Emmanouilidis <aemmanou(a)redhat.com>
wrote:
apologies, just a correction. The security concern is that the database
might contain data which should be NOT visible/accessible to all the
clients.
On Wed, 2013-12-11 at 16:22 +0100, Apostolos Emmanouilidis wrote:
+1 for starting with an implementation similar to CouchDB. This looks
like the Offline Lock pattern where there are version stamps and a
client cannot fulfill a transaction if he has an old current version
stamp.
Regarding the concepts that Summers sent
https://neil.fraser.name/writing/sync/ I feel that are some Open Source
implementations of these concepts out there.
e.g google-diff-match-patch[1]
And something more, there is a security parameter in the data sync
concept. The database might contain data which should be
visible/accessible to all the clients.
[1]: https://code.google.com/p/google-diff-match-patch/
On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
This conflict would the have to be handled by the client. Could we
start out with something similar and as simple as this and evolve it?
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:24 p.m.
Interesting link to understand 'a la couchDB':
http://docs.couchbase.com/couchbase-lite/cbl-concepts/#introduction
that comes will iOS/Android client implementation.
++
Corinne
On Dec 11, 2013, at 5:13 PM, Sebastien Blanc <scm.blanc(a)gmail.com> wrote:
On Wed, Dec 11, 2013 at 4:33 PM, Lucas Holmquist <lholmqui(a)redhat.com> wrote:
wanted to dump some thoughts i was having
Client Side Perspective
let me try to dump some of the client side thoughts i have, which will probably be more
javascript centric
• i see data sync as keeping DataManager stores in sync with the server side
• not sure if it makes sense without it?
• I can see using Notifier using sockJS connecting to a netty server(?) to do the
"real-time" updates
• Sync should be plugable, so an existing REST application using DataManager can just
add it in or turn it off easily
• the sync should have some events that the user can "listen" for, to deal
with conflict/changes and such
• there is another type of sync that i'm also thinking off.
Since Pipeline and Datamanager are not tied to each other, do we want to think about the
possiblity of "syncing" them. that is when i do a read currently on a pipe, i
have to manually store the data, should that be automatic, this used to happen in the way
begining of pipeline, but we removed it since we wanted to keep it modular
Yes, I think that makes totally sense. If you it should be automatic, well, we could
configure it to be or not. In the same spirit, I was thinking of some kind of
storeFallback option we can pass to a pipe. If we call a read on a pipe and there is no
connection, we fallback automatically on the local store to retrieve the data (with some
metadata/flag saying it's local data).
On Dec 11, 2013, at 10:26 AM, Apostolos Emmanouilidis <aemmanou(a)redhat.com> wrote:
> apologies, just a correction. The security concern is that the database
> might contain data which should be NOT visible/accessible to all the
> clients.
>
> On Wed, 2013-12-11 at 16:22 +0100, Apostolos Emmanouilidis wrote:
>> +1 for starting with an implementation similar to CouchDB. This looks
>> like the Offline Lock pattern where there are version stamps and a
>> client cannot fulfill a transaction if he has an old current version
>> stamp.
>>
>> Regarding the concepts that Summers sent
>> https://neil.fraser.name/writing/sync/ I feel that are some Open Source
>> implementations of these concepts out there.
>> e.g google-diff-match-patch[1]
>>
>> And something more, there is a security parameter in the data sync
>> concept. The database might contain data which should be
>> visible/accessible to all the clients.
>>
>> [1]: https://code.google.com/p/google-diff-match-patch/
>>
>>
>> On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
>>
>>
>>> This conflict would the have to be handled by the client. Could we
>>> start out with something similar and as simple as this and evolve it?
>>
>>
>>
>> _______________________________________________
>> aerogear-dev mailing list
>> aerogear-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
5:26 p.m.
So i know with PouchDB, they do long polling when the "continuous" option is on.
they sync against couchDB
On Dec 11, 2013, at 11:24 AM, Corinne Krych <corinnekrych(a)gmail.com> wrote:
Interesting link to understand 'a la couchDB':
http://docs.couchbase.com/couchbase-lite/cbl-concepts/#introduction
that comes will iOS/Android client implementation.
++
Corinne
On Dec 11, 2013, at 5:13 PM, Sebastien Blanc <scm.blanc(a)gmail.com> wrote:
>
>
>
> On Wed, Dec 11, 2013 at 4:33 PM, Lucas Holmquist <lholmqui(a)redhat.com> wrote:
> wanted to dump some thoughts i was having
> Client Side Perspective
>
> let me try to dump some of the client side thoughts i have, which will probably be
more javascript centric
>
> • i see data sync as keeping DataManager stores in sync with the server side
>
> • not sure if it makes sense without it?
>
> • I can see using Notifier using sockJS connecting to a netty server(?) to do the
"real-time" updates
>
> • Sync should be plugable, so an existing REST application using DataManager can
just add it in or turn it off easily
>
> • the sync should have some events that the user can "listen" for, to deal
with conflict/changes and such
>
> • there is another type of sync that i'm also thinking off.
>
> Since Pipeline and Datamanager are not tied to each other, do we want to think about
the possiblity of "syncing" them. that is when i do a read currently on a pipe,
i have to manually store the data, should that be automatic, this used to happen in the
way begining of pipeline, but we removed it since we wanted to keep it modular
>
> Yes, I think that makes totally sense. If you it should be automatic, well, we could
configure it to be or not. In the same spirit, I was thinking of some kind of
storeFallback option we can pass to a pipe. If we call a read on a pipe and there is no
connection, we fallback automatically on the local store to retrieve the data (with some
metadata/flag saying it's local data).
>
>
> On Dec 11, 2013, at 10:26 AM, Apostolos Emmanouilidis <aemmanou(a)redhat.com>
wrote:
>
>> apologies, just a correction. The security concern is that the database
>> might contain data which should be NOT visible/accessible to all the
>> clients.
>>
>> On Wed, 2013-12-11 at 16:22 +0100, Apostolos Emmanouilidis wrote:
>>> +1 for starting with an implementation similar to CouchDB. This looks
>>> like the Offline Lock pattern where there are version stamps and a
>>> client cannot fulfill a transaction if he has an old current version
>>> stamp.
>>>
>>> Regarding the concepts that Summers sent
>>> https://neil.fraser.name/writing/sync/ I feel that are some Open Source
>>> implementations of these concepts out there.
>>> e.g google-diff-match-patch[1]
>>>
>>> And something more, there is a security parameter in the data sync
>>> concept. The database might contain data which should be
>>> visible/accessible to all the clients.
>>>
>>> [1]: https://code.google.com/p/google-diff-match-patch/
>>>
>>>
>>> On Wed, 2013-12-11 at 10:37 +0100, Daniel Bevenius wrote:
>>>
>>>
>>>> This conflict would the have to be handled by the client. Could we
>>>> start out with something similar and as simple as this and evolve it?
>>>
>>>
>>>
>>> _______________________________________________
>>> aerogear-dev mailing list
>>> aerogear-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>
>>
>> _______________________________________________
>> aerogear-dev mailing list
>> aerogear-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
_______________________________________________
aerogear-dev mailing list
aerogear-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev
3789
days inactive
3796
days old
32 comments
8 participants
participants (8)
-
Apostolos Emmanouilidis -
Bruno Oliveira -
Corinne Krych -
Daniel Bevenius -
Erik Jan de Wit -
Lucas Holmquist -
Sebastien Blanc -
Summers Pittman