On Mon 07 Jul 2014 12:14:17 PM EDT, Matt Wringe wrote: > > Initial email to get some discussion going around the LiveOak SDK and > AeroGear collaboration. > > Essentially in LiveOak we are going to need a few different SDK types > > - Client Application SDK > This is the code which will run on the users device. Initial targets > here are javascript (+ cordova support), iOS, Android. This type of SDK > will handle things like getting and sending resources to and from the > server, handling login/logout/registration, etc. Probably some other > things like device registration would be needed as well. > > Not sure if we want to provide support for some other things outside of > communicating with the server or not (eg access to device components (eg > camera, location, etc)) or if these would be best handled by using the > native environment's SDK instead. > > - Server Side SDK > This is code that runs on the server side, written in JavaScript by the > application developer. This will need to be familiar to the client > application's JavaScript SDK, but may not be exactly the same. This type > of SDK will be able to access the same resources as the client side > JavaScript, as well as other internal resources and libraries. > > > I am not sure how to collaborate between the LiveOak and AeroGear teams > here. AeroGear makes really awesome SDKs for various mobile platforms, > but with LiveOak we are dealing with a specific type of application. The > AeroGear SDKs tend to handle the more generic case, which I don't > necessarily think makes sense for a LiveOak SDK. > > I do think it makes sense that the LiveOak SDK uses the AeroGear SDK > internally, but I don't know if we want to expose these AeroGear > components to a LiveOak developer or not. > > > For me, I envision something like the admin setting up their application > in the LiveOak console which then generates a json configuration file > (url locations, resources available, KeyCloak settings, UPS settings, > etc). The application developer then drops this json file in to their > application, the LiveOak SDK reads the json file to set it self up and > then its really easy for the developer to start using it. > > [there are also some really cool things we could be doing here as well > if we can get awesome data sync support for AeroGear. It might be > interesting to be able to fetch a resource from the server and > automatically sync its state across between the client and server. This > way the object appears as a local object: if the resource changes on the > server, it changes locally as well, if it changes locally, that change > is pushed to the server. This way you are just dealing with an object, > and not having to fetch and then push object back and forth between the > server manually] > > Anyone have any thoughts on this? One of the things which may be useful is aerogear-ios and aerogear-android are both modularizing their libraries. IN theory this means that the liveOAK SDK's could be extensions of those modules instead of a single monolothic thing. Sync is still an ongoing discussion in AeroGear, but I think right now the current group thought is that we will focus on 409 Conflict events (JPA versioning on server side and utilities on the client side) and not worrying about realtime sync. Some notes from out last meeting on the topic : Client API Strawman : https://github.com/secondsun/aerogear-android-sync/tree/master/android Client/Server workflow : https://docs.google.com/drawings/d/1E4NDEh3NQCdoEHNNHba4TR2akNrppvV5zDlk5... Also we are looking at doing something with diff merge patch as well as adding in push based data changed notifications later. > _______________________________________________ > aerogear-dev mailing list > aerogear-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev -- Summers Pittman >> Phone:404 941 4698 >> Java is my crack. _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev

On 08/07/14 09:11 AM, Burr Sutter wrote: > At a minimum, the same HTTP error codes used to indicate "sync errors" via JAX-RS + JPA should be the same for LiveOak's REST APIs - therefore the client-code can be reusable across the two endpoints. We should come up with sync strategies - levels 1, 2 and 3 - where 1 is simply the clever use of HTTP error codes and level 3 is real-time/off-line/auto-conflict resolution. Perhaps there are 4 levels :-) > > Anyone care to articulate the possible strategies? Some thoughts: 1) No conflict resolution. The way it handled in LiveOak right now, whoever is the last person to push to the server overwrites any other changes. Obviously not ideal, but may be acceptable in some cases. (or enabled via a 'force' option)b 2) Only allowed to update from the latest version: only allow updates if the resource being updated has not already been modified by someone else. If it has been updated, return HTTP error code 3) Non-conflicting field updates allowed: allow partial, non-conflicting updates to fields; otherwise return HTTP error code (Eg userA & userB both fetch resourceA from the server, userA makes a change to the 'foo' field and pushes it to the server. UserB makes a change to the 'bar' field and tries to push to the server. Since its a change to a different field, it is allowed). 4) Non-conflicting updates allowed: expands on the non-conflicting field updates. A smarter diff type system where its can handle more than just modifications to different fields. Eg say there is an 'article' field, UserA and UserB checkout the resource. UserA fixes a typo and pushes it to the server. UserB fixes the same typo and fixes 2 other typos. UserB can commit his change since his changes don't conflict with the previous update. [I am sure there are probably some nice diff and merge libraries around that we could use on the server side to take care of handling the conflict logic] The above is all stuff which needs to be done on the server side. And there are a few interesting things here where, except for the first option, we could require the data on the server to include special fields for conflict resolution (or something like a md5sum of the resources state). Having special fields like this breaks some data features we currently have in LiveOak, but its probably something we can make configurable and have a compromise. Client side stuff: 1) offline support: I am assuming this is about having some sort of local data cache so that when offline we can get the cached objects. All without having to resort to the whole fetch from a URL then save to local storage manually. Eg liveoak.get("/foo/bar") would fetch and cache locally if online, if offline liveoak.get("/foo/bar") would just get it from the cache. Some interesting stuff would need to be done here on the client side. [any plan for encryption and security in this local storage? Any ability to wipe the cache from the console if the device is lost or stolen?]

2) real-time: this is where I think things get more interesting. If we had conflict resolution (and partial updates) we could almost do this already in LiveOak (eg register to receive changes to a particular resource when modified on the server, push partial updates when the resource is modified locally. These steps of course would be handled by the SDK and not expected to be manually handled by the developer). This would also tie into the offline support, so when the device goes offline and then comes back, it would need to be able to handle the conflicts and push results back to the server. Other considerations - exposing conflicts to the developer so they can manually handle the issue and/or notify the user. - allowing the developer to specify what kind of conflict resolution they want for what resources - how to configure local storage (eg only cache objects already accessed, prefetch a list of resources, never cache certain resources, etc....) Anything else? > > > On Jul 8, 2014, at 9:02 AM, Summers Pittman <supittma(a)redhat.com&gt; wrote: > >> On Mon 07 Jul 2014 12:14:17 PM EDT, Matt Wringe wrote: >>> >>> Initial email to get some discussion going around the LiveOak SDK and >>> AeroGear collaboration. >>> >>> Essentially in LiveOak we are going to need a few different SDK types >>> >>> - Client Application SDK >>> This is the code which will run on the users device. Initial targets >>> here are javascript (+ cordova support), iOS, Android. This type of SDK >>> will handle things like getting and sending resources to and from the >>> server, handling login/logout/registration, etc. Probably some other >>> things like device registration would be needed as well. >>> >>> Not sure if we want to provide support for some other things outside of >>> communicating with the server or not (eg access to device components (eg >>> camera, location, etc)) or if these would be best handled by using the >>> native environment's SDK instead. >>> >>> - Server Side SDK >>> This is code that runs on the server side, written in JavaScript by the >>> application developer. This will need to be familiar to the client >>> application's JavaScript SDK, but may not be exactly the same. This type >>> of SDK will be able to access the same resources as the client side >>> JavaScript, as well as other internal resources and libraries. >>> >>> >>> I am not sure how to collaborate between the LiveOak and AeroGear teams >>> here. AeroGear makes really awesome SDKs for various mobile platforms, >>> but with LiveOak we are dealing with a specific type of application. The >>> AeroGear SDKs tend to handle the more generic case, which I don't >>> necessarily think makes sense for a LiveOak SDK. >>> >>> I do think it makes sense that the LiveOak SDK uses the AeroGear SDK >>> internally, but I don't know if we want to expose these AeroGear >>> components to a LiveOak developer or not. >>> >>> >>> For me, I envision something like the admin setting up their application >>> in the LiveOak console which then generates a json configuration file >>> (url locations, resources available, KeyCloak settings, UPS settings, >>> etc). The application developer then drops this json file in to their >>> application, the LiveOak SDK reads the json file to set it self up and >>> then its really easy for the developer to start using it. >>> >>> [there are also some really cool things we could be doing here as well >>> if we can get awesome data sync support for AeroGear. It might be >>> interesting to be able to fetch a resource from the server and >>> automatically sync its state across between the client and server. This >>> way the object appears as a local object: if the resource changes on the >>> server, it changes locally as well, if it changes locally, that change >>> is pushed to the server. This way you are just dealing with an object, >>> and not having to fetch and then push object back and forth between the >>> server manually] >>> >>> Anyone have any thoughts on this? >> >> One of the things which may be useful is aerogear-ios and >> aerogear-android are both modularizing their libraries. IN theory this >> means that the liveOAK SDK's could be extensions of those modules >> instead of a single monolothic thing. >> >> Sync is still an ongoing discussion in AeroGear, but I think right now >> the current group thought is that we will focus on 409 Conflict events >> (JPA versioning on server side and utilities on the client side) and not >> worrying about realtime sync. >> >> >> Some notes from out last meeting on the topic : >> Client API Strawman : >> https://github.com/secondsun/aerogear-android-sync/tree/master/android >> Client/Server workflow : >> https://docs.google.com/drawings/d/1E4NDEh3NQCdoEHNNHba4TR2akNrppvV5zDlk5... >> >> >> Also we are looking at doing something with diff merge patch as well as >> adding in push based data changed notifications later. >> >> >>> _______________________________________________ >>> aerogear-dev mailing list >>> aerogear-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> >> >> >> -- >> Summers Pittman >>>> Phone:404 941 4698 >>>> Java is my crack. >> >> _______________________________________________ >> aerogear-dev mailing list >> aerogear-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/aerogear-dev >

On Jul 8, 2014, at 10:23 AM, Matt Wringe <mwringe(a)redhat.com&gt; wrote: > On 08/07/14 09:11 AM, Burr Sutter wrote: >> At a minimum, the same HTTP error codes used to indicate "sync errors" via JAX-RS + JPA should be the same for LiveOak's REST APIs - therefore the client-code can be reusable across the two endpoints. We should come up with sync strategies - levels 1, 2 and 3 - where 1 is simply the clever use of HTTP error codes and level 3 is real-time/off-line/auto-conflict resolution. Perhaps there are 4 levels :-) >> >> Anyone care to articulate the possible strategies? > > Some thoughts: > > 1) No conflict resolution. The way it handled in LiveOak right now, whoever is the last person to push to the server overwrites any other changes. Obviously not ideal, but may be acceptable in some cases. (or enabled via a 'force' option)b > > 2) Only allowed to update from the latest version: only allow updates if the resource being updated has not already been modified by someone else. If it has been updated, return HTTP error code The prototype that Erik worked up includes the ability to return the "latest" data - so the end-user can make a decision to overwrite the server copy with the client version OR to overwrite the client copy with the server version. Based on a single HTTP 409 I believe. > > 3) Non-conflicting field updates allowed: allow partial, non-conflicting updates to fields; otherwise return HTTP error code (Eg userA & userB both fetch resourceA from the server, userA makes a change to the 'foo' field and pushes it to the server. UserB makes a change to the 'bar' field and tries to push to the server. Since its a change to a different field, it is allowed). I like that basic merge strategy. > > 4) Non-conflicting updates allowed: expands on the non-conflicting field updates. A smarter diff type system where its can handle more than just modifications to different fields. Eg say there is an 'article' field, UserA and UserB checkout the resource. UserA fixes a typo and pushes it to the server. UserB fixes the same typo and fixes 2 other typos. UserB can commit his change since his changes don't conflict with the previous update. > > [I am sure there are probably some nice diff and merge libraries around that we could use on the server side to take care of handling the conflict logic] > > The above is all stuff which needs to be done on the server side. And there are a few interesting things here where, except for the first option, we could require the data on the server to include special fields for conflict resolution (or something like a md5sum of the resources state). I like it - in a previous life I had a "checksum" like solution to know if something changed during an optimistic-locking scenario - this was much faster than comparing each individual field bit by bit. Once you found out something had changed, you could then iterate through the list of fields/values but the default operation was to first verify the checksum. > Having special fields like this breaks some data features we currently have in LiveOak, but its probably something we can make configurable and have a compromise. > > Client side stuff: > > 1) offline support: I am assuming this is about having some sort of local data cache so that when offline we can get the cached objects. All without having to resort to the whole fetch from a URL then save to local storage manually. Eg liveoak.get("/foo/bar") would fetch and cache locally if online, if offline liveoak.get("/foo/bar") would just get it from the cache. Some interesting stuff would need to be done here on the client side. > > [any plan for encryption and security in this local storage? Any ability to wipe the cache from the console if the device is lost or stolen?] > > 2) real-time: this is where I think things get more interesting. If we had conflict resolution (and partial updates) we could almost do this already in LiveOak (eg register to receive changes to a particular resource when modified on the server, push partial updates when the resource is modified locally. These steps of course would be handled by the SDK and not expected to be manually handled by the developer). This would also tie into the offline support, so when the device goes offline and then comes back, it would need to be able to handle the conflicts and push results back to the server. > > Other considerations > - exposing conflicts to the developer so they can manually handle the issue and/or notify the user. > - allowing the developer to specify what kind of conflict resolution they want for what resources > - how to configure local storage (eg only cache objects already accessed, prefetch a list of resources, never cache certain resources, etc....) > > Anything else? Anybody have comments? We should try to capture this in a wiki/gist :-)

> > >> >> >> On Jul 8, 2014, at 9:02 AM, Summers Pittman <supittma(a)redhat.com&gt; wrote: >> >>> On Mon 07 Jul 2014 12:14:17 PM EDT, Matt Wringe wrote: >>>> >>>> Initial email to get some discussion going around the LiveOak SDK and >>>> AeroGear collaboration. >>>> >>>> Essentially in LiveOak we are going to need a few different SDK types >>>> >>>> - Client Application SDK >>>> This is the code which will run on the users device. Initial targets >>>> here are javascript (+ cordova support), iOS, Android. This type of SDK >>>> will handle things like getting and sending resources to and from the >>>> server, handling login/logout/registration, etc. Probably some other >>>> things like device registration would be needed as well. >>>> >>>> Not sure if we want to provide support for some other things outside of >>>> communicating with the server or not (eg access to device components (eg >>>> camera, location, etc)) or if these would be best handled by using the >>>> native environment's SDK instead. >>>> >>>> - Server Side SDK >>>> This is code that runs on the server side, written in JavaScript by the >>>> application developer. This will need to be familiar to the client >>>> application's JavaScript SDK, but may not be exactly the same. This type >>>> of SDK will be able to access the same resources as the client side >>>> JavaScript, as well as other internal resources and libraries. >>>> >>>> >>>> I am not sure how to collaborate between the LiveOak and AeroGear teams >>>> here. AeroGear makes really awesome SDKs for various mobile platforms, >>>> but with LiveOak we are dealing with a specific type of application. The >>>> AeroGear SDKs tend to handle the more generic case, which I don't >>>> necessarily think makes sense for a LiveOak SDK. >>>> >>>> I do think it makes sense that the LiveOak SDK uses the AeroGear SDK >>>> internally, but I don't know if we want to expose these AeroGear >>>> components to a LiveOak developer or not. >>>> >>>> >>>> For me, I envision something like the admin setting up their application >>>> in the LiveOak console which then generates a json configuration file >>>> (url locations, resources available, KeyCloak settings, UPS settings, >>>> etc). The application developer then drops this json file in to their >>>> application, the LiveOak SDK reads the json file to set it self up and >>>> then its really easy for the developer to start using it. >>>> >>>> [there are also some really cool things we could be doing here as well >>>> if we can get awesome data sync support for AeroGear. It might be >>>> interesting to be able to fetch a resource from the server and >>>> automatically sync its state across between the client and server. This >>>> way the object appears as a local object: if the resource changes on the >>>> server, it changes locally as well, if it changes locally, that change >>>> is pushed to the server. This way you are just dealing with an object, >>>> and not having to fetch and then push object back and forth between the >>>> server manually] >>>> >>>> Anyone have any thoughts on this? >>> >>> One of the things which may be useful is aerogear-ios and >>> aerogear-android are both modularizing their libraries. IN theory this >>> means that the liveOAK SDK's could be extensions of those modules >>> instead of a single monolothic thing. >>> >>> Sync is still an ongoing discussion in AeroGear, but I think right now >>> the current group thought is that we will focus on 409 Conflict events >>> (JPA versioning on server side and utilities on the client side) and not >>> worrying about realtime sync. >>> >>> >>> Some notes from out last meeting on the topic : >>> Client API Strawman : >>> https://github.com/secondsun/aerogear-android-sync/tree/master/android >>> Client/Server workflow : >>> https://docs.google.com/drawings/d/1E4NDEh3NQCdoEHNNHba4TR2akNrppvV5zDlk5... >>> >>> >>> Also we are looking at doing something with diff merge patch as well as >>> adding in push based data changed notifications later. >>> >>> >>>> _______________________________________________ >>>> aerogear-dev mailing list >>>> aerogear-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >>> >>> >>> >>> -- >>> Summers Pittman >>>>> Phone:404 941 4698 >>>>> Java is my crack. >>> >>> _______________________________________________ >>> aerogear-dev mailing list >>> aerogear-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/aerogear-dev >> > _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev