[wildfly-dev] update on WildFly NoSQL prototype integration...

Tue May 10 10:17:33 EDT 2016

On 05/10/2016 05:26 AM, Stefano Maestri wrote:
>
>
> On Thu, May 5, 2016 at 7:48 PM, Jesper
> Pedersen <jesper.pedersen at redhat.com
> <mailto:jesper.pedersen at redhat.com>> wrote:
>
>     Hi,
>
>     On 05/04/2016 02:16 PM, Scott Marlow wrote:
>     > Below is an update on the WildFly NoSQL integration project.  The goal
>     > is for deployed applications to have access to NoSQL databases (via
>     > Hibernate OGM or native APIs).  Items 1-4, should be finished in our
>     > first pass, with as much of the others items as we can do as well.
>     >
>
>     Some general comments:
>
>     - Good start :)
>     - Maybe refactor from org.jboss.as <http://org.jboss.as/> to org.wildfly
>
>
> +1
>
>
>     - Maybe use version identifiers in the package name since NoSQL APIs
>     change a lot
>
>
> Well even if it is different we had bad experience on this w/ IJ and
> removed those version identifier from package name. I think package name
> should not have version ids, even if it could be acceptable in this
> first phase we should try to hide this kind of infos to users of OURS
> API even if APIs from NoSQL impl would be dependents from specific release.
>
>
>     - Add some additional asserts to the test cases to verify input / output
>     from the stores
>
>     > 1. connection management will deal with obtaining NoSQL connections for
>     > application use.
>     >
>     >    - borrow/share Hibernate OGM connection configuration setup code
>     >    - authentication integration
>     >    - support transport level security
>     >
>
>     Connection management is a huge area to tackle. I think it is best to
>     start with what each NoSQL implementation offer.
>
>
> Yup it would be a good start, even if the other question is "why a user
> should use WF integration instead of direct use of various
> implementation?". One of the possible answer could be "to have a common
> connection management" for sure. So the (final) effort should be to have
> a common connection management.

I wonder if any of the NoSQL database development communities would be 
interested in establishing a common connection management client API.

>
>
>
>     The big question is if the connections should be truly managed which
>     would give the option to no-op dangerous API calls out. There aren't any
>     spec requirements for this, so it is really a matter of how the APIs
>     should be presented.
>
>
> Yup exactly.
>
>
>
>     You may have to proxy some of the classes in order to pass the security
>     information into the stores. I think people would expect to use WildFly
>     security domains for this (SubjectFactory/Subject).
>
>
> Yes it's another possible question to "why use WF integration?"
>
>
>
>     > 2. CDI programming simplifications will make it easy to inject NoSQL
>     > data into your application classes.
>     >
>     >    - https://github.com/antoinesd/javaee-nosql is initial idea
>     >
>     > 3. You will easily get a native NoSQL connection from the specified
>     > NoSQL profile and use the native NOSQL (Cassandra/MongoDB/other) API in
>     > your application.
>     >
>
>     I think this is key, as the APIs change almost per release.
>
>
> Agree 100%. And it gives us also another requirement: should be easily
> to upgrade this Wildfly subsystem (or at least underlying NoSQL
> implementations) because NoSQL users would expect to use latest "cool"
> features of their NoSQL impl and don't want to wait for WildFLy (or EAP)
> release cycle.
>
>
>
>
>     > 4. You will also be able to easily use Hibernate OGM with the defined
>     > NoSQL profiles (exactly how is TBD but will be awesome :-).
>     >    - Hibernate OGM static module is included.
>     >    - need to align with OGM dependencies (e.g. Hibernate ORM + other
>     > dependencies).
>     >    - as mentioned above, OGM already has some connection setup code,
>     > which might be good to share for WildFly + standalone NoSQL use.
>     >    - once WildFly has a common NoSQLSource (not a DataSource) that OGM
>     > can use, OGM will be enhanced to use it.
>     >
>     > 5. How best for the WildFly NoSQL subsystem to be optional?
>     >    - Is it enough to not run the wildfly/testsuite/nosql tests by default?
>     >    - Or do we need to start a separate https://github.com/wildfly/nosql
>     > project for the NoSQL subsystems?
>     >
>     > 6. transaction enlistment
>     >
>
>     This will vary for each store, and there are many loopholes that you may
>     want to plug, f.ex. sharing a connection between 2 transactions.
>
>     For 1-phase, you will have to insert a LocalXAResource instance into the
>     transaction when the "connection" is obtained, and return it on the
>     boundary (afterCompletion). Same deal for 2-phase basically.
>
>     The LocalXAResource implementation will of course be different for each
>     store.
>
>     You may want a transaction option for the stores that supports this such
>     that people can choose the "level" of enlistment (ala jta="false").
>
>     One task I see is that people will have access to the transactional
>     methods in the API, and you don't want them to call these methods in
>     their apps unless the transaction setting allows this.
>
>     I think you can leave out the corner-cases in the 1st iteration, like
>     deferred enlistment (get connection, start transaction).
>
>     > 7. compensating transactions
>     >
>
>     Another huge area.
>
>     I would let the use-cases from WildFly/Swarm drive this.
>
>     A simple way to start is to have a SPI that apps can implement in case
>     the transaction is MARKED_FOR_ROLLBACK (hooks to an enlisted
>     XAResource).
>
>
> We implemented those in IJ (even if in much more standard JCA
> environment), and maybe we could share and eventually copy/paste some
> code.

Check out "MongoDB integration test with compensating transactions"
https://github.com/scottmarlow/wildfly/commit/9bd67964a9259416add73cac47328cec5127d25c, 
which is a good start.  We can work forward from what we have and ideas 
from IJ.

>
>
>
>
>     > 8. runtime application monitoring
>     >
>
>     As each store is different I would only look into what the subsystem can
>     expose, plus whatever the store can display.
>
>
> Maybe a plugin approach could be taken as we did for datasource and RA
> subsystems reading statistics available during deployment and runtime
> and exposing them as pure runtime metrics in DMR.

Great idea and great examples.

>
> Just my 2 cents.
>
> Best regards
> S.
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>