[wildfly-dev] update on WildFly NoSQL prototype integration...

[Stefano Maestri]

On Thu, May 5, 2016 at 7:48 PM, Jesper Pedersen <jesper.pedersen at redhat.com>
 wrote:

> Hi,
>
> On 05/04/2016 02:16 PM, Scott Marlow wrote:
> > Below is an update on the WildFly NoSQL integration project.  The goal
> > is for deployed applications to have access to NoSQL databases (via
> > Hibernate OGM or native APIs).  Items 1-4, should be finished in our
> > first pass, with as much of the others items as we can do as well.
> >
>
> Some general comments:
>
> - Good start :)
> - Maybe refactor from org.jboss.as to org.wildfly
>

+1


> - Maybe use version identifiers in the package name since NoSQL APIs
> change a lot
>

Well even if it is different we had bad experience on this w/ IJ and
removed those version identifier from package name. I think package name
should not have version ids, even if it could be acceptable in this first
phase we should try to hide this kind of infos to users of OURS API even if
APIs from NoSQL impl would be dependents from specific release.


> - Add some additional asserts to the test cases to verify input / output
> from the stores
>
> > 1. connection management will deal with obtaining NoSQL connections for
> > application use.
> >
> >    - borrow/share Hibernate OGM connection configuration setup code
> >    - authentication integration
> >    - support transport level security
> >
>
> Connection management is a huge area to tackle. I think it is best to
> start with what each NoSQL implementation offer.
>

Yup it would be a good start, even if the other question is "why a user
should use WF integration instead of direct use of various
implementation?". One of the possible answer could be "to have a common
connection management" for sure. So the (final) effort should be to have a
common connection management.


>
> The big question is if the connections should be truly managed which
> would give the option to no-op dangerous API calls out. There aren't any
> spec requirements for this, so it is really a matter of how the APIs
> should be presented.
>

Yup exactly.


>
> You may have to proxy some of the classes in order to pass the security
> information into the stores. I think people would expect to use WildFly
> security domains for this (SubjectFactory/Subject).
>

Yes it's another possible question to "why use WF integration?"


>
> > 2. CDI programming simplifications will make it easy to inject NoSQL
> > data into your application classes.
> >
> >    - https://github.com/antoinesd/javaee-nosql is initial idea
> >
> > 3. You will easily get a native NoSQL connection from the specified
> > NoSQL profile and use the native NOSQL (Cassandra/MongoDB/other) API in
> > your application.
> >
>
> I think this is key, as the APIs change almost per release.
>

Agree 100%. And it gives us also another requirement: should be easily to
upgrade this Wildfly subsystem (or at least underlying NoSQL
implementations) because NoSQL users would expect to use latest "cool"
features of their NoSQL impl and don't want to wait for WildFLy (or EAP)
release cycle.



>
> > 4. You will also be able to easily use Hibernate OGM with the defined
> > NoSQL profiles (exactly how is TBD but will be awesome :-).
> >    - Hibernate OGM static module is included.
> >    - need to align with OGM dependencies (e.g. Hibernate ORM + other
> > dependencies).
> >    - as mentioned above, OGM already has some connection setup code,
> > which might be good to share for WildFly + standalone NoSQL use.
> >    - once WildFly has a common NoSQLSource (not a DataSource) that OGM
> > can use, OGM will be enhanced to use it.
> >
> > 5. How best for the WildFly NoSQL subsystem to be optional?
> >    - Is it enough to not run the wildfly/testsuite/nosql tests by
> default?
> >    - Or do we need to start a separate https://github.com/wildfly/nosql
> > project for the NoSQL subsystems?
> >
> > 6. transaction enlistment
> >
>
> This will vary for each store, and there are many loopholes that you may
> want to plug, f.ex. sharing a connection between 2 transactions.
>
> For 1-phase, you will have to insert a LocalXAResource instance into the
> transaction when the "connection" is obtained, and return it on the
> boundary (afterCompletion). Same deal for 2-phase basically.
>
> The LocalXAResource implementation will of course be different for each
> store.
>
> You may want a transaction option for the stores that supports this such
> that people can choose the "level" of enlistment (ala jta="false").
>
> One task I see is that people will have access to the transactional
> methods in the API, and you don't want them to call these methods in
> their apps unless the transaction setting allows this.
>
> I think you can leave out the corner-cases in the 1st iteration, like
> deferred enlistment (get connection, start transaction).
>
> > 7. compensating transactions
> >
>
> Another huge area.
>
> I would let the use-cases from WildFly/Swarm drive this.
>
> A simple way to start is to have a SPI that apps can implement in case
> the transaction is MARKED_FOR_ROLLBACK (hooks to an enlisted XAResource).
>

We implemented those in IJ (even if in much more standard JCA environment),
and maybe we could share and eventually copy/paste some code.



>
> > 8. runtime application monitoring
> >
>
> As each store is different I would only look into what the subsystem can
> expose, plus whatever the store can display.
>

Maybe a plugin approach could be taken as we did for datasource and RA
subsystems reading statistics available during deployment and runtime and
exposing them as pure runtime metrics in DMR.

Just my 2 cents.

Best regards
S.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20160510/1a308527/attachment-0001.html