[wildfly-dev] update on WildFly NoSQL prototype integration...

[Scott Marlow]

On 05/05/2016 01:48 PM, Jesper Pedersen wrote:
> Hi,
>
> On 05/04/2016 02:16 PM, Scott Marlow wrote:
>> Below is an update on the WildFly NoSQL integration project.  The goal
>> is for deployed applications to have access to NoSQL databases (via
>> Hibernate OGM or native APIs).  Items 1-4, should be finished in our
>> first pass, with as much of the others items as we can do as well.
>>
>
> Some general comments:
>
> - Good start :)
> - Maybe refactor from org.jboss.as to org.wildfly
Done.

> - I think you can leave out the "subsystem" part of the package name
Done.

> - Maybe use version identifiers in the package name since NoSQL APIs
> change a lot
I'm not yet sure about versioning and about life cycle as well.

> - Add some additional asserts to the test cases to verify input / output
> from the stores

Will do.

>
>> 1. connection management will deal with obtaining NoSQL connections for
>> application use.
>>
>>    - borrow/share Hibernate OGM connection configuration setup code
>>    - authentication integration
>>    - support transport level security
>>
>
> Connection management is a huge area to tackle. I think it is best to
> start with what each NoSQL implementation offer.

It could be interesting to ping the various NoSQL communities to get 
some dialog going on how to do platform level connection management.

>
> The big question is if the connections should be truly managed which
> would give the option to no-op dangerous API calls out. There aren't any
> spec requirements for this, so it is really a matter of how the APIs
> should be presented.
>
> You may have to proxy some of the classes in order to pass the security
> information into the stores. I think people would expect to use WildFly
> security domains for this (SubjectFactory/Subject).

Good point, users will want to use WildFly security.

>
>> 2. CDI programming simplifications will make it easy to inject NoSQL
>> data into your application classes.
>>
>>    - https://github.com/antoinesd/javaee-nosql is initial idea
>>
>> 3. You will easily get a native NoSQL connection from the specified
>> NoSQL profile and use the native NOSQL (Cassandra/MongoDB/other) API in
>> your application.
>>
>
> I think this is key, as the APIs change almost per release.
>
>> 4. You will also be able to easily use Hibernate OGM with the defined
>> NoSQL profiles (exactly how is TBD but will be awesome :-).
>>    - Hibernate OGM static module is included.
>>    - need to align with OGM dependencies (e.g. Hibernate ORM + other
>> dependencies).
>>    - as mentioned above, OGM already has some connection setup code,
>> which might be good to share for WildFly + standalone NoSQL use.
>>    - once WildFly has a common NoSQLSource (not a DataSource) that OGM
>> can use, OGM will be enhanced to use it.
>>
>> 5. How best for the WildFly NoSQL subsystem to be optional?
>>    - Is it enough to not run the wildfly/testsuite/nosql tests by default?
>>    - Or do we need to start a separate https://github.com/wildfly/nosql
>> project for the NoSQL subsystems?
>>
>> 6. transaction enlistment
>>
>
> This will vary for each store, and there are many loopholes that you may
> want to plug, f.ex. sharing a connection between 2 transactions.
>
> For 1-phase, you will have to insert a LocalXAResource instance into the
> transaction when the "connection" is obtained, and return it on the
> boundary (afterCompletion). Same deal for 2-phase basically.
>
> The LocalXAResource implementation will of course be different for each
> store.

Any links to existing IronJacamar code to share here?  I think that we 
could prototype new transaction enlistment handling code, based on what 
we currently have.

>
> You may want a transaction option for the stores that supports this such
> that people can choose the "level" of enlistment (ala jta="false").
>
> One task I see is that people will have access to the transactional
> methods in the API, and you don't want them to call these methods in
> their apps unless the transaction setting allows this.
>
> I think you can leave out the corner-cases in the 1st iteration, like
> deferred enlistment (get connection, start transaction).
>
>> 7. compensating transactions
>>
>
> Another huge area.
>
> I would let the use-cases from WildFly/Swarm drive this.
>
> A simple way to start is to have a SPI that apps can implement in case
> the transaction is MARKED_FOR_ROLLBACK (hooks to an enlisted XAResource).
>
>> 8. runtime application monitoring
>>
>
> As each store is different I would only look into what the subsystem can
> expose, plus whatever the store can display.
>
>> 9. How soon can we make an evaluation distribution available for use on
>> OpenStack/OpenShift?
>>    - Would be great if we could do some load testing with all NoSQL
>> components.
>>    - Would be great if we could enable others to also test.
>>
>> 10. Are there any problems with our WildFly NoSQL subsystem injecting
>> MongoDatabase connections via:
>>
>>      @Resource(lookup = "java:jboss/mongodb/test") MongoDatabase db;
>>
>>    - No @Resource support expected for standalone Java, TBD is whether a
>> runtime library can be used.
>>    - Any problems expected on other EE application servers if this
>> approach becomes popular?
>>
>
> This ties into connection management. I like what you have now for the
> 1st pass.
>
>> 11. WIP topic branch is at
>> https://github.com/scottmarlow/wildfly/tree/nosql-dev9.  Note that every
>> once in a while, commits are squashed and pushed to nosql-devN+1.
>>
>> 12. Add proper unit tests
>>    - multi-threaded NoSQL access to show that works at all.
>>    - use NoSQL from different EE components (e.g. JAX-RS).
>>    - other use cases that represent how NoSQL could be used from WildFly.
>>
>
> I think you can look at test cases for EE components that support @Resource.

Using @Inject may be another way, once we have 
https://github.com/antoinesd/javaee-nosql going.  Will see which way 
survives the prototype stage to what is merged into WildFly. :-)

>
> Best regards,
>   Jesper
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>