[wildfly-dev] update on WildFly NoSQL prototype integration...

[Jesper Pedersen]

Hi,

On 05/04/2016 02:16 PM, Scott Marlow wrote:
> Below is an update on the WildFly NoSQL integration project.  The goal
> is for deployed applications to have access to NoSQL databases (via
> Hibernate OGM or native APIs).  Items 1-4, should be finished in our
> first pass, with as much of the others items as we can do as well.
>

Some general comments:

- Good start :)
- Maybe refactor from org.jboss.as to org.wildfly
- I think you can leave out the "subsystem" part of the package name
- Maybe use version identifiers in the package name since NoSQL APIs 
change a lot
- Add some additional asserts to the test cases to verify input / output 
from the stores

> 1. connection management will deal with obtaining NoSQL connections for
> application use.
>
>    - borrow/share Hibernate OGM connection configuration setup code
>    - authentication integration
>    - support transport level security
>

Connection management is a huge area to tackle. I think it is best to 
start with what each NoSQL implementation offer.

The big question is if the connections should be truly managed which 
would give the option to no-op dangerous API calls out. There aren't any 
spec requirements for this, so it is really a matter of how the APIs 
should be presented.

You may have to proxy some of the classes in order to pass the security 
information into the stores. I think people would expect to use WildFly 
security domains for this (SubjectFactory/Subject).

> 2. CDI programming simplifications will make it easy to inject NoSQL
> data into your application classes.
>
>    - https://github.com/antoinesd/javaee-nosql is initial idea
>
> 3. You will easily get a native NoSQL connection from the specified
> NoSQL profile and use the native NOSQL (Cassandra/MongoDB/other) API in
> your application.
>

I think this is key, as the APIs change almost per release.

> 4. You will also be able to easily use Hibernate OGM with the defined
> NoSQL profiles (exactly how is TBD but will be awesome :-).
>    - Hibernate OGM static module is included.
>    - need to align with OGM dependencies (e.g. Hibernate ORM + other
> dependencies).
>    - as mentioned above, OGM already has some connection setup code,
> which might be good to share for WildFly + standalone NoSQL use.
>    - once WildFly has a common NoSQLSource (not a DataSource) that OGM
> can use, OGM will be enhanced to use it.
>
> 5. How best for the WildFly NoSQL subsystem to be optional?
>    - Is it enough to not run the wildfly/testsuite/nosql tests by default?
>    - Or do we need to start a separate https://github.com/wildfly/nosql
> project for the NoSQL subsystems?
>
> 6. transaction enlistment
>

This will vary for each store, and there are many loopholes that you may 
want to plug, f.ex. sharing a connection between 2 transactions.

For 1-phase, you will have to insert a LocalXAResource instance into the 
transaction when the "connection" is obtained, and return it on the 
boundary (afterCompletion). Same deal for 2-phase basically.

The LocalXAResource implementation will of course be different for each 
store.

You may want a transaction option for the stores that supports this such 
that people can choose the "level" of enlistment (ala jta="false").

One task I see is that people will have access to the transactional 
methods in the API, and you don't want them to call these methods in 
their apps unless the transaction setting allows this.

I think you can leave out the corner-cases in the 1st iteration, like 
deferred enlistment (get connection, start transaction).

> 7. compensating transactions
>

Another huge area.

I would let the use-cases from WildFly/Swarm drive this.

A simple way to start is to have a SPI that apps can implement in case 
the transaction is MARKED_FOR_ROLLBACK (hooks to an enlisted XAResource).

> 8. runtime application monitoring
>

As each store is different I would only look into what the subsystem can 
expose, plus whatever the store can display.

> 9. How soon can we make an evaluation distribution available for use on
> OpenStack/OpenShift?
>    - Would be great if we could do some load testing with all NoSQL
> components.
>    - Would be great if we could enable others to also test.
>
> 10. Are there any problems with our WildFly NoSQL subsystem injecting
> MongoDatabase connections via:
>
>      @Resource(lookup = "java:jboss/mongodb/test") MongoDatabase db;
>
>    - No @Resource support expected for standalone Java, TBD is whether a
> runtime library can be used.
>    - Any problems expected on other EE application servers if this
> approach becomes popular?
>

This ties into connection management. I like what you have now for the 
1st pass.

> 11. WIP topic branch is at
> https://github.com/scottmarlow/wildfly/tree/nosql-dev9.  Note that every
> once in a while, commits are squashed and pushed to nosql-devN+1.
>
> 12. Add proper unit tests
>    - multi-threaded NoSQL access to show that works at all.
>    - use NoSQL from different EE components (e.g. JAX-RS).
>    - other use cases that represent how NoSQL could be used from WildFly.
>

I think you can look at test cases for EE components that support @Resource.

Best regards,
  Jesper