[wildfly-dev] WildFly NoSQL integration prototype

[Scott Marlow]

Are you interested in allowing NoSQL databases access from WildFly 
application deployments?  This email is about an integration effort to 
allow WildFly applications to use NoSQL.  Feedback is welcome on this 
effort, as well as help in improving [1].  Some basic unit tests are 
already added that show a session bean reading/writing MongoDB [2] + 
Cassandra [3] databases.  In order for the tests to pass, the local 
machine must already be running MongoDB or Cassandra databases.

1. Things that currently (seems to be) working in the prototype:

* During WildFly startup, MongoDB/Cassandra databases are connected to 
based on settings in their respective subsystems.  See the configuration 
example [4].

* Applications can access native MongoDB/Cassandra objects that 
represent database connections (with internal native connection 
pooling).  See @Resource example [2][3].  Will see how the requirements 
evolve going forward and whether @Resource is the right way and/or 
whether other annotations are needed.

2.  Currently not working in the prototype:

* Multiple hosts/ports cannot be specified yet for target database.

* Protection against applications closing pooled connections.

* NoSQL drivers currently may create threads in EE application threads 
which could leak ClassLoaders/AccessControlContexts.  One solution might 
be to contribute a patch that allows WildFly to do the thread creation 
in some fashion for the NoSQL drivers.

* We have not (yet) tried using (Java) security manager support with the 
NoSQL driver clients.

* Additional NoSQL connection attributes need to be added to the NoSQL 
subsystems.

* Native NoSQL class objects are bound to JNDI currently (e.g. 
MongoClient).  We might want to bind wrapper or proxy objects so that we 
can extend the NoSQL classes or in some cases, prevent certain actions 
(e.g. prevent calls to MongoClient.close()).  Perhaps we will end up 
with a mixed approach, where we could extend the NoSQL driver if that is 
the only way to manage it, or contribute a listener patch for WildFly to 
get control during certain events (e.g. for ignoring close of pooled 
database connections).

* The prototype currently gives all (WildFly) deployments access to the 
Cassandra/MongoDB driver module classloaders.  This is likely to change 
but not yet sure to what.

3.  The Weld (CDI) project is also looking at NoSQL enhancements, as is 
the Narayana project.  There is also the Hibernate OGM project that is 
pushing on JPA integration and will also help contribute changes to the 
NoSQL drivers that are needed for WildFly integration (e.g. introduce 
alternative way for NoSQL drivers manage thread creation for background 
task execution).

4.  We will need a place to track issues for NoSQL integration.  If the 
NoSQL integration changes are merged directly into WildFly, perhaps we 
could have a nosql category under https://issues.jboss.org/browse/WFLY.

5. You can view outstanding issues in the MongoDB server [5], Java 
driver [6] to get feel for problems that others have run into (just like 
you would with WildFly).  You can view outstanding issues in the 
Cassandra server [7] and Java driver [8] to get a feel for problems as well.

6.  Infinispan [9] integration in WildFly is still going strong. 
Infinispan is still the backbone of WildFly clustering and also 
available for applications to use as a datasource.

7.  The standalone.xml settings [4] will soon change (would like to 
eliminate the "name=default", add more attributes and get the multiple 
host/ports wired in).

8.  If the NoSQL unit tests do stay in the WildFly repo, they will need 
to be disabled by default, as most WildFly developers will not have a 
NoSQL database running.  Speaking of which, we need to wire the unit 
tests to update the standalone.xml to contain the MongoDB/Cassandra 
subsystem settings [4].

9.  What version of NoSQL databases will work with the WildFly NoSQL 
integration?  At this point, we will only work with one version of each 
NoSQL database that is integrated with.  Because we are likely to need 
some changes in the NoSQL client drivers, we will work with the upstream 
communities to ensure the NoSQL driver code can run in an EE container 
thread, without causing leaks.  First we have to identity the changes 
that we need (e.g. find some actual leaks that I only suspect will 
happen at this point and propose some changes).  The Hibernate OGM team 
is going to help with the driver patches (thanks Hibernate OGM team! :-)

10.  Going forward, how can WildFly extend the NoSQL (client driver 
side) capabilities to improve the different application life cycles 
through development, test, production?

Scott

[1] https://github.com/scottmarlow/wildfly/tree/nosql-dev

[2] 
https://github.com/scottmarlow/wildfly/blob/nosql-dev/testsuite/compat/src/test/java/org/jboss/as/test/compat/nosql/mongodb/StatefulTestBean.java#L23

[3] 
https://github.com/scottmarlow/wildfly/blob/nosql-dev/testsuite/compat/src/test/java/org/jboss/as/test/compat/nosql/cassandra/StatefulTestBean.java#L19

[4] https://gist.github.com/scottmarlow/b8196bdc56431bb171c8

[5] https://jira.mongodb.org/browse/SERVER

[6] https://jira.mongodb.org/browse/JAVA

[7] https://issues.apache.org/jira/browse/CASSANDRA

[8] https://datastax-oss.atlassian.net/browse/JAVA

[9] http://infinispan.org