[wildfly-dev] update on WildFly NoSQL prototype integration...
Scott Marlow
smarlow at redhat.com
Wed May 11 10:02:44 EDT 2016
On 05/10/2016 06:34 AM, Gunnar Morling wrote:
> Hi Scott,
>
> Thanks for driving this initiative!
>
> Some comments/questions inline.
>
> 2016-05-04 20:16 GMT+02:00 Scott Marlow <smarlow at redhat.com
> <mailto:smarlow at redhat.com>>:
>
> Hi,
>
> Below is an update on the WildFly NoSQL integration project. The goal
> is for deployed applications to have access to NoSQL databases (via
> Hibernate OGM or native APIs). Items 1-4, should be finished in our
> first pass, with as much of the others items as we can do as well.
>
> 1. connection management will deal with obtaining NoSQL connections for
> application use.
>
> - borrow/share Hibernate OGM connection configuration setup code
> - authentication integration
> - support transport level security
>
>
> Any ideas how this code sharing could look like?
>
> Hibernate OGM should still be usable without WF; So maybe there should
> be a separate project/repo which defines an SPI to obtain/manage
> connections and implementations for different NoSQL stores?
Excellent suggestion, perhaps the SPI could be under
https://github.com/jboss, which is a common area for sharing. Possible
locations for creating the per NoSQL store implementations could be
https://github.com/jboss or https://github.com/hibernate or
https://github.com/wildfly.
>
> I think it'd be beneficial to define some sort of connection URL syntax
> which can be used in NoSQL datasource definitions:
>
>
> nosql:mongodb://192.168.1.100:27017[,another-host:another-port]/some_db?customParam1=xyz&customParam2=abc&...
The WildFly NoSQL subsystem configuration, would likely need separate
fields for NoSQL datasource profiles/definition, instead of one long
URL. Internally, the WildFly subsystem could build a connection URL, if
that becomes our common format.
>
> This common syntax might do for most cases, for others it might be
> needed to support specialized schemes handled by store-specific SPI
> implementations.
What are some examples that might need specialized schemes? Would an
embedded or in-process Infinispan/Cassandra be considered one of the
specialized schemes because there are no remote connections?
>
> Regarding pooling, many NoSQL drivers do it in one way or another, so
> maybe leave that to drivers, at least in the first iteration?
>
> 2. CDI programming simplifications will make it easy to inject NoSQL
> data into your application classes.
>
> - https://github.com/antoinesd/javaee-nosql is initial idea
>
>
> +1 Sounds nice.
>
>
> 3. You will easily get a native NoSQL connection from the specified
> NoSQL profile and use the native NOSQL (Cassandra/MongoDB/other) API in
> your application.
>
>
> Would you take measures to e.g. prevent closing a connection in
> application code?
I think that this should be on the wish list, just not sure that we
should always introduce the overhead of wrapping/proxying NoSQL
connections to allow the check if the application. Perhaps a "platform
level" listener in the native NoSQL client code, that allows the
platform to prevent applications from closing connections in some way.
>
> 4. You will also be able to easily use Hibernate OGM with the defined
> NoSQL profiles (exactly how is TBD but will be awesome :-).
> - Hibernate OGM static module is included.
>
>
> WDYM by "static module"? The Hibernate OGM Core module?
By "static module", I meant what we add to wildfly/modules/*. Excellent
question about which OGM modules, I meant any OGM that depends on
Hibernate ORM, which I think is currently OGM Core.
>
> How would people add the OGM backend module for the NoSQL store of their
> choice? I suppose you don't mean to add all the backends we have to WF
> by default (as this would increase size quite a bit), so would people
> still use the module ZIP we provide?
If we use the OGM module ZIP, that should be extracted into a different
folder than wildfly/modules/system/layers, to avoid confusing the EAP
patch tool. Having said that, I'm not sure how we will patch the OGM
modules, if they are not all included in WildFly. Perhaps, we might
decide to include only certain ones.
Just for reference, my local
hibernate-ogm-modules-wildfly10-5.0.0-SNAPSHOT.zip is 53mb (including
OGM core).
>
>
> - need to align with OGM dependencies (e.g. Hibernate ORM + other
> dependencies).
>
>
> Note we aim at the ORM version part of WF at a given time. But there are
> times where we need a newer ORM version for a new OGM release (when
> using new SPIs not part in the ORM version coming with the currently
> released WF), so there should be a way to achieve that.
Are you thinking of the current Hibernate ORM master or the ORM 5.1
branch? Or perhaps a new branch based on 5.1, that doesn't have all of
the ORM master branch changes?
>
>
> - as mentioned above, OGM already has some connection setup code,
> which might be good to share for WildFly + standalone NoSQL use.
>
>
> I'm not sure how re-usable this is. It's geared towards OGM's needs, I
> feel having a separate SPI dealing just with connections would be more
> useful.
+1
>
>
> - once WildFly has a common NoSQLSource (not a DataSource) that OGM
> can use, OGM will be enhanced to use it.
>
>
> +1
>
> 5. How best for the WildFly NoSQL subsystem to be optional?
> - Is it enough to not run the wildfly/testsuite/nosql tests by
> default?
> - Or do we need to start a separate https://github.com/wildfly/nosql
> project for the NoSQL subsystems?
>
>
> Would there be one OGM sub-system? Or one per NoSQL store?
I don't yet see a need for an OGM sub-system (I don't think OGM needs a
separate WildFly deployment processer). I think that some of the OGM
deployment hacks will be done in Jipijapa (assuming we can change the
JPA deployer to allow that). I'm not exactly sure what your asking here
with regard to question #5, about one OGM sub-system or one per NosQL
store. Do you mean "static module" instead of sub-system?
I think that there will be one WildFly sub-system per NoSQL store but
that is based on the current prototype and the discussion so far.
> And how would
> people add them (see above)? As said before in this thread, there should
> be a way to update modules of specific stores to get the latest and
> greatest.
Not sure yet, maybe with-in minor releases it will be possible.
>
> 6. transaction enlistment
>
> 7. compensating transactions
>
> 8. runtime application monitoring
>
> 9. How soon can we make an evaluation distribution available for use on
> OpenStack/OpenShift?
> - Would be great if we could do some load testing with all NoSQL
> components.
> - Would be great if we could enable others to also test.
>
> 10. Are there any problems with our WildFly NoSQL subsystem injecting
> MongoDatabase connections via:
>
> @Resource(lookup = "java:jboss/mongodb/test") MongoDatabase db;
>
> - No @Resource support expected for standalone Java, TBD is whether a
> runtime library can be used.
> - Any problems expected on other EE application servers if this
> approach becomes popular?
>
> 11. WIP topic branch is at
> https://github.com/scottmarlow/wildfly/tree/nosql-dev9. Note that every
> once in a while, commits are squashed and pushed to nosql-devN+1.
>
>
> Any pointers where to start looking to see what's working already?
https://github.com/scottmarlow/wildfly/tree/nosql-dev9 is still active. :)
>
> 12. Add proper unit tests
> - multi-threaded NoSQL access to show that works at all.
> - use NoSQL from different EE components (e.g. JAX-RS).
> - other use cases that represent how NoSQL could be used from WildFly.
>
> Feedback/help is welcome!
>
> Thanks,
> Scott
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org <mailto:wildfly-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
More information about the wildfly-dev
mailing list