[infinispan-dev] Backwards compatibility issues with Infinispan 9.x and Hibernate 2LC

Radim Vansa rvansa at redhat.com
Fri Feb 17 03:28:38 EST 2017 

On 02/16/2017 05:41 PM, Galder Zamarreño wrote:
> --
> Galder Zamarreño
> Infinispan, Red Hat
>
>> On 16 Feb 2017, at 12:28, Sanne Grinovero <sanne at hibernate.org> wrote:
>>
>> Have fun with the rewriting!
>> ;)
> Thx ;)
>
>> More seriously, it might be worth to have a second module for Hibernate ORM 5.x to address Infinispan 9?
>> At least in depth testing, stressing and performance aspect won't be left to the last minute integration step.
> Hmmmmm, and what do we do with Infinispan 10? Have yet another module? Not sure that model scales well...

2LC uses Infinispan very extensively, and we can't keep backwards 
compatibility at that scale forever. Especially when this involves 
moving to async interceptor stack.

>
>> Also, would it be worth to move such code to Infinispan? In such cases neither repository is ideal, but I'd tend to prefer to integrate with the one which has the more stable and well defined integration point.
> Moving the code to Infinispan might be a good idea. Given how complex the integration has become (for good reasons: ease of use (no tx, all strategies) and perf), this might be a good idea. The benefit here is that if we'd update Infinispan within Wildfly, the 2LC provider shipped would be updated.

I was already thinking about pushing some functionality from 2LC to 
Infinispan, since I see the same problems as I've dealt with in 2LC 
being solved in other projects (e.g. Keycloak) and users in the forums 
(most recently [1]. However, I don't think that exposing just a cache 
mode (or modifying the current invalidation mode behaviour) is 
sufficient - I think that we need different API to be distilled from 2LC 
SPI.

The problems that pop up are rather complex and require extensive 
testing up to the point that I would recommend deprecating invalidation 
modes when this gets implemented. If you look on the history of JBoss 
Cache/Infinispan, we have forgotten too many edge cases, resulting in 
low reliability.

[1] 
https://developer.jboss.org/message/968821?et=watches.email.thread#968821

>
> The major Hibernate version shipped in Wildfly is normally tied based on EE requirements, whereas Infinispan versions can more freely be upgraded, so I can't see the opposite problem happen that easily: the potential problem of having Infinispan 9.0.x having to work with both Hibernate 5 and 6...

The reason WF 10+ integrates with Infinispan 8 while Hibernate 5.0 was 
compiled against Infinispan 7 is that we wanted to avoid supporting both 
7 and 8 branches in the long term, while Hibernate 5.0 could not move to 
Infinispan 8 which was not released soon enough. I don't see why should 
Hibernate 5 ever work with Infinispan 9. Especially since Hibernate 6 
will likely change some core concepts in hibernate-infinispan 
integration [2].

[2] https://hibernate.atlassian.net/browse/HHH-11356

>
> An important thing would be to make sure we don't put barriers that makes consumption by user harder.
>
>> Historically that has been Hibernate, so hosting this code closer to Infinispan and then merely integrate with Hibernate would have been useful.
>>
>> Alternatively, let's see if we can embrace JCache all-in. The architecture I had discussed with Alex Snaps for the Hibernate /JCache integration would allow for "vendor specific extensions" to bypass the spec on some specific points as needed.

Sanne, could you share details?

> Not sure if I've seen the details of such architecture, but given JCache's stable API, maybe Hibernate should maybe just ship this provider, and any other providers be provided by the implementations themselves (as opposed to be included in Hibernate)

I wonder if you could get JCache-facing implementation to be as fast as 
vendor specific, and if it would be reliable regardless of implementation.

Current implementation seems to assume replicated cache, storing 
expirable locks that work as tombstones. Few notes:

* Regrettably some quirks of ORM API (such as not calling 
afterUpdate/unlockItem after transaction rollback) cause these to stay 
somewhat longer than necessary, but this is something that can be fixed.
* I am unsure if the algorithm would work perfectly with reads that use 
different LockMode; and generally there's no stress test in the testsuite.
* The implementation uses loop of get/putIfAbsent/replace which could be 
replaced by EntryProcessors to achieve guaranteed max 2 RPC roundtrip of 
each operation. Infinispan should reimplement JCache's EntryProcessors 
to use the functional API. We'd need vendor specific extensions for 
effective marshalling.
* We would need above mentioned vendor specific extensions to simulate 
PFERs with their fire&forget semantics.
* 2LC can use operations where the order of updates is not important (it 
can be resolved in-situ), therefore strict ordering which we require for 
regular replicated caches is not required.
* With JCache, you can't parallelize operations within transaction 
either as JCache does not have async ops (currently not utilized by 
Infinispan/by ORM either). Please don't try to mention running each 
operation as a task in a threadpool.
* Some operations (after TX commit) are fine to run asynchronously [1] - 
similar to the fire&forget, reducing the latency.

I admit that invalidation-mode in 2LC implementation is rather 
overcomplicated as it stores the lock-like structures in a separate 
cache (pending-puts), and has to provide custom RPCs for modifying this 
information on the remote side. I hope that in 9.x we'll expose a way to 
make tombstones non-evictable and we could get rid of the invalidation 
mode completely, as it does not provide any advantage to replicated mode 
and increases the complexity - being able to have eviction configured is 
IMO the last thing that holds it there (actually, some users might 
prefer to have the entries that are putFromLoaded available only 
locally, not occupying the cache on other nodes, but this is easy to 
simulate on repl cache, too).

Radim

[1] Galder, could you please review and merge 
https://github.com/hibernate/hibernate-orm/pull/1734 ? I completely 
forgot that it's still hanging in the PR queue.

> Cheers,
>
>>
>> On 16 Feb 2017 11:00, "Galder Zamarreño" <galder at redhat.com> wrote:
>> Hi all,
>>
>> As I've mentioned, I'm working on trying to integrate Hibernate 2LC (5.x branch at the moment) with Infinispan 9.
>>
>> To start with, I tried to see if I could just run Hibernate 2LC 5.x, compiled with Infinispan 8, with an Infinispan 9 runtime. The first problem here was to do with changes to PrioritizedMethodMetadata [1].
>>
>> However, that above is the least of our problems... once I tried to compile with Infinispan 9, there are a lot of compilation errors.
>>
>> Here's a list of what I've found out so far, based on the work in [2] which includes compilation errors and runtime issues I've discovered:
>>
>> 1. Plenty of interceptors have been moved from org.infinispan.interceptors.base and org.infinispan.interceptors packages to org.infinispan.interceptors.impl package.
>>
>> 2. ModuleCommandFactory.fromStream now passes a cache name in ByteString instead of String.
>>
>> 3. DataWriteCommand.setMetadata() method is gone. The reason for this is that FlagAffectedCommand does no longer extend MetadataAwareCommand.
>>
>> 4. Interceptors cannot invoke invokeNextInterceptor() any more in parent, the method has been renamed to invokeNext() (in different class, AsyncInterceptor)
>>
>> 5. A lot of interceptors now take flags as long instead of Set<Flag> which results in compilation error.
>>
>> 6. BaseRpcInterceptor subclasses are now force abstract protected method getLog() to be implemented, again a compilation error.
>>
>> 7. CallInterceptor no longer contains visit... methods, so all interceptors extending it need to extend CommandInterceptor and be placed just before CallInterceptor.
>>
>> 7.1. As a result of that, interceptor positioning calls need to be changed.
>>
>> 8. AdvancedCache.filterEntries() is gone, so need to find an alternative way to do the same.
>>
>> 9. WriteCommand.getAffectedKeys() returns Collection instead of Set now.
>>
>> 10. org.infinispan.filter.NullValueConverter is gone. I removed that as part of marshalling changes since it was not used anywhere within Infinispan repo, but Hibernate 2LC actually uses it.
>>
>> 11. BeginInvalidationCommand and EndInvalidationCommand write directly the lockOwner via `output.writeObject(lockOwner)`, but this causes problem when the lockOwner is a CommandInvocationId since there's no externalizer for it any more. The reason for not having an externalizer is that CommandInvocationId is written via static CommandInvocationId.writeTo() calls.
>>
>> 12. org.infinispan.commands.module.ExtendedModuleCommandFactory is gone.
>>
>> 13. ReplicableCommand.setParameters() method is gone.
>>
>> 14. BaseRpcCommand constructor takes a ByteString instead of String.
>>
>> 15. ReplicableCommand implementations need to implement writeTo() and readFrom() methods.
>>
>> 16. (test) BlockingInterceptor no longer can be added via AdvancedCache.addInterceptor() call because it does not extend CommandInterceptor any more.
>>
>> 17. (test) org.infinispan.util.concurrent.ConcurrentHashSet has been moved.
>>
>> 18. (test) TestingEntityCacheKey must be made extend ExternalPojo so that it can be externally marshalled.
>>
>> 19. (test) 2lc-test-tcp.xml contains attributes that are no longer found by JGroups 4.x and throws errors.
>>
>> The question here is whether we should work towards making Infinispan 9 backwards compatible with Infinispan 8 as far as Hibernate 2LC integration is concerned.
>>
>> In theory, Infinispan 9 should be integrated with Hibernate 6.x onwards, but as always, Wildfly might have different opinions... @Paul?
>>
>> If we need to do something, the time to do it is now, before 9.Final.
>>
>> Cheers,
>>
>> p.s. A lot of tests still failing, so the work in [2] is nowhere near finished.
>>
>> [1] https://gist.github.com/galderz/e26ea9d4838a965500906a6df87e064a
>> [2] https://github.com/galderz/hibernate-orm/commit/5e36a021db4eaad75d835d321282eee6c62cc7c6
>> --
>> Galder Zamarreño
>> Infinispan, Red Hat
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team

More information about the infinispan-dev mailing list