CMR collection gets corrupted in 2LC on certain concurrent access ----------------------------------------------------------------- Key: WFLY-7127 URL: https://issues.jboss.org/browse/WFLY-7127 Project: WildFly Issue Type: Bug Components: EJB, JPA / Hibernate, Transactions Affects Versions: 10.1.0.Final Environment: Ubuntu Linux 15.10. Java 1.8.0_66 Reporter: Jari Juslin Assignee: Scott Marlow Attachments: infinispan_cmr_corruption_repro.tar.gz, standalone.xml With certain parallel read and write access to same CMR collection it gets corrupted in Hibernate second level cache. After the event of corruption, any read access to that CMR collection ends up in Exception until the cache is cleared manually or Wildfly restarted. Steps to reproduce: 1) Make sure the CMR collection is not in cache. 2) Start a read-only transaction accessing the said collection. 3) In a separate transaction delete an element from that collection. 4) End the read-only transaction. 5) Read the resulting collection. It now throws EntityNotFoundException. My understanding is that what happens is that the read-only transaction is read-only from the perspective of the database, so the system handles it as such. However, because for the 2LC it is actually a write transaction, when the transaction ends, the contents of the CMR collection visible to this transaction get committed to 2LC. If the write transaction ends before the read-only transaction, the old version of the collection stays in the 2LC. Now, as the CMR collection contains an element already deleted from the DB, any attempt to manifest the collection results in EntityNotFoundException as the child element no longer exists in the DB. For sake of repeatibility the test case here manually spawns threads to cause the effect. In real use, of course, this happens randomly when two different threads spawned by Wildfly happen to interleave with exactly a right kind of pattern. The tar ball contains everything needed to reproduce the bug, including SQL dump of the MySQL/InnoDB database. The easiest way to run the testcase is to take Wildfly 10.1.0.Final and use the provided standalone.xml, add MySQL driver, import the DB with the provided SQL and then deploy the included packaged testcase.ear.