[JIRA] (HHH-16225) Lazy-loading with parallel stream in a separate thread leads to exception with Hibernate 6
by KevinT (JIRA)
KevinT ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *created* an issue
Hibernate ORM ( https://hibernate.atlassian.net/browse/HHH?atlOrigin=eyJpIjoiMjljMWQ2NTcy... ) / Bug ( https://hibernate.atlassian.net/browse/HHH-16225?atlOrigin=eyJpIjoiMjljMW... ) HHH-16225 ( https://hibernate.atlassian.net/browse/HHH-16225?atlOrigin=eyJpIjoiMjljMW... ) Lazy-loading with parallel stream in a separate thread leads to exception with Hibernate 6 ( https://hibernate.atlassian.net/browse/HHH-16225?atlOrigin=eyJpIjoiMjljMW... )
Issue Type: Bug Affects Versions: 6.1.7 Assignee: Unassigned Attachments: hibernate-lazy-loading-async.zip Created: 23/Feb/2023 06:40 AM Priority: Major Reporter: KevinT ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... )
I have migrated from Hibernate 5 to Hibernate 6, and facing an issue while accessing an attribute lazy-loaded via a combination of ForkJoinPool and parallel stream.
I have attached a project to reproduce the issue, just launch org.hibernate.bugs.JPAUnitTestCase#test.
Basically, my data model is the following:
@Entity
@AllArgsConstructor
@NoArgsConstructor
@Data
@EqualsAndHashCode(onlyExplicitlyIncluded = true )
@ToString(onlyExplicitlyIncluded = true )
public class RiskScenario {
@Id
@GeneratedValue
@ToString.Include
private Long id;
@EqualsAndHashCode.Include
@Setter(AccessLevel.PRIVATE)
private UUID uuid = UUID.randomUUID();
@ToString.Include
private String name;
@OneToMany(cascade = ALL, fetch = FetchType.LAZY, mappedBy = "riskScenario" , orphanRemoval = true )
private Set<Treatment> treatments = new HashSet<>();
public void addTreatment(Treatment treatment) {
if (treatments == null ) {
treatments = new HashSet<>();
}
treatments.add(treatment);
treatment.setRiskScenario( this );
}
}
@Entity
@AllArgsConstructor
@NoArgsConstructor
@Data
@EqualsAndHashCode(onlyExplicitlyIncluded = true )
@ToString(onlyExplicitlyIncluded = true )
public class SecurityMeasure {
@Id
@GeneratedValue
@ToString.Include
private Long id;
@EqualsAndHashCode.Include
@Setter(AccessLevel.PRIVATE)
private UUID uuid = UUID.randomUUID();
@ToString.Include
private String name;
@OneToMany(cascade = ALL, fetch = FetchType.LAZY, mappedBy = "securityMeasure" , orphanRemoval = true )
private Set<Treatment> treatments = new HashSet<>();
public void addTreatment(Treatment treatment) {
if (treatments == null ) {
treatments = new HashSet<>();
}
treatments.add(treatment);
treatment.setSecurityMeasure( this );
}
}
@Entity
@AllArgsConstructor
@NoArgsConstructor
@Data
@EqualsAndHashCode(onlyExplicitlyIncluded = true )
@ToString(onlyExplicitlyIncluded = true )
public class Treatment {
@Id
@GeneratedValue
@ToString.Include
private Long id;
@EqualsAndHashCode.Include
@Setter(AccessLevel.PRIVATE)
private UUID uuid = UUID.randomUUID();
@ToString.Include
private String name;
@ManyToOne(cascade = {PERSIST, MERGE}, optional = false )
@JoinColumn(name = "risk_scenario_id" )
@ToString.Include
private RiskScenario riskScenario;
@ManyToOne(cascade = {PERSIST, MERGE}, optional = false )
@JoinColumn(name = "security_measure_id" )
@ToString.Include
private SecurityMeasure securityMeasure;
}
The test itself that you can find in the project is as followed:
@Test
public void test() {
entityManager.getTransaction().begin();
SecurityMeasure firstMeasureRefreshed = entityManager.find(SecurityMeasure.class, this.firstMeasure.getId());
SecurityMeasure secondMeasureRefreshed = entityManager.find(SecurityMeasure.class, this.secondMeasure.getId());
forkJoinPool
.submit(() ->
List.of(firstMeasureRefreshed, secondMeasureRefreshed)
.parallelStream()
.map(SecurityMeasure::getTreatments)
.flatMap(Collection::stream)
.map(Treatment::getRiskScenario)
.collect(Collectors.toList()))
.join();
entityManager.getTransaction().commit();
}
The tricky part is that the error thrown by the test is somehow random:
* Sometimes the test succeeds (but it is quite a rare case)
* Sometimes the test fails with the below exception:
java.util.NoSuchElementException
: java.util.NoSuchElementException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
at java.base/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:672)
at org.hibernate.bugs.JPAUnitTestCase.test(JPAUnitTestCase.java:93)
Caused by: java.util.NoSuchElementException
at java.base/java.util.ArrayDeque.removeFirst(ArrayDeque.java:362)
at org.hibernate.internal.util.collections.StandardStack.pop(StandardStack.java:54)
at org.hibernate.sql.results.spi.LoadContexts.deregister(LoadContexts.java:45)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:207)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:443)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:166)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.list(JdbcSelectExecutorStandardImpl.java:91)
at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:31)
* Sometimes the test fails with the below exception (and this case is the one I managed to reproduce more often):
java.lang.IllegalStateException: java.lang.IllegalStateException: Illegal pop() with non-matching JdbcValuesSourceProcessingState
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
at java.base/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:672)
at org.hibernate.bugs.JPAUnitTestCase.test(JPAUnitTestCase.java:93)
Caused by: java.lang.IllegalStateException: Illegal pop() with non-matching JdbcValuesSourceProcessingState
at org.hibernate.sql.results.spi.LoadContexts.deregister(LoadContexts.java:47)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:207)
at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:443)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:166)
at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.list(JdbcSelectExecutorStandardImpl.java:91)
at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:31)
( https://hibernate.atlassian.net/browse/HHH-16225#add-comment?atlOrigin=ey... ) Add Comment ( https://hibernate.atlassian.net/browse/HHH-16225#add-comment?atlOrigin=ey... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100217- sha1:7bcbf31 )
1 year, 10 months
[JIRA] (HHH-16224) Refactor discovery of exact JDBC drivers, avoid static state in specialized types
by Sanne Grinovero (JIRA)
Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *updated* an issue
Hibernate ORM ( https://hibernate.atlassian.net/browse/HHH?atlOrigin=eyJpIjoiMWYxMzBhNTYw... ) / Task ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiMWYxMz... ) HHH-16224 ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiMWYxMz... ) Refactor discovery of exact JDBC drivers, avoid static state in specialized types ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiMWYxMz... )
Change By: Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... )
We need to re-think how JDBC drivers are detected to being “present” , as the current approach exposes a number of problems; I’m referring to, for example, [https://github.com/hibernate/hibernate-orm/blob/f2deb8f58e990ee87b6e00a73... {{OracleArrayJdbcType}} is also an interesting example as it shows some more design issues. But we need a general pattern, and these two types I’ve called out here are only _examples_ of the issue I’m seeing; the pattern is rather wide-spread.
# Hibernate Reactive (for example) will use the PostgreSQL Dialect, but won’t use PGJDBC; there is an enum {{PostgreSQLDriverKind}} but it’s not protecting things effectively - for example some of the custom types are still having their class initialized (by accident?) even when it’s not PGJDBC, triggering warnings.
# The Hibernate ORM’s classloader won’t necessarily match the application classloader; most commonly in application servers the application will have access to the JDBC driver, but the ORM will not. It’s also possible that the ORM’s classloader is unique while there are multiple applications using it, each on a different classloader - some of which will have access to the driver and some might not.
# The example of the {{OracleArrayJdbcType}} exposes another issue: reflection is not defined upfront during class initialization, but a lazy function is being defined. Ideally if we could ensure all such functions were computed during metadata & Dialect initialization that would be great; alternatively such code will need to emit some kind of event to trigger GraalVM metadata registrations.
My suggestion would be to avoid storing such “presence flags” information in static, classloader bound state; especialy especially as the class definitions will be shared by multiple, different persistence units.
Also, one needs to be careful in separating the notion of “the class can be loaded” from “we’re actually connecting using that driver”; the two concepts are easily conflated, as they will be in our typical integration tests, but they need to be handled strictly separately.
For example it might be possible:
* to load types from the driver as it’s accessible, but we’re using a different database
* we’re using the matching database (e.g. it matches via {{DatabaseMetaData}}#{{getDriverName}}), but the driver is not accessible
An aspect of concern is that the mismatch driver access / metadata would require some craft with integration tests.
Finally, since the pattern is rather wide-spread among a number of custom types, I wonder if we need some tooling help to spot such issues; e.g. reflective calls could theoretically be flagged by forbidden-apis, so to then specifically allow only the ones that have been vetted.
( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... ) Add Comment ( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100217- sha1:7bcbf31 )
1 year, 10 months
[JIRA] (HHH-16224) Refactor discovery of exact JDBC drivers, avoid static state in specialized types
by Sanne Grinovero (JIRA)
Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *updated* an issue
Hibernate ORM ( https://hibernate.atlassian.net/browse/HHH?atlOrigin=eyJpIjoiNThlZGI2Mzg0... ) / Task ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiNThlZG... ) HHH-16224 ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiNThlZG... ) Refactor discovery of exact JDBC drivers, avoid static state in specialized types ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiNThlZG... )
Change By: Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... )
We need to re-think how JDBC drivers are detected to being “present” , as the current approach exposes a number of problems; I’m referring to, for example, [https://github.com/hibernate/hibernate-orm/blob/f2deb8f58e990ee87b6e00a73... {{OracleArrayJdbcType}} is also an interesting examples example as it shows some more design issues. But we need a general pattern, and these two types I’ve called out here are only _examples_ of the issue I’m seeing; the pattern is rather wide-spread.
# Hibernate Reactive (for example) will use the PostgreSQL Dialect, but won’t use PGJDBC; there is an enum {{PostgreSQLDriverKind}} but it’s not protecting things effectively - for example some of the custom types are still having their class initialized (by accident?) even when it’s not PGJDBC, triggering warnings.
# The Hibernate ORM’s classloader won’t necessarily match the application classloader; most commonly in application servers the application will have access to the JDBC driver, but the ORM will not. It’s also possible that the ORM’s classloader is unique while there are multiple applications using it, each on a different classloader - some of which will have access to the driver and some might not.
# The example of the {{OracleArrayJdbcType}} exposes another issue: reflection is not defined upfront during class initialization, but a lazy function is being defined. Ideally if we could ensure all such functions were computed during metadata & Dialect initialization that would be great; alternatively such code will need to emit some kind of event to trigger GraalVM metadata registrations.
My suggestion would be to avoid storing such “presence flags” information in static, classloader bound state; especialy as the class definitions will be shared by multiple, different persistence units.
Also, one needs to be careful in separating the notion of “the class can be loaded” from “we’re actually connecting using that driver”; the two concepts are easily conflated, as they will be in our typical integration tests, but they need to be handled strictly separately.
For example it might be possible:
* to load types from the driver as it’s accessible, but we’re using a different database
* we’re using the matching database (e.g. it matches via {{DatabaseMetaData}}#{{getDriverName}}), but the driver is not accessible
An aspect of concern is that the mismatch driver access / metadata would require some craft with integration tests.
Finally, since the pattern is rather wide-spread among a number of custom types, I wonder if we need some tooling help to spot such issues; e.g. reflective calls could theoretically be flagged by forbidden-apis, so to then specifically allow only the ones that have been vetted.
( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... ) Add Comment ( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100217- sha1:7bcbf31 )
1 year, 10 months
[JIRA] (HHH-16224) Refactor discovery of exact JDBC drivers, avoid static state in specialized types
by Sanne Grinovero (JIRA)
Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *created* an issue
Hibernate ORM ( https://hibernate.atlassian.net/browse/HHH?atlOrigin=eyJpIjoiYjdjMTY2Y2U3... ) / Task ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiYjdjMT... ) HHH-16224 ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiYjdjMT... ) Refactor discovery of exact JDBC drivers, avoid static state in specialized types ( https://hibernate.atlassian.net/browse/HHH-16224?atlOrigin=eyJpIjoiYjdjMT... )
Issue Type: Task Assignee: Unassigned Components: hibernate-core Created: 23/Feb/2023 06:32 AM Fix Versions: 6.wishlist Priority: Major Reporter: Sanne Grinovero ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... )
We need to re-think how JDBC drivers are detected to being “present” , as the current approach exposes a number of problems; I’m referring to, for example, https://github.com/hibernate/hibernate-orm/blob/f2deb8f58e990ee87b6e00a73... ( https://github.com/hibernate/hibernate-orm/blob/f2deb8f58e990ee87b6e00a73... ) and OracleArrayJdbcType is also an interesting examples as it shows some more design issues. But we need a general pattern, and these two types I’ve called out here are only examples of the issue I’m seeing; the pattern is rather wide-spread.
* Hibernate Reactive (for example) will use the PostgreSQL Dialect, but won’t use PGJDBC; there is an enum PostgreSQLDriverKind but it’s not protecting things effectively - for example some of the custom types are still having their class initialized (by accident?) even when it’s not PGJDBC, triggering warnings.
* The Hibernate ORM’s classloader won’t necessarily match the application classloader; most commonly in application servers the application will have access to the JDBC driver, but the ORM will not. It’s also possible that the ORM’s classloader is unique while there are multiple applications using it, each on a different classloader - some of which will have access to the driver and some might not.
* The example of the OracleArrayJdbcType exposes another issue: reflection is not defined upfront during class initialization, but a lazy function is being defined. Ideally if we could ensure all such functions were computed during metadata & Dialect initialization that would be great; alternatively such code will need to emit some kind of event to trigger GraalVM metadata registrations.
My suggestion would be to avoid storing such “presence flags” information in static, classloader bound state; especialy as the class definitions will be shared by multiple, different persistence units.
Also, one needs to be careful in separating the notion of “the class can be loaded” from “we’re actually connecting using that driver”; the two concepts are easily conflated, as they will be in our typical integration tests, but they need to be handled strictly separately.
For example it might be possible:
* to load types from the driver as it’s accessible, but we’re using a different database
* we’re using the matching database (e.g. it matches via DatabaseMetaData # getDriverName ), but the driver is not accessible
An aspect of concern is that the mismatch driver access / metadata would require some craft with integration tests.
Finally, since the pattern is rather wide-spread among a number of custom types, I wonder if we need some tooling help to spot such issues; e.g. reflective calls could theoretically be flagged by forbidden-apis, so to then specifically allow only the ones that have been vetted.
( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... ) Add Comment ( https://hibernate.atlassian.net/browse/HHH-16224#add-comment?atlOrigin=ey... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100217- sha1:7bcbf31 )
1 year, 10 months