[Hibernate-JIRA] Created: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
3:55 a.m.
Hilo optimizer problem in case of multiple threads accessing the sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work and
will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:40 p.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Tara Simpson commented on HHH-3628:
-----------------------------------
This is not a major issue, it is absolutely critical.
Steve Ebersole has 'highly recommended' (in italics) that one should use the
enhanced sequence generators - see http://blog.hibernate.org/Bloggers/Steve. However,
both this and the pooled optimizer strategies do not work in a clustered /
multi-jvm'ed environment - and what good is a sequence generator if it cannot be used
in more than one JVM.
Btw, this bug results in a ConstraintViolationException on the primary key when
inserting. It is easily reproducible on even moderately busy systems.
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:22 a.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Montagnon Cyril commented on HHH-3628:
--------------------------------------
1) Tara, Steve Eversole 'highly recommends' not to use the NON enhanced
TableGenerator.
I'm not sure he doesn't recommend using the enhanced TableGenerator.
"The second is the more risky replacement because of the big difference between the
two. But we've all along discouraged direct use of the current "table"
generator so I think we should be safe there. I am still uncertain when that replacement
will happen (probably 4.0?), but in the meantime, the new generators are available and
highly recommended for use. "
I agree with you : it's critical!
2) The pooled optimizer works in a multi-jvm'ed environment. The generate() method is
correct (the 'value' is reset in the else if part ) :
public synchronized Serializable generate(AccessCallback callback) {
if ( hiValue < 0 ) {
value = callback.getNextValue();
if ( value < 1 ) {
// unfortunately not really safe to normalize this
// to 1 as an initial value like we do the others
// because we would not be able to control this if
// we are using a sequence...
log.info( "pooled optimizer source reported [" + value + "] as the
initial value; use of 1 or greater highly recommended" );
}
hiValue = callback.getNextValue();
}
else if ( value >= hiValue ) {
hiValue = callback.getNextValue();
value = hiValue - incrementSize;
}
return make( value++ );
}
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:25 a.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Tara Simpson commented on HHH-3628:
-----------------------------------
My original comment was a little narky, I apologise....but Steve's blog does recommend
using the new generators. "The new generators are available and highly recommended
for use" is fairly Plain English.
Regarding the PooledOptimizer issue, see HHH-3608.
In meantime we've replaced all use of Hibernate's sequencers with our own working
implementation - so for us the criticality of the issue is somewhat mitigated, but for
many unsuspecting users it is not.
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:09 a.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Montagnon Cyril commented on HHH-3628:
--------------------------------------
No probIem Tara and thank you for your comments!
1) I am using the new generators (org.hibernate.id.enhanced.TableGenerator) with the
"Pooled" optimizer (org.hibernate.id.enhanced.OptimizerFactory$PooledOptimizer).
I was just saying that Steve doesn't recommend not using it. Am I wrong?
In my first post, I was reporting that the
org.hibernate.id.enhanced.OptimizerFactory$HiLoOptimizer doesn't work
2) regarding the Pooled HHH-3608 issue, I don't understand "A fix that works for
multiple JVMs would be to invoke the "callback.getNextValue()" maximum once per
"optimizer.generate()" call:"
According to me, 'callback.getNextValue()' can't be called twice per
'optimizer.generate()' call in the following code :
public synchronized Serializable generate(AccessCallback callback) {
if ( hiValue < 0 ) {
value = callback.getNextValue();
if ( value < 1 ) {
// unfortunately not really safe to normalize this
// to 1 as an initial value like we do the others
// because we would not be able to control this if
// we are using a sequence...
log.info( "pooled optimizer source reported [" + value + "] as the initial
value; use of 1 or greater highly recommended" );
}
hiValue = callback.getNextValue();
}
else if ( value >= hiValue ) {
hiValue = callback.getNextValue();
value = hiValue - incrementSize;
}
return make( value++ );
}
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:40 a.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Hassan Hammoud commented on HHH-3628:
-------------------------------------
Hello Cyril :), I still think that in the ticket they didn't describe the case
correctly. I don't think they meant in one call of generate() but instead two or more
threads are calling the generate().
In brief,
1- Thread one called "value = callback.getNextValue()" inside the
"else if ( value >= hiValue )";
2- Thread two called "value = callback.getNextValue()" inside the
"if ( hiValue < 0 )" hence resetting the value variable;
3- Thread one called "hiValue = callback.getNextValue();" inside the
"else if ( value >= hiValue )";
check the attached PooledOptimizerTest.java in HHH-3608 and run it you will understand
what they mean.
Although the generate() is a synchronized function it is still not static -> so only
synchronized inside the same PooledOptimizer Object. If another PooledOptimizer in
another thread calls the generate(), it will execute. It's only that the value
variable is common to the two objects through the DB or the callback.getNextValue();
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:41 a.m.
New subject: [Hibernate-JIRA] Commented: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Montagnon Cyril commented on HHH-3628:
--------------------------------------
Ok I understand the HHH-3608 issue. There is no fix against this issue (that can only
happen when the hivalue is <0, ie when the Optimizer hasn't retrieve any value for
this sequence yet)??? Strange....
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:05 p.m.
New subject: [Hibernate-JIRA] Updated: (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
[
http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628?page=c...
]
Cyril Deverson updated HHH-3628:
--------------------------------
Attachment: OptimizerFactory.java
I fixed the issue with this update (see line 204)
Hilo optimizer problem in case of multiple threads accessing the
sequence table
-------------------------------------------------------------------------------
Key: HHH-3628
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3628
Project: Hibernate Core
Issue Type: Bug
Affects Versions: 3.2.6
Environment: Sybase
Reporter: Montagnon Cyril
Attachments: OptimizerFactory.java
If 2 (or more) threads access the table storing the ids, this optimizer won't work
and will try to insert entities with twice the same idea.
The problem is the way the HiLoOptimizer class generates the id :
public synchronized Serializable generate(AccessCallback callback) {
if ( lastSourceValue < 0 ) {
lastSourceValue = callback.getNextValue();
while ( lastSourceValue <= 0 ) {
lastSourceValue = callback.getNextValue();
}
hiValue = ( lastSourceValue * incrementSize ) + 1;
value = hiValue - incrementSize;
}
else if ( value >= hiValue ) {
lastSourceValue = callback.getNextValue();
hiValue = ( lastSourceValue * incrementSize ) + 1;
}
return make( value++ );
}
In the 'else if' part, the 'value' variable isn't reaffected, which
means the current thread will try to insert entities with an id that has already been used
by another thread. The value should be reset with hiValue - incrementSize.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:35 a.m.
New subject: [Hibernate-JIRA] (HHH-3628) Hilo optimizer problem in case of multiple threads accessing the sequence table
4208
days inactive
5642
days old
hibernate-issues@lists.jboss.org
8 comments
5 participants
participants (5)
-
Cyril Deverson (JIRA) -
Hassan Hammoud (JIRA)
-
Mark Jenner (JIRA)
-
Montagnon Cyril (JIRA)
-
Tara Simpson (JIRA)