[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-4577?page=c... ] Guenther Demetz updated HHH-4577: --------------------------------- Attachment: Hibernate3_3_2_ActionQueue_patch.jar Previous attachment contained a bug (NPE when 2L Query Cache disabled), please consider only this. best regards G.D. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-4577?page=c... ] Guenther Demetz commented on HHH-4577: -------------------------------------- Another suggestion: if making this improvement for Hibernate 3.5 then maybe it would be the proper opportunity to generally change signature of Serializable[] getPropertySpaces() into java.util.Set<Serializable> getPropertySpaces() in interfaces org.hibernate.action.Executable & org.hibernate.persister.entity.EntityPersister This would help to reach more readable and reduced code in the relative implementor classes. regards Guenther Demetz -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-4577?page=c... ] Guenther Demetz commented on HHH-4577: -------------------------------------- Furthermore the preinvalidate call is only necessary if accessing the database in READ UNCOMMITTED isolation level. Most modern applications use higher isolation levels and here the preinvalidate calls are only overhead. Why not check for the current hibernate isolation level set, and call preinvalidate only if hibernate.connection.isolation==1 ? Snippet of ActionQueue.java version 3.5.0.Beta3: public void execute(Executable executable) { try { executable.execute(); } finally { beforeTransactionProcesses.register( executable.getBeforeTransactionCompletionProcess() ); if ( session.getFactory().getSettings().isQueryCacheEnabled() ) { final String[] spaces = (String[]) executable.getPropertySpaces(); afterTransactionProcesses.addSpacesToInvalidate( spaces ); session.getFactory().getUpdateTimestampsCache().preinvalidate( executable.getPropertySpaces() ); } afterTransactionProcesses.register( executable.getAfterTransactionCompletionProcess() ); } } -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-4577?page=c... ] Steve Ebersole commented on HHH-4577: ------------------------------------- The problem with doing this for pre-invalidate is where to do it? For invalidate there is a natural place... after the transaction. I cannot really think of such a thing for pre-invalidate. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.... - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-4577?page=c... ] Strong Liu updated HHH-4577: ---------------------------- Description: Analyzing what our application is mainly doing during flush/commit, we saw that very often the concerning thread is executing in jbosscache stuff (see 2 sample stacktraces below). Analyzing the code we discovered that, if doing 1000 inserts for a determinate entity: -on flush UpdateTimestampsCache.preinvalidate() is called 1000 times for the same space (=table) -on commit UpdateTimestampsCache.invalidate() is called another 1000 times again for the same space (=table) It would be much smarter to collect the interested spaces once per flush and once per commit and then execute the (pre)invalidate once per single space. In the case above it would reduce the calls to one single UpdateTimestampsCache.preinvalidate()call on flush and one single UpdateTimestampsCache.invalidate() on commit. For the commit the enhancement could be following: {code:title=ActionQueue.java} public void afterTransactionCompletion(boolean success) { int size = executions.size(); final boolean invalidateQueryCache = session.getFactory().getSettings().isQueryCacheEnabled(); Set<Serializable> spaces = invalidateQueryCache ? new HashSet<Serializable>() : null; try { for ( int i = 0; i < size; i++ ) { try { Executable exec = ( Executable ) executions.get( i ); if ( invalidateQueryCache ) { Serializable[] nspaces = exec.getPropertySpaces(); for (int j=0; j < nspaces.length; j++) spaces.add(nspaces[j]); } exec.afterTransactionCompletion( success ); } catch ( CacheException ce ) { log.error( "could not release a cache lock", ce ); // continue loop } catch ( Exception e ) { throw new AssertionFailure( "Exception releasing cache locks", e ); } } } finally { session.getFactory().getUpdateTimestampsCache().invalidate(spaces.toArray(new Serializable[]{})); } executions.clear(); } {code} Stack traces: java.lang.Thread.isInterrupted(Native Method) java.lang.Thread.interrupted(Thread.java:873) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1134) java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:312) java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:241) org.jboss.cache.RegionImpl.registerEvictionEvent(RegionImpl.java:249) org.jboss.cache.RegionImpl.registerEvictionEvent(RegionImpl.java:234) org.jboss.cache.interceptors.EvictionInterceptor.registerEvictionEventToRegionManager(EvictionInterceptor.java:252) org.jboss.cache.interceptors.EvictionInterceptor.visitPutKeyValueCommand(EvictionInterceptor.java:109) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:102) org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301) org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:119) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178) org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutKeyValueCommand(InvocationContextInterceptor.java:82) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287) org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:555) org.hibernate.cache.jbc2.util.CacheHelper.put(CacheHelper.java:212) org.hibernate.cache.jbc2.timestamp.TimestampsRegionImpl.put(TimestampsRegionImpl.java:128) org.hibernate.cache.UpdateTimestampsCache.preinvalidate(UpdateTimestampsCache.java:70) - locked org.hibernate.cache.UpdateTimestampsCache@1afcaae org.hibernate.engine.ActionQueue.execute(ActionQueue.java:275) org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263) org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167) org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) Stack trace: java.lang.Thread.isInterrupted(Native Method) java.lang.Thread.interrupted(Thread.java:873) java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1158) java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:416) org.jboss.cache.util.concurrent.locks.AbstractSharedLockContainer.acquireLock(AbstractSharedLockContainer.java:94) org.jboss.cache.lock.MVCCLockManager.lockAndRecord(MVCCLockManager.java:133) org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:157) org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217) org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186) org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101) org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301) org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:119) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178) org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutKeyValueCommand(InvocationContextInterceptor.java:82) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287) org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:555) org.hibernate.cache.jbc2.util.CacheHelper.put(CacheHelper.java:212) org.hibernate.cache.jbc2.timestamp.TimestampsRegionImpl.put(TimestampsRegionImpl.java:128) org.hibernate.cache.UpdateTimestampsCache.invalidate(UpdateTimestampsCache.java:85) - locked org.hibernate.cache.UpdateTimestampsCache@1afcaae org.hibernate.engine.ActionQueue.afterTransactionCompletion(ActionQueue.java:202) was: Analyzing what our application is mainly doing during flush/commit, we saw that very often the concerning thread is executing in jbosscache stuff (see 2 sample stacktraces below). Analyzing the code we discovered that, if doing 1000 inserts for a determinate entity: -on flush UpdateTimestampsCache.preinvalidate() is called 1000 times for the same space (=table) -on commit UpdateTimestampsCache.invalidate() is called another 1000 times again for the same space (=table) It would be much smarter to collect the interested spaces once per flush and once per commit and then execute the (pre)invalidate once per single space. In the case above it would reduce the calls to one single UpdateTimestampsCache.preinvalidate()call on flush and one single UpdateTimestampsCache.invalidate() on commit. For the commit the enhancement could be following: ActionQueue.java: public void afterTransactionCompletion(boolean success) { int size = executions.size(); final boolean invalidateQueryCache = session.getFactory().getSettings().isQueryCacheEnabled(); Set<Serializable> spaces = invalidateQueryCache ? new HashSet<Serializable>() : null; try { for ( int i = 0; i < size; i++ ) { try { Executable exec = ( Executable ) executions.get( i ); if ( invalidateQueryCache ) { Serializable[] nspaces = exec.getPropertySpaces(); for (int j=0; j < nspaces.length; j++) spaces.add(nspaces[j]); } exec.afterTransactionCompletion( success ); } catch ( CacheException ce ) { log.error( "could not release a cache lock", ce ); // continue loop } catch ( Exception e ) { throw new AssertionFailure( "Exception releasing cache locks", e ); } } } finally { session.getFactory().getUpdateTimestampsCache().invalidate(spaces.toArray(new Serializable[]{})); } executions.clear(); } Stack traces: java.lang.Thread.isInterrupted(Native Method) java.lang.Thread.interrupted(Thread.java:873) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1134) java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:312) java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:241) org.jboss.cache.RegionImpl.registerEvictionEvent(RegionImpl.java:249) org.jboss.cache.RegionImpl.registerEvictionEvent(RegionImpl.java:234) org.jboss.cache.interceptors.EvictionInterceptor.registerEvictionEventToRegionManager(EvictionInterceptor.java:252) org.jboss.cache.interceptors.EvictionInterceptor.visitPutKeyValueCommand(EvictionInterceptor.java:109) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:102) org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301) org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:119) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178) org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutKeyValueCommand(InvocationContextInterceptor.java:82) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287) org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:555) org.hibernate.cache.jbc2.util.CacheHelper.put(CacheHelper.java:212) org.hibernate.cache.jbc2.timestamp.TimestampsRegionImpl.put(TimestampsRegionImpl.java:128) org.hibernate.cache.UpdateTimestampsCache.preinvalidate(UpdateTimestampsCache.java:70) - locked org.hibernate.cache.UpdateTimestampsCache@1afcaae org.hibernate.engine.ActionQueue.execute(ActionQueue.java:275) org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263) org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167) org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) Stack trace: java.lang.Thread.isInterrupted(Native Method) java.lang.Thread.interrupted(Thread.java:873) java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1158) java.util.concurrent.locks.ReentrantLock.tryLock(ReentrantLock.java:416) org.jboss.cache.util.concurrent.locks.AbstractSharedLockContainer.acquireLock(AbstractSharedLockContainer.java:94) org.jboss.cache.lock.MVCCLockManager.lockAndRecord(MVCCLockManager.java:133) org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:157) org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217) org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186) org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101) org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301) org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283) org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:119) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178) org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutKeyValueCommand(InvocationContextInterceptor.java:82) org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100) org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287) org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:555) org.hibernate.cache.jbc2.util.CacheHelper.put(CacheHelper.java:212) org.hibernate.cache.jbc2.timestamp.TimestampsRegionImpl.put(TimestampsRegionImpl.java:128) org.hibernate.cache.UpdateTimestampsCache.invalidate(UpdateTimestampsCache.java:85) - locked org.hibernate.cache.UpdateTimestampsCache@1afcaae org.hibernate.engine.ActionQueue.afterTransactionCompletion(ActionQueue.java:202) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira