[JBoss JIRA] (ISPN-10657) Handle async-xsite in parallel
by Pedro Ruivo (Jira)
[ https://issues.redhat.com/browse/ISPN-10657?page=com.atlassian.jira.plugi... ]
Pedro Ruivo updated ISPN-10657:
-------------------------------
Fix Version/s: 9.4.18.Final
(was: 9.1.8.Final)
> Handle async-xsite in parallel
> ------------------------------
>
> Key: ISPN-10657
> URL: https://issues.redhat.com/browse/ISPN-10657
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication
> Affects Versions: 10.0.0.CR2
> Reporter: Pedro Ruivo
> Assignee: Pedro Ruivo
> Priority: Major
> Fix For: 10.0.0.Final, 9.4.18.Final
>
>
> Asynchronous cross-site replication relies on the ordering of the write operation to ensure data consistency. This ordering is provided by JGroups; it only delivers the next request after the first one is completed.
> This order contains updates for multiple caches and keys. What is proposed here is to handle different keys/cache updates in parallel.
> It should improve the response time, however, there is no free lunch. The following is expected:
> * An increase of CPU utilization: it may spawn new threads to handle different keys update in parallel
> * An increase in memory: it chains the updates (queue)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 2 months
[JBoss JIRA] (ISPN-10367) Possible loss of (pessimistic) lock if a transaction will reach timeout and/or is removed
by Pedro Ruivo (Jira)
[ https://issues.redhat.com/browse/ISPN-10367?page=com.atlassian.jira.plugi... ]
Pedro Ruivo updated ISPN-10367:
-------------------------------
Sprint: DataGrid Sprint #33, DataGrid Sprint #34, DataGrid Sprint #35, DataGrid Sprint #36, DataGrid Sprint #37, DataGrid Sprint #38 (was: DataGrid Sprint #33, DataGrid Sprint #34, DataGrid Sprint #35, DataGrid Sprint #36, DataGrid Sprint #37, DataGrid Sprint #38, DataGrid Sprint #39)
> Possible loss of (pessimistic) lock if a transaction will reach timeout and/or is removed
> -----------------------------------------------------------------------------------------
>
> Key: ISPN-10367
> URL: https://issues.redhat.com/browse/ISPN-10367
> Project: Infinispan
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 10.0.0.Beta3, 9.4.15.Final
> Environment: Infinispan with pessimistic locking enabled
> Reporter: Wolf-Dieter Fink
> Assignee: Pedro Ruivo
> Priority: Critical
> Fix For: 10.1.0.Final
>
> Attachments: StressApp.zip
>
>
> If entries are locked, no matter whether it was done by FORCE_WRITE_LOCK flag or getAdvancedCache().lock(key), and the lock is hold longer than the current Tx timeout setting (.completedTxTimeout(...) ) the transacaction might be removed
> - if the node is blocked and expelled from the cluster (and join back later)
> - the thread processing the lock will take longer than the Tx-timeout setting
> This force to remove the Tx and free the lock.
> An indicator is the Exception below which will be shown if the Tx is timing out, it is not a (remote) access timout.
> If the originator is back after this this (ongoing) Tx is assumed new and it will continue by accident without lock.
> This can cause unexpected inconsistency!
> ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (EJB timer - 13) ISPN000136: Error executing command LockControlCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Replication timeout for lt-33828
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:803)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:641)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> ERROR [org.jboss.as.ejb3.timer] (EJB timer - 13) WFLYEJB0020: Error invoking timeout for timer: [id=8a53d2c3-190d-4c74-9327-8e7554e1df2c timedObjectId=embeddedStressTest-ejb.embeddedStressTest-ejb.CacheAccessSingletonBean auto-timer?:false persistent?:true timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@72c41b07 initialExpiration=Fri Jun 28 10:56:16 CEST 2019 intervalDuration(in milli sec)=1 nextExpiration=Fri Jun 28 10:56:43 CEST 2019 timerState=IN_TIMEOUT info=org.infinispan.wfink.stress.TimerInfo@47ae2053]: javax.ejb.EJBException: org.infinispan.util.concurrent.TimeoutException: Replication timeout for lt-33828
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:246)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:388)
> at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:146)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81)
> at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
> at org.jboss.as.ejb3.component.singleton.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:106)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438)
> at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:619)
> at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57)
> at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
> at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53)
> at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:99)
> at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:109)
> at org.jboss.as.ejb3.timerservice.TimerTask.invokeBeanMethod(TimerTask.java:189)
> at org.jboss.as.ejb3.timerservice.TimerTask.callTimeout(TimerTask.java:185)
> at org.jboss.as.ejb3.timerservice.TimerTask.run(TimerTask.java:159)
> at org.jboss.as.ejb3.timerservice.TimerServiceImpl$Task$1.run(TimerServiceImpl.java:1304)
> at org.wildfly.extension.requestcontroller.RequestController$QueuedTask$1.run(RequestController.java:494)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> Caused by: org.infinispan.util.concurrent.TimeoutException: Replication timeout for lt-33828
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:803)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$1(JGroupsTransport.java:641)
> at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
> at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
> at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
> at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
> at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 2 months
[JBoss JIRA] (ISPN-11071) Support container server driver image troubleshooting
by Ryan Emerson (Jira)
[ https://issues.redhat.com/browse/ISPN-11071?page=com.atlassian.jira.plugi... ]
Ryan Emerson resolved ISPN-11071.
---------------------------------
Fix Version/s: 11.0.0.Alpha1
Assignee: Diego Lovison
Resolution: Done
> Support container server driver image troubleshooting
> ------------------------------------------------------
>
> Key: ISPN-11071
> URL: https://issues.redhat.com/browse/ISPN-11071
> Project: Infinispan
> Issue Type: Enhancement
> Components: Test Suite
> Affects Versions: 10.1.0.CR1
> Reporter: Diego Lovison
> Assignee: Diego Lovison
> Priority: Critical
> Fix For: 11.0.0.Alpha1
>
>
> When running on
> {noformat}
> -bash-4.2$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.7 (Maipo)
> -bash-4.2$ docker -v
> Docker version 1.13.1, build 4ef4b30/1.13.1
> -bash-4.2$
> {noformat}
> The test ResilienceIT is failing due
> {noformat}
> ?? Checking the system...
> ? Docker version should be at least 1.6.0
> ? Docker environment should have more than 2GB free disk space
> [0] STDERR: Exception in thread "main" java.lang.RuntimeException: java.nio.file.AccessDeniedException: /opt/infinispan/server/lib
> [0] STDERR: at org.infinispan.server.loader.Loader.classLoaderFromPath(Loader.java:79)
> [0] STDERR: at org.infinispan.server.loader.Loader.main(Loader.java:45)
> [0] STDERR: Caused by: java.nio.file.AccessDeniedException: /opt/infinispan/server/lib
> [0] STDERR: at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
> [0] STDERR: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> [0] STDERR: at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
> {noformat}
> As QE, I would like to have the docker image created to start my researches and see why the server is failing.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 2 months