[JBoss JIRA] (ISPN-10908) Client timeout Exception should log the connection additional to the key
by Pedro Zapata Fernandez (Jira)
[ https://issues.jboss.org/browse/ISPN-10908?page=com.atlassian.jira.plugin... ]
Pedro Zapata Fernandez reassigned ISPN-10908:
---------------------------------------------
Assignee: Tristan Tarrant
> Client timeout Exception should log the connection additional to the key
> ------------------------------------------------------------------------
>
> Key: ISPN-10908
> URL: https://issues.jboss.org/browse/ISPN-10908
> Project: Infinispan
> Issue Type: Enhancement
> Components: Hot Rod
> Affects Versions: 10.0.0.Final
> Environment: HotRod client
> Reporter: Wolf-Dieter Fink
> Assignee: Tristan Tarrant
> Priority: Major
>
> In case of communication timeout with the server the client should log the connection details as the key will not give enough to analyse the issue.
> ~~~
> ERROR [stderr] (pool-12-thread-3) org.infinispan.client.hotrod.exceptions.TransportException:: java.net.SocketTimeoutException: GetOperation{TestCache, key=[B0x800127656A6B2F6C..[103], flags=0} timed out after 60000 ms
> ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.Util.rewrap(Util.java:54)
> ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.Util.await(Util.java:27)
> ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.RemoteCacheImpl.get(RemoteCacheImpl.java:418)
> ...
> ERROR [stderr] (pool-12-thread-3) Caused by: java.net.SocketTimeoutException: GetOperation{TestCache, key=[B0x800127656A6B2F6C..[103], flags=0} timed out after 60000 ms
> ERROR [stderr] (pool-12-thread-3) at org.infinispan.client.hotrod.impl.operations.HotRodOperation.run(HotRodOperation.java:172)
> ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
> ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
> ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
> ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
> ERROR [stderr] (pool-12-thread-3) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:322)
> ERROR [stderr] (pool-12-thread-3) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
> ERROR [stderr] (pool-12-thread-3) ... 3 more
> ~~~
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11028) org.infinispan:infinispan-spring5-common missing from infinispan-bom
by Katia Aresti (Jira)
[ https://issues.jboss.org/browse/ISPN-11028?page=com.atlassian.jira.plugin... ]
Katia Aresti updated ISPN-11028:
--------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> org.infinispan:infinispan-spring5-common missing from infinispan-bom
> --------------------------------------------------------------------
>
> Key: ISPN-11028
> URL: https://issues.jboss.org/browse/ISPN-11028
> Project: Infinispan
> Issue Type: Bug
> Components: Build
> Affects Versions: 9.4.16.Final, 10.1.0.Beta1
> Reporter: Pedro Ruivo
> Assignee: Pedro Ruivo
> Priority: Major
> Fix For: 9.4.17.Final, 10.1.0.CR1
>
>
> {{org.infinispan:infinispan-spring5-common}} is missing from {{infinispan-bom}}.
> It causes problems in {{infinispan-spring-boot-starter}} project as maven downloads the wrong version (as shown below):
> {code}
> [INFO] org.infinispan:infinispan-spring-boot-starter-embedded:jar:2.1.8-SNAPSHOT
> [INFO] \- org.infinispan:infinispan-spring5-embedded:jar:9.4.17-SNAPSHOT:compile
> [INFO] \- org.infinispan:infinispan-spring5-common:jar:9.4.16.Final:compile
> {code}
> {{org.infinispan:infinispan-spring5-common}} was added to {{spring-boot-dependencies}} bom and since it is missing from our {{infinispan-bom}}, maven choose whatever version is declared in {{spring-boot-dependencies}}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-2980) sqlite support
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-2980?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant closed ISPN-2980.
---------------------------------
Resolution: Won't Do
> sqlite support
> --------------
>
> Key: ISPN-2980
> URL: https://issues.jboss.org/browse/ISPN-2980
> Project: Infinispan
> Issue Type: Feature Request
> Components: Loaders and Stores
> Affects Versions: 5.2.5.Final
> Reporter: Aleksandar Kostadinov
> Priority: Major
> Labels: cache-loader, cache-store, jdbc, sqlite
> Attachments: sqlite-jdbc-3.7.15-SNAPSHOT-f18.jar
>
>
> It would be very nice is we have SQLite support for infinispan. SQLite is a powerful database supporting terabyte sized databases in a file with competitive performance.
> I tried to use it as a JDBC store but the best driver I find in the internet ([xerial sqlite jdbc driver|https://bitbucket.org/xerial/sqlite-jdbc]) does not implement full jdbc specification and trying to use it results in exceptions.
> I think that perhaps using the [non-jdbc wrapper sqlite4java|http://code.google.com/p/sqlite4java/] may make sense for infinispan because:
> 1. it promises better performance
> 2. it allows using the sqlite library from OS (xerial driver uses a customized build of sqlite)
> FYI here is how I setup sqlite for infinispan (unsuccessfully):
> {code}jboss as cli commands:
> /subsystem=datasources/jdbc-driver=sqlite:add(driver-name="sqlite",driver-module-name="org.xerial",driver-class-name=org.sqlite.JDBC)
> data-source add --name=SQLiteDS --connection-url="jdbc:sqlite:${sqlite.database.string}" --jndi-name=java:jboss/datasources/SQLiteDS --driver-name="sqlite"
> /subsystem=datasources/data-source=SQLiteDS/connection-properties=journal_mode:add(value="WAL")
> /subsystem=datasources/data-source=SQLiteDS:enable
> {code}
> {code}JBoss AS module definition (modules/org/xerial/main/module.xml):
> <?xml version="1.0" encoding="UTF-8"?>
> <module xmlns="urn:jboss:module:1.0" name="org.xerial">
> <resources>
> <resource-root path="sqlite-jdbc.jar" />
> </resources>
> <dependencies>
> <module name="javax.api" />
> <module name="javax.transaction.api"/>
> </dependencies>
> </module>
> {code}
> {code}cache store/loader configuration snippet:
> <stringKeyedJdbcStore xmlns="urn:infinispan:config:jdbc:5.2" fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false" key2StringMapper="com.jboss.datagrid.chunchun.util.TwoWayKey2StringChunchunMapper">
> <dataSource jndiUrl="java:jboss/datasources/SQLiteDS" />
> <stringKeyedTable dropOnExit="false" createOnStart="true" prefix="ispn">
> <idColumn name="ID_COLUMN" type="VARCHAR(255)" />
> <dataColumn name="DATA_COLUMN" type="BLOB" />
> <timestampColumn name="TIMESTAMP_COLUMN" type="BIGINT" />
> </stringKeyedTable>
> </stringKeyedJdbcStore>
> </loaders>
> {code}
> sql driver needs to be copied in the same directory as module.xml
> *UPDATE:* the exception is fixed with latest dev code of xerial jdbc driver, please look at comments to see remaining problems.
> The Exception I'm getting is:{code}
> 12:53:10,683 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (MSC service thread 1-3) ISPN000136: Execution error: org.infinispan.loaders.CacheLoaderException: Error while storing string key to database; key: 'user41', buffer size of value: 4918 bytes
> at org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore.storeLockSafe(JdbcStringBasedCacheStore.java:253) [infinispan-cachestore-jdbc-5.2.5.Final.jar:5.2.5.Final]
> ...
> Caused by: java.sql.SQLException: not implemented by SQLite JDBC driver
> at org.sqlite.Unused.unused(Unused.java:29) [sqlite-jdbc-3.7.2.jar:]
> at org.sqlite.Unused.setBinaryStream(Unused.java:60) [sqlite-jdbc-3.7.2.jar:]
> at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.setBinaryStream(WrappedPreparedStatement.java:871)
> at org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore.storeLockSafe(JdbcStringBasedCacheStore.java:247) [infinispan-cachestore-jdbc-5.2.5.Final.jar:5.2.5.Final]
> ... 73 more{code}
> The driver [does not support|http://code.google.com/p/xerial/issues/detail?id=99] setBinaryStream(), only setBytes(). Not sure if there are any other methods required by infinispan but not implemented.
> As a simple comparison between JDBC and direct storage, I tried an app that caches 3000 records of around 5k and 60000 records of around 0.5k (total of less than 60MiB). Bdbje store operation completes in less than a minute. With a local mysql server it takes 10 minutes. And this is on a machine with plenty of CPU and memory over an SSD. Unfortunately bdbje does not work clustered for me (ISPN-2968).
> So my point is that a local disk based, fast, reliable, transactional engine is highly needed.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-5184) The option "shared=true" in custom persistance storage does not work
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-5184?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reassigned ISPN-5184:
-------------------------------------
Assignee: Will Burns
> The option "shared=true" in custom persistance storage does not work
> --------------------------------------------------------------------
>
> Key: ISPN-5184
> URL: https://issues.jboss.org/browse/ISPN-5184
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.0.3.Final
> Reporter: Pavlo Pohrebnyi
> Assignee: Will Burns
> Priority: Major
>
> When I'm tried to enable option "shared=true" in custom persistance storage, nothing changes - all nodes updates storage (should update only node, that put cache entry).
> This is part of documentation:
> shared - (false by default) indicates that the cache loader is shared among different cache instances, for example where all instances in a cluster use the same JDBC settings to talk to the same remote, shared database. Setting this to true prevents repeated and unnecessary writes of the same data to the cache loader by different cache instances.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11004) SQL server exception purging data
by Tristan Tarrant (Jira)
[ https://issues.jboss.org/browse/ISPN-11004?page=com.atlassian.jira.plugin... ]
Tristan Tarrant reassigned ISPN-11004:
--------------------------------------
Assignee: Ryan Emerson
> SQL server exception purging data
> ---------------------------------
>
> Key: ISPN-11004
> URL: https://issues.jboss.org/browse/ISPN-11004
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 10.0.1.Final
> Reporter: Megan van de Valk
> Assignee: Ryan Emerson
> Priority: Major
>
> When using string keyed jdbc store with dialect="SQL_SERVER" a SQL exception of 'FOR UPDATE clause allowed only for DECLARE CURSOR' is thrown.
> {code:java}
> 14:57:39,444 WARN [org.infinispan.PERSISTENCE] (expiration-thread--p7-t1) ISPN000026: Caught exception purging data container!: org.infinispan.persistence.spi.PersistenceException: Failed clearing string based JDBC store
> at org.infinispan.persistence.jdbc.stringbased.JdbcStringBasedStore.purge(JdbcStringBasedStore.java:502)
> at org.infinispan.persistence.spi.SegmentedAdvancedLoadWriteStore.purge(SegmentedAdvancedLoadWriteStore.java:291)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.lambda$purgeExpired$8(PersistenceManagerImpl.java:522)
> at java.util.ArrayList.forEach(ArrayList.java:1257)
> at org.infinispan.persistence.manager.PersistenceManagerImpl.purgeExpired(PersistenceManagerImpl.java:528)
> at org.infinispan.persistence.support.DelegatingPersistenceManager.purgeExpired(DelegatingPersistenceManager.java:90)
> at org.infinispan.expiration.impl.ExpirationManagerImpl.processExpiration(ExpirationManagerImpl.java:120)
> at org.infinispan.expiration.impl.ExpirationManagerImpl$ScheduledTask.run(ExpirationManagerImpl.java:282)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Line 1: FOR UPDATE clause allowed only for DECLARE CURSOR.
> at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
> at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:404)
> at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:350)
> at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
> at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
> at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(SQLServerPreparedStatement.java:285)
> at org.infinispan.persistence.jdbc.stringbased.JdbcStringBasedStore.purge(JdbcStringBasedStore.java:467)
> ... 14 more
> {code}
> The method initSelectOnlyExpiredRowsSql in [AbstractTableManager|https://github.com/infinispan/infinispan/blob/master...] includes a FOR UPDATE:
> {code:java}
> return String.format("%1$s WHERE %2$s < ? AND %2$s > 0 FOR UPDATE", getLoadAllRowsSql(), config.timestampColumnName());
> {code}
> It should probably be overwritten in [SQLServerTableManger|https://github.com/infinispan/infinispan/blob/master...] with
> {code:java}
> return String.format("%1$s WITH(UPDLOCK) WHERE %2$s < ? AND %2$s > 0", getLoadAllRowsSql(), config.timestampColumnName());
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11030) ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11030?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes commented on ISPN-11030:
------------------------------------------
Introduced by ISPN-10967
> ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-11030
> URL: https://issues.jboss.org/browse/ISPN-11030
> Project: Infinispan
> Issue Type: Bug
> Components: REST
> Affects Versions: 10.1.0.Beta1
> Environment: 10 clients, each client with 10 threads
> Reporter: Diego Lovison
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> {noformat}
> [0m19:08:46,465 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t2) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561], phase READ_OLD_WRITE_ALL, topology id 2
> [0m[0m19:08:46,680 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3
> [0m[0m19:08:46,718 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t13) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4
> [0m[31m19:08:46,607 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-9) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[0m19:08:46,791 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561], topology id 5
> [0m[0m19:08:46,804 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], phase READ_OLD_WRITE_ALL, topology id 6
> [0m[0m19:08:46,949 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 7
> [0m[0m19:08:47,014 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 8
> [0m[0m19:08:47,023 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], topology id 9
> 19:08:47,143 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [0m[40.227s][info ][gc] GC(6) Pause Young (Normal) (G1 Evacuation Pause) 838M->143M(16384M) 95.766ms
> 19:08:56,735 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [45.080s][info ][gc] GC(7) Pause Young (Normal) (G1 Evacuation Pause) 855M->246M(16384M) 132.612ms
> [49.699s][info ][gc] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 958M->348M(16384M) 116.067ms
> 19:09:06,731 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [53.220s][info ][gc] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 1060M->459M(16384M) 100.669ms
> [56.705s][info ][gc] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 1171M->578M(16384M) 112.650ms
> [31m19:09:10,384 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-15) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[60.055s][info ][gc] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 1290M->693M(16384M) 109.505ms
> 19:09:16,737 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [63.456s][info ][gc] GC(12) Pause Young (Normal) (G1 Evacuation Pause) 1405M->807M(16384M) 109.216ms
> [31m19:09:17,139 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-7) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[66.872s][info ][gc] GC(13) Pause Young (Normal) (G1 Evacuation Pause) 1519M->925M(16384M) 105.106ms
> [70.354s][info ][gc] GC(14) Pause Young (Normal) (G1 Evacuation Pause) 1653M->1048M(16384M) 112.962ms
> 19:09:26,750 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [74.102s][info ][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 1824M->1176M(16384M) 118.885ms
> [31m19:09:27,781 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-1) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11030) ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11030?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes updated ISPN-11030:
-------------------------------------
Affects Version/s: 10.1.0.Beta1
(was: 10.1.0.CR1)
> ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-11030
> URL: https://issues.jboss.org/browse/ISPN-11030
> Project: Infinispan
> Issue Type: Bug
> Components: REST
> Affects Versions: 10.1.0.Beta1
> Environment: 10 clients, each client with 10 threads
> Reporter: Diego Lovison
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> {noformat}
> [0m19:08:46,465 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t2) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561], phase READ_OLD_WRITE_ALL, topology id 2
> [0m[0m19:08:46,680 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3
> [0m[0m19:08:46,718 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t13) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4
> [0m[31m19:08:46,607 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-9) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[0m19:08:46,791 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561], topology id 5
> [0m[0m19:08:46,804 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], phase READ_OLD_WRITE_ALL, topology id 6
> [0m[0m19:08:46,949 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 7
> [0m[0m19:08:47,014 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 8
> [0m[0m19:08:47,023 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], topology id 9
> 19:08:47,143 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [0m[40.227s][info ][gc] GC(6) Pause Young (Normal) (G1 Evacuation Pause) 838M->143M(16384M) 95.766ms
> 19:08:56,735 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [45.080s][info ][gc] GC(7) Pause Young (Normal) (G1 Evacuation Pause) 855M->246M(16384M) 132.612ms
> [49.699s][info ][gc] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 958M->348M(16384M) 116.067ms
> 19:09:06,731 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [53.220s][info ][gc] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 1060M->459M(16384M) 100.669ms
> [56.705s][info ][gc] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 1171M->578M(16384M) 112.650ms
> [31m19:09:10,384 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-15) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[60.055s][info ][gc] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 1290M->693M(16384M) 109.505ms
> 19:09:16,737 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [63.456s][info ][gc] GC(12) Pause Young (Normal) (G1 Evacuation Pause) 1405M->807M(16384M) 109.216ms
> [31m19:09:17,139 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-7) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[66.872s][info ][gc] GC(13) Pause Young (Normal) (G1 Evacuation Pause) 1519M->925M(16384M) 105.106ms
> [70.354s][info ][gc] GC(14) Pause Young (Normal) (G1 Evacuation Pause) 1653M->1048M(16384M) 112.962ms
> 19:09:26,750 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [74.102s][info ][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 1824M->1176M(16384M) 118.885ms
> [31m19:09:27,781 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-1) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11030) ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11030?page=com.atlassian.jira.plugin... ]
Gustavo Fernandes updated ISPN-11030:
-------------------------------------
Status: Open (was: New)
> ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-11030
> URL: https://issues.jboss.org/browse/ISPN-11030
> Project: Infinispan
> Issue Type: Bug
> Components: REST
> Affects Versions: 10.1.0.CR1
> Environment: 10 clients, each client with 10 threads
> Reporter: Diego Lovison
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> {noformat}
> [0m19:08:46,465 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t2) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561], phase READ_OLD_WRITE_ALL, topology id 2
> [0m[0m19:08:46,680 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3
> [0m[0m19:08:46,718 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t13) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4
> [0m[31m19:08:46,607 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-9) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[0m19:08:46,791 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561], topology id 5
> [0m[0m19:08:46,804 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], phase READ_OLD_WRITE_ALL, topology id 6
> [0m[0m19:08:46,949 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 7
> [0m[0m19:08:47,014 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 8
> [0m[0m19:08:47,023 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], topology id 9
> 19:08:47,143 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [0m[40.227s][info ][gc] GC(6) Pause Young (Normal) (G1 Evacuation Pause) 838M->143M(16384M) 95.766ms
> 19:08:56,735 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [45.080s][info ][gc] GC(7) Pause Young (Normal) (G1 Evacuation Pause) 855M->246M(16384M) 132.612ms
> [49.699s][info ][gc] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 958M->348M(16384M) 116.067ms
> 19:09:06,731 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [53.220s][info ][gc] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 1060M->459M(16384M) 100.669ms
> [56.705s][info ][gc] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 1171M->578M(16384M) 112.650ms
> [31m19:09:10,384 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-15) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[60.055s][info ][gc] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 1290M->693M(16384M) 109.505ms
> 19:09:16,737 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [63.456s][info ][gc] GC(12) Pause Young (Normal) (G1 Evacuation Pause) 1405M->807M(16384M) 109.216ms
> [31m19:09:17,139 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-7) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[66.872s][info ][gc] GC(13) Pause Young (Normal) (G1 Evacuation Pause) 1519M->925M(16384M) 105.106ms
> [70.354s][info ][gc] GC(14) Pause Young (Normal) (G1 Evacuation Pause) 1653M->1048M(16384M) 112.962ms
> 19:09:26,750 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [74.102s][info ][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 1824M->1176M(16384M) 118.885ms
> [31m19:09:27,781 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-1) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11030) ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
by Gustavo Fernandes (Jira)
[ https://issues.jboss.org/browse/ISPN-11030?page=com.atlassian.jira.plugin... ]
Work on ISPN-11030 started by Gustavo Fernandes.
------------------------------------------------
> ConcurrentModificationException - ArrayList$Itr.checkForComodification(ArrayList.java:1042) - regression-cs-rest-dist-(reads|writes)
> ------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-11030
> URL: https://issues.jboss.org/browse/ISPN-11030
> Project: Infinispan
> Issue Type: Bug
> Components: REST
> Affects Versions: 10.1.0.CR1
> Environment: 10 clients, each client with 10 threads
> Reporter: Diego Lovison
> Assignee: Gustavo Fernandes
> Priority: Critical
>
> {noformat}
> [0m19:08:46,465 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t2) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561], phase READ_OLD_WRITE_ALL, topology id 2
> [0m[0m19:08:46,680 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3
> [0m[0m19:08:46,718 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t13) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4
> [0m[31m19:08:46,607 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-9) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[0m19:08:46,791 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561], topology id 5
> [0m[0m19:08:46,804 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100002: Starting rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], phase READ_OLD_WRITE_ALL, topology id 6
> [0m[0m19:08:46,949 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 7
> [0m[0m19:08:47,014 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t3) [Context=rest]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 8
> [0m[0m19:08:47,023 INFO [org.infinispan.CLUSTER] (remote-thread--p3-t5) [Context=rest]ISPN100010: Finished rebalance with members [edg-perf06-12311, edg-perf04-40561, edg-perf03-42053, edg-perf07-41186, edg-perf05-21163, edg-perf02-59957, edg-perf08-16062, edg-perf01-57801], topology id 9
> 19:08:47,143 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [0m[40.227s][info ][gc] GC(6) Pause Young (Normal) (G1 Evacuation Pause) 838M->143M(16384M) 95.766ms
> 19:08:56,735 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [45.080s][info ][gc] GC(7) Pause Young (Normal) (G1 Evacuation Pause) 855M->246M(16384M) 132.612ms
> [49.699s][info ][gc] GC(8) Pause Young (Normal) (G1 Evacuation Pause) 958M->348M(16384M) 116.067ms
> 19:09:06,731 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [53.220s][info ][gc] GC(9) Pause Young (Normal) (G1 Evacuation Pause) 1060M->459M(16384M) 100.669ms
> [56.705s][info ][gc] GC(10) Pause Young (Normal) (G1 Evacuation Pause) 1171M->578M(16384M) 112.650ms
> [31m19:09:10,384 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-15) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[60.055s][info ][gc] GC(11) Pause Young (Normal) (G1 Evacuation Pause) 1290M->693M(16384M) 109.505ms
> 19:09:16,737 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [63.456s][info ][gc] GC(12) Pause Young (Normal) (G1 Evacuation Pause) 1405M->807M(16384M) 109.216ms
> [31m19:09:17,139 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-7) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
> at io.netty.handler.codec.http.cors.CorsHandler.getForOrigin(CorsHandler.java:125)
> at io.netty.handler.codec.http.cors.CorsHandler.channelRead(CorsHandler.java:85)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
> at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:326)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:300)
> at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
> at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
> at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
> at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:502)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407)
> at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> [0m[66.872s][info ][gc] GC(13) Pause Young (Normal) (G1 Evacuation Pause) 1519M->925M(16384M) 105.106ms
> [70.354s][info ][gc] GC(14) Pause Young (Normal) (G1 Evacuation Pause) 1653M->1048M(16384M) 112.962ms
> 19:09:26,750 INFO [org.radargun.service.InfinispanRestAPI] (pool-2-thread-1) CacheManagerInfo{clusterMembers=[edg-perf01-57801, edg-perf03-42053, edg-perf02-59957, edg-perf04-40561, edg-perf05-21163, edg-perf07-41186, edg-perf06-12311, edg-perf08-16062], clusterSize=8}
> [74.102s][info ][gc] GC(15) Pause Young (Normal) (G1 Evacuation Pause) 1824M->1176M(16384M) 118.885ms
> [31m19:09:27,781 ERROR [org.infinispan.rest.RestRequestHandler] (SINGLE_PORT-ServerIO-8-1) ISPN012006: Uncaught exception in the pipeline: java.util.ConcurrentModificationException
> at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0
[JBoss JIRA] (ISPN-11023) Indexless query fails with missing component in the registry
by Dan Berindei (Jira)
[ https://issues.jboss.org/browse/ISPN-11023?page=com.atlassian.jira.plugin... ]
Dan Berindei commented on ISPN-11023:
-------------------------------------
Actually that stack trace is wrong because of ISPN-11022. The stack trace on the remote node is
{noformat}
13:56:08,905 WARN (jgroups-6,Test-NodeB-29188) [NonTotalOrderPerCacheInboundInvocationHandler] ISPN000071: Caught exception when handling command InitialPublisherCommand{cacheName='search-rest'}
org.infinispan.commons.CacheConfigurationException: Unable to inject dependencies for component class org.infinispan.query.dsl.embedded.impl.IckleFilterAndConverter, path null
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:349) ~[classes/:?]
at org.infinispan.factories.impl.BasicComponentRegistryImpl.wireDependencies(BasicComponentRegistryImpl.java:242) ~[classes/:?]
at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:88) ~[classes/:?]
at org.infinispan.filter.CacheFilters$FilterConverterAsValueFunction.inject(CacheFilters.java:183) ~[classes/:?]
at org.infinispan.filter.CorePackageImpl$4.wire(CorePackageImpl.java:65) ~[classes/:?]
at org.infinispan.filter.CorePackageImpl$4.wire(CorePackageImpl.java:63) ~[classes/:?]
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:334) ~[classes/:?]
at org.infinispan.factories.impl.BasicComponentRegistryImpl.wireDependencies(BasicComponentRegistryImpl.java:242) ~[classes/:?]
at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:88) ~[classes/:?]
at org.infinispan.stream.impl.intops.object.FlatMapOperation.handleInjection(FlatMapOperation.java:35) ~[classes/:?]
at org.infinispan.stream.impl.CacheStreamIntermediatePublisher.inject(CacheStreamIntermediatePublisher.java:60) ~[classes/:?]
at org.infinispan.reactive.publisher.impl.commands.batch.InitialPublisherCommand.invokeAsync(InitialPublisherCommand.java:108) ~[classes/:?]
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:116) ~[classes/:?]
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.invoke(BaseBlockingRunnable.java:100) ~[classes/:?]
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.lambda$runAsync$0(BaseBlockingRunnable.java:91) ~[classes/:?]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) [?:?]
at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) [?:?]
at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) [?:?]
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runAsync(BaseBlockingRunnable.java:74) [classes/:?]
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:41) [classes/:?]
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.handleRunnable(BasePerCacheInboundInvocationHandler.java:175) [classes/:?]
at org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler.handle(NonTotalOrderPerCacheInboundInvocationHandler.java:78) [classes/:?]
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:118) [classes/:?]
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:76) [classes/:?]
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1389) [classes/:?]
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1313) [classes/:?]
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:129) [classes/:?]
at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1461) [classes/:?]
at org.jgroups.JChannel.up(JChannel.java:775) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:920) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.FRAG3.up(FRAG3.java:171) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.FlowControl.up(FlowControl.java:346) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.FlowControl.up(FlowControl.java:354) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.tom.TOA.up(TOA.java:119) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:855) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.Discovery.up(Discovery.java:300) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.protocols.TP.passMessageUp(TP.java:1275) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87) [jgroups-4.1.6.Final.jar:4.1.6.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.infinispan.commons.CacheException: Expected component not found in registry: org.infinispan.query.remote.impl.ObjectReflectionMatcher
at org.infinispan.query.dsl.embedded.impl.IckleFilterAndConverter.injectDependencies(IckleFilterAndConverter.java:80) ~[classes/:?]
at org.infinispan.query.dsl.embedded.impl.QueryPackageImpl$1.wire(QueryPackageImpl.java:26) ~[classes/:?]
at org.infinispan.query.dsl.embedded.impl.QueryPackageImpl$1.wire(QueryPackageImpl.java:24) ~[classes/:?]
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:334) ~[classes/:?]
... 46 more
org.infinispan.commons.CacheConfigurationException: Unable to inject dependencies for component class org.infinispan.query.dsl.embedded.impl.IckleFilterAndConverter, path null
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:349)
at org.infinispan.factories.impl.BasicComponentRegistryImpl.wireDependencies(BasicComponentRegistryImpl.java:242)
at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:88)
at org.infinispan.filter.CacheFilters$FilterConverterAsValueFunction.inject(CacheFilters.java:183)
at org.infinispan.filter.CorePackageImpl$4.wire(CorePackageImpl.java:65)
at org.infinispan.filter.CorePackageImpl$4.wire(CorePackageImpl.java:63)
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:334)
at org.infinispan.factories.impl.BasicComponentRegistryImpl.wireDependencies(BasicComponentRegistryImpl.java:242)
at org.infinispan.factories.AbstractComponentRegistry.wireDependencies(AbstractComponentRegistry.java:88)
at org.infinispan.stream.impl.intops.object.FlatMapOperation.handleInjection(FlatMapOperation.java:35)
at org.infinispan.stream.impl.CacheStreamIntermediatePublisher.inject(CacheStreamIntermediatePublisher.java:60)
at org.infinispan.reactive.publisher.impl.commands.batch.InitialPublisherCommand.invokeAsync(InitialPublisherCommand.java:108)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokeCommand(BasePerCacheInboundInvocationHandler.java:116)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.invoke(BaseBlockingRunnable.java:100)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.lambda$runAsync$0(BaseBlockingRunnable.java:91)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883)
at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runAsync(BaseBlockingRunnable.java:74)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:41)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.handleRunnable(BasePerCacheInboundInvocationHandler.java:175)
at org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler.handle(NonTotalOrderPerCacheInboundInvocationHandler.java:78)
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:118)
at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:76)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processRequest(JGroupsTransport.java:1389)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.processMessage(JGroupsTransport.java:1313)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.access$300(JGroupsTransport.java:129)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport$ChannelCallbacks.up(JGroupsTransport.java:1461)
at org.jgroups.JChannel.up(JChannel.java:775)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:920)
at org.jgroups.protocols.FRAG3.up(FRAG3.java:171)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:346)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:354)
at org.jgroups.protocols.tom.TOA.up(TOA.java:119)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:855)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:243)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1049)
at org.jgroups.protocols.UNICAST3.addMessage(UNICAST3.java:772)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:753)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:405)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:592)
at org.jgroups.protocols.Discovery.up(Discovery.java:300)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1275)
at org.jgroups.util.SubmitToThreadPool$SingleMessageHandler.run(SubmitToThreadPool.java:87)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.infinispan.commons.CacheException: Expected component not found in registry: org.infinispan.query.remote.impl.ObjectReflectionMatcher
at org.infinispan.query.dsl.embedded.impl.IckleFilterAndConverter.injectDependencies(IckleFilterAndConverter.java:80)
at org.infinispan.query.dsl.embedded.impl.QueryPackageImpl$1.wire(QueryPackageImpl.java:26)
at org.infinispan.query.dsl.embedded.impl.QueryPackageImpl$1.wire(QueryPackageImpl.java:24)
at org.infinispan.factories.impl.BasicComponentRegistryImpl.invokeInjection(BasicComponentRegistryImpl.java:334)
... 46 more
{noformat}
> Indexless query fails with missing component in the registry
> ------------------------------------------------------------
>
> Key: ISPN-11023
> URL: https://issues.jboss.org/browse/ISPN-11023
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 10.0.1.Final
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 4 months
- 1
- 0