[JBoss JIRA] (ISPN-9313) ISPN000136: Execution error org.infinispan.remoting.RpcException: No more valid responses. Received invalid responses from all of [Sender
by Nilesh Deshpande (JIRA)
Nilesh Deshpande created ISPN-9313:
--------------------------------------
Summary: ISPN000136: Execution error org.infinispan.remoting.RpcException: No more valid responses. Received invalid responses from all of [Sender
Key: ISPN-9313
URL: https://issues.jboss.org/browse/ISPN-9313
Project: Infinispan
Issue Type: Bug
Components: Configuration
Affects Versions: 7.2.2.Final
Environment: Windows
Reporter: Nilesh Deshpande
We have 2 node cluster. If we stop the other node, I get continuous errors in logs with following traces. Restarting the node has resolved errors but I want to understand root cause and find if any configuration can fix the issue.
ERROR 06/20 19:01:40.303 [ComponentInit_2] o.i.i.InvocationContextInterceptor ISPN000136: Execution error
org.infinispan.remoting.RpcException: No more valid responses. Received invalid responses from all of [Sender{address=tsps-blr-win02-11856, responded=true}]
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$FutureCollator.getResponseList(CommandAwareRpcDispatcher.java:656)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processCalls(CommandAwareRpcDispatcher.java:493)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommands(CommandAwareRpcDispatcher.java:152)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:564)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:287)
at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.invokeClusterGetCommandRemotely(BaseDistributionInterceptor.java:177)
at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.retrieveFromRemoteSource(BaseDistributionInterceptor.java:165)
at org.infinispan.interceptors.distribution.TxDistributionInterceptor.remoteGet(TxDistributionInterceptor.java:363)
at org.infinispan.interceptors.distribution.TxDistributionInterceptor.remoteGetBeforeWrite(TxDistributionInterceptor.java:346)
at org.infinispan.interceptors.distribution.TxDistributionInterceptor.handleTxWriteCommand(TxDistributionInterceptor.java:304)
at org.infinispan.interceptors.distribution.TxDistributionInterceptor.visitPutKeyValueCommand(TxDistributionInterceptor.java:109)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:388)
at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:453)
at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:195)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitDataWriteCommand(OptimisticLockingInterceptor.java:147)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.visitPutKeyValueCommand(AbstractTxLockingInterceptor.java:68)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:367)
at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:263)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.statetransfer.StateTransferInterceptor.handleTxWriteCommand(StateTransferInterceptor.java:258)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:247)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:108)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
at org.infinispan.interceptors.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:107)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1097)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1089)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:437)
at com.bmc.truesight.api.lookup.ContainerLookupCache.addCiToCiTypeCiCache(ContainerLookupCache.java:468)
at com.bmc.truesight.api.lookup.ContainerLookupCache.populateCiToServiceIfServiceCI(ContainerLookupCache.java:530)
at com.bmc.truesight.platform.etl.CI.SyncTSIMCIs.lambda$null$113(SyncTSIMCIs.java:130)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at com.bmc.truesight.platform.etl.CI.SyncTSIMCIs.lambda$processResponseMsgs$114(SyncTSIMCIs.java:125)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at com.bmc.truesight.platform.etl.CI.SyncTSIMCIs.processResponseMsgs(SyncTSIMCIs.java:118)
at com.bmc.truesight.platform.etl.CI.SyncTSIMCIs.sendSyncCIRequest(SyncTSIMCIs.java:97)
at com.bmc.truesight.platform.etl.CI.SyncTSIMCIs.syncCis(SyncTSIMCIs.java:63)
at com.bmc.truesight.platform.message.InitializeTSIMServers.syncDataFromChildServer(InitializeTSIMServers.java:184)
at com.bmc.truesight.platform.message.InitializeTSIMServers.initializeChildBPPMserver(InitializeTSIMServers.java:93)
at com.bmc.truesight.platform.etl.component.tsim.TSIMWSUtility.performTSIMUpdateOrInitilize(TSIMWSUtility.java:274)
at com.bmc.truesight.platform.etl.component.tsim.TSIMWSUtility.initializeTSIM(TSIMWSUtility.java:331)
at com.bmc.truesight.platform.etl.component.management.types.TSIMComponent.initializationNotification(TSIMComponent.java:130)
at com.bmc.truesight.platform.etl.component.management.tasks.ComponentManagementTask.runInitialization(ComponentManagementTask.java:297)
at com.bmc.truesight.platform.etl.component.management.tasks.ComponentManagementTask.process(ComponentManagementTask.java:91)
at com.bmc.truesight.api.qmgr.TSQueue.run(TSQueue.java:85)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
*Our Cache.conf used is as below*
<?xml version="1.0" encoding="UTF-8"?>
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:7.0 http://www.infinispan.org/schemas/infinispan-config-7.0.xsd"
xmlns="urn:infinispan:config:7.0">
<jgroups>
<stack-file name="configurationFile" path="cache-replicator.conf"/>
</jgroups>
<cache-container default-cache="defaultCache">
<!--
Enables collection of global JMX statistics, and allows you to customize the JMX domain name under which
MBeans are registered.
-->
<jmx>
<property name="enabled">false</property>
<property name="jmxDomain">org.infinispan</property>
<property name="cacheManagerName">CacheManager</property>
<property name="statistics">false</property>
</jmx>
<!--
If the transport is omitted, there is no way to create distributed or clustered caches.
There is no added cost to defining a transport and not creating a cache that uses one, since the transport
is created and initialized lazily. By default, the JGroupsTransport is used.
Note that the JGroups transport uses sensible defaults if no configuration property is defined.
See:
http://community.jboss.org/wiki/ClusteredConfigurationQuickStart
http://docs.jboss.org/infinispan/5.0/apidocs/org/infinispan/remoting/tran...
-->
<transport cluster="TrueSight-Cache-cluster" stack="configurationFile">
</transport>
<!--
Used to register JVM shutdown hooks.
hookBehavior: DEFAULT, REGISTER, DONT_REGISTER
-->
<!-- <shutdown hookBehavior="DONT_REGISTER"/> -->
<local-cache name="defaultCache" statistics="false">
<!-- <storeAsBinary /> -->
<!--
Tunes the locking and concurrency characteristics of the cache.
See:
http://community.jboss.org/wiki/LockingandConcurrency
-->
<locking
isolation="READ_COMMITTED"
acquire-timeout="20000"
write-skew="false"
concurrency-level="5000"
striping="false"
/>
<!--
Used to register JMX statistics in any available MBean server
<jmxStatistics enabled="false"/>-->
<!--
Used to control whether objects stored in Infinispan nodes are primarily stored as rehydrated Objects or as
raw byte arrays. See:
http://community.jboss.org/wiki/Marshalling#Lazy_Deserialization
-->
<!-- <storeAsBinary enabled="true"/> -->
<!--
Used to enable invocation batching and allow the use of Cache.startBatch()/endBatch() methods.
http://community.jboss.org/wiki/Batching
<invocationBatching enabled="true"/> -->
<transaction mode="BATCH"/>
<!--
This element specifies that the cache is clustered.
modes supported: distribution (d), replication (r) or invalidation (i).
http://community.jboss.org/wiki/Clusteringmodes
-->
</local-cache>
<replicated-cache name="defaultCache" mode="ASYNC" queue-size="500" queue-flush-interval="10000">
<state-transfer await-initial-transfer="false"
timeout="240000"
enabled="true"
chunk-size="10000"
/>
<!--Default write-skew is false, concurrency-level is 16 and acquire-timeout is 20 seconds in infinispan. Increasing -->
<locking concurrency-level="1000" striping="true" acquire-timeout="240000"
isolation="READ_COMMITTED" write-skew="false"/>
<!-- <sync replTimeout="20000"/> -->
<!-- async replication. -->
</replicated-cache>
</cache-container>
</infinispan>
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-9312) Better API for registering remote listeners
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-9312:
--------------------------------------
Summary: Better API for registering remote listeners
Key: ISPN-9312
URL: https://issues.jboss.org/browse/ISPN-9312
Project: Infinispan
Issue Type: Enhancement
Components: Listeners, Remote Protocols
Reporter: Galder Zamarreño
Annotation based approach is cumbersome to use and some parameters cannot be defined at runtime. ISPN-9151 is a stop-gap solution to make those parameters more dynamic, but a better API is needed.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-9306) Unguarded lazy initialization of SQL statements in JDBC CacheStore
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-9306?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero commented on ISPN-9306:
---------------------------------------
Example, taking some code from {{AbstractTableManager}}:
{code:java}
protected String deleteRowSql;
public String getDeleteRowSql() {
if (deleteRowSql == null) {
deleteRowSql = String.format("DELETE FROM %s WHERE %s = ?", getTableName(), config.idColumnName());
}
return deleteRowSql;
}
{code}
It's not strictly wrong - especially it might seem to work fine in single threaded tests or servers with a limited amount of cores, but it becomes a huge red flag for the kind of situations Infinispan is used for.
Assuming I understand correctly, the method {{getDeleteRowSql()}} is invoked by multiple independent threads sharing the same instance of the table manager, which makes it troublesome.
The simplest solution is to either mark the method as {{synchronized}} or the field as {{volatile}} yet both could have other annoying effects on performance optimisations; best would be to initialize all such things into a final field, but that requires a bit of refactoring to ensure you have the table names, column names, etc.. during the contractor invocation.
> Unguarded lazy initialization of SQL statements in JDBC CacheStore
> ------------------------------------------------------------------
>
> Key: ISPN-9306
> URL: https://issues.jboss.org/browse/ISPN-9306
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 9.3.0.CR1
> Reporter: Sanne Grinovero
> Assignee: Ryan Emerson
>
> Several fields in {{org.infinispan.persistence.jdbc.table.management.AbstractTableManager}} are lazily initialized by its child implementations, yet without any barrier.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-9311) HotRod/Rest interoperability not working for protobuf caches with non string keys
by Gustavo Fernandes (JIRA)
Gustavo Fernandes created ISPN-9311:
---------------------------------------
Summary: HotRod/Rest interoperability not working for protobuf caches with non string keys
Key: ISPN-9311
URL: https://issues.jboss.org/browse/ISPN-9311
Project: Infinispan
Issue Type: Bug
Components: Server
Affects Versions: 9.3.0.CR1, 9.2.5.Final
Reporter: Gustavo Fernandes
Assignee: Gustavo Fernandes
When requesting an entry via REST for a cache storing protobuf, the request header "Key-Content-Type" is ignored and the response is 404 since all keys are assumed String
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (ISPN-9310) HotRod/Rest interoperability not working for protobuf caches with non string keys
by Gustavo Fernandes (JIRA)
Gustavo Fernandes created ISPN-9310:
---------------------------------------
Summary: HotRod/Rest interoperability not working for protobuf caches with non string keys
Key: ISPN-9310
URL: https://issues.jboss.org/browse/ISPN-9310
Project: Infinispan
Issue Type: Bug
Components: Server
Affects Versions: 9.3.0.CR1, 9.2.5.Final
Reporter: Gustavo Fernandes
Assignee: Gustavo Fernandes
When requesting an entry via REST for a cache storing protobuf, the request header "Key-Content-Type" is ignored and the response is 404 since all keys are assumed String
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months