[JBoss JIRA] (ISPN-5208) Avoid invalid topology
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5208?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño commented on ISPN-5208:
----------------------------------------
Added a new PR with a fix.
> Avoid invalid topology
> ----------------------
>
> Key: ISPN-5208
> URL: https://issues.jboss.org/browse/ISPN-5208
> Project: Infinispan
> Issue Type: Enhancement
> Components: Server
> Reporter: Takayoshi Kimura
> Assignee: Galder Zamarreño
> Labels: jdg641
> Fix For: 7.2.0.Final
>
> Attachments: ISPN_5208.java
>
>
> We've seen some invalid topology propagated to client and it causes ArrayIndexOutOfBoundsException:
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
> at org.infinispan.client.hotrod.impl.transport.tcp.RoundRobinBalancingStrategy.getServerByIndex(RoundRobinBalancingStrategy.java:68) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.transport.tcp.RoundRobinBalancingStrategy.nextServer(RoundRobinBalancingStrategy.java:44) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.nextServer(TcpTransportFactory.java:220) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:194) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:27) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:48) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:535) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.RemoteCacheManager.ping(RemoteCacheManager.java:635) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:616) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:527) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:523) [infinispan-client-hotrod-6.1.0.Final-redhat-4.jar:6.1.0.Final-redhat-4]
> {noformat}
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
> at org.infinispan.client.hotrod.impl.consistenthash.SegmentConsistentHash.getServer(SegmentConsistentHash.java:33)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:204)
> at org.infinispan.client.hotrod.impl.operations.AbstractKeyOperation.getTransport(AbstractKeyOperation.java:40)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:48)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:237)
> at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
> at sample.Main.main(Main.java:16)
> {noformat}
> It happens on both Hot Rod 2 and 1.3 clients.
> It's really hard to reproduce this state and we don't have a consistent way to reproduce it. However when this happens there is always view change happening so it's related to view change.
> Judging from the stack trace, the client receives numOwners=0 or numSegments=0 topology from the server.
> Also we are unable to find to recover this situation. Rebooting random nodes don't help and keep getting this exceptions on client side.
> Until we can find the root cause, I think it's better to add a guard to avoid this kind invalid topology stored in the server side and propagated to the clients.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5174) Transaction cannot be recommitted after ownership changes
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5174?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5174:
-----------------------------------
Affects Version/s: 7.1.1.Final
> Transaction cannot be recommitted after ownership changes
> ---------------------------------------------------------
>
> Key: ISPN-5174
> URL: https://issues.jboss.org/browse/ISPN-5174
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.0.CR2, 7.1.1.Final
> Reporter: Radim Vansa
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 7.2.0.Beta2, 7.2.0.Final
>
>
> Once transaction is completed, it cannot commit again. If it should commit more keys since it has become an owner of some new keys modified in this transaction, it just ignores the further commit.
> There is a race with state transfer which can bring an old value (with StateResponseCommand sent before it is commited) but the value is not set by the ongoing transaction either.
> This results with stale value stored on one node.
> In my case, The problematic part is transaction <edg-perf01-62141>:15066 (consisting of 10 modifications) which got prepared and committed on edg-perf04 in topology 25. Before the originator finishes, topology changes and 04 requests ongoing transactions:
> {code}
> 11:06:11,369 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (transport-thread-17) Replication task sending StateRequestCommand{cache=testCache, origin=edg-perf04-35097, type=GET_TRANSACTIONS, topologyId=28, segments=[275, 1, 278, 9, 282, 286, 17, 259, 25, 267, 171, 169, 33, 306, 175, 173, 310, 172, 314, 41, 167, 165, 318, 187, 290, 49, 185, 191, 294, 189, 179, 298, 57, 177, 183, 302, 181, 343, 205, 201, 338, 203, 336, 351, 197, 349, 199, 347, 193, 345, 195, 326, 85, 87, 322, 93, 332, 95, 330, 89, 91, 103, 101, 99, 506, 97, 105, 357, 359, 353, 355, 361]} to single recipient edg-perf01-62141 with response mode GET_ALL
> 11:06:11,495 DEBUG [org.infinispan.statetransfer.StateConsumerImpl] (transport-thread-17) Applying 6 transactions for cache testCache transferred from node edg-perf01-62141
> {code}
> However I don't see how these are applied, since PrepareCommand is not created again - from the code I see only that backup locks are added. Not sure if the transaction is registered at all, since it was already completed on this node (but at that time it did not own key_00000000000002EB).
> After originator stores the entry, it sends one more CommitCommand with topology 28:
> {code}
> 11:06:11,619 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-2) Replication task sending CommitCommand {gtx=GlobalTransaction:<edg-perf01-62141>:15066:local, cacheName='testCache', topologyId=28} to addresses [edg-perf03-20530, edg-perf04-35097] with response mode GET_ALL
> {code}
> 04 receives several CommitCommands (both from originator and forwards), but all of them are ignored as the transaction is completed.
> I don't see the logs where state transfer is assembled, but it's probably before the entry is stored on originator as the state transfer contains the old entry:
> {code}
> 11:06:13,449 TRACE [org.infinispan.statetransfer.StateConsumerImpl] (remote-thread-91) Received chunk with keys [key_000000000000065B, key_00000000000006BE, key_FFFFFFFFFFFFE62F, key_0000000000001F42, key_000000000000027B, key_000000000000159D, key_00000000000002EB, key_00000000000002BB] for segment 343 of cache testCache from node edg-perf01-62141
> 11:06:13,454 TRACE [org.infinispan.container.DefaultDataContainer] (remote-thread-91) Store ImmortalCacheEntry{key=key_00000000000002EB, value=[2 #7: 366, 544, 576, 804, 1061, 1181, 1290, ]} in container
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5174) Transaction cannot be recommitted after ownership changes
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-5174?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-5174:
-----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 7.2.0.Beta2
7.2.0.Final
Resolution: Done
> Transaction cannot be recommitted after ownership changes
> ---------------------------------------------------------
>
> Key: ISPN-5174
> URL: https://issues.jboss.org/browse/ISPN-5174
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.1.0.CR2, 7.1.1.Final
> Reporter: Radim Vansa
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 7.2.0.Beta2, 7.2.0.Final
>
>
> Once transaction is completed, it cannot commit again. If it should commit more keys since it has become an owner of some new keys modified in this transaction, it just ignores the further commit.
> There is a race with state transfer which can bring an old value (with StateResponseCommand sent before it is commited) but the value is not set by the ongoing transaction either.
> This results with stale value stored on one node.
> In my case, The problematic part is transaction <edg-perf01-62141>:15066 (consisting of 10 modifications) which got prepared and committed on edg-perf04 in topology 25. Before the originator finishes, topology changes and 04 requests ongoing transactions:
> {code}
> 11:06:11,369 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (transport-thread-17) Replication task sending StateRequestCommand{cache=testCache, origin=edg-perf04-35097, type=GET_TRANSACTIONS, topologyId=28, segments=[275, 1, 278, 9, 282, 286, 17, 259, 25, 267, 171, 169, 33, 306, 175, 173, 310, 172, 314, 41, 167, 165, 318, 187, 290, 49, 185, 191, 294, 189, 179, 298, 57, 177, 183, 302, 181, 343, 205, 201, 338, 203, 336, 351, 197, 349, 199, 347, 193, 345, 195, 326, 85, 87, 322, 93, 332, 95, 330, 89, 91, 103, 101, 99, 506, 97, 105, 357, 359, 353, 355, 361]} to single recipient edg-perf01-62141 with response mode GET_ALL
> 11:06:11,495 DEBUG [org.infinispan.statetransfer.StateConsumerImpl] (transport-thread-17) Applying 6 transactions for cache testCache transferred from node edg-perf01-62141
> {code}
> However I don't see how these are applied, since PrepareCommand is not created again - from the code I see only that backup locks are added. Not sure if the transaction is registered at all, since it was already completed on this node (but at that time it did not own key_00000000000002EB).
> After originator stores the entry, it sends one more CommitCommand with topology 28:
> {code}
> 11:06:11,619 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (DefaultStressor-2) Replication task sending CommitCommand {gtx=GlobalTransaction:<edg-perf01-62141>:15066:local, cacheName='testCache', topologyId=28} to addresses [edg-perf03-20530, edg-perf04-35097] with response mode GET_ALL
> {code}
> 04 receives several CommitCommands (both from originator and forwards), but all of them are ignored as the transaction is completed.
> I don't see the logs where state transfer is assembled, but it's probably before the entry is stored on originator as the state transfer contains the old entry:
> {code}
> 11:06:13,449 TRACE [org.infinispan.statetransfer.StateConsumerImpl] (remote-thread-91) Received chunk with keys [key_000000000000065B, key_00000000000006BE, key_FFFFFFFFFFFFE62F, key_0000000000001F42, key_000000000000027B, key_000000000000159D, key_00000000000002EB, key_00000000000002BB] for segment 343 of cache testCache from node edg-perf01-62141
> 11:06:13,454 TRACE [org.infinispan.container.DefaultDataContainer] (remote-thread-91) Store ImmortalCacheEntry{key=key_00000000000002EB, value=[2 #7: 366, 544, 576, 804, 1061, 1181, 1290, ]} in container
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-4919) Configuration templates
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/ISPN-4919?page=com.atlassian.jira.plugin.... ]
Paul Ferraro commented on ISPN-4919:
------------------------------------
One issue that will need to be addressed for server is that undefined attributes of a given resource have default values. From WF's perspective, this is a requirement of the management model (i.e. to indicate the assumed value for a given attribute if undefined). However, this makes the concept of defined vs undefined attributes difficult to decipher when handling extending configuration. This suggests that we need separate schema/attribute definitions for base configurations vs extended configurations.
> Configuration templates
> -----------------------
>
> Key: ISPN-4919
> URL: https://issues.jboss.org/browse/ISPN-4919
> Project: Infinispan
> Issue Type: Feature Request
> Components: Configuration
> Reporter: Tristan Tarrant
> Assignee: Tristan Tarrant
>
> Currently there is a 1:1 relationship between configuration and named caches. While the programmatic API does have the ability to .read() an existing configuration to create a new one, the declarative config does not.
> We should introduce the concept of configuration inheritance, e.g.:
> {code}
> <local-cache name="eviction-cache">
> <eviction strategy="LIRS" maxEntries="10000"/>
> </local-cache>
> <local-cache name="mycache" template="eviction-cache" />
> {code}
> Possibly, cache templates should be made "abstract" so that they cannot be instantiated as named caches directly, e.g.:
> {code}
> <local-cache name="eviction-cache" abstract="true">
> ...
> </local-cache>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5321) StringBasedStoreMultinodeIT.testFetchState fails randomly
by Vojtech Juranek (JIRA)
Vojtech Juranek created ISPN-5321:
-------------------------------------
Summary: StringBasedStoreMultinodeIT.testFetchState fails randomly
Key: ISPN-5321
URL: https://issues.jboss.org/browse/ISPN-5321
Project: Infinispan
Issue Type: Bug
Components: Test Suite - Server
Reporter: Vojtech Juranek
Fails as {{k1}} enrtyr is evicted instead of expected {{k2}}:
{noformat}
java.lang.AssertionError: expected null, but was:<[B@636e6f73>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotNull(Assert.java:664)
at org.junit.Assert.assertNull(Assert.java:646)
at org.junit.Assert.assertNull(Assert.java:656)
at org.infinispan.server.test.cs.jdbc.multinode.StringBasedStoreMultinodeIT.testFetchState(StringBasedStoreMultinodeIT.java:60)
{noformat}
see e.g. [this CI build|http://ci.infinispan.org/viewLog.html?buildId=24519&tab=buildResult...]
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5310) Produce and distribute module org.hibernate.search.infinispan-storage
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5310?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes commented on ISPN-5310:
-----------------------------------------
Please take a look at the preview PR. The module is being produced solely on the migrated hibernate search directory provider and is called {{org.infinispan.hibernate-search.directory-provider}} slot {{ispn-7.2}}.
Do you have a hard requirement on the name {{org.hibernate.search.infinispan-storage}} ?
> Produce and distribute module org.hibernate.search.infinispan-storage
> ---------------------------------------------------------------------
>
> Key: ISPN-5310
> URL: https://issues.jboss.org/browse/ISPN-5310
> Project: Infinispan
> Issue Type: Task
> Components: Build process, Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Gustavo Fernandes
> Priority: Blocker
> Fix For: 7.2.0.Beta2
>
>
> Hibernate Search now expects to load the module {{org.hibernate.search.infinispan-storage}} to bind to the Infinispan integration.
> This module should be produced by the Infinispan build, and distributed by the Infinispan releases.
> When downloading the Infinispan modules zip for WildFly, this module should be included.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years
[JBoss JIRA] (ISPN-5310) Produce and distribute module org.hibernate.search.infinispan-storage
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPN-5310?page=com.atlassian.jira.plugin.... ]
Gustavo Fernandes updated ISPN-5310:
------------------------------------
Status: Open (was: New)
> Produce and distribute module org.hibernate.search.infinispan-storage
> ---------------------------------------------------------------------
>
> Key: ISPN-5310
> URL: https://issues.jboss.org/browse/ISPN-5310
> Project: Infinispan
> Issue Type: Task
> Components: Build process, Embedded Querying
> Reporter: Sanne Grinovero
> Assignee: Gustavo Fernandes
> Priority: Blocker
> Fix For: 7.2.0.Beta2
>
>
> Hibernate Search now expects to load the module {{org.hibernate.search.infinispan-storage}} to bind to the Infinispan integration.
> This module should be produced by the Infinispan build, and distributed by the Infinispan releases.
> When downloading the Infinispan modules zip for WildFly, this module should be included.
--
This message was sent by Atlassian JIRA
(v6.3.11#6341)
11 years