December 2015 - infinispan-issues

[JBoss JIRA] (ISPN-5093) Granularity of remote event listener implementations doing the same job

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5093?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5093: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Granularity of remote event listener implementations doing the same job > ----------------------------------------------------------------------- > > Key: ISPN-5093 > URL: https://issues.jboss.org/browse/ISPN-5093 > Project: Infinispan > Issue Type: Enhancement > Components: Remote Protocols > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Fix For: 8.2.0.Alpha1 > > > Currently, if N clients add the same listener to a cache that does the same job, e.g. keeping a near cache consistent, this results in N server-side cluster listeners created, each potentially installed in different nodes. If one of those nodes fails, all clients that had a listener registered to that node will have to find a different node for this listener. > The downsides of this approach is that there are as many cluster listeners installed as clients have added listeners (or have near cache enabled), which might not very efficient. If a node goes down, all clients that have cluster listeners there need to failover to some other node. > The advantage of this approach is simplicity of the approach to decide where to add the listener and where to failover to. > For this type of scenarios, an alternative set up might be worth exploring: > If all these client side listeners are interested in exactly the same events, and the client ID would be exposed via the RemoteCache API, a server side cluster listener multi-plexing between all these clients could be potentially built. In other words, instead of having N clients register N cluster listeners, the first client would register the cluster listener with a client listener ID, and if more registrations were added with the same client listener ID, the connections would be added to the existing cluster listener implementation. > The maximise the efficiency of this solution, all clients (even running in different JMVs), given the same client listener ID, should agree upon the node to add the listener in. For a distributed cache, hashing on the cache name would work. For replicated caches, since there's no hashing available, the first node of the view could be used. > Since the logic to be executed server-side varies between being the first node adding the client listener vs the others, synchronization would be added to make sure that the first invocation only creates the cluster listener, and the others simply add the channel to the listener. > Failover is a bit more tricky too, because if the node with the cluster listener goes down, all the clients have to failover, which again exposes a 1st vs the others type of logic. > Advantages of this approach is the reduction in number of cluster listeners and potentially efficiency coming from a single cluster listener implementation server side. > The disadvantages come from the server side logic to add/failover a cluster listener, which need to take into account if the listener is present or not. Other disadvantages come from needing the clients to use some specific routing for adding listeners for same node. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5077) Custom remote events can be slightly inefficient

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5077?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5077: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Custom remote events can be slightly inefficient > ------------------------------------------------ > > Key: ISPN-5077 > URL: https://issues.jboss.org/browse/ISPN-5077 > Project: Infinispan > Issue Type: Enhancement > Components: hot, Remote Protocols > Affects Versions: 7.0.2.Final > Reporter: Galder Zamarreño > Fix For: 8.2.0.Alpha1 > > > Something we might want to improve for Hot Rod 3.0 protocol: > [16:40] <galderz> i've been thinking further about converters, and I think i've found a slight mismatch between what converter means for embedded listeners vs remote listeners > [16:40] <wburns> oh yeah? > [16:40] <galderz> for embedded listeners, it essentially transforms what you see as `value` > [16:41] <galderz> with the knowledge that key and metadata information will be shipped > [16:41] <galderz> the way i mapped converter to remote listeners is that whatever the converter returns, we ship that, as is, to the client > [16:41] <galderz> so, if a remote listener wants a custom event that includes key + value > [16:41] <galderz> it needs to develop a converter impl that returns bytes containing key + value > [16:41] <galderz> which is inefficient because you are passing around the key twice > [16:42] <galderz> once as part of the event itself, and again inside the converted value > [16:42] <galderz> inefficient from the POV of shipping stuff around from other nodes to where the cluster listener is located > [16:44] <wburns> yeah makes sense > [16:44] <galderz> not a major issue but not easy to fix without changing semantics or public protocol -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5076) Pessimistic transactions can lose their locks when the primary owner changes

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5076?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5076: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Pessimistic transactions can lose their locks when the primary owner changes > ---------------------------------------------------------------------------- > > Key: ISPN-5076 > URL: https://issues.jboss.org/browse/ISPN-5076 > Project: Infinispan > Issue Type: Bug > Components: Core, State Transfer > Affects Versions: 7.0.2.Final, 7.1.0.Alpha1 > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Labels: 7.0 > Fix For: 8.2.0.Alpha1 > > > In a pessimistic cache, if a transaction {{T1}} has a {{put(k, v)}} operation and the primary owner of the key is the originator, the lock is acquired on the originator but it is not replicated to on the backup(s). > If one of the backup owners becomes the primary owner, it will allow another transaction {{T2}} to lock (and update) key {{k}} before it receives the one-phase prepare command from the originator of {{T1}}. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5454) XSite: RetryMechanismTest random failures

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5454?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5454: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > XSite: RetryMechanismTest random failures > ----------------------------------------- > > Key: ISPN-5454 > URL: https://issues.jboss.org/browse/ISPN-5454 > Project: Infinispan > Issue Type: Bug > Components: Cross-Site Replication > Affects Versions: 7.2.1.Final > Reporter: Dan Berindei > Priority: Blocker > Labels: testsuite_stability > Fix For: 8.2.0.Alpha1 > > > {{ClusteredCacheBackupReceiver.awaitRemoteTask()}} doesn't respect the state push command's timeout, at least when it's smaller than the sync replication timeout in the target cache. When that happens, the state provider will resend the state, and there will be 2 state push commands executing at the same time. > RetryMechanismTest changes the state push timeout to 2 seconds, but the sync replication timeout stays at 15 seconds. This causes failures in {{testRetryLocally}} and {{testFailRetryLocally}}, if it takes more than 2 seconds to suspect the killed node. > {noformat} > 10:02:13,007 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)] > 10:02:16,008 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)] > 10:02:16,040 TRACE (asyncTransportThread-4,NodeP:) [RpcManagerImpl] replication exception: > org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected > 10:02:16,040 TRACE (asyncTransportThread-0,NodeP:) [RpcManagerImpl] replication exception: > org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected > 10:02:19,147 ERROR (testng-RetryMechanismTest:) [UnitTestTestNGListener] Test testFailRetryLocally(org.infinispan.xsite.statetransfer.failures.RetryMechanismTest) failed. > java.lang.AssertionError: expected:<2> but was:<3> > at org.testng.AssertJUnit.fail(AssertJUnit.java:59) > at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364) > at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80) > at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245) > at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252) > at org.infinispan.xsite.statetransfer.failures.RetryMechanismTest.testFailRetryLocally(RetryMechanismTest.java:227) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5465) Replace the Hash function with a segment mapper

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5465?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5465: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Replace the Hash function with a segment mapper > ----------------------------------------------- > > Key: ISPN-5465 > URL: https://issues.jboss.org/browse/ISPN-5465 > Project: Infinispan > Issue Type: Feature Request > Components: Configuration, Core > Affects Versions: 7.2.1.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 8.2.0.Alpha1 > > > Currently, we allow the user to customize the mapping of keys to CH segments with a custom {{Hash}} function. But the Hash function doesn't give the user direct control over where a key is mapped, the ultimate location depends on the CH implementation. The CH implementation is also customizable, but it's much harder for the user to get right. > We should replace the Hash with something like this: > {code:java} > public interface SegmentMapper { > public int getSegment(Object key, int numSegments); > {code} > This should also be easier to implement than the {{Grouper}} interface we have now, when the user only needs co-location and doesn't need additional grouping features like {{cache.getGroup(name)}}. > I think this should also help internally, e.g. to replace the {{GroupingConsistentHash}} that needs to be re-created on every topology update with a constant {{SegmentMapper}} wrapper. It might also help to compute the segment of a key only once and save it in the context, instead of computing it every time we need to know the location of a key. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5453) X-site state transfer: retry locally if local state push throws a SuspectException

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5453?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5453: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > X-site state transfer: retry locally if local state push throws a SuspectException > ---------------------------------------------------------------------------------- > > Key: ISPN-5453 > URL: https://issues.jboss.org/browse/ISPN-5453 > Project: Infinispan > Issue Type: Bug > Components: Core, Cross-Site Replication > Affects Versions: 7.2.1.Final > Reporter: Dan Berindei > Priority: Minor > Fix For: 8.2.0.Alpha1 > > > When the local state push command throws a {{SuspectException}}, we currently retry remotely unless the target node was already removed from the cache topology. This will fail with another {{SuspectException}}, so it would be better to retry locally the first time. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5427) Change getAll to return ordered map

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5427?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5427: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Change getAll to return ordered map > ----------------------------------- > > Key: ISPN-5427 > URL: https://issues.jboss.org/browse/ISPN-5427 > Project: Infinispan > Issue Type: Enhancement > Components: Core, Remote Protocols > Reporter: William Burns > Assignee: William Burns > Fix For: 8.2.0.Alpha1 > > > Currently our getAll returns a map of entries that exist in the cache in any order. We could enhance this to return a map where the entries are in the same iteration order of the Set (important when the user uses a LinkedHashSet). We could even do something like List<V> getAll(List<K>) to show better ordering as an option too. > This has 2 immediate advantages. > 1. Remote getAll no longer requires to serialize all the keys in the response since it knows which values map to which keys based on what it sent. > 2. Query results need ordered List as well. See query/src/main/java/org/infinispan/query/impl/EntityLoader.java -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5451) Data Container Segment Striping

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5451?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5451: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Data Container Segment Striping > ------------------------------- > > Key: ISPN-5451 > URL: https://issues.jboss.org/browse/ISPN-5451 > Project: Infinispan > Issue Type: Feature Request > Components: Core > Reporter: William Burns > Assignee: William Burns > Fix For: 8.2.0.Alpha1 > > > Today each node owns a given number of segments at a time. There are many operations that are only concerned with entries of a given segment. Unfortunately this means that we must iterate over the entire contents of the cache to find only the entries that match given segments. Note this JIRA is only for the in memory data container. The cache store requires additional work and should be done separately as it may involve reworking many more areas of the code base. > # State Transfer entry removal (this removes all entries of given segments) > # Entry retrieval (if we could iterate by segment instead of over all values we could complete segments sooner which would reduce how much memory a client must hold) > # Indexing may want to index by segment at some point as well -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5241) Cache topology updates should use the NO_FC flag

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5241?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5241: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Cache topology updates should use the NO_FC flag > ------------------------------------------------ > > Key: ISPN-5241 > URL: https://issues.jboss.org/browse/ISPN-5241 > Project: Infinispan > Issue Type: Bug > Components: Core, State Transfer > Affects Versions: 7.1.0.Final > Reporter: Dan Berindei > Assignee: Dan Berindei > Priority: Critical > Fix For: 8.2.0.Alpha1 > > > Topology updates are sent while holding the ClusterCacheStatus lock, so they should never block. However, when MFC is present, the topology update can block waiting for enough credits. As most CacheTopologyControlCommands need to acquire the ClusterCacheStatus lock, this can easily lead to a full remote-executor pool (and OOB pool) and the appearance of a deadlock. > What's more, if one node is not responsive, it can block all the other nodes from receiving further topology updates. Topology updates should be as prompt as possible, so we should use the NO_FC flag to ensure that each node receives topology updates as soon as possible. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

[JBoss JIRA] (ISPN-5415) Expose protobuf entries to scripting

by Adrian Nistor (JIRA)

[ https://issues.jboss.org/browse/ISPN-5415?page=com.atlassian.jira.plugin.... ] Adrian Nistor updated ISPN-5415: -------------------------------- Fix Version/s: 8.2.0.Alpha1 (was: 8.1.0.Final) > Expose protobuf entries to scripting > ------------------------------------ > > Key: ISPN-5415 > URL: https://issues.jboss.org/browse/ISPN-5415 > Project: Infinispan > Issue Type: Feature Request > Components: Remote Querying > Affects Versions: 8.0.0.Final > Reporter: Adrian Nistor > Assignee: Adrian Nistor > Fix For: 8.2.0.Alpha1 > > > We need an alternative API for Protostream marshalling that is easy to consume from scripting languages. The messages need to be unmarshalled into a map-like object that can be accessed easily from scripting languages. No marshaller implementation code should be provided by users, also no annotations. -- This message was sent by Atlassian JIRA (v6.4.11#64026)

8 years, 10 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues December 2015