May 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-4190) Use Message.Flag.DONT_LOOPBACK instead of RequestOptions.setExclusionList

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-4190?page=com.atlassian.jira.plugin.... ] William Burns reopened ISPN-4190: --------------------------------- The first pass at this didn't quite work out. However a newer version of JGroups should allow this to work with the fix included in JGRP-1835. > Use Message.Flag.DONT_LOOPBACK instead of RequestOptions.setExclusionList > ------------------------------------------------------------------------- > > Key: ISPN-4190 > URL: https://issues.jboss.org/browse/ISPN-4190 > Project: Infinispan > Issue Type: Task > Components: Core > Affects Versions: 7.0.0.Alpha2 > Reporter: Dan Berindei > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha4 > > > We are currently using {{RequestOptions.setExclusionList(channel.getAddress())}} in CommandAwareRpcDispatcher, to prevent the sender of an RPC to receive that message. We can't use {{Channel.setDiscardOwnMessages(true)}}, because TO requires messages to be delivered to self. > JGroups 3.5.0.Beta2 has added a {{DONT_LOOPBACK}} flag, which should be a lot more efficient than using the exclusion list. We should start using it. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4216) Use IGNORE_RETURN_VALUES flag for inserting keys/values in M/R intermediate cache

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4216?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4216: ----------------------------------------------- Dan Berindei <dberinde(a)redhat.com> changed the Status of [bug 1096878|https://bugzilla.redhat.com/show_bug.cgi?id=1096878] from NEW to MODIFIED > Use IGNORE_RETURN_VALUES flag for inserting keys/values in M/R intermediate cache > --------------------------------------------------------------------------------- > > Key: ISPN-4216 > URL: https://issues.jboss.org/browse/ISPN-4216 > Project: Infinispan > Issue Type: Bug > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha4 > > > Whenever data gets inserted in intermediate cache unsafeReturnValues setting is ignored and previous value from cache is returned every time we do cache#put. We did not experience this slowdown before because we only inserted once per key in intermediate cache put. Now that we do partial batching and transfer of these keys/values, we really hit the wall there with multiple insertions. Adding IGNORE_RETURN_VALUES flag to put invocation is the key to resolve this issue. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4216) Use IGNORE_RETURN_VALUES flag for inserting keys/values in M/R intermediate cache

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4216?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration updated ISPN-4216: ------------------------------------------ Bugzilla Update: Perform Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1096878 > Use IGNORE_RETURN_VALUES flag for inserting keys/values in M/R intermediate cache > --------------------------------------------------------------------------------- > > Key: ISPN-4216 > URL: https://issues.jboss.org/browse/ISPN-4216 > Project: Infinispan > Issue Type: Bug > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha4 > > > Whenever data gets inserted in intermediate cache unsafeReturnValues setting is ignored and previous value from cache is returned every time we do cache#put. We did not experience this slowdown before because we only inserted once per key in intermediate cache put. Now that we do partial batching and transfer of these keys/values, we really hit the wall there with multiple insertions. Adding IGNORE_RETURN_VALUES flag to put invocation is the key to resolve this issue. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4131) Lock acquired forever with delayed PrepareCommand

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4131?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4131: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1080314|https://bugzilla.redhat.com/show_bug.cgi?id=1080314] from ASSIGNED to MODIFIED > Lock acquired forever with delayed PrepareCommand > ------------------------------------------------- > > Key: ISPN-4131 > URL: https://issues.jboss.org/browse/ISPN-4131 > Project: Infinispan > Issue Type: Bug > Components: Transactions > Affects Versions: 6.0.2.Final, 7.0.0.Alpha1 > Reporter: Radim Vansa > Assignee: Dan Berindei > Priority: Critical > Labels: 630betablocker > > Distributed transactional cache: > 1. A sends Prepare to B > 2. B receives Prepare, but due to ongoing ST it is blocked > 3. B replication timeout elapses > 4. B sends Rollback, this does not find the TX as Prepare was not executed yet. The transaction is put into completedTransactions. > 5. Completed transactions timeout elapses. This is by default 15 seconds, way shorter than ST timeout (due to which the Prepare was blocked) > 6. Prepare is executed on B, acquiring lock on K > Nobody will rollback the TX as originator thinks it was already rolled back. > Result: key K will be locked forever, all attempts to update/remove it will fail. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4239) Commands received before the first topology where the local node is a member are not ignored

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4239?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4239: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096073|https://bugzilla.redhat.com/show_bug.cgi?id=1096073] from ASSIGNED to MODIFIED > Commands received before the first topology where the local node is a member are not ignored > -------------------------------------------------------------------------------------------- > > Key: ISPN-4239 > URL: https://issues.jboss.org/browse/ISPN-4239 > Project: Infinispan > Issue Type: Bug > Components: Core, State Transfer > Affects Versions: 6.0.2.Final, 7.0.0.Alpha3 > Reporter: Dan Berindei > Assignee: Dan Berindei > Labels: testsuite_stability > Fix For: 7.0.0.Alpha4, 7.0.0.Final > > > The ISPN-3731 fix was supposed to prevent a joiner from running commands broadcasted by other members before the joiner became a member in the cache topology. > However, the fix was incomplete, and commands received before the joiner became a member were still executed. This is causing failures in {{StateTransferReplicationQueueTest}}, which is issuing a put(k, v) and a remove(k) and expecting the key to be removed. It can happen that the put(k, v) command is executed before the joiner becomes a member, but the remove(k) command is rejected because it came from a topology where the joiner was not a member: > {noformat} > 12:54:41,412 TRACE (testng-StateTransferReplicationQueueTest:nbst-replqueue) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686], numSegments=1, primaryOwners=[0]}, pendingCH=null} on cache nbst-replqueue > 12:54:41,412 TRACE (testng-StateTransferReplicationQueueTest:nbst-replqueue) [StateTransferLockImpl] Signalling topology 3 is installed > 12:54:41,433 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [InboundInvocationHandlerImpl] Calling perform() on MultipleRpcCommand{commands=[..., PrepareCommand {modifications=[PutKeyValueCommand{key=test58, value=org.infinispan.statetransfer.StateTransferReplicationQueueTest$PojoValue@59, flags=null, putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}], onePhaseCommit=true, gtx=GlobalTransaction:<StateTransferReplicationQueueTest-NodeA-8686>:6969:local, cacheName='nbst-replqueue', topologyId=3}], cacheName='nbst-replqueue'} > 12:54:41,439 TRACE (remote-thread-StateTransferReplicationQueueTest-NodeC-p3980-t1:nbst-replqueue) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686], numSegments=1, primaryOwners=[0]}, pendingCH=ReplicatedConsistentHash{members=[StateTransferReplicationQueueTest-NodeA-8686, StateTransferReplicationQueueTest-NodeC-12962], numSegments=1, primaryOwners=[0]}} on cache nbst-replqueue > 12:54:41,439 TRACE (remote-thread-StateTransferReplicationQueueTest-NodeC-p3980-t1:nbst-replqueue) [StateTransferManagerImpl] This is the first topology in which the local node is a member > 12:54:41,462 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:nbst-replqueue) [ReadCommittedEntry] Updating entry (key=test58 removed=false valid=true changed=true created=true loaded=false value=org.infinispan.statetransfer.StateTransferReplicationQueueTest$PojoValue@59 metadata=EmbeddedMetadata{version=null}, providedMetadata=null) > 12:54:41,491 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [CommandAwareRpcDispatcher] Attempting to execute command: MultipleRpcCommand{commands=[PrepareCommand {modifications=[RemoveCommand{key=test58, value=null, flags=null, valueMatcher=MATCH_ALWAYS}], onePhaseCommit=true, gtx=GlobalTransaction:<StateTransferReplicationQueueTest-NodeA-8686>:6970:local, cacheName='nbst-replqueue', topologyId=3}, ...], cacheName='nbst-replqueue'} [sender=StateTransferReplicationQueueTest-NodeA-8686] > 12:54:41,493 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [StateTransferLockImpl] Waiting for transaction data for topology 3, current topology is 3 > 12:54:41,493 TRACE (Incoming-1,StateTransferReplicationQueueTest-NodeC-12962:) [InboundInvocationHandlerImpl] Ignoring command sent before the local node was a member (command topology id is 3) > {noformat} -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4240) Migrate one intermediate key/value per maxCollectorSize threshold

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4240?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4240: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096467|https://bugzilla.redhat.com/show_bug.cgi?id=1096467] from POST to MODIFIED > Migrate one intermediate key/value per maxCollectorSize threshold > ----------------------------------------------------------------- > > Key: ISPN-4240 > URL: https://issues.jboss.org/browse/ISPN-4240 > Project: Infinispan > Issue Type: Enhancement > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha4 > > > When we hit the maxCollectorSize threshold, we remove all the values from the collector and we start transferring them into the intermediary cache. But the other mapper threads keep working, and event with large collector sizes, it's very likely that the collector will be full again before we finished sending the previous batch. So we could end up with a lot of threads trying to insert maxCollectorSize values in the intermediary cache in parallel, and a lot more intermediary values in memory on each mapper node than the user would expect. > In order to alleviate this observed phenomena Dan proposed an idea to keep the maxCollectorSize threshold, but to only move a single key at a time from the collector to the intermediary cache when the threshold is hit (the least recently used one). That way, the other keys would have time to collect more values (or just run the combiner a few more times). > This way we still transfer some intermediate keys/values during map/combine phase rather than all at once at the end of that phase thus alleviating possibility of OOM. At the end of map/combine phase all remaining keys/values are of course transferred. Application users can fine tune their requirements and a specific use case using maxCollectorSize threshold. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4218) Increase default maxCollectorSize for MapReduceTask

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4218?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4218: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096468|https://bugzilla.redhat.com/show_bug.cgi?id=1096468] from POST to MODIFIED > Increase default maxCollectorSize for MapReduceTask > --------------------------------------------------- > > Key: ISPN-4218 > URL: https://issues.jboss.org/browse/ISPN-4218 > Project: Infinispan > Issue Type: Enhancement > Components: Distributed Execution and Map/Reduce > Reporter: Vladimir Blagojevic > Assignee: Vladimir Blagojevic > Priority: Minor > Fix For: 7.0.0.Alpha4, 7.0.0.Final > > > During performance testing we have found that the value of around 10K leads to a better performance of large map/reduce tasks containing more than 100K of cache entries. We have concluded that the current default value of 1024 is too small. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4160) Do not allow the state transfer chunk size to be <= 0

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-4160?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-4160: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096466|https://bugzilla.redhat.com/show_bug.cgi?id=1096466] from POST to MODIFIED > Do not allow the state transfer chunk size to be <= 0 > ----------------------------------------------------- > > Key: ISPN-4160 > URL: https://issues.jboss.org/browse/ISPN-4160 > Project: Infinispan > Issue Type: Task > Components: Configuration, State Transfer > Affects Versions: 7.0.0.Alpha1 > Reporter: Dan Berindei > Assignee: Vladimir Blagojevic > Fix For: 7.0.0.Alpha4 > > > The state transfer chunk size is used in other places as well, and the case of {{chunkSize <= 0}} has to be handled everywhere independently. StateTransferConfigurationBuilder should validate that {{chunkSize > 0}}, so that users of the chunk size setting don't need a default value. > The users will still be able to force state transfer to transfer everything at once with {{chunkSize == Integer.MAX_VALUE}}. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3877) SingleFileStore continues to consume space on the disk even if the free entries at end of the file are not in use. This space can be released back to the filesystem

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-3877?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-3877: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096726|https://bugzilla.redhat.com/show_bug.cgi?id=1096726] from NEW to MODIFIED > SingleFileStore continues to consume space on the disk even if the free entries at end of the file are not in use. This space can be released back to the filesystem > -------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: ISPN-3877 > URL: https://issues.jboss.org/browse/ISPN-3877 > Project: Infinispan > Issue Type: Patch > Components: Loaders and Stores > Affects Versions: 6.0.0.Final > Environment: Oracle JDK 1.7, Windows Server 2008 R2 x64 > Reporter: Rajesh Jangam > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha4 > > Attachments: PersistentQueue.java > > > We are testing the latest Infinispan 6.0 SingleFileStore version for our application. > We tend to create a lot of entries in the store and remove them when not required. > However, what we have observed is that the size of the file keeps on increasing as more data is being added/removed and that it does not get reclaimed. > > This was not seen with earlier versions like 5.3.0. There the size of the cache folder would reduce as entries got removed. > We have attempted to create a fix for this issue and tested it in our environment. It keeps the space usage under control as compared to the original implementation. > As a part of this fix, we truncate the file (as a part of purge cycle) if there are free entries found towards the end of the file. Also these free entries are removed from the freeList. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3894) SingleFileStore can optimize space usage by coalescing adjacent free entry blocks.

by RH Bugzilla Integration (JIRA)

[ https://issues.jboss.org/browse/ISPN-3894?page=com.atlassian.jira.plugin.... ] RH Bugzilla Integration commented on ISPN-3894: ----------------------------------------------- Tristan Tarrant <ttarrant(a)redhat.com> changed the Status of [bug 1096727|https://bugzilla.redhat.com/show_bug.cgi?id=1096727] from NEW to MODIFIED > SingleFileStore can optimize space usage by coalescing adjacent free entry blocks. > ---------------------------------------------------------------------------------- > > Key: ISPN-3894 > URL: https://issues.jboss.org/browse/ISPN-3894 > Project: Infinispan > Issue Type: Patch > Components: Loaders and Stores > Affects Versions: 6.0.0.Final > Environment: Oracle JDK 1.7, Windows Server 2008 R2 > Reporter: Rajesh Jangam > Assignee: Dan Berindei > Fix For: 7.0.0.Alpha4 > > Attachments: CoalesceEntries.java > > > This issue is related to issue ISPN-3877. > Here we have a use case where the size of the entries gradually keeps on increasing and then finally those entries expire. > 1. Smaller entries expire and new bigger entries start getting added > 2. These new bigger entries will not fit into the free slots (entries) created due to the expiry of smaller entries. > 3. Thus, these new bigger entries only get allocated at the end of the file resulting in growth of file size. > The optimization proposed is to: > 1. Coalesce/Combine adjacent free entries during the periodic purge cycle to create bigger/larger free entries. > 2. While allocating from an existing free entry, check if the size of the free entry is more than required for the new request being allocated. If yes, then, split the free entry into two free entries: > a. First part equal to the length being requested > This is returned as a part of the allocation activity. > b. Second part equal to the remainder of the space. > This added back as a free entry to the freeList. > Thus free space created due to expiry of smaller entries becomes available for consumption for the larger entries to some extent. > This reduces file size of the store. -- This message was sent by Atlassian JIRA (v6.2.3#6260)

10 years, 5 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues May 2014