Re: [infinispan-dev] Parallel M/R
by Mircea Markus
Thanks Vladimir, I like the hands on approach!
Adding -dev, there's a lot of interest around the parallel M/R so I think others will have some thoughts on it as well.
So what you're basically doing in your branch is iterate over all the keys in the cache and then for each key invoke the mapping in a separate thread. Whilst this would work, I think it has some drawbacks:
- the iteration over the keys in the container happens in sequence, albeit the mapping phases happening in parallel. This speeds things up a bit but not as much as having the iteration
happening in parallel, especially when the mapper is fast, which I think it's pretty common.
- the StatelessTask + some smaller objects are being created for each iterated key. That's a lot of noise for the GC imo
I think delegating the parallel iteration to the DataContainer (similar to AdvancedCacheLoader.process (Executor)) would be a better approach IMO:
- the logic is reusable for other components as well, such as querying (to implement full-scan-like search, or a general purpose parallel iterator over the keys
- object creation is reduced
- the DefaultDetaContainer uses an EquivalentConcurrentHashMapV8 for holding the entries, which already supports parallel iteration so the heavy lifting is already in place
On Dec 4, 2013, at 5:16 PM, Vladimir Blagojevic <vblagoje(a)redhat.com> wrote:
> Here is my M/R parallel execution solution updated to master https://github.com/vblagoje/infinispan/tree/t_2284_new
>
> Now, I'll work on your solution which I am starting to like actually the more I think about it. Although I have to admit that I would eviscerate some of your interfaces like these KeyFilters into more prominent packages so we can all use the same interfaces. Also I would see if we can genericize some of your interfaces and implementations.
>
> Will keep you updated.
>
> Vladimir
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 4 months
Cost of inheritance
by Sanne Grinovero
In a particular benchmark I'm running, the bottleneck of my system
seems to be the object allocation rate.
More specifically, in just some minutes I've got about 55GB allocated
just of instances of SingleKeyNonTxInvocationContext
(yes I literally mean GigaBytes)
and 45GB of org.infinispan.commands.read.GetKeyValueCommand.
To be clear: these high amounts are caused only by the "shallow" class
instance of these two types, not counting key nor value sizes being
actually moved into/out Infinispan.
Of course it means we're talking about many of these instances, but
also the size of each of them matters, so I've been taking a look at
their definitions.
Running 64-bit HotSpot VM.
Using compressed references with 3-bit shift.
Objects are 8 bytes aligned.
# org.infinispan.context.SingleKeyNonTxInvocationContext
offset size type description
0 12 (assumed to be the object header + first
field alignment)
12 1 byte AbstractInvocationContext.contextFlags
13 3 (alignment/padding gap)
16 4 Address AbstractInvocationContext.origin
20 4 WeakReference AbstractInvocationContext.classLoader
24 1 boolean SingleKeyNonTxInvocationContext.isOriginLocal
25 1 boolean SingleKeyNonTxInvocationContext.isLocked
26 2 (alignment/padding gap)
28 4 Object SingleKeyNonTxInvocationContext.key
32 4 CacheEntry SingleKeyNonTxInvocationContext.cacheEntry
36 4 (loss due to the next object alignment)
40 (object boundary, size estimate)
I notice two things in here:
a) we could refactor maybe some code to have less fields in such a
hot allocated type
b) Lots of space is being wasted in padding!
If you count the bytes lost because of the various alignment rules: 9
That's almost 25%, about 13GB of used memory!
Why are there two separate object alignment gaps? That's because the
fields of the parent class need to be grouped, then the child class's
fields are aligned after it.
In other words, if I move the fields from the parent class to the
bottom class I get:
org.infinispan.context.SingleKeyNonTxInvocationContext
offset size type description
0 12 (assumed to be the object header + first
field alignment)
12 1 byte SingleKeyNonTxInvocationContext.contextFlags
13 1 boolean SingleKeyNonTxInvocationContext.isOriginLocal
14 1 boolean SingleKeyNonTxInvocationContext.isLocked
15 1 (alignment/padding gap)
16 4 Address SingleKeyNonTxInvocationContext.origin
20 4 ClassLoader SingleKeyNonTxInvocationContext.classLoader
24 4 Object SingleKeyNonTxInvocationContext.key
28 4 CacheEntry SingleKeyNonTxInvocationContext.cacheEntry
32 (object boundary, size estimate)
which recovers 20% ..
Looking forward to see simpler class hierarchies in the code ;-)
Sanne
10 years, 4 months
Denormalizing hashes
by Radim Vansa
Hi Galder,
as I am trying to debug some problem in C++ client, I was looking into
the server code. And I am not sure whether I understand the code
correctly, but it seems to me that the server denormalizes the
consistent hash for each client anew (after each topology change or
client joining). Is this true? Looking into trace logs, I can see stuff
like
18:15:17,339 TRACE [org.infinispan.server.hotrod.Encoders$Encoder12$]
(HotRodServerWorker-12) Writing hash id 639767 for 192.168.11.101:11222
From denormalizeSegmentHashIds() method I see that this means that we
have executed the hash function 639768 times just to notify one client.
Is my understanding correct?
Also, there is nothing like the concept of primary owner, is this right?
I thought that every first request in HotRod will go to primary owner,
so that the PUT does not have to do the first hop and is executed
directly on the primary. But it seems to me that it goes to any of the
owners (practically random one, as you are only looking for the numOwner
ids in leeway = on the beginning of the range - then, 99.98% or more
requests should go to the server with last position in the leeway). This
looks pretty suboptimal for writes, isn't it?
Cheers
Radim
PS: for every line of code you write in Scala, God kills a kitten
--
Radim Vansa <rvansa(a)redhat.com>
JBoss DataGrid QA
10 years, 4 months
The monthly failing tests newsletter
by Sanne Grinovero
Failed tests:
ThreadLocalLeakTest.testCheckThreadLocalLeaks:87 IllegalState Thread
locals st...
ExplicitUnlockTest.testLock:43->doTestLock:80 All worker should
complete without exceptions
ExplicitUnlockTest.testLockNoExplicitUnlock:51->doTestLock:80 All
worker should complete without exceptions
ExplicitUnlockTest.testLockNoExplicitUnlockTwoTasks:55->doTestLock:80
All worker should complete without exceptions
UNORDEREDEvictionFunctionalTest>BaseEvictionFunctionalTest.testSimpleExpirationMaxIdle:54
cache size should be zero: 128
StateTransferReplicationQueueTest.testStateTransferWithNodeRestartedAndBusy:81->thirdWritingCacheTest:119
null
Tests run: 3808, Failures: 6, Errors: 0, Skipped: 0
10 years, 4 months
rethinking ISPN transactions
by Mircea Markus
Hey guys,
Several things were discussed lately([1],[2],[3],[4]) around our transaction support. Here's some some thoughts I have around re-modeling transactions for 7.0:
1. Async options for commit/rollback
- they don't really make sense as a user you don't get any guarantee on the status of the transaction
- they complicate the code significantly
- I think they should be removed
2. READ_COMMITTED
- it has the same performance as REPEATABLE_READ, but offers less guarantees.
- unlike REPEATABLE_READ, it also behaves inconsistently when the data is owned by transaction originator
- I think it should be removed
3. Optimistic tx without Write Skew Check (WSC)
- well, without WSC the transactions are not optimistic by definition
- they are something else: an batch update of multiple key/values. If the batch is successful you know the update was atomic. If it failed you don't get any guarantee
- suggestion: optimistic tx should *always* have WSC enabled (no option to configure it)
- build our batching functionality on top of what currently is optimistic tx without WSC and document it as such
4. Remove 1PC option
- I'm not totally sure about it, but does it really make sense to have 1PC as an option? they don't offer any consistency guarantees so async API + non tx do about the same thing
[1] http://markmail.org/thread/a7fjko4dyejxqgdy
[2] https://github.com/infinispan/infinispan/pull/2177
[3] http://infinispan.markmail.org/thread/nl2bs7rjvayjcybv
[4] http://infinispan.markmail.org/thread/vbg6g4otu7djazbc
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 4 months
Re: [infinispan-dev] CI run for the Cpp client
by Mircea Markus
Thanks, Ion!
On Dec 2, 2013, at 9:01 AM, isavin(a)redhat.com wrote:
> On 11/19/2013 11:27 PM, Mircea Markus wrote:
>> Hi,
>>
>> Ion has access to my AWS account so he can launch new instances.
>>
>> From the top of my head, for the Cpp client CI we'd need:
>> - an RHEL5 build on every check in
>> - an RHEL6 build on every check in
>> - an Windows 7 (?) build on every check in
>> Same for pull every pull requests.
>>
>> Work to integration with the teamcity is also need. (Ion you should have credentials for connecting to ci.infinispan.org.)
>>
>> For cost reasons, these instances should only be launched when there's activity.
>> Please also give a cost estimate (monthly + initial) so that we can get an idea of the budget needed.
>>
>> Cheers,
>
> Hi Mircea,
>
> Here are some cost estimates:
>
> RHEL5:
> * m1.small RHEL 5.9 x86_64 (initial: $0; per hour: $0.12; ~$30 per month with 8h/day usage)
>
> RHEL6:
> * m1.small RHEL 6.4 x86_64 (initial: $0; per hour: $0.12; ~$30 per month with 8h/day usage)
>
> Windows Build Configuration (could use this for HotRod C# also):
> * m1.small with Windows Server 2012 x86_64 (initial: $0; per hour $0.091; ~$23 per month with 8h/day usage)
> * Visual Studio 2013 perpetual license (without MSDN subscription): (initial: ~$1000 after: $0)
What about running just the .NET tests (when available) on the windows machine? I assume we won't need a full VS2013 license for that, but just the .NET framework which is free (?)
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 5 months
