New partial replication protocol providing serializability guarantees in Infinispan
by Paolo Romano
Hi,
within the context Cloud-TM project we have developed a new partial
replication algorithm (corresponding to distribution mode of Infinispan)
that guarantees serializability in a very scalable fashion. We have
called the algorithm GMU, Genuine Multiversion Update Serializability,
and we've integrated it into Infinispan (5.0.0).
The source code is available on github:
http://github.com/cloudtm/infinispan-5.0.0.SERIALIZABLE
GMU's key features are:
1. unlike any other partial replication protocol we are aware of, GMU is
the first distributed multi-versioned based partial replication protocol
that does not rely on a single global clock in order to determine
consistent snapshots. Conversely, the protocol guarantees to involve
only the nodes that maintain data accessed by a committing transaction T
(a property that is known in literature as "genuineness"). This is a
property that is crucial, in our opinion, to achieve high scalability.
2. read-only tranasctions are never aborted, and do not need to be
validated at commit time, making them very fast. Read-only transactions
are guaranteed to observe a consistent snapshot of the data using a
novel mechanism based on vector clocks. Note that in order to achieve
this results we integrated in ISPN a multiversion concurrency control,
very similar to the one used in PostgreSQL or JVSTM, that maintains
multiple data item versions tagged with scalars per each key.
3. The consistency guarantees ensured by GMU are a variant of classic
1-Copy-Serialiability (1CS), and, more precisely, "extended update
serializable" (EUS). You can check the tech. report in attach for more
details on this, but, roughly speaking, US guarantees that update
transactions execute according to 1CS. Concurrent read-only
transactions, instead, may observe the updates generated by two
*non-conflicting* update transactions in different order.
In practice, we could not think of any realistic application for which
the schedules admitted by US would represent an issue, which leads us to
argue that US is, in practical settings, as good as 1CS, but brings the
key advantage of allowing way more scalable (genuine) implementations.
We have evaluated GMU performance using up to 20 physical machines in
our in-house cluster, and in 40 VMs in the FutureGrid (and we are
currently trying to use more VMs in FutureGrid to see if we can make it
scale up to hundreds of machines... we'll keep you posted on this!) with
the YCSB (https://github.com/brianfrankcooper/YCSB/wiki) and TPC-C
benchmarks.
Our experimental results show that in low conflict scenarios, the
protocol performs as good as the existing Repeatable Read
implementation... and actually, in some scenarios, even slightly better,
given that GMU spares the cost of saving the values read in the
transactional context, unlike the existing Repeatable Read implementation.
In high contention scenarios, GMU does pay a higher toll in terms of
aborts, but it still drastically outperform classic non-genuine MVCC
implementations as the size of the system grows. Also, we've a bunch of
ideas on how to improve GMU performance in high contention scenarios...
but that's another story!
You find the technical report at this url:
http://www.inesc-id.pt/ficheiros/publicacoes/7549.pdf
Comments are more than welcome of course!
Cheers,
Paolo
--
Paolo Romano, PhD
Coordinator of the Cloud-TM ICT FP7 Project (www.cloudtm.eu)
Senior Researcher @ INESC-ID (www.inesc-id.pt)
Invited Professor @ Instituto Superior Tecnico (www.ist.utl.pt)
Rua Alves Redol, 9
1000-059, Lisbon Portugal
Tel. + 351 21 3100300
Fax + 351 21 3145843
Webpage http://www.gsd.inesc-id.pt/~romanop
14 years
benchmarking 5.1 vs 5.0
by Mircea Markus
Hi,
Now that all the 5.1 major work is done I plan to run some benchmarks comparing 5.1 with 5.0. It's not only tx stuff I'd like to compare, as some other were made, so here's my plan of action.
Each of the following benchmarks will be run on local, distributed and replicated caches:
1. non transactional: web session replication[1]
2. transactional (both optimistic and pessimistic): web session replication[2]
3. transactional (both optimistic and pessimistic): tpcc [3]
The difference between 2 and 3 is the fact that 3 induces some key contention between transactions.
Any other suggestion for benchmarking?
Cheers,
Mircea
[1] http://sourceforge.net/apps/trac/radargun/wiki/WebSessionBenchmark
[2] http://sourceforge.net/apps/trac/radargun/wiki/BenchmarkingTransactions
[3] tpcc is Radargun extension developed by Sebastiano Peluso from the CloudTM team. It's now integrated in Radargun, I'm working on adding some documentation around it atm.
14 years
tomcat infinispan session manager
by Zdeněk Henek
Hi,
I am working on prototype of https://issues.jboss.org/browse/ISPN-465.
Currently works basic functionality like create session, add,update,
remove values from session, remove session.
Session could be with or without jvmRoute ... see details
here:https://github.com/zvrablik/tomcatInfinispanSessionManager)
I have a few questions related to infinispan.
1. classloading issue when used only one shared classloader
============================================
Current master (https://github.com/zvrablik/tomcatInfinispanSessionManager)
creates
cache manager per war application. I would like use one shared cache manager.
Is it possible?
I have created branch
(https://github.com/zvrablik/tomcatInfinispanSessionManager/tree/classloader)
where only first InfinispanSessionManager create DefaultCacheManager.
All created caches set explicitly class loader to configuration and
DecoratedCache is used to access cache.
See InfinispanSessionManager.initInfinispan method.
my infinispan configuration (all created caches use default settings):
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.0
http://www.infinispan.org/schemas/infinispan-config-5.0.xsd"
xmlns="urn:infinispan:config:5.0">
<global>
<transport clusterName="tomcatSession"/>
<globalJmxStatistics enabled="true" allowDuplicateDomains="true"/>
</global>
<!-- use only default cache for all caches created by session manger -->
<!-- to specify custom parameters to one cache create named cache with
name _session_attrContainerName
where ContainerName is name of war application -->
<default>
<jmxStatistics enabled="true"/>
<clustering mode="distribution">
<l1 enabled="false" lifespan="600000"/>
<hash numOwners="2" rehashRpcTimeout="6000"/>
<sync/>
</clustering>
<invocationBatching enabled="true"/>
</default>
</infinispan>
I use tomcat 6.0.29 and Infinispan 5.1.Beta5, java 6 update 26 on
Debian stable 64bit.
I get exception on node which should replicate state sent from another node.
See attached tomcat log.
The missing class is available only through testLB war application classloader.
2. locking and distributed transactions
============================
I use FineGrainedAtomicMap to store session attriubtes. I don't use
any locking nor XA transactions.
Do I have to use locking or XA transactions? I think autocommit mode
could be better in this case
and tomcat doesn't have XA manager by default. I am using distributed
transactions only in relational databases.
3. propagating session events to other nodes, sharing session metadata
==============================================
Is it possible to send (broadcast) custom events through Infinispan?
Currently it is possible to remove session from all nodes when session
is removed in any node,
but there must be done more. Session timeout, session custom events ...
other possible aproach is to create separated cache with session metadata.
I think creating new cache for metadata is better approach. I think
there will be
less network traffic when nodes will get information only when requested.
session object could have assigned listeners and broadcast events
which are not related to session attributes.
Thanks for help.
Regards,
Zdenek Henek
14 years
trace vs log.isTraceEnabled
by Mircea Markus
Hi,
I'm not aware of any convention on using trace vs log.isTraceEnabled() to guard the trace statements.
if (trace) log.trace("some request related stuff")
vs
if (log.isTraceEnabled()) log.trace("some request related stuff");
The former is more efficient, but cannot be managed at runtime. It seems to be the preferred form, so shall we stick with it?
Cheers,
Mircea
14 years
Shipping logical address String in Hot Rod protocol v1.1?
by Galder Zamarreño
Hi,
We've been having a discussion this morning with regards to the Hot Rod changes introduced in 5.1 with regards to hashing.
When Hot Rod server is deployed in AS, in order to start correctly, it requires the Hot Rod server to start before any other (clustered) caches in AS. This is because any hashing can only be done once the hash has been calculated on the hot rod endpoint.
Although this can be fixed in a hacky way (have all caches configured to start lazily and let the Hot Rod server start the topology cache and then all defined caches, ugh), we're considering a slight change to Hot Rod protocol v 1.1 (https://docs.jboss.org/author/display/ISPN/Hot+Rod+Protocol+-+Version+1.1) that would solve this problem.
Instead of hashing on the hot rod endpoint address (host:port), we could hash on JGroups's Address UTF-8 toString representation, which is the logical address. The advantage here is any cache can calculate the hash on this from the start, no need to wait for Hot Rod server to start. The downside is that Hot Rod clients need to be aware of this UTF-8 string in order to hash to the same thing, so it'd mean shipping this back to the clients alongside the hot rod endpoint info.
In spite of the added bytes, there's a benefit to this, and that is the fact that we're not tying a specific format to the String. That is, we're just telling the clients to take this UTF-8 string and hash on that. So, it's internal representation could evolve over time with no impact to the client. In the current v 1.1 protocol, clients and servers assume a String of format host:port to do the hashing.
A third way to fix this would have been to have Hot Rod servers run with a different cache manager to the rest (but still same JGroups channel), but according to Dan, AS7 is not capable of doing this.
So, what does people think of this protocol change in v1.1? This version was introduced in 5.1 and since we still have not released yet, this is the right time to consider protocol changes like this. Personally, I'm in favour.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years
Fwd: [JBoss JIRA] (ISPN-1562) Alternative needed for Cache.getConfiguration()
by Pete Muir
All,
Any ideas on the below? Issue is that the sane name for this method is getConfiguration() but this name is already taken. Options I see are:
1) Use another name (ugh)
2) Swap the return types with no deprecation stage (ugh)
Any better ideas?
Begin forwarded message:
> From: Galder Zamarreño (Created) (JIRA) <jira-events(a)lists.jboss.org>
> Subject: [JBoss JIRA] (ISPN-1562) Alternative needed for Cache.getConfiguration()
> Date: 24 November 2011 09:51:41 GMT
> To: pmuir(a)bleepbleep.org.uk
>
> Alternative needed for Cache.getConfiguration()
> -----------------------------------------------
>
> Key: ISPN-1562
> URL: https://issues.jboss.org/browse/ISPN-1562
> Project: Infinispan
> Issue Type: Bug
> Components: Configuration, Core API
> Affects Versions: 5.1.0.BETA5
> Reporter: Galder Zamarreño
> Assignee: Pete Muir
> Fix For: 5.1.0.CR1
>
>
> Provide an alternative way of retrieving a Cache's configuration instead of deprecated Cache.getConfiguration()
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
14 years
PutForExternalRead and autoCommit
by Galder Zamarreño
Hi,
Forcing caches to be either transactional or non transactional caches causes some issues with operations such as putForExternalRead with default configuration options.
Assuming we have a transactional cache, if autoCommit is on (default), putForExternalRead will:
1. Suspend the ongoing transaction
2. Will create a brand new transaction due to implicit transaction creation logic in auto commit.
This is not good.
I don't think we can just disable autoCommit globally if someone calls putForExternalRead because there might other operations not called within a transaction, but there's a point to be made here:
The whole point of calling PFER is to suspend on-going transactions, so it kinda implies that transactions are managed externally already.
If we don't disable autoCommit globally, there's a few things that we should consider doing:
1. Print a WARN if PFER is called and autoCommit is on?
2. Apart from the message, some kind of way for putForExternalRead to instruct the implicit transaction logic to avoid creating a new transaction in this case.
WDYT?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years
Fwd: [jgroups-dev] Incompatible change in 3.1.0
by Bela Ban
FYI
-------- Original Message --------
Subject: [jgroups-dev] Incompatible change in 3.1.0
Date: Mon, 28 Nov 2011 17:36:49 +0100
From: Bela Ban <belaban(a)yahoo.com>
To: jg-users <javagroups-users(a)lists.sourceforge.net>, jg-dev
<javagroups-development(a)lists.sourceforge.net>
I'm working on an issue that requires a change in Message and
RequestOptions: I want to change the 'flags' field from a byte to a
short. I'd changethe follwoing methods and fields:
- Message.OOB, DONT_BUNDLE, NO_FC, SCOPED, NO_RELIABILITY,
NO_TOTAL_ORDER, NO_RELAY would become a short from a byte
- void Message.setFlag(byte flag) --> void Message.setFlag(short flag);
- void Message.clearFlag(byte flag) --> void Message.clearFlag(short flag)
- boolean Message.isFlagSet(byte flag) --> boolean
Message.isFlagSet(short flag)
A typical use case such as:
Message msg=new Message(...);
msg.setFlag(Message.OOB);
would *not* require any change at all.
However, if you have:
byte flags=Message.OOB;
msg.setFlag(flags);
This wouldn't work as 'flags' would have to be a short (or downcast to a
byte).
Also, in RequestOptions, the 'flags' field would have to be changed to a
short (from a byte). RequestOptions.setFlags()/getFlags()/clearFlags()
would be affected. Again, typical code like this:
RequestOptions opts=new RequestOptions(ResponseMode.GET_ALL, 5000,
false).setFlags(Message.OOB)
would not have to be changed at all.
What do people think ? I know this is an API change, although a minor
one, and I wanted to see if anyone's code would break.
I checked Infinispan (master) and this doesn't cause any code breakage.
[1] https://issues.jboss.org/browse/JGRP-1250
14 years
5.1.0.CR1 update
by Galder Zamarreño
Hi all,
How are things looking for CR1? What's currently in your plate for this?
I want to release CR1 asap to test with Hibernate 2LC and use1PcForAutoCommitTransactions.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
14 years
Syncing Infinispan to central
by Pete Muir
All,
Does anyone want to volunteer to run some analysis on Infinispan dependencies to make this happen (it's the only blocker now).
Basically, we need a list of Infinispan dependencies *not* in central.
I can help you with the tricks needed to do this if you have time.
Pete
Begin forwarded message:
> So, I think I've finally fixed all the filters for the org.infinispan artifacts. The *archetype artifacts with SNAPSHOT dependencies have all been filtered and only versions starting with 5.1.0.ALPHA2 and later are going to be synced.
>
> I'm not sure if you want to run some dependency analysis for these artifacts, since, even with the reduced number of version, there are still a large number of them.
>
> Anyway, I'll wait for your go-ahead on the org.infinispan group. In the meantime, I've got some weld and weld dependency groupIds to work through.
14 years, 1 month
