7:11 p.m.
Hello,
As part of the LEADS project (http://www.leads-project.eu), I
implemented a factory of atomic objects on top of Infinispan. The code
is available in the classes AtomicObject*.java stored in the directory
https://github.com/otrack/Leads-infinispan/tree/master/core/src/main/java...
of my Github clone of infinispan. The tests are written in the class
AtomicObjectFactoryTest.java, available under
core/src/test/java/org/infinispan/atomic. Besides, you may find below a
simple code snippet:
AtomicObjectFactory factory = new AtomicObjectFactory(c1); // c1 is both
synchronous and transactional
Set set = (Set) factory.getOrCreateInstanceOf(HashSet.class, "k"); // k
is the key to store the variable set inside the cache c1
set.add("smthing"); // some call examples
System.out.println(set.toString())
set.addAll(set);
factory.disposeInstanceOf(HashSet.class, "set", true); // to store the
object in a persistent way
The pattern I followed to implement this facility is the state machine
replication approach. More precisely, the factory is built on top of the
transactional facility of Infinispan. When the factory creates an
object, it stores a local copy of the object and registers a proxy as a
cache listener. A call to some method of an object is serialized in a
transaction consisting of a single put operation. When the call is
de-serialized it is applied to the local copy, and in case the calling
process was local, the result of the call is returned (this mechanism is
implemented via a future object). Notice that this implies that all the
arguments of the methods of the object are serializable, as well as the
object itself. The current implementation supports the retrieval of
persisted objects, an optimization mechanism for read operations, and is
elastic (an atomic object supports on the fly the addition/removal of
local and/or distant threads accessing it).
I hope that this work might interest you.
Best,
Pierre
10:36 a.m.
Hi Pierre,
First of all, thanks a lot for your involvement and for implementing this interesting new
functionality :)
I've made some comments below, but really, the best way to discuss this would be for
you to send us a pull request and we use that as a way to comment on a line-per-line
basis. Much easier :)
On Aug 19, 2013, at 7:11 PM, Pierre Sutra <pierre.sutra(a)unine.ch> wrote:
Hello,
As part of the LEADS project (http://www.leads-project.eu), I
implemented a factory of atomic objects on top of Infinispan. The code
is available in the classes AtomicObject*.java stored in the directory
https://github.com/otrack/Leads-infinispan/tree/master/core/src/main/java...
of my Github clone of infinispan.
^ It'll be easier for you to keep up with the latest master changes in upstream if you
work on topic branches instead of your fork's master [1]
The tests are written in the class
AtomicObjectFactoryTest.java, available under
core/src/test/java/org/infinispan/atomic. Besides, you may find below a
simple code snippet:
AtomicObjectFactory factory = new AtomicObjectFactory(c1); // c1 is both
synchronous and transactional
Set set = (Set) factory.getOrCreateInstanceOf(HashSet.class, "k"); // k
is the key to store the variable set inside the cache c1
^ Rather than pushing casting to users, it'd be better to do it internally, i.e. with
a definition like this:
<T> T getOrCreateInstanceOf(Class<T> clazz,…
Also, getOrCreateInstanceOf name feels a bit verbose to me. Why not just call it
'create'? Alternatively, we already have code to create atomic maps in
AtomicMapLookup. We might wanna merge both into a single factory/lookup class. Not a
biggie though...
set.add("smthing"); // some call examples
System.out.println(set.toString())
set.addAll(set);
factory.disposeInstanceOf(HashSet.class, "set", true); // to store the
object in a persistent way
The pattern I followed to implement this facility is the state machine
replication approach. More precisely, the factory is built on top of the
transactional facility of Infinispan.
^ Why do you need transactions? Can you just call put?
When the factory creates an
object, it stores a local copy of the object and registers a proxy as a
cache listener.
Hmmm, javassist based proxying. This needs some thought and consideration. I'm not a
huge expert on this topic, but we have Hibernate crew listening here which might be able
to provide feedback on how to best bridge over, since it's something they've been
doing since day one.
A call to some method of an object is serialized in a
transaction consisting of a single put operation.
If you're just calling a single cache operation, don't see the reason for a
transaction. You still get notifications from other servers if something fails. The only
difference is lack of of rollback if something goes wrong in one of the servers and it
works fine in others. Not using transactions makes things work faster.
When the call is
de-serialized it is applied to the local copy, and in case the calling
process was local, the result of the call is returned (this mechanism is
implemented via a future object).
^ Why a future?
Notice that this implies that all the
arguments of the methods of the object are serializable, as well as the
object itself.
Serializable, or somehow marshallable… [2]
The current implementation supports the retrieval of
persisted objects, an optimization mechanism for read operations,
^ Can you explain this read optimization in more detail? Why do you need it?
I'm also seeing a lot of marshalling in your code, which I don't understand why
you need it at all
and is
elastic (an atomic object supports on the fly the addition/removal of
local and/or distant threads accessing it).
I hope that this work might interest you.
Cheers,
Best,
Pierre
[1]
https://docs.jboss.org/author/display/ISPN/Contributing+to+Infinispan#Con...
[2]
https://docs.jboss.org/author/display/ISPN/Plugging+Infinispan+With+User+...
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
12:33 p.m.
Hello,
^ It'll be easier for you to keep up with the latest master
changes in
upstream if you work on topic branches instead of your fork's master [1]
If I
understand correctly, this does not provide me with a copy on a
remote stable storage. Besides, I am not the only one to work on these
modifications.
^ Rather than pushing casting to users, it'd be better to do it
internally, i.e. with a definition like this: <T> T
getOrCreateInstanceOf(Class<T> clazz,…
It was not meant to be the final API,
and I will change this. (despite I
am not found of Java generics)
Also, getOrCreateInstanceOf name feels a bit verbose to me. Why not just call it
'create'? Alternatively, we already have code to create atomic maps in
AtomicMapLookup. We might wanna merge both into a single factory/lookup class. Not a
biggie though...
It is named getOrCreateInstanceOf because you may create a new object
but also, in case the object exists already in the system, receive a
consistent copy of it.
The two approaches of building maps are radically different, and they
will give radically different results. Nevertheless, I agree that
interfacing the two constructs with a single API would be of interest.
This is something I will eventually investigate.
> set.add("smthing"); // some call examples
> System.out.println(set.toString())
> set.addAll(set);
> factory.disposeInstanceOf(HashSet.class, "set", true); // to store the
> object in a persistent way
>
> The pattern I followed to implement this facility is the state machine
> replication approach. More precisely, the factory is built on top of the
> transactional facility of Infinispan.
^ Why do you need transactions? Can you just call put?
The approach I am following to implement linearizable (aka. atomic)
objects [1] is a variation of state machine replication [2]. At core,
state machine replication requires total order broadcast to propagate
the modifications - a property given by a push in transactional mode in
Infinispan.
> When the factory creates an
> object, it stores a local copy of the object and registers a proxy as a
> cache listener.
Hmmm, javassist based proxying. This needs some thought and consideration. I'm not a
huge expert on this topic, but we have Hibernate crew listening here which might be able
to provide feedback on how to best bridge over, since it's something they've been
doing since day one.
This approach seems to do a fairly good job and is both fast and
invisible to the user.
If you're just calling a single cache operation, don't see
the reason for a transaction. You still get notifications from other servers if something
fails. The only difference is lack of of rollback if something goes wrong in one of the
servers and it works fine in others. Not using transactions makes things work faster.
I could use a non-transactional cache, provided I have access to an
atomic broadcast primitive. For the moment, I did not find a
non-intrusive solution to this issue.
> When the call is
> de-serialized it is applied to the local copy, and in case the calling
> process was local, the result of the call is returned (this mechanism is
> implemented via a future object).
^ Why a future?
The result of the call is received via a call-back triggered by
modifications on the object's key.
> Notice that this implies that all the
> arguments of the methods of the object are serializable, as well as the
> object itself.
Serializable, or somehow marshallable… [2]
I will add this feature to the code
(e.g., by giving a marshaller object
to the factory).
> The current implementation supports the retrieval of
> persisted objects, an optimization mechanism for read operations,
^ Can you explain this read optimization in more detail? Why do you need it?
I'm also seeing a lot of marshalling in your code, which I don't understand why
you need it at all
The idea is to optimistically execute each operation locally
(marshalling here is employed to obtain a deep copy of the object). If
the operation does not modify the object, then it is safe to return the
result of its tentative execution to the caller. Notice here that,
strictly speaking, activating this feature makes the object not atomic
but sequentially consistent [3].
Best,
Pierre
[1] http://en.wikipedia.org/wiki/Linearizability
[2] http://en.wikipedia.org/wiki/State_machine_replication
[3] http://en.wikipedia.org/wiki/Sequential_consistency
2:05 p.m.
On 31 août 2013, at 12:33, Pierre Sutra <pierre.sutra(a)unine.ch> wrote:
> ^ It'll be easier for you to keep up with the latest master
changes in
> upstream if you work on topic branches instead of your fork's master [1]
If I understand correctly, this does not provide me with a copy on a
remote stable storage. Besides, I am not the only one to work on these
modifications.
I am not following your remark about remote stable storage completely but working on a
topic branch is not different than working on master. It is literally just a different
name. The big thing is that you then are able to work on different topics. Wen you need to
work on a new topic, create a branch off upstream's master.
Wen you are ready to share - ie asap - simply push the topic branch on your github fork.
The reviewer will then take your branch, rebase it off master if changes have happened
since then, and push the new master.
If others work on your topic branch, they can add your remote repo, push their changes and
offer you a pull request on your fork. I many ways, it is no different than the initial
topic branch.
A few useful links
http://in.relation.to/Bloggers/HibernateMovesToGitGitTipsAndTricks
http://emmanuelbernard.com/blog/2010/05/31/git-how-my-life-has-improved-s...
3898
days inactive
3910
days old
infinispan-dev@lists.jboss.org
3 comments
3 participants
participants (3)
-
Emmanuel Bernard -
Galder Zamarreño -
Pierre Sutra