Re: [infinispan-dev] Removing Infinispan dependency on the Hibernate-Infinispan module in 4.x
by Galder Zamarreño
Scott, what do you suggest doing instead then? Without the commands, evictAll invalidation won't work.
Are you suggesting that I revert back to using the cache as a notification bus so that regions are invalidated?
On Feb 8, 2012, at 4:13 PM, Scott Marlow wrote:
> http://lists.jboss.org/pipermail/infinispan-dev/2012-February/010125.html has more context.
>
> Since there are no easy/quick fixes that can be applied at this time, to remove the AS7 Infinispan dependency on the Hibernate-Infinispan module, I think we should avoid depending on the service loader way to supply the custom commands (in the Hibernate-Infinispan module), at least until this can be addressed elsewhere.
>
> I propose that the Hibernate-Infinispan second level cache should not use the Service Loader to pass custom commands into Infinispan. If we agree, I'll create a jira for this.
>
> Scott
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
10 years, 11 months
Map Reduce 2.0
by Vladimir Blagojevic
Hey guys,
Before moving forward with next iteration of map reduce I wanted to hear
your thoughts about the following proposal. After we agree on the
general direction I will transcribe the agreed design on a wiki page and
start implementation.
Shortcoming of current map reduce implementation
While our current map reduce implementation is more than a proof of a
concept there are several drawbacks preventing it from being an
industrial grade map reduce solution. The main drawback is the inability
of the current solution to deal with a large data (in GB/TB) map reduce
problems. This shortcoming is mainly around our reduce phase execution.
Reduce phase, as you might know, is currently done on a single
Infinispan master task node; reduce phase of map reduce problems we can
support (data size wise) is therefore shrunk to a working memory of a
single node.
Proposed solution
The proposed solution involves distributing execution of reduce phase
tasks across the cluster thus effectively achieving higher reduce task
parallelization and at the same time removing the above mentioned reduce
phase restriction. Effectively leveraging our consistent hashing
solution even further we can parallelize reduce phase and elevate our
map reduce solution to an industrial level. Here is how we can achieve that.
Map phase
MapReduceTask, as it currently does, will hash task input keys and group
them by execution node N they are hashed to. For each node N and its
grouped input KIn keys MapReduceTask creates a MapCombineCommand which
is migrated to an execution target node N. MapCombineCommand is similar
to current MapReduceCommand. MapCombineCommand takes an instance of a
Mapper and an instance of a Reducer, which is a combiner [1].
Once loaded into target execution node MapCombineCommand takes each
local KIn key and executes Mapper method void map(KIn key, VIn value,
Collector<KOut, VOut> collector). Results are collected to a common
Collector<KOut, VOut> collector and combine phase is initiated. A
Combiner, if specified, takes KOut keys and imediatelly invokes reduce
phase on keys. The result of mapping phase executed on each node is
<KOut, VOut> map M. There will be one resulting M map per execution node N.
At the end of combine phase instead of returning map M to the master
task node (as we currently do), we now hash each KOut in map M and group
KOut keys by the execution node N they are hashed to. Each group of KOut
keys and its VOut values, hashed to the same node, is wrapped with a new
command Migrate. Command Migrate, which is very similar to
PutKeyValueCommand,executed on Infinispan target node N esentially
maintains KOut K -> List<VOut> mapping, i.e all KOut/VOut pairs from all
executed MapCombineCommands will be collocated on a node N where KOut is
hashed to and value for KOut will be a list of all VOut values. We
essentially collect all VOut values under each KOut for all executed
MapCombineCommands.
At this point MapCombineCommand has finished its execution; list of KOut
keys is returned to a master node and its MapReduceTask. We do not
return VOut values as we do not need them at master task node.
MapReduceTask is ready to start with reduce phase.
Reduce phase
MapReduceTask initializes ReduceCommand with a user specified Reducer.
For each key KOut collected from a map phase we group them by execution
node N they are hashed to. For each node N and its grouped input KOut
keys MapReduceTask creates a ReduceCommand and sends it to a node N
where KOut keys are hashed. Once loaded on target execution node,
ReduceCommand for each KOut key grabs list of values VOut and invokes:
VOut reduce(KOut reducedKey, Iterator<VOut> iter).
A result of ReduceCommand is a map M where each key is KOut and value is
VOut. Each Infinispan execution node N returns one map M where each key
KOut is hashed to N and each VOut is KOut's reduced value.
When all ReduceCommands return to a calling node, MapReduceTask simply
combines all these M maps and returns final Map<KOut, VOut> as a result
of MapReduceTask. All intermediate KOut->List<VOut> maps left on
Infinispan cluster are then cleaned up.
[1] See section 4.3 of http://research.google.com/archive/mapreduce.html
11 years, 11 months
Design of temote event handling in Hot Rod
by Galder Zamarreño
Hi all,
Re: https://community.jboss.org/docs/DOC-17571
Over the past week and a bit I've been working on a rough prototype for remote event handling in Hot Rod that covers the server side (I've not done any work on the Hot Rod client).In the link above you can find my design notes.
I wanted to get some feedback on the minimum requirements explained and I wanted to discuss the need of the optional requirements, in particular the 1st of the optional requirements.
The idea is that at a logical level, it'd be interesting to know what the origin of a modification for a couple of reasons:
- It allows clients be able to know whether the modification is originated locally from a logical point of view. If it sounds to abstract, think about near caches (see preso in https://www.jboss.org/dms/judcon/presentations/London2011/day1track2sessi...) and imagine a local cache (near cache) configured with a remote cache store (a Java hot rod client with N channels open at the same time). Remote listeners could decide to act differently if the modification was originated locally or not, i.e. if it's not local then remove the key from cache, if local, it means the modification comes from this remote cache store and so I have the latest data in memory. This is a very nice optimisation for at least this use case.
- This can be extended further. If all channels opened with can be associated with a logical origin, we could optimise sending back events. For example, imagine a remote cache store (it has 1 remote cache manager) that has N channels open with the server. There's no need for all N channels to receive a notification for a cache removal. As long as one of the channels gets it event, it's good enough to act to on the local cache.
As you can see, what I'm heading towards is that for each remote cache manager started to be id'd uniquely, and this id to be shipped with all Hot Rod operations. It could be possible to limit the operations that carry such an id, but this could complicate the protocol.
Thoughts?
Also, any thoughts on the need for the 4th optional requirement? For near caches, remote events are important, but you could limit the need for it with an aggressive eviction policy in the near cache to cover against lost events.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
12 years, 1 month
size of distribution zip
by Sanne Grinovero
It seems the distribution zip is now 130MB, almost twice as it used to be.
There are several duplicate jars: many non-core modules share the same
dependencies, and they are bundled over and over in each module
directory/lib.
Also when one module depends on another, it re-packages again the same
Infinispan module with all of its dependencies again.
It also seems we're shipping the full Cassandra dependency set? How
should we deal with it?
-- Sanne
12 years, 2 months
Infinispan listener events not very intuitive
by Galder Zamarreño
Hi all,
When are we gonna switch to JSR-107 listeners?
As I mention in https://community.jboss.org/message/719245#719245, cache entry created and modified notifications are not very intuitive if you want to do something on cache entry created that you don't wanna do on cache modified, and viceversa, and you want to do such thing when isPre=false (the majority of times tbh).
To give you an real example, Hot Rod remote notifications. I want to send a different notification when a cache entry is created (=insert) as opposed to when cache entry is modified (=update), and of course, I wanna do so *after* the event. How do I code this?
Option 1:
- Implement @CacheEntryCreated and in it send a notification for cache entry created when isPre=false.
- Implement @CacheEntryModified and in it send a notification for cache entry modified when isPre=false.
Problem? If the entry is created, this will generate two notifications (one for created, and one for modified) and @CacheEntryModified can only differentiate between cache entry created vs modified with isPre=true.
Option 2:
- Implement @CacheEntryModified and in it, check whether value is null. If null, entry was created, if not null, entry was modified.
Problem? Only works with isPre=true and this is not good cos you're sending the remote notification before the actual event.
Any other solution requires the listener to keep some state which is ugly to say the least.
If you have any simpler ideas I might have missed, I'm all ears.
Btw, a similar problems happen with cache entry removed and cache entry modified. How do you differentiate between them two when isPre=false?
I haven't checked the JSR-107 listeners yet, but I hope they're more sane in this area.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
12 years, 2 months
CPU and GC diagrams
by Galder Zamarreño
Hey,
First of all, thx for incorporating my memory and CPU usage metrics :)
I have a question though. In http://goo.gl/n65ib diagram, wouldn't it be more useful if you had two lines per test? One for CPU usage and one for GC usage? That way, you could figure out which part of that CPU usage is GC and hence you might be able to map this with a reduction in thoughtput.
In fact, what might be better is if you add GC usage (only!) to throughput diagram and then you can see immediately see if a drop in throughput is due to GC or not.
WDYT?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
12 years, 2 months
Verbose exceptions in test suite runs
by Manik Surtani
These don't cause tests to fail, but I do see a heck of a lot of the following.
Dan, (a) I presume these are harmless? (b) is it absolutely necessary to be this alarming/verbose? :)
Cheers
Manik
2012-02-25 11:35:10,129 ERROR [CacheViewsManagerImpl] (CacheViewInstaller-1,ConcurrentOverlappingLeaveTest-NodeA-41379) ISPN000172: Failed to prepare view CacheView{viewId=5, members=[ConcurrentOverlappingLeaveTest-NodeA-41379, ConcurrentOverlappingLeaveTest-NodeB-58370, ConcurrentOverlappingLeaveTest-NodeC-61261]} for cache dist, rolling back to view CacheView{viewId=4, members=[ConcurrentOverlappingLeaveTest-NodeA-41379, ConcurrentOverlappingLeaveTest-NodeB-58370, ConcurrentOverlappingLeaveTest-NodeC-61261, ConcurrentOverlappingLeaveTest-NodeD-39762]}
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: dist: Received cache view prepare request after the local node has already shut down
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
at java.util.concurrent.FutureTask.get(FutureTask.java:91)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:319)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250)
at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.IllegalStateException: dist: Received cache view prepare request after the local node has already shut down
at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:488)
at org.infinispan.commands.control.CacheViewControlCommand.perform(CacheViewControlCommand.java:125)
at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:95)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:178)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:158)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:447)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:354)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:230)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:542)
at org.jgroups.JChannel.up(JChannel.java:715)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1015)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
at org.jgroups.protocols.FC.up(FC.java:499)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:884)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)
at org.jgroups.protocols.RSVP.up(RSVP.java:189)
at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:435)
at org.jgroups.protocols.pbcast.NAKACK2.handleMessage(NAKACK2.java:690)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:526)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140)
at org.jgroups.protocols.FD.up(FD.java:273)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282)
at org.jgroups.protocols.Discovery.up(Discovery.java:355)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1194)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1749)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1731)
... 3 more
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org
12 years, 2 months
