UserTransactionLookup for JSR-107?
by Galder Zamarreño
Hi all,
I'm back with a more food for thought wrt JSR-107 impl. Our CacheManager adapter needs to implement:
UserTransaction getUserTransaction();
The problem there is that there's no standard way of getting a UserTransaction given a JTA TransactionManager.
It really is down to each TransactionManager provider to give a UserTransaction instance (whether JNDI, static…etc).
So, we need a way to lookup a UserTransaction.
One option is to add a getUserTransaction to TransactionManagerLookup, but that will break existing clients.
Alternatively, define a org.infinispan.transaction.lookup.UserTransactionLookup interface which is configurable. We'd then need to implement for existing TML classes.
If anyone has any other ideas, let us know.
Cheers,
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
12 years, 7 months
Rolling upgrades improvement proposal
by Tristan Tarrant
Hi all,
the current implementation of rolling upgrades is lacking in a few areas
which I think warrant urgent attention in the scope of JDG 6.1 (i.e. for
the 5.2.x-cycle):
* support for rolling upgrades from JDG 6.0.x to JDG 6.1.x
* efficiency of the bulk key synchronization (all keys are serialized
into a single key which the target cluster uses to *suck* all data
from the source cluster).
During the Infinispan 5.2 cycle I added support in
RemoteCache+RemoteCacheStores to be able to fetch the entries complete
with metadata (i.e. expiration/maxIdle). Unfortunately this feature is
not available in Infinispan 5.1, and therefore the implementation may
only be able to fetch naked key/values, losing mortality information.
To partially overcome this issue, and also to solve the above efficiency
issue, I would like the synchronization phase of the rolling upgrade to
be a push operation from the source cluster to the target cluster:
instead of dumping the keyset into a single key, the source cluster
would initiate a distributed task which would cycle through the entire
keyset and PUT to the target cluster complete with the correct
expiration information (if the source cluster uses JDG 6.1 this could
probably be a GET, which would trigger the usual getWithMetadata).
Comments, thoughts.
Tristan
12 years, 7 months
Trigger NBST for Auto-Placer
by Pedro Ruivo
Hi,
I'm working in a way to rebase auto-placer on top of NBST and I have one
question...
If you have already forgot, auto-placer analyzes the workload and tries
to move the most remote accessed keys to the corresponding requester.
After calculating the new mappings, I want to trigger the NBST with this
mapping. I'm thinking to add a new method in the ClusterTopologyManager,
something like:
triggerAutoPlacer(String cacheName, Mappings newMappings);
and this method it will be a duplicate of triggerRebalance but instead
of doing chFactory.rebalance(CH) (in the startRebalance() method) I'm
thinking to do chFactory.autoPlacer(CH, Mappings). The last method will
override the defautl CH location.
Question: will this solution trigger the NBST or do I have to create the
triggerAutoPlacer() method in another class?
ps. forget the methods names... I will think in better names later
Thanks!!
Cheers,
Pedro
12 years, 7 months
Threadpools in a large cluster
by Radim Vansa
Hi guys,
after dealing with the large cluster for a while I find the way how we use OOB threads in synchronous configuration non-robust.
Imagine a situation where node which is not an owner of the key calls PUT. Then the a RPC is called to the primary owner of that key, which reroutes the request to all other owners and after these reply, it replies back.
There are two problems:
1) If we do simultanously X requests from non-owners to the primary owner where X is OOB TP size, all the OOB threads are waiting for the responses and there is no thread to process the OOB response and release the thread.
2) Node A is primary owner of keyA, non-primary owner of keyB and B is primary of keyB and non-primary of keyA. We got many requests for both keyA and keyB from other nodes, therefore, all OOB threads from both nodes call RPC to the non-primary owner but there's noone who could process the request.
While we wait for the requests to timeout, the nodes with depleted OOB threadpools start suspecting all other nodes because they can't receive heartbeats etc...
You can say "increase your OOB tp size", but that's not always an option, I have currently set it to 1000 threads and it's not enough. In the end, I will be always limited by RAM and something tells me that even nodes with few gigs of RAM should be able to form a huge cluster. We use 160 hotrod worker threads in JDG, that means that 160 * clusterSize = 10240 (64 nodes in my cluster) parallel requests can be executed, and if 10% targets the same node with 1000 OOB threads, it stucks. It's about scaling and robustness.
Not that I'd have any good solution, but I'd really like to start a discussion.
Thinking about it a bit, the problem is that blocking call (calling RPC on primary owner from message handler) can block non-blocking calls (such as RPC response or command that never sends any more messages). Therefore, having a flag on message "this won't send another message" could let the message be executed in different threadpool, which will be never deadlocked. In fact, the pools could share the threads but the non-blocking would have always a few threads spare.
It's a bad solution as maintaining which message could block in the other node is really, really hard (we can be sure only in case of RPC responses), especially when some locks come. I will welcome anything better.
Radim
-----------------------------------------------------------
Radim Vansa
Quality Assurance Engineer
JBoss Datagrid
tel. +420532294559 ext. 62559
Red Hat Czech, s.r.o.
Brno, Purkyňova 99/71, PSČ 612 45
Czech Republic
12 years, 8 months
Emergency release 5.2.1.Final
by Mircea Markus
Hi,
Erik Salter found and fixes (thanks!) two memory leaks in Infinispan 5.2.0.Final:
- ISPN-2803 Memory leak on every cache write.operation
- ISPN-2804 Memory Leak: TransactionTable never cleans up complete transactions
These issue significantly impact the usability of Infinispan 5.2.0.Final release in any real-life deployment, so we'll release 5.2.1.Final today.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
12 years, 8 months
Fwd: Moving to Java 7?
by Galder Zamarreño
Thoughts?
Begin forwarded message:
> From: Brian Oliver <brian.oliver(a)oracle.com>
> Subject: Moving to Java 7?
> Date: February 5, 2013 9:13:35 PM GMT+01:00
> To: jsr107(a)googlegroups.com
> Reply-To: jsr107(a)googlegroups.com
>
> Currently we are targeting Java 7 for JSR-107.
>
> Given that:
>
> a). Java 6 is end-of-life at the end of February 2013 (see http://www.oracle.com/technetwork/java/eol-135779.html)
> b). We're meant to be working with Java EE 7 (ok, we've slipped but Java EE7 is Java 7+)
>
> I'd like to propose Java 7 as the minimum requirement for JSR-107 (although it may work on Java 6, we just wouldn't guarantee it).
>
> Thoughts?
>
> -- Brian
>
> --
> You received this message because you are subscribed to the Google Groups "jsr107" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jsr107+unsubscribe(a)googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
12 years, 8 months
Codename for Infinispan 5.3.0
by Mircea Markus
Following the tradition, please bring your suggestion of beer-code name for the new Infinispan release. Then we'll vote.
Mine is Guinness :-)
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
12 years, 8 months
scala code in Infinispan
by Mircea Markus
Hi,
The REST module is written in Scala (both main + tests). We have some *test* contributions written in Java (thanks mlinhard).
There was an IRC discussion on whether it's worth migrating the Java contribution to Scala code or not.
Pros for migrating the contribution from Java to Scala:
- the REST module is written in Scala. Contributing these tests in Java would make the module bi-lingual, potentially confusing future contributors
- even though this is not the case with this particular contribution, there might be code duplications between the scala test suite and java test suite.
Cons for migrating the contribution from Java to Scala:
- there are contributors that are not familiar with Scala or are more proficient with Java(such as mlinhard). Forcing them to contribute in a language they are not familiar with would put them off
- my general feeling over time was that people (including me) are not very enthusiastic about debugging and extending Scala code. So IMO if there's a choice between scala and java (in the scope of the scala modules) we should stick to Java wherever possible (such as this contribution).
This email is not about the migration of scala code to java, but about the very specific contribution described above (even thought the mix of scala+java code in ISPN is a a very interesting topic by itself).
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
12 years, 8 months
