introducing an artificial network delay
by Sanne Grinovero
Hello,
I'm running some performance tests; my goal is to alter the code for
the Lucene Directory to use optimal batching and Flags, but to make
sure I'm improving performance for DIST and REPL configurations the
only option I'm having is to run it on a real cluster, which is not
doable right now.
So as I'm running tests on my laptop, network IO is not real and I
suppose it's not representative of real life bottlenecks.
Is there some option I could set in the jgroups configuration to
introduce an arbitrary delay, or even some degree of random packet
losses?
Or should I extend the transport to introduce some sleep() in
interesting points?
Maybe adding a slow-down interceptor around remote invocations?
suggestions?
thanks in advance,
Sanne
16 years, 1 month
Securing access to Infinispan REST server
by Galder Zamarreño
Hi,
During my REST/Cloud presentation, I got a particularly interesting question about the Infinispan REST server.
As it is, once the REST module is deployed, anyone can access it as shown in http://community.jboss.org/wiki/AccessingdatainInfinispanviaRESTfulinterface
Now, how would you go about authentication/authorization to access Infinispan via REST?
Since at the end of the day the REST module is a war, users would need to tweak it accordingly in order to configure the security constraints under its web.xml defining the corresponding roles and authentication methods. Wouldn't they?
I don't think it's possible for Infinispan to provide a more restricted Infinispan REST module, but instead some guidelines on how to secure it would be handy.
Thoughts?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years, 1 month
eager locking and deadlocks
by Mircea Markus
Hi,
Whit eager locking enabled, cache.put(k,v) acquires locks as follows:
1. acquire local lock on k
2. acquire remote lock on k
3. if 2 fails release local lock on k
Previous sequence (i.e. local then remote) makes more sense to me than the reversed:
1. acquire remote lock on k
2. acquire local lock on k
3. if 2 fails release remote lock on k
This is because it doesn't make sense to go remotely just then to realise that you cannot acquire a local lock.
However, this approach does not work with deadlock detection[1] which requires the 2nd approach.
Here are my options:
1. forbid deadlock detection on eager locking
2. allow users to use deadlock detection with eager locking and if DLD is on to use the 2nd locking approach, otherwise to use the first one.
3.?
2 is doable, and I think DLD might have a good impact on eager locking, as in this situation the transactions hold locks longer, hence the chance for deadlock is higher.
Wdyt?
Cheers,
Mircea
[1] here is why:
tx1 and tx2 two transactions.
tx1 originates on node A, tx2 originates on node B
Step 1 (simultaneously):
tx1: A.put(k1) -> lock(tx1) = {A_k1, B_k1}
tx2: B.put(k2) -> lock(tx2) = {A_k2, B_k2}
Step 2:
tx1: A.put(k2) -> status(tx1) = Holds lock on {A_k1, B_k1} and tries to lock A_k2
tx2: B.put(k1) -> status(tx2) = Holds lock on {A_k2, B_k2} and tries to lock A_k1
DLD on Tx1 cannot progress: it tries to lock A_k2 and cannot, it is aware that A_k2 is locked by tx2, but it doesn't know that tx2's intention is to lock A_k1(that would mean a deadlock). Why? because tx2 hasn't replicated its intention to the other node as it's waiting to acquire lock on A_k1.
Similar to Tx2.
16 years, 1 month
HotRod client - optimize serialization
by Mircea Markus
Hi,
On HR we serialize the (key, value) pairs before sending them to the client.
Current approach is we transfer them into an byte[] using an ByteArrayInputStream/ByteArrayOutputStream and then write them over the network with socket.getOutputStream().write(byte[])
What I'm looking for a better way of serializing, by reusing byte arrays.
One approach would be to use an pool of ExposedByteArrayOutputStream, pool's size being == number of tcp connections between client and server. My concern with this approach is that if one is using a large value (e.g. 100MB ) once in a blue moon, than I'll always keep an 100MB array in memory, cached, even though I don't want it.
Another approach would be to use use existing code for keys(i.e. serialize them into an byte[]), which are expected to be smaller, and for values to write directly in the socket, through socket.getOutputStream(). This way I won't have the 100MB issue and also I won't create an byte[] for each value (I need to do that for keys though, as I need access to key's byet[] for computing its hash code).
Any suggestions much appreciated!
Cheers,
Mircea
16 years, 1 month
Branch 4.1.x is open for business :)
by Galder Zamarreño
After releasing 4.1.0.CR1, 4.1.x branch is now unfrozen.
Enjoy :)
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years, 1 month
eager locking strategy
by Mircea Markus
Hi,
Whit eager locking enabled, cache.put(k,v) acquires locks as follows:
1. acquire local lock on k
2. acquire remote lock on k
3. if 2 fails release local lock on k
Previous sequence (i.e. local then remote) makes more sense to me than the reversed:
1. acquire remote lock on k
2. acquire local lock on k
3. if 2 fails release remote lock on k
This is because it doesn't make sense to go remotely just then to realise that you cannot acquire a local lock.
However, this approach does not work with deadlock detection[1] which requires the 2nd approach.
Here are my options:
1. forbid deadlock detection on eager locking
2. allow users to use deadlock detection with eager locking and if DLD is on to use the 2nd locking approach, otherwise to use the first one.
3.?
2 is doable, and I think DLD might have a good impact on eager locking, as in this situation the transactions hold locks longer, hence the chance for deadlock is higher.
Wdyt?
Cheers,
Mircea
[1] here is why:
tx1 and tx2 two transactions.
tx1 originates on node A, tx2 originates on node B
Step 1 (simultaneously):
tx1: A.put(k1) -> lock(tx1) = {A_k1, B_k1}
tx2: B.put(k2) -> lock(tx2) = {A_k2, B_k2}
Step 2:
tx1: A.put(k2) -> status(tx1) = Holds lock on {A_k1, B_k1} and tries to lock A_k2
tx2: B.put(k1) -> status(tx2) = Holds lock on {A_k2, B_k2} and tries to lock A_k1
DLD on Tx1 cannot progress: it tries to lock A_k2 and cannot, it is aware that A_k2 is locked by tx2, but it doesn't know that tx2's intention is to lock A_k1(that would mean a deadlock). Why? because tx2 hasn't replicated its intention to the other node as it's waiting to acquire lock on A_k1.
Similar to Tx2.
16 years, 1 month
Changing managed method names
by Galder Zamarreño
Hey,
As a result of changing RpcManagerImpl getAddress method name to getNodeAddress, the following test is failing:
http://hudson.qa.jboss.com/hudson/view/Infinispan/job/Infinispan-4.1.x-JD...
Be careful about changing method names, particularly when these methods are annotated with @ManagedAttribute or JOPR annotations because they affect statistic MBean property names.
By the way, what is the reason why you changed the method name and added a new getAddress method to RpcManager interface and RpcManagerImpl?
That information is available via rpcManager.getTransport.getAddress. Is it to provide some kind of shortcut? DefaultCacheManager uses transport.getAddress() without problems.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years, 1 month
