[Design of JBossCache] - Eager state push on failure with Buddy Replication
by manik.surtani@jboss.com
At the moment, when using buddy replication and a data owner fails, the backup resides on a buddy instance and is only gravitated into the primary tree of an instance when someone makes a request for that data.
It has been designed this way to minimise network traffic and load during the crash of a server, which could cause a network storm if there is a lot of state that would need to be reorganised across a cluster.
Despite that, people have asked for an option to allow an eager push of backup state to a new data owner - or even the assigned buddy taking on the data as though it were the owner - and creating appropriate backups.
Here are some thoughts on how this can be implemented:
* When an instance fails, this option forces the buddy to take ownership of the failed nodeâs state.
* Should wait a defined amount of time first, to allow for gravitation calls to organically move data.
* Donât need to block data gravitation calls when taking ownership since DG will look in both primary and backup trees
* Could be in chunks to prevent a network storm (since the new node taking ownership will be backing stuff up as well). Would need some additional "hints" as to which subtrees should be considered "related" data though.
* When an instance failure is detected, buddies should rename the region such to prevent the original instance re-appearing and overwriting backed up state. E.g., rename /_B_B_/CacheA/ to /_B_B_/CacheA:dead_n/ where n is a counter since A could die and rejoin several times before the state is re-owned.
Thoughts and comments? How important/useful do you think this is in the first place?
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4135090#4135090
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4135090
18 years, 1 month
[Design of JBossCache] - Eager deadlock detection with 2 phase commits
by manik.surtani@jboss.com
At JBoss World, Jason had proposed a mechanism for eager deadlock detection when using a 2 phase commit and pessimistic locking or MVCC.
A description of the problem:
* Server 1 updates /a, /b and broadcasts a prepare.
* Server 2 updates /a, /c and broadcasts a prepare at the same time.
---> Deadlock, even though both txs succeed in getting local locks.
With OL, one will fail, with PL, both will fail after TIMING OUT (LockAcquisitionTimeout millisecs), which can be unnecessarily slow. MVCC will have the same behaviour as PL since MVCCâs write lock strategy is similar to PL.
Eager Deadlock Detection (EDD) requires:
* lock() method on the lock manager to inspect other owners of the lock attempting to be acquired if the lock CANNOT be acquired.
* Can be achieved with acquire(someSmallNumber); in a loop to achieve the real timeout required:
| lock(int timeout)
| {
| while (currentTime < startTime + timeout)
| {
| if (acquire(smallTimeout)) break;
| testEDD();
| }
| }
|
Assume we have 2 servers, S1 and S2. S1 has thread TL1 which is a thread pertaining to a local transaction. S1-TL1 will have a corresponding thread, S2-TR1 pertaining to the remote prepare broadcast to S2. Similarly, S1-TL2 is a local prepare on S2, and corresponds to S1-TR2.
This process will be followed by either S1-TR2 or S2-TR1.
1) If the lock is owned and the current thread originates remotely, inspect the owner.
2) If the owner is a LOCAL tx, inspect itâs state. If it is COMMITTING or COMMITTED, then there will definitely be a deadlock, since the remote tx wants a lock on a node that has already been locally locked and is committing - which means the local tx thread will fail on other remote nodes for the same reason.
3) If the deadlock is detected abort the local transaction. (e.g., if the current thread is S1-TR2, S2-TL2 will be forced to roll back). This will allow S2-TR1 to get the remote locks it needs, allowing S1-TL1 to succeed.
4) Only one instance should do this with the local transaction, not both (since otherwise both will not succeed). I.e., only S2-TR1 or S1-TR2 should follow this process; not both.
5) Sufficiently randomise the probability of "winning" in a deadlock by each node sending out a "coin toss" - a random number generated for each transaction and stored in the TransactionEntry. This way all cache instances know, deterministically, whether it will yield it's local transaction during a deadlock. I.e., both instances know which of the two threads, S1-TR2 or S2-TR1, will yield.
The upside of such an approach is that the deadlock can be determined very quickly (milliseconds?) and one of the participants in the deadlock forced to abort, allowing at least one transaction to complete quickly.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4135088#4135088
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4135088
18 years, 1 month
[Design of JBossCache] - Fqns containing just Strings
by manik.surtani@jboss.com
We've spoken about this at length in the past, and got to the point where we'd be happy with Fqns containing just Strings and Java primitives.
After discussing things further at JBoss World, we really should just limit Fqn elements to Strings. If people wish to use other types, they could implement their own Fqn subclasses that can encode/decode primitives, etc. into String representations.
Here is a summary of the discussions - please feel free to comment.
Purpose: partly to improve performance with Fqn usage in Maps and the calculation of Fqn hash codes, as well as frequent creation.
1. Remove Fqn generics!
This was introduced in JBC 2.0.0 and has added an unnecessary layer of complexity for little benefit as far as Fqns are concerned.
2. Fqns should only contain String elements
If this were to contain primitives, these would be encoded as Strings internally anyway, so may as well let users do this themselves
Perhaps as a subclass of Fqn (but make sure we make some internal methods final)
3. Change Node.name to be a String as well instead of an Object, since Fqns would only contain Strings.
4. Fqn.fromString() should escape "invalid" characters such as "/" to disambiguate from a String containing such characters.
5. All Fqn constructors to be private (or protected). Instantiation via factory methods only.
Reduces unnecessary object construction if we maintain a weak hashmap of pre-known Fqns and we can reuse these since Fqns are immutable.
Need to be careful of and test for mem leaks!
6. Profile to see if a search-and-get or search-and-instantiate-and-put is actually faster (on average) than outright construction, to validate the above approach.
7. Fqn factory method to allow for String varargs and Fqn + String vararg signatures.
8. All escaping of Fqn strings happens internally.
9. Fqns hold a ref to the String elements (ArrayList) as well as a complete String rep, for quick hashCode() calculations and equals() comparisons.
10. Expose an âintern()â method akin to String.intern() with the same effects and caveats about permgen space.
11. Always intern Fqn.ROOT and other internal Fqn constructs like /_BUDDY_BACKUP_/ by default.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4135087#4135087
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4135087
18 years, 1 month
[Design the new POJO MicroContainer] - Re: Updated Codebase
by adrian@jboss.org
"adrian(a)jboss.org" wrote :
| * Alias tests (expected)
|
I've fixed this so now the testsuite is back at 100% for Kernel!!! :-)
I've reworked the way aliases are parsed.
Instead of the crippled:
| <alias class="whatever">cant do much</alias>
|
| |
| | you can now do
| |
| | | <alias><javabean ...></alias>
| | |
| |
| | NO POINT RE-INVENTING THE WHEEL :-)
| |
| | Which means you can parse anything you like,
| | it still seems pointless as feature to me. ;-)
| |
| | However, to make it work, I had to put a horrible hack in the AbstractKernelControllerContext, because the aliases need to be determined
| | early and the basic BeanMetaData only accepts plain Objects.
| |
| |
| | |
| | | /**
| | | * Determine the aliases
| | | *
| | | * @return the aliases
| | | */
| | | private static Set<Object> determineAliases(BeanMetaData metaData)
| | | {
| | | if (metaData == null)
| | | return null;
| | |
| | | // FIXME THIS IS HACK
| | | if (metaData instanceof AbstractBeanMetaData)
| | | {
| | | AbstractBeanMetaData abmd = (AbstractBeanMetaData) metaData;
| | | Set<AliasMetaData> aliasMetaDatas = abmd.getAliasMetaData();
| | | if (aliasMetaDatas != null && aliasMetaDatas.isEmpty() == false)
| | | {
| | | Set<Object> aliases = abmd.getAliases();
| | | if (aliases == null)
| | | {
| | | aliases = new HashSet<Object>();
| | | abmd.setAliases(aliases);
| | | }
| | | for (AliasMetaData aliasMetaData : aliasMetaDatas)
| | | aliases.add(aliasMetaData.getAliasValue());
| | | }
| | | }
| | | return metaData.getAliases();
| | | }
| | |
| |
| | I'd suggest that if we are going to continue to support AliasMetaData
| | then BeanMetaData becomes
| |
| | | - Set<Object> getAliases();
| | | + Set<AliasMetaData> getAliases();
| | |
| |
| | Personally, I'd rather just drop it and have aliases as plain strings in the xml. ;-)
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4135072#4135072
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4135072
18 years, 1 month