Configuration visitor - Re: [JBoss JIRA] Commented: (ISPN-145) No transport and singleton store enabled should not be allowed
by Vladimir Blagojevic
Hi,
Galder and I talked about this offline. Time to involve you guys!
I just completed visitor pattern for our configuration objects. Visitor
is passed from root of configuration - InfinispanConfiguration object.
InfinispanConfiguration class has a new method:
public void accept(ConfigurationBeanVisitor v)
How do we want to integrate this visitor into existing structure?
1) We add a new factory method to InfinispanConfiguration with
additional ConfigurationBeanVisitor parameter
2) We leave everything as is and if there is a need to pass some visitor
we pass it to InfinispanConfiguration instance directly (from
DefaultCacheManager)
DefaultCacheManager will pass ValidationVisitor to
InfinispanConfiguration that will verify configuration semantically.
Regards,
Vladimir
On 09-09-09 10:19 AM, Galder Zamarreno wrote:
> Good idea :)
>
> On 09/09/2009 04:13 PM, Vladimir Blagojevic wrote:
>> Yeah,
>>
>> I was thinking that we can make a visitor for configuration tree and
>> then you can do verification of any node and other things as well. Use
>> cases will come up in the future for sure.
>>
>> Cheers
>>
>>
>>
>> On 09-09-09 3:29 AM, Galder Zamarreno (JIRA) wrote:
>>> [
>>> https://jira.jboss.org/jira/browse/ISPN-145?page=com.atlassian.jira.plugi...
>>>
>>> ]
>>>
>>> Galder Zamarreno commented on ISPN-145:
>>> ---------------------------------------
>>>
>>> Not sure I understand what you mean by generic though. You mean any
>>> component to have a validation step of some sort?
>>>
>>> Thanks for taking this on :)
>>>
>>>> No transport and singleton store enabled should not be allowed
>>>> --------------------------------------------------------------
>>>>
>>>> Key: ISPN-145
>>>> URL: https://jira.jboss.org/jira/browse/ISPN-145
>>>> Project: Infinispan
>>>> Issue Type: Bug
>>>> Components: Loaders and Stores
>>>> Affects Versions: 4.0.0.ALPHA6
>>>> Reporter: Galder Zamarreno
>>>> Assignee: Vladimir Blagojevic
>>>> Priority: Minor
>>>> Fix For: 4.0.0.CR1
>>>>
>>>>
>>>> Throw configuration exception if singleton store configured without
>>>> transport having been configured.
>>>> It makes no sense to have singleton store enabled when there's no
>>>> transport.
>>
>
12 years, 6 months
abstracting key locating, asymmetric clusters
by Christian Kulenkampff
Hi all,
I just want to share some thoughts on Infinispan. Probably you already
had similar ideas, maybe you can provide some keywords, so I can
search for discussions in the archive.
As I understand, the ConsistentHash works just as a consistent hash
based "locator" for keys in the cluster. Basically one could implement
full replication with a special ConsistentHash that always returns all
addresses in the cluster. A ConsistentHash/"locator" has to be uniform
over the entire cluster to make it work. In principle this is the only
requirement for a "locator".
Currently there is only one parameter for the "locator" - a set with
all nodes in the cluster. If there would be a way to pass (maybe
dynamically) more parameters to a "locator" in a consistent way, many
possibilities would open up: e.g. an asymmetric cache configuration by
feeding the locator with a distributed registry; switching hash
functions at runtime; subscribable keys; keys, that have to stay at a
special node... Also it may be possible to remove the replication mode
and instead provide a special locator. To make this work rebalancing
would have to be more fine grained. E.g. it should be possible for the
locator (or its manager?) to ignore a view change or to trigger some
rebalance work per cache.
What do you think about it?
Cheers,
Christian
12 years, 9 months
Master is now 5.1.0; 5.0.0 is on branch 5.0.x
by Manik Surtani
Guys, pls note $SUBJECT.
You should:
1. Update any scripts you have to be aware of 5.0.x as a branch
2. Synchronize your fork using sync_with_upstream from https://github.com/maniksurtani/githelpers (make sure you update the script as per step 1)
3. Rebase any of your topic branches. Stash any unsaved work, $ git rebase master if your topic branch is for 5.1. You don't need to rebase if your topic branch was for 5.0.
4. Un-stash your work.
Check all of your branches using
$ git log --pretty=oneline --graph --abbrev-commit --decorate
to make sure they are what you expect. :)
Cheers
Manik
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Lead, Infinispan
http://www.infinispan.org
12 years, 9 months
Data center replication
by Erik Salter
Hi all,
Bela was kind enough to have a discussion with me last week regarding my data center replication requirements.
At a high level, I have 3 independent data centers (sites A, B, C). The latency between data centers is high. (~200ms round trip) So initially I was thinking about using a backing store (like Cassandra) to handle the replication between data centers. Each center would have its own individual grid to manage "local" resources. So when a local TX is committed successfully, it is replicated to the stores in the other data centers. That way, on a data center failure, the requests can be directed to the other data centers by loading from the backing store.
The largest drawback: Certain distributed applications require highly serialized access to resources in the grid. This means lots of explicit locking of keys in a single transaction. In the event that a request is directed to, say, Data Center B because of an intermittent failure of Data Center A, as it stands there exists the possibility that a stale resource could still be resident in that grid. It naturally follows that there will have to be application logic to for the grid in each data center to know which resources it "owns". And once the backing store gets an update from another data center, it will need to aggressively evict non-owned resources from the grid.
I (and the customer) would like to use a single data grid across multiple data centers. Bela detailed an option based off of JGroups RELAY that is a candidate solution.
- When doing a 2PC, Infinispan broadcasts the PREPARE to all nodes (in A, B and C). *However*, it only expects responses from *local* nodes (in this case nodes in data center A). Infinispan knows its own siteId and can extract the siteId from every address, so it can grab the current view (say A1, A2, A3... A10, B1-B10, C1-C10) and remove non-local nodes, to arrive at a sanitized list A1-10. This means it expects responses to its PREPARE message only from A1-10. When it receives a response from non-local nodes, it simply discards them.
- On rollback, a ROLLBACK(TX) message is broadcast to the entire virtual cluster (A, B and C)
- On commit, a COMMIT(TX) is broadcast to the entire virtual cluster (A, B and C).
The downside here is that the 2PC won't be atomic, in the sense that it is only atomic for A, but not for B or C. A PREPARE might fail on a node in B and C, but the 2PC won't get rolled back as long as all nodes in A sent back a successful PREPARE-OK response. This is the same though in the current solution.
Comments? Thoughts?
Erik Salter
esalter(a)bnivideo.com
Software Architect
BNI Video
Cell: (404) 317-0693
________________________________
The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message.
12 years, 9 months
poll on self-tuning ISPN / JGROUPS paramters
by Paolo Romano
Hi,
in the last Cloud-TM's meeting in Rome we have discussed the idea of
posting a poll asking to ISPN's users which parameters they consider
more complex to tune, or which parameters they consider as more
important to be self-tuned.
Before we post the poll, however, we thought to first ask you if you
think that we could suggest also different parameters.
The current list of parameters includes:
- size of the marshalling buffer
- frequency with which replicationQueue (for asynchronous acks) is
processed:
- clustering->async-> replQueueMaxElements
- clustering->async-> replQueueInterval
-number of max concurrent transactions (currently this is
unbounded, but we could self-tune the multi-programming level to
maximize performance depending on the number of available cores, the
actual conflict rate among transactions etc.)
-lock striping, also adapting size of lock pool (striping can lead
to false sharing, but reduce the memory footprint and the size of the
exchanged messages)
-adaptive timeout for deadlock detection, based on the avg duration
of a transaction/conflict rate etc
Any suggestions?
Cheers,
Paolo
12 years, 9 months
State transfer testing
by Galder Zamarreño
Hi all,
I've noticed a problem with the way we test state transfer in our testsuite. For example, take https://github.com/infinispan/infinispan/blob/master/core/src/test/java/o...
This test checks that when a new node is started, state transfer happens. But, it could happen that a merge happens instead of a join, so if a merge happens no state transfer occurs.
Now, the problem is that waiting for the view to be set happens in the main thread, and the callback to merge view listener happens in a different thread. So, in an unfortunate situation, the following can happen:
1. [main-thread] waits for view to be set.
2. [main-thread] view is set due to a merge and main thread carries on.
3. [main-thread] checks the merge view listener and sees that
4. [callback-thread] calls MergedViewListener.mergedView and sets merged=true.
I've seen this failure happening in my local machine when trying to replicate other random failures.
So, I'm solving this issue by having a listener that listens for both merge and view changes, and then having a latch that can waits for either one of the two callbacks to happen. Clearly, the countDown() would happen after either merged (boolean) or viewChanged(boolean) have been set, so that gives the guarantees that either a merge happened or not and then I can check the initial data if necessary. I'll send a pull req later on today with this.
Btw, you might be wondering how on earth a merge would happen with our new TEST_PING? I have that question too and seems like sometimes Discovery.sendGetMembersRequest does not get called and my TEST_PING implementation assumes that will definitely get called. I've sent an email to Bela and I'm gonna try to add some more debugging to find out how on earth this happens.
Now, what I'm wondering here is whether this is something the end users would be interested as well cos they might be running their own testing to check whether state transfer works as expected.
It's at this point that I miss tuples in java cos I think it'd be handy to have a getMembers() call that returns not only a List<Address> but also an enum value indicating whether the last view change was a merge or view change, or more simply a boolean indicating whether the last view change was a merge or not:
(boolean, List<Members>) getMembers();
Unfortunately Java does not make it easy to return things like this. Having a separate method to find out if the last view change was a merge or not would be clunky cos you'd want a single method that can provide the guarantees with regards to the member list returned.
Any thoughts on this API enhancement? Would it be useful? In the Java world, it would require creating a new type which is a bit of a deterrent for me.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
12 years, 9 months
State transfer is transferring the same data twice
by Dan Berindei
Hi guys
While fixing https://issues.jboss.org/browse/ISPN-1243 I found that in
certain cases state transfer will copy the in-memory data twice: first
when copying the in-memory data and then again while copying the
persistent data.
I was getting duplicate key exceptions in
StateTransferManagerImpl.applyPersistentState(), so I changed
StateTransferManagerImpl.applyInMemoryState() to skip writing to the
cache store if persistent state transfer is enabled and passivation is
disabled. That seems to work, but I'm wondering if there are other
cases that I missed.
For 5.1, since we're going to use the rehashing code, we can check if
a key is in memory and not include it again in the persistent state,
but that seems too intrusive to include in 5.0. WDYT?
Cheers
Dan
12 years, 9 months
New documentation type - Glossary
by Pete Muir
I've added a new docs category - Glossary. This is for defining terminology we use that new users (and experienced users) might have trouble with. "Data Grid" would be a great example.
For each item in the glossary, create a page underneath Glossary, and then we can link it in using an include. An item in the Glossary should use an h2. heading and include the heading in the text body as well as have a page title - this just makes the item reusable.
12 years, 9 months
