multi-mapping with indexing - do we need big-table
by kapil nayar
We have two data sets {A1, A2, A3...} and {B1, B2, B3...}
Each B has some associated data {C1, C2, C3....} which has 1:1 mapping.
The mappings would be something like (assume that C would be stored along
side B):
A1-> B1, B2
A2-> B3, B5
A3-> B4, B6, B7
Now, we would need the following indexes:
A->B and B->A
Notice, that both are unique mappings. However, as shown A has multiple
mappings to B.
The big-table type of data structure allow this and make it pretty easy off
the shelf.
Now, I am trying to explore if we can implement these mappings with
Infinispan.
We may need a basic multi-map - to store multiple values for the same key in
the cache.
1. The "get" would return the complete list of the values.
2. The "put" would add the new value without replacing the existing value.
3. The "remove" would remove a specific value or optionally all values
associated with the key.
4. These operations (especially "put") on the same key can occur
simultaneously from multiple nodes.
I know there is an atomic map option in Infinispan which may be applicable,
but AFAIK it requires transactions (which we want to avoid..).
Alternatively, perhaps Infinispan (in combination with lucene) can be used.
1. We should be able to create data structure {B, C} and store A-> {B,C}
with indexes defined for B.
2. Also, the key A could be structured as a combination of A+B to store
multiple entries like A1B1->{B1,C1} and A1B2->{B2,C2}. Lucene would allow
wild carded searches. e.g. To look for all A1 values we could do something
like A1* which should return both A1B1 and A2B2....I may be making some
assumptions here (feel free to correct!)
3. There seems to be one bottleneck though - since the cache mode is
"distribution", it seems it is mandatory to use a backend DB to store these
indexes and moreover the DB needs to be shared. This requirement actually
seems to defeat the purpose of using Infinispan.
Any ideas for achieving this implementation would be greatly appreciated.
For reference this email is further to the user forum thread
http://community.jboss.org/message/622996#622996
Thanks,
Kapil
12 years, 7 months
build broken
by Sanne Grinovero
Hi all,
I know nobody cares about Query.. but some of the commits of
today/yesterday broke it.
I think both the pull-requestor and the reviewer should run a full
build before allowing any change to be pushed upstream; with a script
like this one:
https://gist.github.com/789588
it won't even prevent you to work on a different branch while tests
test are run, so while it takes some CPU, you can still look at other
stuff.
Sanne
12 years, 7 months
batching and auto-commit
by Mircea Markus
Hi,
ATM I cannot enable both batching and auto-commit[1] because the way the batching is implemented:
- it starts a tx, suspends it and and holds it in a thread local so that when a put arrives it can resume it
- when I do a put in a batch, the auto-commit code which runs first doesn't see any tx associated with the thread and starts and commits a new tx
Is there any reasons why the batch container/interceptor doesn't want to expose the batch induced transaction to the outside world? The only drawback I see with that is if some other XA resource is used within the batch, it will participate in the dist tx.
[1] auto-commit is a new feature in 5.1 which injects a tx for transactional caches so that user won't have to start/stop one for single key operations.
Cheers,
Mircea
12 years, 7 months
Infinispan security?
by Joni Hahkala
Hi,
I was reading and watching presentations of Infinispan and it seems that
currently it is intended for use in secure environment, like data center
behind a firewall with other datacenters connected through secure links,
if I understood correctly. But deploying it in more open environment,
e.g. public cloud, could pose security risks. Manik said in a
presentation that the underlying Jgroups uses certificates (or can be
configured to use), and I would assume SSL. So, there is at least some
security in the Infinispan joins, leaves etc. Manik also told that there
has been some talk/plans already about the security in general.
I would be interested in hearing about these plans for security and to
see if there is possibilities for cooperation. I'm currently searching
for a PhD subject, I have background in grid security, and this work
sounds like it could be useful and interesting.
Cheers,
Joni
12 years, 7 months
configuring transactional caches
by Mircea Markus
Hi,
ATM a cache is marked as transactional by setting the transactionalManagerLookup, transactionalManagerLookupClass or enable batching.
This is a bit hard to explain/clumsy and also doesn't work well with the xml config: if one wants the default cache to be transactional and a named cache not to be, it would have to specify an empty transactionManagerLookup attribute - not nice.
So an alternative would be to have an explicit "isTransactional" configuration.
One way to do it is to add a "mode" attribute under the transactional element:
<transaction mode="NON_TRANSACTIONAL | TRANSACTIONAL_SYNCHRONIZED | TRANSACTIONAL_FULL_XA | TRANSACTIONAL_XA_NO_RECOVERY" />
Somehow similar to the clustering element's mode attribute.
Pete is there an JSR-107 for the above enum? I looked for it in both the spec and API but couldn't find one...
How does this sound?
Cheers,
Mircea
12 years, 8 months
mixing optimistic and explicit locking
by Mircea Markus
Hi,
Do we want to support explicit locking within optimistic locking scheme? This is something that is currently supported, i.e. one can used the advanced cache's lock method on an optimistic cache.
My concern is mainly related to the existing use cases, is there a need for allowing this?
Cheers,
Mircea
12 years, 8 months
