TimedOutExceptions from CassandraCacheLoader
by James Aley
Hey guys,
Not sure if this is the most appropriate list - please let me know if this
question is best asked elsewhere!
I'm using Infinispan's Lucene Directory with a Cassandra CacheStore backing
it, version 5.3.0.Final. Everything seems to be more-or-less working fine,
except for some kind of apparent race condition on start-up.
At the first point where my code accesses the InfinispanDirectory, I get
usually get a Cassandra TimedOutException from the Client.get() operation
in the CacheLoader. I've been googling around and all the suggestions on
the web (relating to using Cassandra directly) don't seem particularly
relevant, as the most common cause is reading too much data in single
operations. This is merely a 32kb read.
I was wondering if the problem is that my code is supposed to somehow wait
for the Cache instances to be "ready" before using them? Is there some kind
of listener I'm supposed to use to be notified of the Cache being ready for
read operations?
Any suggestions greatly appreciated!
Thanks,
James.
12 years
Re: [infinispan-dev] Atomic Objects on top of Infinispan
by Pierre Sutra
Hello,
> ^ It'll be easier for you to keep up with the latest master changes in
> upstream if you work on topic branches instead of your fork's master [1]
If I understand correctly, this does not provide me with a copy on a
remote stable storage. Besides, I am not the only one to work on these
modifications.
> ^ Rather than pushing casting to users, it'd be better to do it
> internally, i.e. with a definition like this: <T> T
> getOrCreateInstanceOf(Class<T> clazz,…
It was not meant to be the final API, and I will change this. (despite I
am not found of Java generics)
>
> Also, getOrCreateInstanceOf name feels a bit verbose to me. Why not
just call it 'create'? Alternatively, we already have code to create
atomic maps in AtomicMapLookup. We might wanna merge both into a single
factory/lookup class. Not a biggie though...
It is named getOrCreateInstanceOf because you may create a new object
but also, in case the object exists already in the system, receive a
consistent copy of it.
The two approaches of building maps are radically different, and they
will give radically different results. Nevertheless, I agree that
interfacing the two constructs with a single API would be of interest.
This is something I will eventually investigate.
>
>> set.add("smthing"); // some call examples
>> System.out.println(set.toString())
>> set.addAll(set);
>> factory.disposeInstanceOf(HashSet.class, "set", true); // to store the
>> object in a persistent way
>>
>> The pattern I followed to implement this facility is the state machine
>> replication approach. More precisely, the factory is built on top of the
>> transactional facility of Infinispan.
> ^ Why do you need transactions? Can you just call put?
The approach I am following to implement linearizable (aka. atomic)
objects [1] is a variation of state machine replication [2]. At core,
state machine replication requires total order broadcast to propagate
the modifications - a property given by a push in transactional mode in
Infinispan.
>> When the factory creates an
>> object, it stores a local copy of the object and registers a proxy as a
>> cache listener.
> Hmmm, javassist based proxying. This needs some thought and
consideration. I'm not a huge expert on this topic, but we have
Hibernate crew listening here which might be able to provide feedback on
how to best bridge over, since it's something they've been doing since
day one.
This approach seems to do a fairly good job and is both fast and
invisible to the user.
> If you're just calling a single cache operation, don't see the reason
for a transaction. You still get notifications from other servers if
something fails. The only difference is lack of of rollback if something
goes wrong in one of the servers and it works fine in others. Not using
transactions makes things work faster.
I could use a non-transactional cache, provided I have access to an
atomic broadcast primitive. For the moment, I did not find a
non-intrusive solution to this issue.
>
>> When the call is
>> de-serialized it is applied to the local copy, and in case the calling
>> process was local, the result of the call is returned (this mechanism is
>> implemented via a future object).
> ^ Why a future?
The result of the call is received via a call-back triggered by
modifications on the object's key.
>> Notice that this implies that all the
>> arguments of the methods of the object are serializable, as well as the
>> object itself.
> Serializable, or somehow marshallable… [2]
I will add this feature to the code (e.g., by giving a marshaller object
to the factory).
>
>> The current implementation supports the retrieval of
>> persisted objects, an optimization mechanism for read operations,
> ^ Can you explain this read optimization in more detail? Why do you
need it?
>
> I'm also seeing a lot of marshalling in your code, which I don't
understand why you need it at all
The idea is to optimistically execute each operation locally
(marshalling here is employed to obtain a deep copy of the object). If
the operation does not modify the object, then it is safe to return the
result of its tentative execution to the caller. Notice here that,
strictly speaking, activating this feature makes the object not atomic
but sequentially consistent [3].
Best,
Pierre
ps: it seems that my previous answer get lost.
[1] http://en.wikipedia.org/wiki/Linearizability
[2] http://en.wikipedia.org/wiki/State_machine_replication
[3] http://en.wikipedia.org/wiki/Sequential_consistency
12 years
Where is the LegacyLoaderAdapter interface ?
by Guillaume SCHEIBEL
Hello guys,
I'm updating myself since the cache store have been split into different
projects. I have forked the infinispan-cachestore-monogdb and I can't find
an interface
org.infinispan.configuration.cache.LegacyLoaderAdapter
Does it have been deleted ? Replace (by what) ?
Must / should the MongoDB cache store been updated ?
Thanks
Guillaume
12 years
To squash or not to squash
by Mircea Markus
When issuing pull requests people split the added functionality corresponding to a single JIRA issue into multiple logically related commits. (If they don't they should as this makes reviewer's life much easier).
Do you think these commits should be squashed into a single commit before integrating (1) or should be integrated as such (2)?
Note that for (2), especially for very large features, internediate commits might not compile/pass the suite.
My preference is for (2) as this way the history reflects the flow of developement and makes history easier to read.
Cheers,
Mircea
12 years
ISPN-3051 configuration
by Dan Berindei
Hi guys
As you know, I'm working on ISPN-3051, allowing each node to take a higher
or lower proportion of the entries in the cache. I've implemented this by
adding a float "loadFactor" setting in each node's configuration, with 1
being the default and any positive value being accepted (including 0).
There are two questions I wanted to ask you about the configuration:
1. What do you think about the "loadFactor" name? I started having doubts
about it, since it has a very different meaning in HashMap. I have come up
with a couple alternatives, but I don't love any of them:
"proportionalLoad" and "proportionalCapacity".
2. Where should we put this setting? I have added it as
CacheConfiguration.clustering().hash().loadFactor(), but I can't think of a
reason for having different values for each cache, so we might as well put
it in the global configuration.
Cheers
Dan
12 years
6.0.0 release schedule update
by Mircea Markus
Hi,
I've added another Alpha4 (to be released today) to the release schedule:
- we're still not feature complete
- waiting for Dan's ISPN-3051 to be finished
- ISPN-3477 will drop Cache.compact so should be tackled within the scope of Beta1
- the deprecated API needs to be dropped
The 6.0.0.Final release data has been shifted one week to 11 Oct.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
12 years, 1 month
Indexing for remote query
by Adrian Nistor
Hi devs,
Currently remote caches have byte[] values. With the introduction of
remote query these byte arrays are no longer opaque for search-enabled
caches, they are Protobuf encoded and their schema is known and we have
to somehow help hibernate-search extract the indexable data and index
it. As a side note, the indexing for embedded mode is done by
QueryInterceptor which relies on hibernate-search annotated cache
values. This definitely does not work directly for byte[].
One option is to write another interceptor similar to QueryInterceptor
but capable of making sense of the byte[], but that would duplicate
functionality (and maintenance) and would have to use Lucene directly
because in order to use hibernate-search you would need annotations on
your entities. And if we go this route we might need to also duplicate
MassIndexer.
A second more preferable option is to wrap the byte[] values into a
hsearch FieldBridge or ClassBridge annotated entity so we can continue
to use existing QueryInterceptor. This was implemented in the PR [1],
the wrapping entity being
org.infinispan.query.remote.indexing.ProtobufValueWrapper and the place
where the wrapping happens is a new interceptor,
org.infinispan.query.remote.indexing.RemoteValueWrapperInterceptor.
RemoteValueWrapperInterceptor uses a mechanism similar to
TypeConverterInterceptor (of compat mode), so a common base class was
extracted for them.
The wrapping approach has received favourable review so far. But I'm
asking myself (and you) maybe there's an even simpler solution that I've
overlooked? Or maybe we can do the wrapping somehow without introducing
a new interceptor?
Thanks!
[1] https://github.com/infinispan/infinispan/pull/2022
12 years, 1 month
