Concurrency API
by Katia Aresti
Hi all,
I've been working last week on a concurrent API design, trying to keep the
scope small (but not too small). I've came up with a proposal (WIP) while
I've been having a look to what competitors do today.
I've shared a small design proposal concerning an Infinispan Global API
Object too. TBH nothing very exotic; but I believe that this object could
add a stone for user API usability.
Please, be kind with me, it's not super detailed (forgive me Radim !) but I
would like to share it already so you can add your thoughts/suggestions
through comments and help me improve it before heading any implementation.
Here is the PR :
https://github.com/infinispan/infinispan-designs/pull/8
Cheers
Katia
7 years, 7 months
JDK 9 EA Build 170 is available on jdk.java.net
by Rory O'Donnell
Hi Galder
*
JDK 9 Early Access* build 170 is available at the new location : -
jdk.java.net/9/
A summary of all the changes in this build are listed here
<http://download.java.net/java/jdk9/changes/jdk-9+170.html>.
Changes which were introduced since the last availability email that may
be of interest :
* b168 - JDK-8175814: Update default HttpClient protocol version and
optional request version
o related to JEP 110 : HTTP/2 Client.
* b169 - JDK-8178380 : Module system implementation refresh (5/2017)
o changes in command line options
* b170 - JDK-8177153 : LambdaMetafactory has default
constructorIncompatible change,
o release note: JDK-8180035
*New Proposal - Mark Reinhold has asked for comments on the jigsaw-dev
mailing list *[1]
* Proposal: Allow illegal reflective access by default in JDK 9
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
Rgds,Rory
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland
7 years, 7 months
Deprecation of Index.LOCAL
by Gustavo Fernandes
Hi, the Index.LOCAL setting was introduced eons ago to allow indexing to
occur once cluster-wide;
thus it's recommended when using an IndexManager such as
InfinispanIndexManager and ElasticsearchIndexManager that is shared among
all nodes.
Furthermore, Index.LOCAL suits ClusteredQueries [1] where each node has its
own "private" index and query is broadcasted to each individual node, and
aggregated in the caller before returning the results.
The issue with Index.LOCAL is when a command is originated in a NON_OWNER
(this happens in DIST caches), where there is no context available that
prevents obtention of previous values needed certain commands. This makes
fixing [2] complex as it requires fiddling with more than a couple of
interceptors, and it'd require remote fetching of values. This extra fetch
could be avoided if indexing always occurs in the owners.
tl;dr
The proposal is to deprecate Index.LOCAL, and map it internally to
Index.PRIMARY_OWNER
Everything should work as before, except if someone is relying to find a
certain entry indexed in a specific local index where the put was issued:
the ClusteredQuery test suite does that, but I don't think this is a
realistic use case.
Any objections?
Thanks,
Gustavo
[1]
http://infinispan.org/docs/stable/user_guide/user_guide.html#query.cluste...
[2] https://issues.jboss.org/browse/ISPN-7806
7 years, 7 months
S3_PING and GOOGLE_PING deprecated
by Bela Ban
Hi folks,
I've deprecated S3_PING [1] and GOOGLE_PING [3]. The replacements are
NATIVE_S3_PING [2] and GOOGLE_PING2 [4].
S3_PING started out as a copy of sample code by Amazon a long time ago,
and - because I never wanted Amazon's dependencies - is quite fat, as it
contains a bunch of dependent classes.
While Amazon's client code has evolved and become more robust, S3_PING
has never changed much, so I suspect it contains a number of bugs and is
possibly also not very efficient.
GOOGLE_PING is a subclass of S3_PING and uses Google's AWS compatibility
library. Thus, GOOGLE_PING inherits all of S3_PING's deficiencies.
A couple of week's ago, I therefore created NATIVE_S3_PING (a port of
Zalando's original protocol of the same name for 3.x), which uses the
AWS client SDK to access S3 storage.
This week, I also committed a first version of GOOGLE_PING2, which doe
not extend S3_PING any longer, but instead uses Google's client library
to access Google Cloud Storage directly.
The benefit of using an official client library instead of the
copy&paste kludge that's called S3_PING is that client libs are
maintained / updated / offer new functionality / yada yada yada.
Since both protocols have depedencies, they're located in the
jgroups-extras repo (https://github.com/jgroups-extras). I want to core
JGroups repo to be free of any dependencies.
S3_PING and GOOGLE_PING will be removed from JGroups in the next major
version (5.x).
If you have any concerns that functionality currently present in either
S3_PING or GOOGLE_PING is not available in NATIVE_S3_PING or
GOOGLE_PING2, let me know.
Cheers,
[1] http://www.jgroups.org/manual4/index.html#_s3_ping
[2] https://github.com/jgroups-extras/native-s3-ping
[3] http://www.jgroups.org/manual4/index.html#_google_ping
[4] https://github.com/jgroups-extras/jgroups-google
7 years, 7 months
to be a command, or not to be a command, that is the question
by Katia Aresti
Hi all,
As you might know I'm working since my arrival, among other things, on
ISPN-5728 Jira [1], where the idea is to override the default ConcurrentMap
methods that are missing in CacheImpl (merge, replaceAll, compute ... )
I've created a pull-request [2] for compute, computeIfAbsent and
computeIfPresent methods, creating two new commands. By the way, I did the
same thing for the merge method in a branch that I haven't pull requested
yet.
There is an opposite view between Radim and Will concerning the
implementation of these methods. To make it short :
In one side Will considers compute/merge best implementation should be as a
new Command (so what is already done)
In the other side, Radim considers adding another command is not necessary
as we could simple implement these methods using ReadWriteKeyCommand
The detailed discussion and arguments of both sides is on GitHub [2]
Before moving forward and making any choice by myself, I would like to hear
your opinions. For the record, it doesn't bother me redoing everything if
most people think like Radim because working on commands has helped me to
learn and understand more about infinispan internals, so this hasn't been a
waste of time for me.
Katia
[1] https://issues.jboss.org/browse/ISPN-5728
[2] https://github.com/infinispan/infinispan/pull/5046
7 years, 7 months
Need to understand, ClusteredCacheWithElasticsearchIndexManagerIT
by Katia Aresti
Hi all,
I'm really struggling with something in order to finish the compute methods.
I added a test in *ClusteredCacheWithElasticsearchIndexManagerIT*
public void testToto() throws Exception {
SearchManager searchManager = Search.getSearchManager(cache2);
QueryBuilder queryBuilder = searchManager
.buildQueryBuilderForClass(Person.class)
.get();
Query allQuery = queryBuilder.all().createQuery();
String key = "newGoat";
Person person4 = new Person(key, "eats something", 42);
cache2.putIfAbsent(key, person4);
StaticTestingErrorHandler.assertAllGood(cache1, cache2);
List<Person> found = searchManager.<Person>getQuery(allQuery,
Person.class).list();
assertEquals(1, found.size());
assertTrue(found.contains(person4));
}
I put some logs in the processPutKeyValueCommand method in the
*QueryInterceptor* to explain what is happening.
*2 threads*
Sometimes two threads get involved.
= Thread 72 First (or second) call
It happens from a non local Node. The so the shouldModifyIndexes says "no,
you should not modify any index" because the IndexModificationStrategy is
set to "LOCAL ONLY". [1]
72 ctx.getOrigin() =
ClusteredCacheWithElasticsearchIndexManagerIT-NodeB-19565
72 should modify false
72 previousValue null
72 putValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null} // value in the command
72 contextValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null} //value in the invocation context
= Thread 48 Second (or first) call
the origin is null, and this is considered as a LOCAL in the
SingleKeyNonTxInvocationContext.
[2] In this case, the index is modified correctly, the value in the context
has already been set up by the PutKeyValueCommand and the index get's
correctly updated.
48 ctx.getOrigin() = null
48 should modify true
48 previousValue null
48 putValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
48 contextValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
And everything is ok. Everything is fine too in the case of a compute
method instead of the put method.
But sometimes, this is not executed like that.
*3 threads*
What is a bit more weird to me is this second scenario where the commands
are executed both from non local nodes (A and B). And so the index is not
updated.
But just later, another thread get's involved and calls the
QueryInterceptor with a invocation context where the command has not been
executed (the value is not inside the context and the debugger does not
enter in the perform method, this has happened just twice before). This
call is coming like from a callback? in the QueueAsyncInvocationStage.
80 ctx.getOrigin() =
ClusteredCacheWithElasticsearchIndexManagerIT-NodeA-65110
80 should modify false
80 prev null
80 putValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
80 contextValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
38 ctx.getOrigin() =
ClusteredCacheWithElasticsearchIndexManagerIT-NodeB-35919
38 should modify false
38 prev null
38 putValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
38 contextValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
48 ctx.getOrigin() = null
48 should modify true
48 prev null
48 putValue Person{name='newGoat', blurb='eats something', age=42,
dateOfGraduation=null}
48 contextValue null
This execution works perfectly with PutKeyValueCommand. But don't work wth
compute.
The "computed value" is not inside the Command like put, replace or others.
It is computed in the perform method (if needed). So, the first time the
command is executed in A, the computed value is in the context, but the
index is not updated. Second call, executed in B, value in context, but the
index is not updated. The magic callback is executed, but the computed
value is nowhere because the command is not executed a third time, so the
context is null.
Can somebody please give me some light on this and explain to me what am I
missing ? Other tests are failing for the same problem,
like org.infinispan.query.blackbox.ClusteredCacheWithInfinispanDirectoryTest
Thank you very much for your help !
Katia
[1]
https://github.com/infinispan/infinispan/blob/master/query/src/main/java/...
[2]
https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
7 years, 7 months
