Reactive Streams + RxJava
by William Burns
I was thinking more about [1] and I find that I was going to implement
basically reactive streams. What we have now in github is similar but it
uses a very crude method of blocking the thread to prevent back pressure.
This can then cause severe issues as many users have found out when they
don't close iterator.
Unfortunately reactive streams is just a spec. I am proposing to add RxJava
[2] as a dependency [2] in the core module to provide access to reactive
streams and the various conversion methods. This library adds a bunch of
support for built in back pressure, transformations and much more which
would reduce the amount of code I would need to write substantially.
In regards to timing, I am thinking this is too close for 9.1, so maybe 9.2
or higher.
What do you guys think?
[1] https://issues.jboss.org/browse/ISPN-7865
[2] https://github.com/ReactiveX/RxJava
[3] https://mvnrepository.com/artifact/io.reactivex.rxjava2/rxjava/2.1.0
7 years, 6 months
please IGNORE 9.0.2 Release
by Vittorio Rigamonti
Hi All,
by mistake I've published a 9.0.2.Final which is actually a copy of
9.1.0.Beta1 release.
Please ignore it, next good release for 9.0.x series will be 9.0.3.Final...
coming soon.
Sorry for the added entropy.
--
Vittorio Rigamonti
Senior Software Engineer
Red Hat
<https://www.redhat.com>
Milan, Italy
vrigamon(a)redhat.com
irc: rigazilla
<https://red.ht/sig>
7 years, 6 months
Hibernate Search v 5.8 and plans for beyond!
by Sanne Grinovero
TLDR: the new Search SPI requires using the "free form" types - but
don't expect to be able to take advantage of it just yet.
Hi all,
we just released Hibernate Search 5.8.0.Beta3 and it's meant to be our
last beta!
Progress and pace are looking very good.
The details of the release can be found on our blog [1] as usual, but
I'd like to call out on several changes relevant for Infinispan.
First and foremost, know that we'll be working on 6.0 as next big
thing; at that stage the API will see a significant redesign, focused
on making Lucene an optional implementation detail.
This makes this 5.8 significant for Infinispan as it's the last one
you'll have until you're ready to break API.
We've been working hard on addressing the needs of Infinispan, but in
several cases we had to make - and will still make - dramatic changes
in the SPI.
Some benefits relevant to you all:
- the "ram" directory changed name as you suggested just recently
- statistics now include index size (how large is it to store it)
- a proper API to define Analyzers programmatically, you could now
add "out of the box" default definitions for Infinispan server.
- the same for Normalizers (a new concept, see the blog [1])
- tons of other improvements and bugfixes ;) such as:
-- better performance by finally being able to use JDK8 features
-- Java 9 compatibility
-- a critical performance improvement in Spatial bridges
-- better Spring and CDI improvements
# A new type system is coming !
And this is the main point.. however, it's not ready yet as it's a huge change.
The new type system - also named "free form" in all the meetings in
which we wished we had it - needs to start from the SPI and that's
partially included in this Beta.
(We postponed this for way too long by trying to match a good time to
break SPIs.. we'll just break it or it will never happen)
I will soon send a PR to update Infinispan to *work fine* with the
current SPI changes, but I will refrain from refactoring anything yet,
so that we can validate it all in small steps.
Still, many other existing SPI methods have been deprecated:
As soon as that's merged, I'll need help to migrate away from all
those deprecated methods as I simply can't make progress on the Search
side without starting to delete a good amount of these deprecated
methods.
So please know this: most of those deprecated methods will be removed
from Hibernate Search soon; we can of course keep some of them around
if we identify strong needs, but identifying these points can only be
done by getting started.
Please note that you can't yet plug-in a new type system to benefit
REST, Protobuf or other systems:
- several components still make assumptions about this being backed
by annotated POJOs
- there's no SPI yet to plugin alternative metadata
There's also no API to extract indexed values from an alternative
system, although I'm sure you could keep using the current hacks for
this to work out: it's not possible to get this in 5.8 so it will have
to be 6+.
Thanks,
Sanne
1 - http://in.relation.to/2017/06/13/hibernate-search-5-8-0-Beta3/
7 years, 6 months
Using load balancers for Infinispan in Kubernetes
by Sebastian Laskawiec
Hey guys!
Over past few weeks I've been working on accessing Infinispan cluster
deployed inside Kubernetes from the outside world. The POC diagram looks
like the following:
[image: pasted1]
As a reminder, the easiest (though not the most effective) way to do it is
to expose a load balancer Service (or a Node Port Service) and access it
using a client with basic intelligence (so that it doesn't try to update
server list based on topology information). As you might expect, this won't
give you much performance but at least you could access the cluster.
Another approach is to use TLS/SNI but again, the performance would be even
worse.
During the research I tried to answer this problem and created "External IP
Controller" [1] (and corresponding Pull Request for mapping
internal/external addresses [2]). The main idea is to create a controller
deployed inside Kubernetes which will create (and destroy if not needed) a
load balancer per Infinispan Pod. Additionally the controller exposes
mapping between internal and external addresses which allows the client to
properly update server list as well as consistent hash information. A full
working example is located here [3].
The biggest question is whether it's worth it? The short answer is yes.
Here are some benchmark results of performing 10k puts and 10k puts&gets
(please take them with a big grain of salt, I didn't optimize any server
settings):
- Benchmark app deployed inside Kuberenetes and using internal addresses
(baseline):
- 10k puts: 674.244 ± 16.654
- 10k puts&gets: 1288.437 ± 136.207
- Benchamrking app deployed in a VM outside of Kubernetes with basic
intelligence:
- *10k puts: 1465.567 ± 176.349*
- *10k puts&gets: 2684.984 ± 114.993*
- Benchamrking app deployed in a VM outside of Kubernetes with address
mapping and topology aware hashing:
- *10k puts: 1052.891 ± 31.218*
- *10k puts&gets: 2465.586 ± 85.034*
Note that benchmarking Infinispan from a VM might be very misleading since
it depends on data center configuration. Benchmarks above definitely
contain some delay between Google Compute Engine VM and a Kubernetes
cluster deployed in Google Container Engine. How big is the delay? Hard to
tell. What counts is the difference between client using basic intelligence
and topology aware intelligence. And as you can see it's not that small.
So the bottom line - if you can, deploy your application along with
Infinispan cluster inside Kubernetes. That's the fastest configuration
since only iptables are involved. Otherwise use a load balancer per pod
with External IP Controller. If you don't care about performance, just use
basic client intelligence and expose everything using single load balancer.
Thanks,
Sebastian
[1] https://github.com/slaskawi/external-ip-proxy
[2] https://github.com/infinispan/infinispan/pull/5164
[3] https://github.com/slaskawi/external-ip-proxy/tree/master/benchmark
7 years, 6 months
Moving functional API to core
by Radim Vansa
Hi guys,
when the functional API has been outline, the interfaces were put into
infinispan-commons to make it possible to share these between remote
clients and embedded use case. However, it seems that reusing this as-is
impossible or at least impractical as we cannot send the lambdas in a
language neutral way. In the future, we may implement a way to share
functions between client and a server but that will most likely result
in an interface accepting something else than Function<ReadWriteEntry,
R>. Also, it's rather weird to have two EntryVersion interfaces.
Therefore I suggest moving org.infinispan.commons.api.functional to
infinispan-core, package org.infinispan.api.functional
You might say that the server-side code would use the interfaces, but
once it's running on server, it should depend on core (or core-api) -
commons is what is shared with the client, and if the client will in
future register a new function on the server, the user code should
depend on core-api as well (client-hotrod itself does not have to).
If you wonder what led me to this is that I've tried to add
SerializableFunction overloads to the FunctionalMap and found out that
SerializableFunction et all are only in infinispan-core (for good).
Please let me know if you have objections/if there something I have missed.
Radim
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
7 years, 6 months
