Infinispan and change data capture
by Randall Hauch
The Debezium project [1] is working on building change data capture connectors for a variety of databases. MySQL is available now, MongoDB will be soon, and PostgreSQL and Oracle are next on our roadmap.
One way in which Debezium and Infinispan can be used together is when Infinispan is being used as a cache for data stored in a database. In this case, Debezium can capture the changes to the database and produce a stream of events; a separate process can consume these change and evict entries from an Infinispan cache.
If Infinispan is to be used as a data store, then it would be useful for Debezium to be able to capture those changes so other apps/services can consume the changes. First of all, does this make sense? Secondly, if it does, then Debezium would need an Infinispan connector, and it’s not clear to me how that connector might capture the changes from Infinispan.
Debezium typically monitors the log of transactions/changes that are committed to a database. Of course how this works varies for each type of database. For example, MySQL internally produces a transaction log that contains information about every committed row change, and MySQL ensures that every committed change is included and that non-committed changes are excluded. The MySQL mechanism is actually part of the replication mechanism, so slaves update their internal state by reading the master’s log. The Debezium MySQL connector [2] simply reads the same log.
Infinispan has several mechanisms that may be useful:
Interceptors - See [3]. This seems pretty straightforward and IIUC provides access to all internal operations. However, it’s not clear to me whether a single interceptor will see all the changes in a cluster (perhaps in local and replicated modes) or only those changes that happen on that particular node (in distributed mode). It’s also not clear whether this interceptor is called within the context of the cache’s transaction, so if a failure happens just at the wrong time whether a change might be made to the cache but is not seen by the interceptor (or vice versa).
Cross-site replication - See [4][5]. A potential advantage of this mechanism appears to be that it is defined (more) globally, and it appears to function if the remote backup comes back online after being offline for a period of time.
State transfer - is it possible to participate as a non-active member of the cluster, and to effectively read all state transfer activities that occur within the cluster?
Cache store - tie into the cache store mechanism, perhaps by wrapping an existing cache store and sitting between the cache and the cache store
Monitor the cache store - don’t monitor Infinispan at all, and instead monitor the store in which Infinispan is storing entries. (This is probably the least attractive, since some stores can’t be monitored, or because the store is persisting an opaque binary value.)
Are there other mechanism that might be used?
There are a couple of important requirements for change data capture to be able to work correctly:
Upon initial connection, the CDC connector must be able to obtain a snapshot of all existing data, followed by seeing all changes to data that may have occurred since the snapshot was started. If the connector is stopped/fails, upon restart it needs to be able to reconnect and either see all changes that occurred since it last was capturing changes, or perform a snapshot. (Performing a snapshot upon restart is very inefficient and undesirable.) This works as follows: the CDC connector only records the “offset” in the source’s sequence of events; what this “offset” entails depends on the source. Upon restart, the connector can use this offset information to coordinate with the source where it wants to start reading. (In MySQL and PostgreSQL, every event includes the filename of the log and position in that file. MongoDB includes in each event the monotonically increasing timestamp of the transaction.
No change can be missed, even when things go wrong and components crash.
When a new entry is added, the “after” state of the entity will be included. When an entry is updated, the “after” state will be included in the event; if possible, the event should also include the “before” state. When an entry is removed, the “before” state should be included in the event.
Any thoughts or advice would be greatly appreciated.
Best regards,
Randall
[1] http://debezium.io
[2] http://debezium.io/docs/connectors/mysql/
[3] http://infinispan.org/docs/stable/user_guide/user_guide.html#_custom_inte...
[4] http://infinispan.org/docs/stable/user_guide/user_guide.html#CrossSiteRep...
[5] https://github.com/infinispan/infinispan/wiki/Design-For-Cross-Site-Repli...
7 years, 5 months
Spring module - change dependencies to Uber Jars
by Sebastian Laskawiec
Hey!
I'm currently trying to solve a tricky class loading issue connected to
Spring, CDI and Uber Jars. Here's the scenario:
- Remote Uber Jar contains CDI module
- Our Hot Rod client use newer version of JBoss Logging which is present
in Wildfly/EAP modules
- However EAP and Wildfly will load (and make available for deployment)
their own version of JBoss Logging [1]
- The easiest fix for this is to relocate JBoss Logging package in
Uber Jar
- Spring module requires some classes from Infinispan Common and they in
turn need BasicLogger from JBoss Logging
- If we relocate JBoss Logging and will try to use Uber Jar with
Spring - we will end up with classloading issue [2]
So it seems the best approach is to make Spring depend on Uber Jars instead
of "small ones". Of course, users who use small jars will probably be
affected by this change (they would have to either accept using Uber Jars
or exclude them in their poms and add dependencies manually).
Is anyone against this solution? JIRA tracking ticket: [3].
Thanks
Sebastian
[1] Scenario with Weld enabled WAR
https://docs.jboss.org/author/display/AS7/Implicit+module+dependencies+fo...
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1266831#c7
[3] https://issues.jboss.org/browse/ISPN-6132
7 years, 5 months
Multi tenancy support for Infinispan
by Sebastian Laskawiec
Dear Community,
Please have a look at the design of Multi tenancy support for Infinispan
[1]. I would be more than happy to get some feedback from you.
Highlights:
- The implementation will be based on a Router (which will be built
based on Netty)
- Multiple Hot Rod and REST servers will be attached to the router which
in turn will be attached to the endpoint
- The router will operate on a binary protocol when using Hot Rod
clients and path-based routing when using REST
- Memcached will be out of scope
- The router will support SSL+SNI
Thanks
Sebastian
[1]
https://github.com/infinispan/infinispan/wiki/Multi-tenancy-for-Hotrod-Se...
7 years, 8 months
jdk8backported package
by Radim Vansa
Hi,
although we're on Java 8, there's still the package
org.infinispan.*.jdk8backported in our codebase. Is there any plan (and
possibility) to remove these and use implementation provided by runtime?
Or have we tweaked them too much, so shall we rather rename them?
Radim
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
7 years, 9 months
Sequential interceptors API
by Radim Vansa
Hi,
I would like to encourage you to play with the (relatively) new API for
sequential interceptors, and voice your comments - especially you corish
devs, and Galder who has much experience with async invocations and that
kind of stuff from JS-world.
I am now trying to use the asynchronous methods only (the
forkInvocationSync() is only temporary helper!); Dan has made it this
way as he wanted to avoid unnecessary allocations, and I welcome this
GC-awareness, but regrettably I find it rather hard to use, due to its
handler-style nature. For the simplest style interceptors (do this,
invoke next interceptor, and process result) it's fine, but when you
want to do something like:
visitFoo(cmd) {
Object x = null;
if (/* ... */) {
x = invoke(new OtherCommand());
}
invoke(new DifferentCommand(x));
Object retval = invoke(cmd);
return wrap(retval);
}
I find myself passing handlers deep down. There is allocation cost for
closures, so API that does not allocate CompletableFutures does not pay off.
I don't say that I could improve it (I have directed my comments to Dan
on IRC when I had something in particular), I just say that this is very
important API for further Infinispan development and everyone should pay
attention before it gets final.
So please, play with it, and show your opinion.
Radim
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
7 years, 9 months
Persisted state
by Radim Vansa
Hi,
in what situations is the state (ATM just version + cache topologies)
meant to be persisted? I guess it's necessary with non-shared cache
stores, but should it be persisted with shared one, too?
And what are the guarantees during writing that state down? (e.g. can
you make sure that no operations are executed when persisting?)
My problem is that for scattered cache, I need to persist highest
version for each segment, or I have to iterate through cache store when
starting - that's kind of forced preload. It's even worse - during
regular preload, cache topology is not installed yet, and as I've found,
I can't do that in @Start annotated method because I need to find
segment for each key and cache topology can be installed even later than
in STMI.start() (when the persistent state is being loaded, the response
to join may not contain the cache topology).
Radim
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team
7 years, 9 months
Uber client, which means ALPN investigation
by Sebastian Laskawiec
Hey guys!
Recently I've been looking into ALPN support [1] and studying RFC [2] as
well as JEP [3]. In short, the Application Layer Protocol Negotiation -
allows the server and the client to agree which protocol shall be used
after TLS handshake. It will be supported out of the box in JDK9. For JDK8
you need a special Jetty Java agent [4].
With ALPN we could build an Uber Client, which would be able to support
many protocols at the same time (REST, HTTP/2, Hot Rod). We should be able
to select the protocol during client initialization as well as renegotiate
existing connection. This could be very convenient for situations when
connecting to multiple Hot Rod servers and some of them are accessible
using Hot Rod (the same DC or the same Cloud tenant) and some connections
need to get through a firewall (HTTP/2, REST).
Of course implementing this requires major refactoring in the server
endpoint as well as in the client. Possibly this is something for
Infinispan 10 :)
WDYT?
Thanks
Sebastian
[1] https://issues.jboss.org/browse/ISPN-6899
[2] https://tools.ietf.org/html/rfc7301
[3] http://openjdk.java.net/jeps/244
[4] https://github.com/jetty-project/jetty-alpn
7 years, 9 months
Passivation and manual eviction
by Tristan Tarrant
Hi all,
Radoslav just brought to my attention that our eviction configuration
validator prints out a warning when passivation is enabled without
configuring eviction. This was done to make sure users are aware of the
fact that a cache with this configuration will never actually passivate.
In WildFly's case however, eviction is performed manually (by invoking
cache.evict()). In this case the warning is just misleading.
My proposal is therefore to introduce a new eviction strategy "MANUAL"
wihch internally would be handled in the same way as "NONE" but which
would prevent the warning.
Wdyt ?
Tristan
--
Tristan Tarrant
Infinispan Lead
JBoss, a division of Red Hat
7 years, 9 months
Health check use cases
by Sebastian Laskawiec
Dear Community,
I'd like to ask you for help. I'm currently sketching a design for a REST
health check endpoint for Infinispan and I'm trying to imagine possible use
cases.
Could you please give me a hand and tell me what functionalities are
important for you? Would you like to be able to check status per-cache or
maybe a red (not healthy), green (healthy), yellow (healthy, rebalance in
progress) cluster status is sufficient? What kind of information do you
expect to be there?
Thanks
Sebastian
7 years, 10 months
