Proposal - encrypted cache
by Sebastian Laskawiec
Hey!
A while ago I stumbled upon [1]. The article talks about encrypting data
before they reach the server, so that the server doesn't know how to
decrypt it. This makes the data more secure.
The idea is definitely not new and I have been asked about something
similar several times during local JUGs meetups (in my area there are lots
of payments organizations who might be interested in this).
Of course, this can be easily done inside an app, so that it encrypts the
data and passes a byte array to the Hot Rod Client. I'm just thinking about
making it a bit easier and adding a default encryption/decryption mechanism
to the Hot Rod client.
What do you think? Does it make sense?
Thanks
Sebastian
[1] https://eprint.iacr.org/2016/920.pdf
5 years, 10 months
Infinispan Managment Console versioning and releases
by Sebastian Laskawiec
Hey guys,
A while ago was been talking with Ryan and Tristan about automated releases
for Infinispan Management Console. I would like to send the main point for
wider audience.
Long story short, we were considering different versioning schemes, such as
X.Y.Z.SHA1 or using Z as an auto-increment counter for console releases.
The main problem we were trying to solve was how to release the management
console more often.
I would like to propose different approach - Let's stick with a standard
versioning (X.Y.Z.[Alpha|Beta|Fina] for releases and X.Y.Z-SNAPSHOT for
ongoing work). Then we need to embed SHA1 into the MANIFEST.MF to increase
tracability (in other words, here I have an Infinispan build and I need to
know which SHA1 was used to build the console). SNAPSHOTs will be pushed
into JBoss Repository [1] after each commit. Infinispan master branch will
have a SNAPSHOT dependency to the console. The tricky part are releases.
Well, at first we need to release the console (I hope we will automate that
in Team City). Then we can use the version plugin [2] to update the
Infinispan source code to the latest version of the console. Finally, we
can release the Infinispan. As a long-term goal, everything will happen
inside a single staging repository in Nexus (but that's a long-term goal...
first let get this running).
If you agree to my proposal, please change the version in the console into
9.0.0-SNAPSHOT and retrigger [3] (automated builds are disabled at the
moment). Next, I would kindly ask to look into the build logs [4][5] and
give me a hint how to fix it. The NPM plugin is failing with some weird
error. Once we are done with that, I will configure a Pull Request builder
and release job.
Thanks
Sebastian
[1] https://repository.jboss.org/nexus/content/repositories/snapshots/
[2] http://www.mojohaus.org/versions-maven-plugin/
[3]
http://ci.infinispan.org/viewType.html?buildTypeId=Infinispan_ManagmentCo...
[4]
http://ci.infinispan.org/viewLog.html?buildId=46542&buildTypeId=Infinispa...
[5]
http://ci.infinispan.org/viewLog.html?buildId=46543&buildTypeId=Infinispa...
7 years, 4 months
Infinispan and change data capture
by Randall Hauch
The Debezium project [1] is working on building change data capture connectors for a variety of databases. MySQL is available now, MongoDB will be soon, and PostgreSQL and Oracle are next on our roadmap.
One way in which Debezium and Infinispan can be used together is when Infinispan is being used as a cache for data stored in a database. In this case, Debezium can capture the changes to the database and produce a stream of events; a separate process can consume these change and evict entries from an Infinispan cache.
If Infinispan is to be used as a data store, then it would be useful for Debezium to be able to capture those changes so other apps/services can consume the changes. First of all, does this make sense? Secondly, if it does, then Debezium would need an Infinispan connector, and it’s not clear to me how that connector might capture the changes from Infinispan.
Debezium typically monitors the log of transactions/changes that are committed to a database. Of course how this works varies for each type of database. For example, MySQL internally produces a transaction log that contains information about every committed row change, and MySQL ensures that every committed change is included and that non-committed changes are excluded. The MySQL mechanism is actually part of the replication mechanism, so slaves update their internal state by reading the master’s log. The Debezium MySQL connector [2] simply reads the same log.
Infinispan has several mechanisms that may be useful:
Interceptors - See [3]. This seems pretty straightforward and IIUC provides access to all internal operations. However, it’s not clear to me whether a single interceptor will see all the changes in a cluster (perhaps in local and replicated modes) or only those changes that happen on that particular node (in distributed mode). It’s also not clear whether this interceptor is called within the context of the cache’s transaction, so if a failure happens just at the wrong time whether a change might be made to the cache but is not seen by the interceptor (or vice versa).
Cross-site replication - See [4][5]. A potential advantage of this mechanism appears to be that it is defined (more) globally, and it appears to function if the remote backup comes back online after being offline for a period of time.
State transfer - is it possible to participate as a non-active member of the cluster, and to effectively read all state transfer activities that occur within the cluster?
Cache store - tie into the cache store mechanism, perhaps by wrapping an existing cache store and sitting between the cache and the cache store
Monitor the cache store - don’t monitor Infinispan at all, and instead monitor the store in which Infinispan is storing entries. (This is probably the least attractive, since some stores can’t be monitored, or because the store is persisting an opaque binary value.)
Are there other mechanism that might be used?
There are a couple of important requirements for change data capture to be able to work correctly:
Upon initial connection, the CDC connector must be able to obtain a snapshot of all existing data, followed by seeing all changes to data that may have occurred since the snapshot was started. If the connector is stopped/fails, upon restart it needs to be able to reconnect and either see all changes that occurred since it last was capturing changes, or perform a snapshot. (Performing a snapshot upon restart is very inefficient and undesirable.) This works as follows: the CDC connector only records the “offset” in the source’s sequence of events; what this “offset” entails depends on the source. Upon restart, the connector can use this offset information to coordinate with the source where it wants to start reading. (In MySQL and PostgreSQL, every event includes the filename of the log and position in that file. MongoDB includes in each event the monotonically increasing timestamp of the transaction.
No change can be missed, even when things go wrong and components crash.
When a new entry is added, the “after” state of the entity will be included. When an entry is updated, the “after” state will be included in the event; if possible, the event should also include the “before” state. When an entry is removed, the “before” state should be included in the event.
Any thoughts or advice would be greatly appreciated.
Best regards,
Randall
[1] http://debezium.io
[2] http://debezium.io/docs/connectors/mysql/
[3] http://infinispan.org/docs/stable/user_guide/user_guide.html#_custom_inte...
[4] http://infinispan.org/docs/stable/user_guide/user_guide.html#CrossSiteRep...
[5] https://github.com/infinispan/infinispan/wiki/Design-For-Cross-Site-Repli...
7 years, 5 months
Accidental comments on some PRs
by Jiri Holusa
Hi,
some of you might noticed comments on some PRs: "Performance tests run successfully. Link to the results: ${report_url}". I'm sorry about that, it was a misconfiguration.
Please ignore them, thanks. I apologize.
Jiri
7 years, 5 months
New blog post
by Adrian Nistor
Hi all,
I've just published a new blog post that briefly introduces Ickle, the query language of Infinispan [1]. This will be followed soon by another one on defining domain model schemas, configuring model indexing and analysis.
Cheers,
Adrian
[1]http://blog.infinispan.org/2016/12/meet-ickle.html
7 years, 5 months
JDK 9 b148 including a refresh of the module system is available on java.net
by Rory O'Donnell
Hi Galder,
JDK 9 build b148 <https://jdk9.java.net/download/> includes an important
Refresh of the module system [1] , summary of changes are listed here
<http://download.java.net/java/jdk9/changes/jdk-9+148.html>.
*This refresh includes a disruptive change that is important to understand.
*For those that have been trying out modules with regular JDK 9 builds
then be aware that `requires public` changes to `requires transitive`.
In addition, the binary representation of the module declaration
(module-info.class) has changed so that you need to recompile any
modules that were compiled with previous JDK 9 builds.
As things stand today in JDK 9 then you use setAccessible to break into
non-public elements of any type in exported packages. However, it cannot
be used to break into any type in non-exported package. The current
specified behavior was a compromise for the initial integration of the
module system. It is of course not very satisfactory, hence the
#AwkwardStrongEncapsulation issue [2] on the JSR 376 issues list. With
the updated proposal in the JSR, this refresh changes setAccessible
further so that it cannot be used to break into non-public types, or
non-public elements of public types, in exported packages. Code that
uses setAccessible to hack into the private constructor of
java.lang.invoke.MethodHandles.Lookup will be disappointed for example.
This change will expose hacks in many existing libraries and tools. As a
workaround then a new command line option `--add-opens` can be used to
open specific packages for "deep reflection". For example, a really
popular build tool fails with this refresh because it uses setAccessible
+ core reflection to hack into a private field of an unmodifiable
collection so that it can mutate it, facepalm! This code will continue
to work as before when run with `--add-opens
java.base/java.util=ALL-UNNAMED` to open the package java.util in module
java.base to "all unnamed modules" (think class path).
*Any help reporting issues to popular tools and libraries would be
appreciated. *
A debugging aid that is useful to identify issues is to run with
-Dsun.reflect.debugModuleAccessChecks=true to get a stack trace when
setAccessible fails, this is particularly useful when code swallows
exceptions without any logging.
Rgds,Rory
[1]
http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-November/005276.html
<http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-October/000...>
[2]
http://openjdk.java.net/projects/jigsaw/spec/issues/#AwkwardStrongEncapsu...
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland
7 years, 5 months
