Please review proposal for Spring Infinispan as a Spring extension
by Olaf Bergner
I'm currently working on implementing Spring's forthcoming cache SPI for
Infinispan, aiming to contribute the results back to Spring. Since
Spring is unlikely to accept cache providers besides EHCache in
spring-context itself, I originally planned on making Spring Infinispan
a part of Spring Modules.
However, Spring Modules has been phased out, and Spring Extensions has
taken its place. Le roi et mort, vive le roi. In order to have Spring
Infinispan accepted as an official Spring extension I need to fill out a
form detailing scope, purpose and benefit of the proposed extension.
Below is my first attempt at doing so. Since this touches on how
Infinispan will be perceived in the greater developer community, and
since noone has yet accused me of being overly diplomatic, I herewith
ask you to review this form and to not hesitate to offer constructive
criticism.
Cheers,
Olaf
Extension Proposal Pack
This document is part of the Proposal pack for requesting a new
extension. Please modify the following sections and submit back to the
current project lead at SpringSource, or alternatively
extensions(a)springsource.com
Naming
Each Extension has several names, a publicly visible name and an SVN
name. This is actually quite an important aspect, as the name has to be
intuitive and distinct from every other Extension.
The naming convention for the internal name is "se-xxx", if it is a .NET
Extension, the naming convention is "se-xxx.net".
Proposed Name: Spring Infinispan
----------------------------------------------------------------------------------------------
Description and Background
What is the purpose of the Extension. What does it do, why is it needed,
how did it evolve etc.
Description:
Spring Infinispan will primarily provide an implementation of Spring
3.1's cache SPI, namely
* org.springframework.cache.Cache und
* org.springframework.cache.CacheManager,
backed by the high-performance distributed cache INFINISPAN:
http://www.jboss.org/infinispan.
In addition, Spring Infinispan will offer implementations of Spring's
FactoryBean interface for producing
* native Infinispan caches,
* native Infinispan cache managers,
* Infinispan configuration instances,
* ...
within a Spring context.
Spring Infinispan's overarching goal is to make Infinispan a good and
well-behaved citizen is Spring land, thus
* offering Infinispan users the option to use Spring's popular
programming model, and
* offering Spring users access to a mature, performant and actively
maintained distributed open source cache.
----------------------------------------------------------------------------------------------
Stakeholders
Who are the main stakeholders in the Extension? It is important for the
stakeholders to be clear to understand potential conflict of interest
issues.
Stakeholders:
At this stage, the only stakeholders in Spring Infinispan is the
Infinispan development community, led by Manik Surtani of Red Hat.
----------------------------------------------------------------------------------------------
Competitors/Related Projects
Who (if any) are competitors for this Extension. If there are
competitors, why would people use this Extension instead of the competitors?
Competitors:
To the best of my and Google'd knowledge, there is no other effort
underway to integrate Infinispan into the Spring ecosystem. And even if
not, Spring Infinispan has the bonus of being officially supported by
the Infinispan development community.
----------------------------------------------------------------------------------------------
Benefit to SpringSource
Why do we care about this?
SpringSource Benefit:
* According to
http://blog.springsource.com/2011/02/23/spring-3-1-m1-caching/ is Spring
actively encouraging contributors to implement their forthcoming cache
SPI for as many cache providers as possible. Spring Infinispan will be
testament to the community's positive response to this call, and may
thus encourage other cache providers to follow suit, thereby promoting
Spring Cache's popularity.
* Developers invested in Infinispan may find Spring an attractive
programming model and switch to the Spring framework, thereby further
boosting its popularity.
----------------------------------------------------------------------------------------------
Benefit to the Community
What benefit does this Extension bring to the community?
Community Benefit:
* Make a high-performance and rapidly progressing distributed cache
accessible using the familiar Spring programming model.
----------------------------------------------------------------------------------------------
Sponsor
Every Extension must have a SpringSource sponsor. If the SpringSource
sponsor is already known, they should be listed here.
SpringSource Sponsor:
----------------------------------------------------------------------------------------------
Roadmap
What is the expected life time for the Extension. When is the first
release likely to happen? Try and provide a plan for the next 6-12
months. This doesnʼt form part of the formal contract, merely your best
intentions at this time.
Roadmap:
Spring Infinispan has a well circumscribed scope: implement Spring's
Cache SPI and provide some helper classes to make the lifes of
Infinispan users easier in Spring land. Therefore, it is currently
expected that Spring Infinispan will mature rather rapidly. Once it
supports embedded and remote Infinispan caches, its main goals will be
achieved. Beyond that, it will depend largely on community feedback if
and what further enhancements will be needed.
A first version supporting embedded (in-VM) Infinispan caches is likely
to go public within the first two months after this proposal has been
accepted.
A second version additionally supporting remote Infinispan caches should
be available at most two months later.
13 years, 2 months
Fwd: [Infinispan] - Re: Infinispan 5.0.0 on JBoss AS 6.0.0
by Galder Zamarreño
Guys,
Any idea about this? It would appear as if even if Infinispan 5.0 is deployed in isolated mode, the JAXB layer might have issues with previously loaded XSD?
Cheers,
Begin forwarded message:
> From: Bertrand Renuart <do-not-reply(a)jboss.com>
> Date: February 18, 2011 5:31:17 PM GMT+01:00
> To: Galder Zamarreño <galder.zamarreno(a)redhat.com>
> Subject: [Infinispan] - Re: Infinispan 5.0.0 on JBoss AS 6.0.0
>
> JBoss Community
> Re: Infinispan 5.0.0 on JBoss AS 6.0.0
> created by Bertrand Renuart in Infinispan - View the full discussion
> Adding META-INF/jboss-classloading.xml as you propose does the trick - jars contained in my ear are loaded intstead of those provided by JBoss AS.
>
>
> BUT... Infinispan still fails to initialize with the following error:
>
>
> javax.xml.bind.UnmarshalException: unexpected element (uri:urn:infinispan:config:5.0, local:infinispan). Expected elements are <{urn:infinispan:config:4.2}infinispan>
>
>
>
> For your information, this happens only when starting JBoss with the "all" configuration - it does work correctly with the "default" configuration.
>
> My understanding is as follows:
>
> The "all" configuration contains Clustering support which makes use of Infinispan (version 4.2.0). The Clustering features are initialized before any application is deployed;
> Infinispan relies on JAXB to read/parse its configuration - the InfinispanConfiguration class is unmarshalled using JAXB.
> Then comes the initialization of Infinispan within my application - version 5.x and not 4.2.0. For whatever reason, this time JAXB fails to unmarshal it correctly.
>
> The second time JAXB complains with an expected urn of the form :config:4.2 - this looks like a left over from the first unmarshalling (done when JBoss initialized its clustering support). This would also explains why everything works perfectly when deploying under the "default" profile which doesn't contain clustering support...
>
>
> Do you have any idea on how to solve this issue ?
>
> Is there a way to tell JAXB to "flush" anything it may have collected about the previous unmarshalling ?
>
> Reply to this message by going to Community
> Start a new discussion in Infinispan at Community
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
13 years, 2 months
ISPN200 / JSR107
by Israel Lacerra
Hi!
Two unrelated things:
-*ISPN200-* Sanne, I already made most of the changes you suggested for the
ISPN 200. I will show a new version in 20~30 days. I used the strategy that
I said in a earlier email ("when a distributed query is "initialized", each
node creates and keeps a Searcher for this query and return the TopDocs to
the requester node. So, the requester node merge the results, and when he
needs to retrieves the next value, he sends the ScoreDoc.doc to the correct
node. To merge the values I'm using a FieldDocSortedHitQueue class copy,
with some changes.").
*-JSR107* Does anybody knows where I can find some documentation about the
JSR107 spec?? I googled it and I did not have found details about it. I
writing about with Infinispan (for my master thesis) and I'd like to write a
little about the spec.
thanks!
Israel
13 years, 2 months
API stability policy for major vs minor vs patch releases
by Paul Ferraro
In my course of upgrading AS6 from 4.2.0.Final to 4.2.1.CR3, I came
across a NoSuchMethodError. Specifically, a couple AS components
utilize DistributionManager.isLocal(String), which was dropped from the
public API sometime after 4.2.1.CR1.
While the fix is trivial enough (in the end I'll need to perform several
component release to compensate), this raises the larger issue of a API
stability policy for major vs minor vs patch releases. At the very
least, I don't think its wise to remove methods from a public interface
in a patch release. In my opinion, removing methods from a public API
should only happen across major releases, and even then, only after
being formally deprecated.
Thoughts?
Paul
13 years, 2 months
Interested in lending a hand
by Olaf Bergner
I've been quietly following Infinispan's progress and finally made the
decision to offer to help out. Since I'm not an experienced system-level
programmer - I've got a strong background in JEE application development
- I was looking for a not too complex task to start with. I took a look
at ISPN-272 "Implement XA recovery in TransactionXaAdapter" and if I
interpret Manik Surtani's comments correctly it all boils down to
essentially:
* Make it so that Xid[] recover(int flags) returns the Xids of the
LocalTransactions managed by this TransactionXaAdapter that are
currently in state 'PREPARED'
Is this correct? If so I thought about
1. adding a method Set<LocalTransaction>
getLocalTransactionsCurrentlyInStatus(int desiredStatus) to
TransactionTable, and
2. having recover(int flags) on TransactionXaAdapter delegate to this
new method, passing in Status.STATUS_PREPARED.
So:
1. Could you use my help?
2. Is ISPN-272 a good starting point, or are there better tasks for a
newcomer?
3. If ISPN-272 is good starting point, is my proposed solution viable,
or is it lacking?
Regards,
Olaf
13 years, 2 months
rehash and prepared tx
by Mircea Markus
Hi,
This behaviour is unspecified:
Tx is prepared on N1 and N2. Then N2 crashes, and prepared tx is migrated across to N3. What happens with the tx if it cannot acquire locks on N3, because there's another tx2 preparing* on N3?
*preparing - it cannot be finished preparing as there's already a *prepared* tx owning some of its locks. We cannot have two prepared tx that have same lock acquired.
Here is my take on this scenario: if the tx is already prepared on Node1 then same tx should force lock acquisition on Node3 and force rollback of any tx that holds a lock on that node. Why? (1) because Node1's tx is already prepared and it made a commitment to the TxManager to apply state (or manual recovery needs to be done - bad for throughput and user experience. (2) on the other hand the transaction on Node3 is not yet prepared.
Cheers,
Mircea
13 years, 2 months
Distributed execution framework - update
by Vladimir Blagojevic
I want to roughly outline what I believe needs to be done to implement
basic distributed
execution framework for 5.0 Final.
If you recall, Distributed/MapReduce task is a logical work unit
consisting of multiple distributed
executables executed individually across Infinispan cluster. Each
individual task execution
on Infinispan cluster is governed by failover, load balancing and
execution policies.
Failover policy
Failover policy regulates how and if distributed task executables are
migrated to
backup execution nodes in case of failure.
Executable can fail due to:
- exception raised in task implementation during execution
- node crash/leave
- migration failure to/from target execution node
Infinispan will invoke failover mechanism in all above cases except when
exception
is raised by task executable itself. Exception will be returned to
invoker of
distributed task who can act upon it.
By default, there will be two failover policies: failover off, and
failover on.
If failover is on load balancing policy in place will decide where to
migrate
task executable for execution. In case failure is off task invoker will
be notified.
Load balancing policy
Load balancing policy decides how distributed task executables are
dispersed for
execution around Infinispan cluster. By default data collocating load
balancing
policy is used as soon as distributed task is invoked on a set of keys
in cache.
Other, simpler, load balancing policies can be implemented as well if a
need arises.
Execution policy
Execution policy decides how task executable is executed once it has
been migrated
to an execution node. By default priority queue is used for queuing of
execution
task executables. Users can, if needed, fine-tune task priority on per
task basis.
If priority is not changed for all tasks then all their executables are
effectively
queued fifo on execution nodes.
Time permitting, job stealing policy should be implemented taking into
account ideas
from fork/join framework and applying it in a distributed fashion
amongst Infinispan
nodes.
Implementation sketches
In order to implement distributed/mapreduce task execution I believe we
should reuse
existing Infinispan infrastructure (marshalling, remote command
invocation, interceptor
chain, thread pools) as much as possible.
As user submits distributed task we would locate Infinispan nodes where
the input keys
are located and send executables (DistributedCallable/Mapper/Reducer) to
those nodes
using exisiting remote command invocation mechanism. Decision about
migration of
executables is effectively done by load balancing policy, the default
one being
collocating policy.
When executables wrapped into commands arrive to Infinispan nodes they
are handed
off to a special handling object (execution policy) rather than
invocation handler.
Execution policy interacts with execution container and in turn queues
and monitors
executables as they are executed in container's thread pool.
DistributedCallable(s) are
invoked and results returned to invoking node. Mappers are invoked as
well and their
results handed off to Reducers as described in mapreduce algorithm.
Eventually a
result of each Reducer is also returned to task invoker and in turn
Collator is invoked.
In case of task failure due to exception raised in task itself,
exception is returned to
task submitter. In other cases, failover policy along with load
balancing policy decides
how to migrate executable to other Infinispan nodes.
If you think that I omitted something and/or have suggestion let me know.
Regards,
Vladimir
13 years, 2 months
Distributed execution framework - update (formatted)
by Vladimir Blagojevic
I want to roughly outline what I believe needs to be done to implement basic distributed execution framework for 5.0 Final.
If you recall, Distributed/MapReduce task is a logical work unit consisting of multiple distributed executables executed individually across Infinispan cluster. Each individual task execution on Infinispan cluster is governed by failover, load balancing and execution policies.
Failover policy
Failover policy regulates how and if distributed task executables are migrated to backup execution nodes in case of failure.
Executable can fail due to:
- exception raised in task implementation during execution
- node crash/leave
- migration failure to/from target execution node
Infinispan will invoke failover mechanism in all above cases except when exception is raised by task executable itself. Exception will be returned to invoker of distributed task who can act upon it.
By default, there will be two failover policies: failover off, and failover on. If failover is on load balancing policy in place will decide where to migrate task executable for execution. In case failure is off task invoker will be notified.
Load balancing policy
Load balancing policy decides how distributed task executables are dispersed for execution around Infinispan cluster. By default data collocating load balancing policy is used as soon as distributed task is invoked on a set of keys in cache. Other, simpler, load balancing policies can be implemented as well if a need arises.
Execution policy
Execution policy decides how task executable is executed once it has been migrated to an execution node. By default priority queue is used for queuing of execution task executables. Users can, if needed, fine-tune task priority on per task basis. If priority is not changed for all tasks then all their executables are effectively queued fifo on execution nodes.
Time permitting, job stealing policy should be implemented taking into account ideas from fork/join framework and applying it in a distributed fashion amongst Infinispan nodes.
Implementation sketches
In order to implement distributed/mapreduce task execution I believe we should reuse existing Infinispan infrastructure (marshalling, remote command invocation, interceptor chain, thread pools) as much as possible.
As user submits distributed task we would locate Infinispan nodes where the input keys are located and send executables (DistributedCallable/Mapper/Reducer) to those nodes using exisiting remote command invocation mechanism. Decision about migration of executables is effectively done by load balancing policy, the default one being collocating policy.
When executables wrapped into commands arrive to Infinispan nodes they are handed off to a special handling object (execution policy) rather than invocation handler. Execution policy interacts with execution container and in turn queues and monitors executables as they are executed in container's thread pool. DistributedCallable(s) are invoked and results returned to invoking node. Mappers are invoked as well and their results handed off to Reducers as described in mapreduce algorithm. Eventually a result of each Reducer is also returned to task invoker and in turn Collator is invoked.
In case of task failure due to exception raised in task itself, exception is returned to task submitter. In other cases, failover policy along with load balancing policy decides how to migrate executable to other Infinispan nodes.
If you think that I omitted something and/or have suggestion let me know.
Regards,
Vladimir
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
13 years, 2 months
