[JBoss JIRA] (ISPN-7000) Infinispan distributed task execution without serialization
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-7000?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño edited comment on ISPN-7000 at 9/14/16 3:30 AM:
-----------------------------------------------------------------
Cross posted from http://stackoverflow.com/questions/39332737/infinispan-distributed-task-e...
was (Author: galder.zamarreno):
Cross posted from http://stackoverflow.com/questions/39332737/infinispan-distributed-task-e...
> Infinispan distributed task execution without serialization
> ------------------------------------------------------------
>
> Key: ISPN-7000
> URL: https://issues.jboss.org/browse/ISPN-7000
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 8.0.2.Final
> Reporter: mohammad nadeem
>
> 0
> down vote
> favorite
> I am aware of simple distributed task execution (Using Distributed Executor Service), for example [here ](https://github.com/infinispan/infinispan-simple-tutorials/blob/master/distexec/src/main/java/org/infinispan/tutorial/simple/distexec/InfinispanDistExec.java) and [here](https://github.com/infinispan/infinispan-quickstart/tree/master/clu..., However the mechanism used is to serialize the callable and transport it to destination for execution, as I said this works for simple cases, many a times it wont be feasible to searlize every thing, for example from callable lots of classes would be referred to get a task done, in this case everything should be transported (many a times not possible, even if it is possible, it would take humongous time to transport accross, there by killing the purpose), what would be required instead is to assume that the classes exists at the target system and simply invoke it (the way hadoop works), I am not sure if there exists such execution model in infinispan?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-7000) Infinispan distributed task execution without serialization
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-7000?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño commented on ISPN-7000:
----------------------------------------
Cross posted from http://stackoverflow.com/questions/39332737/infinispan-distributed-task-e...
> Infinispan distributed task execution without serialization
> ------------------------------------------------------------
>
> Key: ISPN-7000
> URL: https://issues.jboss.org/browse/ISPN-7000
> Project: Infinispan
> Issue Type: Feature Request
> Components: Distributed Execution and Map/Reduce
> Affects Versions: 8.0.2.Final
> Reporter: mohammad nadeem
>
> 0
> down vote
> favorite
> I am aware of simple distributed task execution (Using Distributed Executor Service), for example [here ](https://github.com/infinispan/infinispan-simple-tutorials/blob/master/distexec/src/main/java/org/infinispan/tutorial/simple/distexec/InfinispanDistExec.java) and [here](https://github.com/infinispan/infinispan-quickstart/tree/master/clu..., However the mechanism used is to serialize the callable and transport it to destination for execution, as I said this works for simple cases, many a times it wont be feasible to searlize every thing, for example from callable lots of classes would be referred to get a task done, in this case everything should be transported (many a times not possible, even if it is possible, it would take humongous time to transport accross, there by killing the purpose), what would be required instead is to assume that the classes exists at the target system and simply invoke it (the way hadoop works), I am not sure if there exists such execution model in infinispan?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6806) Exception serializing L1InternalCacheEntry during state transfer
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6806?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6806:
----------------------------------
Fix Version/s: 8.1.5.Final
8.2.5.Final
> Exception serializing L1InternalCacheEntry during state transfer
> ----------------------------------------------------------------
>
> Key: ISPN-6806
> URL: https://issues.jboss.org/browse/ISPN-6806
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.5.Final, 9.0.0.Alpha2, 8.2.2.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
>
> {{L1InternalCacheEntry}} is not serializable, as only entries properly owned by the node are supposed to be sent via state transfer.
> However, when the last owner of a segment leaves the cluster, one of the other nodes becomes the owner of that segment, and the L1 entries it already has suddenly become part of the state it can send to other nodes. It then fails to send its state with this exception:
> {noformat}
> [org.infinispan.marshall.core.VersionAwareMarshaller] (transport-thread--p19-t14) Object is not serializable: java.io.NotSerializableException: org.infinispan.container.entries.L1InternalCacheEntry
> {noformat}
> This can happen quite often if L1 is enabled and {{numOwners = 1}}.
> Unfortunately, the serialization exception isn't propagated, and all the user sees is an initial state transfer timeout exception when trying to start another node later.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6806) Exception serializing L1InternalCacheEntry during state transfer
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6806?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6806:
----------------------------------
Status: Open (was: New)
> Exception serializing L1InternalCacheEntry during state transfer
> ----------------------------------------------------------------
>
> Key: ISPN-6806
> URL: https://issues.jboss.org/browse/ISPN-6806
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 7.2.5.Final, 9.0.0.Alpha2, 8.2.2.Final
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
>
> {{L1InternalCacheEntry}} is not serializable, as only entries properly owned by the node are supposed to be sent via state transfer.
> However, when the last owner of a segment leaves the cluster, one of the other nodes becomes the owner of that segment, and the L1 entries it already has suddenly become part of the state it can send to other nodes. It then fails to send its state with this exception:
> {noformat}
> [org.infinispan.marshall.core.VersionAwareMarshaller] (transport-thread--p19-t14) Object is not serializable: java.io.NotSerializableException: org.infinispan.container.entries.L1InternalCacheEntry
> {noformat}
> This can happen quite often if L1 is enabled and {{numOwners = 1}}.
> Unfortunately, the serialization exception isn't propagated, and all the user sees is an initial state transfer timeout exception when trying to start another node later.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant commented on ISPN-6857:
---------------------------------------
An Invalidation cache doesn't transfer state, only invalidations, so the "missing" StateTransferInterceptor is not a mistake. However the analysis is correct, the remove should be retried.
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Marek Posolda
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reassigned ISPN-6857:
-------------------------------------
Assignee: Dan Berindei
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Marek Posolda
> Assignee: Dan Berindei
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6857:
----------------------------------
Fix Version/s: 8.1.5.Final
9.0.0.Final
8.2.5.Final
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Marek Posolda
> Assignee: Dan Berindei
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6857) OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-6857?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-6857:
----------------------------------
Status: Open (was: New)
> OutdatedTopologyException in clustered invalidation cache because StateTransferInterceptor not in the chain
> -----------------------------------------------------------------------------------------------------------
>
> Key: ISPN-6857
> URL: https://issues.jboss.org/browse/ISPN-6857
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Marek Posolda
> Assignee: Dan Berindei
> Fix For: 8.1.5.Final, 9.0.0.Final, 8.2.5.Final
>
> Attachments: OutdatedTopologyExceptionReproducerTest.java
>
>
> I have the following setup:
> - 2 nodes in cluster with mode INVALIDATION_SYNC. No-transaction cache.
> - Node1 is started
> - Called "cache.remove" on some key on node1. At the same time, node2 is starting, which is causing topology change.
> - The "cache.remove" call on node1 is throwing OutdatedTopologyException.
> I found the cause is that StateTransferInterceptor is not added in InterceptorChain during INVALIDATION mode. It's just available during REPLICATION or DISTRIBUTED modes - https://github.com/infinispan/infinispan/blob/master/core/src/main/java/o...
> Indeed when I manually added StateTransferInterceptor to my invalidation cache:
> {code}
> invalidationConfigBuilder.customInterceptors()
> .addInterceptor()
> .before(NonTransactionalLockingInterceptor.class)
> .interceptorClass(StateTransferInterceptor.class);
> {code}
>
> I can see that issue is gone as OutdatedTopologyException is catched and command is retried with new topology.
> I am attaching the Java unit test for reproducing issue. On my laptop when I run it, I can almost always simulate the issue.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6980) Provide a manual method to sync sites
by Alan Field (JIRA)
[ https://issues.jboss.org/browse/ISPN-6980?page=com.atlassian.jira.plugin.... ]
Alan Field commented on ISPN-6980:
----------------------------------
[~NadirX] and [~mgencur], I'm pinging you both in this JIRA, since I reported it under my personal account. That's why it's hard for me to find!
> Provide a manual method to sync sites
> -------------------------------------
>
> Key: ISPN-6980
> URL: https://issues.jboss.org/browse/ISPN-6980
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication
> Affects Versions: 9.0.0.Alpha4, 8.2.4.Final
> Reporter: Alan Field
>
> When switching between sites and especially when the replication between sites it async, cache operators should be able to sync any changes that haven't been replicated before taking the site offline.
> It also might be useful to be able to do a controlled shutdown of a site that sends all pending changes to the backup site, and then shuts down the cluster.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years
[JBoss JIRA] (ISPN-6980) Provide a manual method to sync sites
by Alan Field (JIRA)
[ https://issues.jboss.org/browse/ISPN-6980?page=com.atlassian.jira.plugin.... ]
Alan Field updated ISPN-6980:
-----------------------------
Issue Type: Enhancement (was: Feature Request)
> Provide a manual method to sync sites
> -------------------------------------
>
> Key: ISPN-6980
> URL: https://issues.jboss.org/browse/ISPN-6980
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication
> Affects Versions: 9.0.0.Alpha4, 8.2.4.Final
> Reporter: Alan Field
>
> When switching between sites and especially when the replication between sites it async, cache operators should be able to sync any changes that haven't been replicated before taking the site offline.
> It also might be useful to be able to do a controlled shutdown of a site that sends all pending changes to the backup site, and then shuts down the cluster.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years