[JBoss JIRA] (ISPRK-22) InfinispanRDD is not fault tolerant
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPRK-22?page=com.atlassian.jira.plugin.s... ]
Gustavo Fernandes updated ISPRK-22:
-----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> InfinispanRDD is not fault tolerant
> -----------------------------------
>
> Key: ISPRK-22
> URL: https://issues.jboss.org/browse/ISPRK-22
> Project: Infinispan Spark
> Issue Type: Bug
> Components: RDD
> Affects Versions: 0.3
> Reporter: Vojtech Juranek
> Assignee: Gustavo Fernandes
> Fix For: 0.4
>
>
> When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
> This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
> [Here|https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072ce24e0...] are some basic tests and example of exception thrown by HR client:
> {noformat}
> org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at scala.Option.map(Option.scala:146)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
> at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6874) Merge suite.security and suite.security.manual profiles
by Vojtech Juranek (JIRA)
[ https://issues.jboss.org/browse/ISPN-6874?page=com.atlassian.jira.plugin.... ]
Vojtech Juranek closed ISPN-6874.
---------------------------------
Release Notes Text: Actually this is already done in ISPN.
Resolution: Rejected
> Merge suite.security and suite.security.manual profiles
> -------------------------------------------------------
>
> Key: ISPN-6874
> URL: https://issues.jboss.org/browse/ISPN-6874
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Server
> Reporter: Vojtech Juranek
> Assignee: Vojtech Juranek
>
> Server testsuite profiles {{suite.security}} and {{suite.security.manual}} don't work well, running any of these two profiles will trigger all security tests and tests contained in former or later profile fail. Moreover there's no reason to keep two separate profiles as none of these profiles is run by CI.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months
[JBoss JIRA] (ISPN-6874) Merge suite.security and suite.security.manual profiles
by Vojtech Juranek (JIRA)
Vojtech Juranek created ISPN-6874:
-------------------------------------
Summary: Merge suite.security and suite.security.manual profiles
Key: ISPN-6874
URL: https://issues.jboss.org/browse/ISPN-6874
Project: Infinispan
Issue Type: Task
Components: Test Suite - Server
Reporter: Vojtech Juranek
Assignee: Vojtech Juranek
Server testsuite profiles {{suite.security}} and {{suite.security.manual}} don't work well, running any of these two profiles will trigger all security tests and tests contained in former or later profile fail. Moreover there's no reason to keep two separate profiles as none of these profiles is run by CI.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 6 months