[JBoss JIRA] (ISPN-6593) LevelDB tests fail on Windows
by Sebastian Łaskawiec (JIRA)
Sebastian Łaskawiec created ISPN-6593:
-----------------------------------------
Summary: LevelDB tests fail on Windows
Key: ISPN-6593
URL: https://issues.jboss.org/browse/ISPN-6593
Project: Infinispan
Issue Type: Bug
Components: Loaders and Stores
Environment: Apache Maven 3.3.9
Java version: 1.8.0_91, vendor: Oracle Corporation
Default locale: pl_PL, platform encoding: Cp1250
OS name: "windows 8.1", version: "6.3", arch: "amd64", family: "dos"
Reporter: Sebastian Łaskawiec
Assignee: Sebastian Łaskawiec
LevelDB JNI implementation has some problems with the way that Windows manages files.
Here are 2 major problems there:
# JDK doesn't always clean up files: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4715154
# A cleared DB object has {{null}} references inside and this results in throwing {{DBException}}s
Some stacktraces for investigation:
{noformat}
org.iq80.leveldb.DBException: Closed
at org.fusesource.leveldbjni.internal.JniDB.iterator(JniDB.java:100)
at org.infinispan.persistence.leveldb.LevelDBStore.clear(LevelDBStore.java:227)
at org.infinispan.persistence.manager.PersistenceManagerImpl.clearAllStores(PersistenceManagerImpl.java:399)
at org.infinispan.test.TestingUtil.clearCacheLoader(TestingUtil.java:733)
at org.infinispan.test.TestingUtil.clearContent(TestingUtil.java:680)
at org.infinispan.persistence.BaseStoreFunctionalTest.teardown(BaseStoreFunctionalTest.java:55)
at org.infinispan.test.SingleCacheManagerTest.destroyAfterMethod(SingleCacheManagerTest.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:786)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
{noformat}
org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.preload() on object of type PersistenceManagerImpl
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:172)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:864)
at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:633)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:622)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:547)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:238)
at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:890)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:635)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:585)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:451)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:437)
at org.infinispan.persistence.leveldb.config.ConfigurationTest.testXmlConfig(ConfigurationTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.persistence.spi.PersistenceException: java.lang.RuntimeException: org.fusesource.leveldbjni.internal.NativeDB$DBException: Invalid argument: not an sstable (bad magic number)
at org.infinispan.persistence.leveldb.LevelDBStore.process(LevelDBStore.java:315)
at org.infinispan.persistence.manager.PersistenceManagerImpl.preload(PersistenceManagerImpl.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
... 31 more
Caused by: java.lang.RuntimeException: org.fusesource.leveldbjni.internal.NativeDB$DBException: Invalid argument: not an sstable (bad magic number)
at org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:89)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
at org.infinispan.persistence.leveldb.LevelDBStore.process(LevelDBStore.java:298)
... 37 more
Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: Invalid argument: not an sstable (bad magic number)
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
at org.fusesource.leveldbjni.internal.NativeIterator.checkStatus(NativeIterator.java:121)
at org.fusesource.leveldbjni.internal.NativeIterator.key(NativeIterator.java:169)
at org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:87)
... 40 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPRK-22) InfinispanRDD is not fault tolerant
by Vojtech Juranek (JIRA)
[ https://issues.jboss.org/browse/ISPRK-22?page=com.atlassian.jira.plugin.s... ]
Vojtech Juranek commented on ISPRK-22:
--------------------------------------
you are right, this is not very sensible. In [fa56b5f072|https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072...] I pushed hopefully more meaningful one. However, now it looks like connector looses some data during failover:
{noformat}
[info] - RDD write failover *** FAILED ***
[info] 997 was not equal to 1000 (RDDFailOverSuite.scala:63)
{noformat}
> InfinispanRDD is not fault tolerant
> -----------------------------------
>
> Key: ISPRK-22
> URL: https://issues.jboss.org/browse/ISPRK-22
> Project: Infinispan Spark
> Issue Type: Bug
> Components: RDD
> Affects Versions: 0.3
> Reporter: Vojtech Juranek
> Assignee: Gustavo Fernandes
> Fix For: 0.4
>
>
> When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
> This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
> [Here|https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072ce24e0...] are some basic tests and example of exception thrown by HR client:
> {noformat}
> org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at scala.Option.map(Option.scala:146)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
> at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPRK-22) InfinispanRDD is not fault tolerant
by Vojtech Juranek (JIRA)
[ https://issues.jboss.org/browse/ISPRK-22?page=com.atlassian.jira.plugin.s... ]
Vojtech Juranek updated ISPRK-22:
---------------------------------
Description:
When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
[Here|https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072ce24e0...] are some basic tests and example of exception thrown by HR client:
{noformat}
org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
at scala.Option.map(Option.scala:146)
at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
... 21 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
... 25 more
{noformat}
was:
When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
[Here|https://github.com/vjuranek/infinispan-spark/commit/c4a3101624e76d61...] are some basic tests and example of exception thrown by HR client:
{noformat}
org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
at scala.Option.map(Option.scala:146)
at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
... 21 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
... 25 more
{noformat}
Steps to Reproduce: Some basic tests: https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072ce24e055036... (was: Some basic tests: https://github.com/vjuranek/infinispan-spark/commit/c4a3101624e76d61097c4...)
> InfinispanRDD is not fault tolerant
> -----------------------------------
>
> Key: ISPRK-22
> URL: https://issues.jboss.org/browse/ISPRK-22
> Project: Infinispan Spark
> Issue Type: Bug
> Components: RDD
> Affects Versions: 0.3
> Reporter: Vojtech Juranek
> Assignee: Gustavo Fernandes
> Fix For: 0.4
>
>
> When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
> This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
> [Here|https://github.com/vjuranek/infinispan-spark/commit/fa56b5f072ce24e0...] are some basic tests and example of exception thrown by HR client:
> {noformat}
> org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at scala.Option.map(Option.scala:146)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
> at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPN-6076) Errors when the same column is both a gouping column and an aggregation column
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-6076?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-6076:
-----------------------------------------------
Adrian Nistor <anistor(a)redhat.com> changed the Status of [bug 1333297|https://bugzilla.redhat.com/show_bug.cgi?id=1333297] from NEW to POST
> Errors when the same column is both a gouping column and an aggregation column
> ------------------------------------------------------------------------------
>
> Key: ISPN-6076
> URL: https://issues.jboss.org/browse/ISPN-6076
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 8.1.0.Final, 8.0.2.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 8.2.0.Beta1, 8.2.0.Final, 8.1.1.Final
>
>
> This should not be allowed.
> {code}
> public void testGroupingAndAggregationOnSameField() {
> QueryFactory qf = getQueryFactory();
> Query q = qf.from(getModelFactory().getUserImplClass())
> .select(Expression.count("surname"), Expression.sum("addresses.number"))
> .groupBy("surname")
> .build();
> List<Object[]> list = q.list(); // throws ClassCastException
> }
> {code}
> {code}
> java.lang.ClassCastException: java.lang.String cannot be cast to org.infinispan.objectfilter.impl.aggregation.Counter
> at org.infinispan.objectfilter.impl.aggregation.CountAccumulator.merge(CountAccumulator.java:24)
> at org.infinispan.objectfilter.impl.aggregation.FieldAccumulator.merge(FieldAccumulator.java:42)
> at org.infinispan.objectfilter.impl.aggregation.Grouper.addRow(Grouper.java:133)
> at org.infinispan.query.dsl.embedded.impl.AggregatingQuery.getBaseIterator(AggregatingQuery.java:52)
> at org.infinispan.query.dsl.embedded.impl.HybridQuery$1.<init>(HybridQuery.java:47)
> at org.infinispan.query.dsl.embedded.impl.HybridQuery.getIterator(HybridQuery.java:45)
> at org.infinispan.query.dsl.embedded.impl.BaseEmbeddedQuery.listInternal(BaseEmbeddedQuery.java:65)
> at org.infinispan.query.dsl.embedded.impl.BaseEmbeddedQuery.list(BaseEmbeddedQuery.java:57)
> at org.infinispan.query.dsl.embedded.impl.DelegatingQuery.list(DelegatingQuery.java:45)
> at org.infinispan.query.dsl.embedded.QueryDslConditionsTest.testEmbeddedSum(QueryDslConditionsTest.java:1959)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPN-6076) Errors when the same column is both a gouping column and an aggregation column
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-6076?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration updated ISPN-6076:
------------------------------------------
Bugzilla References: https://bugzilla.redhat.com/show_bug.cgi?id=1333297
Bugzilla Update: Perform
> Errors when the same column is both a gouping column and an aggregation column
> ------------------------------------------------------------------------------
>
> Key: ISPN-6076
> URL: https://issues.jboss.org/browse/ISPN-6076
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 8.1.0.Final, 8.0.2.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 8.2.0.Beta1, 8.2.0.Final, 8.1.1.Final
>
>
> This should not be allowed.
> {code}
> public void testGroupingAndAggregationOnSameField() {
> QueryFactory qf = getQueryFactory();
> Query q = qf.from(getModelFactory().getUserImplClass())
> .select(Expression.count("surname"), Expression.sum("addresses.number"))
> .groupBy("surname")
> .build();
> List<Object[]> list = q.list(); // throws ClassCastException
> }
> {code}
> {code}
> java.lang.ClassCastException: java.lang.String cannot be cast to org.infinispan.objectfilter.impl.aggregation.Counter
> at org.infinispan.objectfilter.impl.aggregation.CountAccumulator.merge(CountAccumulator.java:24)
> at org.infinispan.objectfilter.impl.aggregation.FieldAccumulator.merge(FieldAccumulator.java:42)
> at org.infinispan.objectfilter.impl.aggregation.Grouper.addRow(Grouper.java:133)
> at org.infinispan.query.dsl.embedded.impl.AggregatingQuery.getBaseIterator(AggregatingQuery.java:52)
> at org.infinispan.query.dsl.embedded.impl.HybridQuery$1.<init>(HybridQuery.java:47)
> at org.infinispan.query.dsl.embedded.impl.HybridQuery.getIterator(HybridQuery.java:45)
> at org.infinispan.query.dsl.embedded.impl.BaseEmbeddedQuery.listInternal(BaseEmbeddedQuery.java:65)
> at org.infinispan.query.dsl.embedded.impl.BaseEmbeddedQuery.list(BaseEmbeddedQuery.java:57)
> at org.infinispan.query.dsl.embedded.impl.DelegatingQuery.list(DelegatingQuery.java:45)
> at org.infinispan.query.dsl.embedded.QueryDslConditionsTest.testEmbeddedSum(QueryDslConditionsTest.java:1959)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPRK-22) InfinispanRDD is not fault tolerant
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPRK-22?page=com.atlassian.jira.plugin.s... ]
Gustavo Fernandes updated ISPRK-22:
-----------------------------------
Fix Version/s: 0.4
> InfinispanRDD is not fault tolerant
> -----------------------------------
>
> Key: ISPRK-22
> URL: https://issues.jboss.org/browse/ISPRK-22
> Project: Infinispan Spark
> Issue Type: Bug
> Components: RDD
> Affects Versions: 0.3
> Reporter: Vojtech Juranek
> Assignee: Gustavo Fernandes
> Fix For: 0.4
>
>
> When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
> This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
> [Here|https://github.com/vjuranek/infinispan-spark/commit/c4a3101624e76d61...] are some basic tests and example of exception thrown by HR client:
> {noformat}
> org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at scala.Option.map(Option.scala:146)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
> at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (ISPRK-22) InfinispanRDD is not fault tolerant
by Gustavo Fernandes (JIRA)
[ https://issues.jboss.org/browse/ISPRK-22?page=com.atlassian.jira.plugin.s... ]
Gustavo Fernandes commented on ISPRK-22:
----------------------------------------
With relation to the writes, it is not possible to failover simply by calling
{{rdd1.writeToInfinispan(config)}}
if config has only 1 unreachable server; rdd1 is not an InfinispanRDD, so the config must have at least 1 reachable server
> InfinispanRDD is not fault tolerant
> -----------------------------------
>
> Key: ISPRK-22
> URL: https://issues.jboss.org/browse/ISPRK-22
> Project: Infinispan Spark
> Issue Type: Bug
> Components: RDD
> Affects Versions: 0.3
> Reporter: Vojtech Juranek
> Assignee: Gustavo Fernandes
>
> When primary ISPN server fails during processing InfinispanRDD, Spark is not able to overcome this failure.
> This is caused by re-creating {{RemoteCachManager}} with pre-configured ISPN server address (for read [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...], for writes [here|https://github.com/infinispan/infinispan-spark/blob/master/src/main/...]), so when this server fails during RDD processing and Spark calls some function, which under the hood creates {{RemoteCacheManager}}, it will fail with connection refused exception.
> [Here|https://github.com/vjuranek/infinispan-spark/commit/c4a3101624e76d61...] are some basic tests and example of exception thrown by HR client:
> {noformat}
> org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:395)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:241)
> at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:26)
> at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:53)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:490)
> at org.infinispan.client.hotrod.impl.RemoteCacheImpl.resolveCompatibility(RemoteCacheImpl.java:551)
> at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:341)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:222)
> at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:217)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD$$anonfun$1.apply(InfinispanRDD.scala:52)
> at scala.Option.map(Option.scala:146)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:52)
> at org.infinispan.spark.rdd.InfinispanRDD.compute(InfinispanRDD.scala:66)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /127.0.0.1:11222
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:78)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
> at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
> at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:390)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:111)
> at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:68)
> ... 25 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months