[JBoss JIRA] (ISPN-4318) Infinispan should collect statistics for M/R tasks
by Alan Field (JIRA)
Alan Field created ISPN-4318:
--------------------------------
Summary: Infinispan should collect statistics for M/R tasks
Key: ISPN-4318
URL: https://issues.jboss.org/browse/ISPN-4318
Project: Infinispan
Issue Type: Feature Request
Components: Distributed Execution and Map/Reduce
Affects Versions: 7.0.0.Alpha4
Reporter: Alan Field
Assignee: Dan Berindei
Map/Reduce tasks should collect statistics during the task execution that can be returned to the user to help them determine the optimal settings for the task. Here are some thoughts on useful statistics:
Final status - completed, failed, cancelled, etc.
Duration - either overall, per node, per phase (map, reduce, combine, collate)
Number of nodes participating in the task
Keys in the intermediate cache
Keys in the result map
Node specific statistics:
Status of node - completed, failed, cancelled, etc.
Number of keys processed
Max size of collector
Here are the built in counters that are reported by Hadoop:
https://www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/chapte...
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-3680) Inconsistent way to use the column id in the JdbcBinaryCacheStore
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3680?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3680:
-----------------------------------------------
Ladislav Thon <lthon(a)redhat.com> changed the Status of [bug 1087205|https://bugzilla.redhat.com/show_bug.cgi?id=1087205] from ON_QA to VERIFIED
> Inconsistent way to use the column id in the JdbcBinaryCacheStore
> -----------------------------------------------------------------
>
> Key: ISPN-3680
> URL: https://issues.jboss.org/browse/ISPN-3680
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 5.2.7.Final, 5.3.0.Final, 6.0.0.CR1
> Reporter: Nicolas Filotto
> Assignee: Mircea Markus
> Fix For: 7.0.0.Alpha4, 5.2.9.Final
>
>
> I met some issues with (at least) H2 and JdbcBinaryCacheStore that prevent to modify a value in the cache store which is quite annoying. After a deeper look, I realized that it was due to the method {{JdbcBinaryCacheStore.loadBucket(Integer keyHashCode)}} that uses {{setInt}} instead of {{setString}} like in updateBucket and insertBucket to set the id as parameter to the query. With HSQLDB and MySQL, it works normally but with H2, the result of the query is always empty so it always tries to add a new entry which of course fails because of the integrity constraint on the primary key (aka the id).
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-3689) Preloading fails with JdbcBinaryCacheStore on DB2
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-3689?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-3689:
-----------------------------------------------
Ladislav Thon <lthon(a)redhat.com> changed the Status of [bug 1087209|https://bugzilla.redhat.com/show_bug.cgi?id=1087209] from ON_QA to VERIFIED
> Preloading fails with JdbcBinaryCacheStore on DB2
> --------------------------------------------------
>
> Key: ISPN-3689
> URL: https://issues.jboss.org/browse/ISPN-3689
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Affects Versions: 5.2.7.Final, 6.0.0.CR1
> Environment: DB2 9.7.5
> Reporter: Nicolas Filotto
> Assignee: Mircea Markus
> Fix For: 7.0.0.Alpha4, 5.2.9.Final
>
>
> I use the {{JdbcBinaryCacheStore}} with preloading enabled, when I test it on DB2 I get an exception of type:
> {code}
> 06.11.2013 16:27:51 *ERROR* [main] DataManipulationHelper: ISPN008007: SQL error while fetching all StoredEntries (DataManipulationHelper.java, line 253)
> com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601, SQLERRMC=?;r_quota" FETCH FIRST;<space>, DRIVER=4.13.80
> at com.ibm.db2.jcc.am.id.a(id.java:677)
> at com.ibm.db2.jcc.am.id.a(id.java:60)
> at com.ibm.db2.jcc.am.id.a(id.java:127)
> at com.ibm.db2.jcc.am.fo.c(fo.java:2653)
> at com.ibm.db2.jcc.am.fo.d(fo.java:2641)
> at com.ibm.db2.jcc.am.fo.a(fo.java:2090)
> at com.ibm.db2.jcc.am.go.a(go.java:7639)
> at com.ibm.db2.jcc.t4.cb.h(cb.java:141)
> at com.ibm.db2.jcc.t4.cb.b(cb.java:41)
> at com.ibm.db2.jcc.t4.q.a(q.java:32)
> at com.ibm.db2.jcc.t4.sb.i(sb.java:135)
> at com.ibm.db2.jcc.am.fo.ib(fo.java:2059)
> at com.ibm.db2.jcc.am.go.sc(go.java:3555)
> at com.ibm.db2.jcc.am.go.b(go.java:4344)
> at com.ibm.db2.jcc.am.go.fc(go.java:741)
> at com.ibm.db2.jcc.am.go.executeQuery(go.java:711)
> at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
> at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
> at org.infinispan.loaders.jdbc.DataManipulationHelper.loadSome(DataManipulationHelper.java:245)
> at org.infinispan.loaders.jdbc.binary.JdbcBinaryCacheStore.loadLockSafe(JdbcBinaryCacheStore.java:312)
> at org.infinispan.loaders.LockSupportCacheStore.load(LockSupportCacheStore.java:167)
> at org.infinispan.loaders.CacheLoaderManagerImpl.loadState(CacheLoaderManagerImpl.java:285)
> at org.infinispan.loaders.CacheLoaderManagerImpl.preload(CacheLoaderManagerImpl.java:238)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
> at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:886)
> at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:657)
> at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:646)
> at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:549)
> at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:217)
> at org.infinispan.CacheImpl.start(CacheImpl.java:582)
> at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:686)
> at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)
> at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)
> {code}
> I looks like you cannot use a parameter to set your query limit in case of DB2 9.7.5 at least
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-4317) Optimize server test suite
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-4317?page=com.atlassian.jira.plugin.... ]
Martin Gencur reassigned ISPN-4317:
-----------------------------------
Assignee: Jakub Markos (was: Martin Gencur)
> Optimize server test suite
> --------------------------
>
> Key: ISPN-4317
> URL: https://issues.jboss.org/browse/ISPN-4317
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Server
> Affects Versions: 7.0.0.Alpha4
> Reporter: Martin Gencur
> Assignee: Jakub Markos
>
> The goal is to make the test execution of server test suite shorter and minimize the number of maven profiles so that most of the tests run in a default profile.
> The first thing to do is to minimize the number of server restarts when running the test suite. This will require adding many different cache configurations to the same Infinispan subsystem.
> Also, look for other ways to minimize test execution time.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-4317) Optimize server test suite
by Martin Gencur (JIRA)
Martin Gencur created ISPN-4317:
-----------------------------------
Summary: Optimize server test suite
Key: ISPN-4317
URL: https://issues.jboss.org/browse/ISPN-4317
Project: Infinispan
Issue Type: Task
Components: Test Suite - Server
Affects Versions: 7.0.0.Alpha4
Reporter: Martin Gencur
Assignee: Martin Gencur
The goal is to make the test execution of server test suite shorter and minimize the number of maven profiles so that most of the tests run in a default profile.
The first thing to do is to minimize the number of server restarts when running the test suite. This will require adding many different cache configurations to the same Infinispan subsystem.
Also, look for other ways to minimize test execution time.
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-4283) Unable to setup SASL auth properties
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4283?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4283:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1097283|https://bugzilla.redhat.com/show_bug.cgi?id=1097283] from ON_QA to VERIFIED
> Unable to setup SASL auth properties
> ------------------------------------
>
> Key: ISPN-4283
> URL: https://issues.jboss.org/browse/ISPN-4283
> Project: Infinispan
> Issue Type: Bug
> Components: Server
> Reporter: Vojtech Juranek
> Assignee: Tristan Tarrant
> Fix For: 7.0.0.Beta1
>
>
> It seems that it's not possible to setup SASL auth properies. This is needed e.g. in case of DIGEST-MD5 authentication - {{org.jboss.sasl.digest.DigestMD5Server}}, which handles this auth., expects that properties contain {{com.sun.security.sasl.digest.realm}}, otherwise default to server name as realm name. However, it seems that proprties passed to {{DigestMD5Server}} are always {{null}}. Therefore it's not possible to refer any security realm named other than server name (e.g. standard {{ApplicationRealm}}).
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month
[JBoss JIRA] (ISPN-4288) JGroups kerberos auth is not able to obtain credentials
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4288?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4288:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1097758|https://bugzilla.redhat.com/show_bug.cgi?id=1097758] from NEW to CLOSED
> JGroups kerberos auth is not able to obtain credentials
> -------------------------------------------------------
>
> Key: ISPN-4288
> URL: https://issues.jboss.org/browse/ISPN-4288
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Server
> Reporter: Vojtech Juranek
> Assignee: Mircea Markus
>
> When trying to use kerberos for authentication between ISPN nodes (which uses JGroups authentication) I get following exception:
> {noformat}
> Caused by: java.lang.Exception: connecting to channel "clustered" failed
> at org.jgroups.JChannel._connect(JChannel.java:564)
> at org.jgroups.JChannel.connect(JChannel.java:288)
> at org.jgroups.JChannel.connect(JChannel.java:273)
> at org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded(JGroupsTransport.java:198)
> ... 27 more
> Caused by: java.lang.SecurityException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
> at org.jgroups.protocols.SASL.down(SASL.java:294)
> at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:347)
> at org.jgroups.protocols.pbcast.ClientGmsImpl.sendJoinMessage(ClientGmsImpl.java:243)
> at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:124)
> at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:40)
> at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1082)
> at org.jgroups.protocols.FlowControl.down(FlowControl.java:340)
> at org.jgroups.protocols.FlowControl.down(FlowControl.java:340)
> at org.jgroups.protocols.FRAG2.down(FRAG2.java:136)
> at org.jgroups.protocols.RSVP.down(RSVP.java:142)
> at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1039)
> at org.jgroups.JChannel.down(JChannel.java:785)
> at org.jgroups.JChannel._connect(JChannel.java:558)
> ... 30 more
> Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
> at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) [rt.jar:1.7.0_45]
> at org.jgroups.auth.sasl.SaslClientContext.addHeader(SaslClientContext.java:84)
> at org.jgroups.protocols.SASL.down(SASL.java:289)
> ... 42 more
> Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
> at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) [rt.jar:1.7.0_45]
> at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121) [rt.jar:1.7.0_45]
> at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) [rt.jar:1.7.0_45]
> at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223) [rt.jar:1.7.0_45]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) [rt.jar:1.7.0_45]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) [rt.jar:1.7.0_45]
> at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) [rt.jar:1.7.0_45]
> ... 44 more
> {noformat}
> The test and setup can be downloaded [here|https://github.com/vjuranek/infinispan/tree/ISPN-4273]. Not completely sure, if it's a bug or some kerberos setup issue, but it failed also in case when I tried to use already running kerberos (not ApacheDS run in within the test).
--
This message was sent by Atlassian JIRA
(v6.2.3#6260)
10 years, 1 month