[JBoss JIRA] (ISPN-7580) Use of marsheller is not consistent in all places
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/ISPN-7580?page=com.atlassian.jira.plugin.... ]
Ramesh Reddy commented on ISPN-7580:
------------------------------------
[~anistor] I know exposing the the "impl" interface without moving to a different package without refactoring further is bit hacky, but I did not wanted to do wider changes without knowing the what is public vs private API and potential for regression. If you want to discuss/provide the layer at the Serialization context level, I am willing to bridge the gap with another patch.
> Use of marsheller is not consistent in all places
> -------------------------------------------------
>
> Key: ISPN-7580
> URL: https://issues.jboss.org/browse/ISPN-7580
> Project: Infinispan
> Issue Type: Bug
> Components: Marshalling, Remote Querying
> Reporter: Ramesh Reddy
> Assignee: Adrian Nistor
>
> Usage of extended ProtoStreamMarshaller is not consistent across all the code paths. For the purposes of Teiid translator, I have extended ProtoStreamMarshaller which knows to read/write byte streams in portable fashion for given message type, which are representions of a relational table in Teiid. This works fine, if I just use cache's get/put calls.
> However, the same fails when used with RemoteQuery or Continuous query. The reason is, these classes circumvent extended Marsheller and go directly to serialization context registered to do the wrapping/unwrapping. Not only that there are few places code will type cast the SerializationContext to SerializationContextImpl object. Thus I can not even provide my own Serializer nor I can extend this as SerializationContextImpl is declared as final. These need to be corrected such that extended marsheller is used rather than hard coding them.
> I am guessing this is first time anyone has even done this without using dedicated java classes as marshellers.
> This is extremely critical for me to be fixed to move forward, I can provide the pull request for it?
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (ISPN-6713) RemoteQueryDslConditionsTest.testIsNullNumericWithProjection1 fails
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-6713?page=com.atlassian.jira.plugin.... ]
Adrian Nistor updated ISPN-6713:
--------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/4998
> RemoteQueryDslConditionsTest.testIsNullNumericWithProjection1 fails
> -------------------------------------------------------------------
>
> Key: ISPN-6713
> URL: https://issues.jboss.org/browse/ISPN-6713
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 8.0.0.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 9.0.0.Final
>
>
> {code}
> java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)
> at org.apache.lucene.util.NumericUtils.getPrefixCodedIntShift(NumericUtils.java:216)
> at org.apache.lucene.util.NumericUtils$2.accept(NumericUtils.java:504)
> at org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:232)
> at org.apache.lucene.uninverting.FieldCacheImpl$Uninvert.uninvert(FieldCacheImpl.java:285)
> at org.apache.lucene.uninverting.FieldCacheImpl$LongCache.createValue(FieldCacheImpl.java:552)
> at org.apache.lucene.uninverting.FieldCacheImpl$Cache.get(FieldCacheImpl.java:189)
> at org.apache.lucene.uninverting.FieldCacheImpl.getNumerics(FieldCacheImpl.java:469)
> at org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(UninvertingReader.java:233)
> at org.apache.lucene.index.DocValues.getNumeric(DocValues.java:225)
> at org.apache.lucene.search.FieldComparator$NumericComparator.getNumericDocValues(FieldComparator.java:167)
> at org.apache.lucene.search.FieldComparator$NumericComparator.doSetNextReader(FieldComparator.java:153)
> at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
> at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
> at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:812)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:523)
> at org.hibernate.search.query.engine.impl.LazyQueryState.search(LazyQueryState.java:103)
> at org.hibernate.search.query.engine.impl.QueryHits.updateTopDocs(QueryHits.java:241)
> at org.hibernate.search.query.engine.impl.QueryHits.<init>(QueryHits.java:136)
> at org.hibernate.search.query.engine.impl.QueryHits.<init>(QueryHits.java:105)
> at org.hibernate.search.query.engine.impl.LuceneHSQuery.getQueryHits(LuceneHSQuery.java:313)
> at org.hibernate.search.query.engine.impl.LuceneHSQuery.queryEntityInfos(LuceneHSQuery.java:143)
> at org.infinispan.query.impl.CacheQueryImpl.list(CacheQueryImpl.java:161)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQuery.listInternal(EmbeddedLuceneQuery.java:75)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQuery.list(EmbeddedLuceneQuery.java:69)
> at org.infinispan.query.remote.impl.QueryFacadeImpl.makeResponse(QueryFacadeImpl.java:87)
> at org.infinispan.query.remote.impl.QueryFacadeImpl.query(QueryFacadeImpl.java:64)
> at org.infinispan.server.hotrod.HotRodServer.query(HotRodServer.scala:78)
> at org.infinispan.server.hotrod.ContextHandler.realRead(ContextHandler.java:146)
> at org.infinispan.server.hotrod.ContextHandler.lambda$channelRead0$1(ContextHandler.java:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (ISPN-7580) Use of marsheller is not consistent in all places
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/ISPN-7580?page=com.atlassian.jira.plugin.... ]
Ramesh Reddy commented on ISPN-7580:
------------------------------------
[~anistor] I am trying to integrate Teiid at a lower level then exposed JAVA Marshaller API level in a nutshell.
Current Teiid implementation of JDG translator (with lesser part of Infinispan, as there is no community version), utilizes the JAVA marshall/entity class files created (sometimes generated) for .proto/and annotations, then uses these java class files for CRUD operations. This uses the Query DSL for querying. In the end uses java reflection utilities to scrap the data off the java objects into tabular tuples. With this we have lot of usability issues, in terms whole JAVA entity layer and classloaders and Teiid use of resource adapters and dynamic nature we need for materialization (caching of the table) feature.
So, we decided to rewrite this translator again from ground up. In Teiid, we already have metadata layer and query layer. Now, where given the Protobuf definition and portable nature of Infinispan's hotrod protocol, I have converted the a "message" definition in the .proto file into a relational table definition. So, now I can issue CRUD queries against this table, which are against this message. At the same time, using the rest of the information in .proto file about the tags and data type information, I can generate a dynamic Marshaller that represents table encoding/decoding. Now all I need is way to inject this in/out this marshaller when my usecase in play. Thus the required changes to make it SerializationContext generic and ability to replace stock SerializationContext.
I did have to duplicate small portions of ProtobufReader/Writer classes in my code, but I do not think they are significant enough for concern for us to think about maintainability in future. I have already written such layer and testing it currently. So, what this does is, we can entirely throw away Teiid's dependence of JAVA definition of entity/marshaller classes. Then on top it, I use the new {{Ickle}} for query layer. Once completed, I can simply connect to the Infinispan cache and read the registered .proto files, generate compatible schema (DDL) for Teiid's VDB, and let user issue SQL queries against it right away.
Given a relational schema (DDL), I can even generate (not done yet) an equivalent .proto file and register with Infinispan. Once the remote cache creation feature is available, then I can effectively replace our internal materialization cache with Infinispan implementation as seamless integration.
Hope this gives clear context of what I am trying to do. If you are interested, I can show a demo once at a stable state.
> Use of marsheller is not consistent in all places
> -------------------------------------------------
>
> Key: ISPN-7580
> URL: https://issues.jboss.org/browse/ISPN-7580
> Project: Infinispan
> Issue Type: Bug
> Components: Marshalling, Remote Querying
> Reporter: Ramesh Reddy
> Assignee: Adrian Nistor
>
> Usage of extended ProtoStreamMarshaller is not consistent across all the code paths. For the purposes of Teiid translator, I have extended ProtoStreamMarshaller which knows to read/write byte streams in portable fashion for given message type, which are representions of a relational table in Teiid. This works fine, if I just use cache's get/put calls.
> However, the same fails when used with RemoteQuery or Continuous query. The reason is, these classes circumvent extended Marsheller and go directly to serialization context registered to do the wrapping/unwrapping. Not only that there are few places code will type cast the SerializationContext to SerializationContextImpl object. Thus I can not even provide my own Serializer nor I can extend this as SerializationContextImpl is declared as final. These need to be corrected such that extended marsheller is used rather than hard coding them.
> I am guessing this is first time anyone has even done this without using dedicated java classes as marshellers.
> This is extremely critical for me to be fixed to move forward, I can provide the pull request for it?
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (ISPN-6713) RemoteQueryDslConditionsTest.testIsNullNumericWithProjection1 fails
by Adrian Nistor (JIRA)
[ https://issues.jboss.org/browse/ISPN-6713?page=com.atlassian.jira.plugin.... ]
Adrian Nistor commented on ISPN-6713:
-------------------------------------
The test failure is not longer reproducible with currently used lucene version.
> RemoteQueryDslConditionsTest.testIsNullNumericWithProjection1 fails
> -------------------------------------------------------------------
>
> Key: ISPN-6713
> URL: https://issues.jboss.org/browse/ISPN-6713
> Project: Infinispan
> Issue Type: Bug
> Components: Remote Querying
> Affects Versions: 8.0.0.Final
> Reporter: Adrian Nistor
> Assignee: Adrian Nistor
> Fix For: 9.0.0.Final
>
>
> {code}
> java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)
> at org.apache.lucene.util.NumericUtils.getPrefixCodedIntShift(NumericUtils.java:216)
> at org.apache.lucene.util.NumericUtils$2.accept(NumericUtils.java:504)
> at org.apache.lucene.index.FilteredTermsEnum.next(FilteredTermsEnum.java:232)
> at org.apache.lucene.uninverting.FieldCacheImpl$Uninvert.uninvert(FieldCacheImpl.java:285)
> at org.apache.lucene.uninverting.FieldCacheImpl$LongCache.createValue(FieldCacheImpl.java:552)
> at org.apache.lucene.uninverting.FieldCacheImpl$Cache.get(FieldCacheImpl.java:189)
> at org.apache.lucene.uninverting.FieldCacheImpl.getNumerics(FieldCacheImpl.java:469)
> at org.apache.lucene.uninverting.UninvertingReader.getNumericDocValues(UninvertingReader.java:233)
> at org.apache.lucene.index.DocValues.getNumeric(DocValues.java:225)
> at org.apache.lucene.search.FieldComparator$NumericComparator.getNumericDocValues(FieldComparator.java:167)
> at org.apache.lucene.search.FieldComparator$NumericComparator.doSetNextReader(FieldComparator.java:153)
> at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
> at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
> at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:812)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:523)
> at org.hibernate.search.query.engine.impl.LazyQueryState.search(LazyQueryState.java:103)
> at org.hibernate.search.query.engine.impl.QueryHits.updateTopDocs(QueryHits.java:241)
> at org.hibernate.search.query.engine.impl.QueryHits.<init>(QueryHits.java:136)
> at org.hibernate.search.query.engine.impl.QueryHits.<init>(QueryHits.java:105)
> at org.hibernate.search.query.engine.impl.LuceneHSQuery.getQueryHits(LuceneHSQuery.java:313)
> at org.hibernate.search.query.engine.impl.LuceneHSQuery.queryEntityInfos(LuceneHSQuery.java:143)
> at org.infinispan.query.impl.CacheQueryImpl.list(CacheQueryImpl.java:161)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQuery.listInternal(EmbeddedLuceneQuery.java:75)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQuery.list(EmbeddedLuceneQuery.java:69)
> at org.infinispan.query.remote.impl.QueryFacadeImpl.makeResponse(QueryFacadeImpl.java:87)
> at org.infinispan.query.remote.impl.QueryFacadeImpl.query(QueryFacadeImpl.java:64)
> at org.infinispan.server.hotrod.HotRodServer.query(HotRodServer.scala:78)
> at org.infinispan.server.hotrod.ContextHandler.realRead(ContextHandler.java:146)
> at org.infinispan.server.hotrod.ContextHandler.lambda$channelRead0$1(ContextHandler.java:56)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months