[teiid-issues] [JBoss JIRA] (TEIID-4600) Infinispan DSL Translator metadata import: optional protobuf fields are not nullable in imported tables

Jan Stastny created TEIID-4600: ---------------------------------- Summary: Infinispan DSL Translator metadata import: optional protobuf fields are not nullable in imported tables Key: TEIID-4600 URL: https://issues.jboss.org/browse/TEIID-4600 Project: Teiid Issue Type: Bug Components: Misc. Connectors Affects Versions: 8.12.7.6_3 Reporter: Jan Stastny Assignee: Steven Hawkins Priority: Critical When using Infinispan DSL translator in use case with protobuf descriptor and custom marshaller then optional protobuf fields are not imported as nullable. This means all the columns have to be set when doing an insert. Protobuf descriptor: {code:java|title=SmallA.proto} package org.jboss.qe.jdg.remote.protobuf; message SmallA { required int32 intKey = 1; optional string stringNum = 2; required string stringKey = 3; optional float floatNum = 4; optional string bigIntegerValue = 5; optional int32 shortValue = 6; optional double doubleNum = 7; optional bytes objectValue = 8; optional int32 intNum = 9; optional string bigDecimalValue = 10; optional int64 longNum = 11; optional bool booleanValue = 12; optional int64 timeStampValue = 13; optional int32 byteNum = 14; optional int64 timeValue = 15; optional int64 dateValue = 16; optional int32 charValue = 17; } {code} VDB: {code:xml} <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <vdb name="teiid4590" version="1"> <model name="jdgSource1" type="PHYSICAL"> <source name="jdgSource1" connection-jndi-name="java:/infinispanRemoteDSL" translator-name="infinispan-cache-dsl" /> </model> </vdb> {code} SQL query used: {code:sql} INSERT INTO jdgSource1.SmallA(intKey,stringKey) VALUES(1,'1') {code} Error on execute: {code} 09:47:16,828 INFO [MultiPlatformProcessRunner] 09:47:16,827 WARN [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue0) TEIID30020 Processing exception for request WwSb1MLjGZXu.0 'TEIID30492 Element jdgSource1.SmallA.booleanValue of jdgSource1.SmallA is neither nullable nor has a default value. A value must be specified in the insert.'. Originally QueryValidatorException Request.java:334. Enable more detailed logging to see the entire stacktrace. 09:47:16,875 INFO [TeiidConnectionUtils] Closing connection org.teiid.jdbc.ConnectionImpl@55dfebeb 09:47:16,878 ERROR [LoggingTestListener] Error in test. org.teiid.jdbc.TeiidSQLException: TEIID30492 Element jdgSource1.SmallA.booleanValue of jdgSource1.SmallA is neither nullable nor has a default value. A value must be specified in the insert. at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:721) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:64) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:560) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:268) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:306) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102] at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:98) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at com.sun.proxy.$Proxy31.read(Unknown Source) ~[?:?] at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:405) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:569) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:1063) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:338) ~[teiid-jdbc-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.jboss.qe.dvqe.reproducers.Teiid4590Test.testStagingTableMetadataLoad(Teiid4590Test.java:86) ~[test-classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102] at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85) ~[testng-6.8.21.jar:?] at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:200) ~[testng-6.8.21.jar:?] at org.jboss.qe.tests.listeners.DBAllocatorAwareTestListener.run(DBAllocatorAwareTestListener.java:808) ~[db-annotations-0.1-SNAPSHOT.jar:?] at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:212) ~[testng-6.8.21.jar:?] at org.testng.internal.Invoker.invokeMethod(Invoker.java:689) [testng-6.8.21.jar:?] at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:882) [testng-6.8.21.jar:?] at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1189) [testng-6.8.21.jar:?] at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:124) [testng-6.8.21.jar:?] at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108) [testng-6.8.21.jar:?] at org.testng.TestRunner.privateRun(TestRunner.java:767) [testng-6.8.21.jar:?] at org.testng.TestRunner.run(TestRunner.java:617) [testng-6.8.21.jar:?] at org.testng.SuiteRunner.runTest(SuiteRunner.java:348) [testng-6.8.21.jar:?] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343) [testng-6.8.21.jar:?] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305) [testng-6.8.21.jar:?] at org.testng.SuiteRunner.run(SuiteRunner.java:254) [testng-6.8.21.jar:?] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) [testng-6.8.21.jar:?] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) [testng-6.8.21.jar:?] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224) [testng-6.8.21.jar:?] at org.testng.TestNG.runSuitesLocally(TestNG.java:1149) [testng-6.8.21.jar:?] at org.testng.TestNG.run(TestNG.java:1057) [testng-6.8.21.jar:?] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) [surefire-testng-2.19.1.jar:2.19.1] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112) [surefire-testng-2.19.1.jar:2.19.1] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99) [surefire-testng-2.19.1.jar:2.19.1] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) [surefire-testng-2.19.1.jar:2.19.1] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) [surefire-booter-2.19.1.jar:2.19.1] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) [surefire-booter-2.19.1.jar:2.19.1] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) [surefire-booter-2.19.1.jar:2.19.1] Caused by: org.teiid.api.exception.query.QueryValidatorException: TEIID30492 Element jdgSource1.SmallA.booleanValue of jdgSource1.SmallA is neither nullable nor has a default value. A value must be specified in the insert. at org.teiid.dqp.internal.process.Request.validateWithVisitor(Request.java:334) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.Request.validateQuery(Request.java:290) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.Request.generatePlan(Request.java:418) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.Request.processRequest(Request.java:470) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.RequestWorkItem.processNew(RequestWorkItem.java:642) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:337) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) ~[teiid-engine-8.12.7.6_3-redhat-1.jar:8.12.7.6_3-redhat-1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_102] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_102] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_102] {code} Resulting DDL metadata: {code:sql} CREATE FOREIGN TABLE SmallA ( intKey integer NOT NULL OPTIONS (NAMEINSOURCE 'intKey', SEARCHABLE 'Searchable', NATIVE_TYPE 'int'), stringNum string OPTIONS (NAMEINSOURCE 'stringNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'), stringKey string NOT NULL OPTIONS (NAMEINSOURCE 'stringKey', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'), floatNum float OPTIONS (NAMEINSOURCE 'floatNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'float'), bigIntegerValue biginteger OPTIONS (NAMEINSOURCE 'bigIntegerValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'), shortValue short OPTIONS (NAMEINSOURCE 'shortValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'int'), doubleNum double OPTIONS (NAMEINSOURCE 'doubleNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'double'), objectValue varbinary OPTIONS (NAMEINSOURCE 'objectValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'byte[]'), intNum integer OPTIONS (NAMEINSOURCE 'intNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'int'), bigDecimalValue bigdecimal OPTIONS (NAMEINSOURCE 'bigDecimalValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'java.lang.String'), longNum long OPTIONS (NAMEINSOURCE 'longNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'long'), booleanValue boolean NOT NULL OPTIONS (NAMEINSOURCE 'booleanValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'boolean'), timeStampValue timestamp OPTIONS (NAMEINSOURCE 'timeStampValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'long'), byteNum byte OPTIONS (NAMEINSOURCE 'byteNum', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'int'), timeValue time OPTIONS (NAMEINSOURCE 'timeValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'long'), dateValue date OPTIONS (NAMEINSOURCE 'dateValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'long'), charValue char OPTIONS (NAMEINSOURCE 'charValue', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'int'), CONSTRAINT PK_INTKEY PRIMARY KEY(intKey) ) OPTIONS (UPDATABLE TRUE); {code} In annotated pojos scenario it is the same. {code:java|title=CustomerReport.java} @Indexed public class CustomerReport implements Serializable { private static final long serialVersionUID = 1L; private Integer customerId; private Integer totalAmount; @NumericField @Field(index=Index.YES, store=Store.YES, analyze=Analyze.NO) @ProtoField(number=1,required = true) public Integer getCustomerId() { return customerId; } public void setCustomerId(Integer customerId) { this.customerId = customerId; } @Field(index=Index.YES, store=Store.YES, analyze=Analyze.NO) @ProtoField(number = 2) public Integer getTotalAmount() { return totalAmount; } public void setTotalAmount(Integer totalAmount) { this.totalAmount = totalAmount; } } {code} The totalAmount field is not required, but resulting DDL metadata doesn't allow nullable. {code:sql} CREATE FOREIGN TABLE CustomerReport ( customerId integer NOT NULL OPTIONS (NAMEINSOURCE 'customerId', SEARCHABLE 'Searchable', NATIVE_TYPE 'int'), totalAmount integer OPTIONS (NAMEINSOURCE 'totalAmount', SEARCHABLE 'Unsearchable', NATIVE_TYPE 'int'), CONSTRAINT PK_CUSTOMERID PRIMARY KEY(customerId) ) OPTIONS (UPDATABLE TRUE); {code} -- This message was sent by Atlassian JIRA (v7.2.3#72005)