[JBoss JIRA] (TEIID-5007) Changes to reduce Teiid in the cloud footprint
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5007?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5007:
---------------------------------------
> container aware auto-sizing
That can largely be satisfied by the logic in an entry point script, such as: https://github.com/shawkins/teiid-openshift-templates/blob/master/server-... - it already has appropriate memory limits.
An additional modification is to add a CPU_LIMIT env parameter, which can be used to set the socket max workers.
We need limit buffer disk usage as well.
> Changes to reduce Teiid in the cloud footprint
> ----------------------------------------------
>
> Key: TEIID-5007
> URL: https://issues.jboss.org/browse/TEIID-5007
> Project: Teiid
> Issue Type: Quality Risk
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.1
>
>
> A Teiid instance even as swarm or springboot needs additional considerations to minimize the runtime footprint. This includes:
> * container aware auto-sizing. Detection of the number of cpus and available memory need refined - there are experimental settings being considered for containerized vms to better report these values and there is logic in WildFly and other projects that attempts better auto-detection. We also need to utilize the memory buffer space more and probably as off-heap space (and ideally direct operations on the serialized data)
> * Subsystems required include JTA, webserver, security, which could be satisfied by slimmer alternative versions - especially if we make new assumptions, such as not utilizing xa transactions.
> * Engine dependencies could be application specific - removing xml/xsl support, geometry support, etc.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-5144:
------------------------------------
[~rareddy] the check is there, exactly two lines under the line I sent in my last comment. It already fails in that case. I was thinking we could filter the errors somehow, to find out, whether it was our proto-schema that failed.
If we don't, we fail the deployment even though our proto-schema was possibly registered just fine.
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5143) Infinispan hotrod translator proto-schema registered as ProtoFilePath
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-5143?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-5143:
------------------------------------
I don't follow.
* I use ProtoFilePath, when I have a file physically present on the Teiid instance. Then I set only the ProtoFilePath property.
* When I want to reuse the same proto-schema, but I don't have the file, I can use ProtobufName property. Though currently it has to be exactly the same value as the original ProtoFilePath property before.
Thus I end up using a ProtobufName with value like /a/path/to/protobuf/somewhere/when/it/was/registered/file.proto ?
After I restart the JDG instance and the proto-schema needs to be re-registered, I have to have the file at the exactly same place, if I don't want to break my other vdbs.
> Infinispan hotrod translator proto-schema registered as ProtoFilePath
> ---------------------------------------------------------------------
>
> Key: TEIID-5143
> URL: https://issues.jboss.org/browse/TEIID-5143
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
>
> When using an existing protobuf file to configure source metadata and providing full path via ProtoFilePath, there are issues with how the proto-schema is being registered in JDG.
> When using ProtoFilePath, the proto-schema is being registered under the exact value of this property. This is unexpected and source of many issues. Rather the proto-schema should be registered under the package context.
> This leads to a situation, when using the same file on various machines, or when the file is moved, then the proto-schema is being registered twice with different keys in JDG proto-schema cache, this leads to an exception on JDG's side and non-deployed VDB.
> I expect the ProtobufName to be a complementary to ProtoFilePath:
> # I register a proto-schema using ProtoFilePath property
> # I use that schema in different VDB using ProtobufName property
> Then the ProtobufName used in second case needs to contain exactly the same value as ProtoFilePath in the first case. This is hard to understand.
> Could the change be to:
> # Set both ProtoFilePath and ProtobufName properties (ProtobufName is our logical descriptor) when a user want to register new protobuf file. This way the contents of a file denoted by ProtoFilePath is registered in JDG under ProtobufName key.
> # Using only ProtobufName when an already registered proto-schema is to be used.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5144:
-------------------------------------
Sure we can add the check for .errors before we proceed, but in that case we would just fail to deploy/or make the vdb active.
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
> Priority: Blocker
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Ramesh Reddy updated TEIID-5144:
--------------------------------
Priority: Major (was: Blocker)
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-5144:
------------------------------------
[~rareddy]
Feel free to reduce the priority as you see fit. I wanted to stress the fact, that Teiid's functionality may be adversely affected by another entity accessing shared JDG instance.
> what's the ".errors" cache
My mistake, it is not a cache, but an entry suffix in a metadata cache. See [this line|https://github.com/teiid/jdg/blob/4162be7b01cffc68db232b630a7195c73d...]. The metadata cache can contain ".errors" key suffix but for different protoBufResource (talking about a variable in the code linked above).
> So, how is a "bad" proto-stream file introduced? by User?
Yes, by a user. I could get there by trying register same protobuf file under two different keys in JDG. Which is kinda invalid state, though in my view if there's a way to help Teiid overcome this issue, it's worth pursuing.
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
> Priority: Blocker
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Ramesh Reddy reassigned TEIID-5144:
-----------------------------------
Assignee: Ramesh Reddy (was: Steven Hawkins)
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
> Priority: Blocker
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5144) Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5144?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5144:
-------------------------------------
Interesting, whats in the ".errors" cache? I did not know about it. I guess we can validate before we generate metadata in that case.
Either case is far from a blocker from Teiid perspective. If I were to generate a wrong proto-stream from the DDL file, I would have accepted as a blocker. So, how is a "bad" proto-stream file introduced? by User?
> Infinispan hotrod translator doesn't overcome error in unrelated proto-schema
> -----------------------------------------------------------------------------
>
> Key: TEIID-5144
> URL: https://issues.jboss.org/browse/TEIID-5144
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Blocker
>
> Infinispan Hotrod translator can't overcome an error in any proto-schema on the JDG server.
> It ends with a message, that there's an issue with a proto-schema. Doesn't say which.
> Note, that this can be completely unrelated proto-schema to Teiid. Then the only solution is to reload JDG, or possibly clear proto-schema cache.
> If Teiid is checking for errors in .errors cache, it should be selective and look only for the newly added proto-schema.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5143) Infinispan hotrod translator proto-schema registered as ProtoFilePath
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5143?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5143:
-------------------------------------
Can you not give the same value currently "ProtoFilePath" and accomplish same?
> Infinispan hotrod translator proto-schema registered as ProtoFilePath
> ---------------------------------------------------------------------
>
> Key: TEIID-5143
> URL: https://issues.jboss.org/browse/TEIID-5143
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Priority: Critical
>
> When using an existing protobuf file to configure source metadata and providing full path via ProtoFilePath, there are issues with how the proto-schema is being registered in JDG.
> When using ProtoFilePath, the proto-schema is being registered under the exact value of this property. This is unexpected and source of many issues. Rather the proto-schema should be registered under the package context.
> This leads to a situation, when using the same file on various machines, or when the file is moved, then the proto-schema is being registered twice with different keys in JDG proto-schema cache, this leads to an exception on JDG's side and non-deployed VDB.
> I expect the ProtobufName to be a complementary to ProtoFilePath:
> # I register a proto-schema using ProtoFilePath property
> # I use that schema in different VDB using ProtobufName property
> Then the ProtobufName used in second case needs to contain exactly the same value as ProtoFilePath in the first case. This is hard to understand.
> Could the change be to:
> # Set both ProtoFilePath and ProtobufName properties (ProtobufName is our logical descriptor) when a user want to register new protobuf file. This way the contents of a file denoted by ProtoFilePath is registered in JDG under ProtobufName key.
> # Using only ProtobufName when an already registered proto-schema is to be used.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months
[JBoss JIRA] (TEIID-5143) Infinispan hotrod translator proto-schema registered as ProtoFilePath
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-5143?page=com.atlassian.jira.plugin... ]
Ramesh Reddy reassigned TEIID-5143:
-----------------------------------
Assignee: Ramesh Reddy (was: Steven Hawkins)
> Infinispan hotrod translator proto-schema registered as ProtoFilePath
> ---------------------------------------------------------------------
>
> Key: TEIID-5143
> URL: https://issues.jboss.org/browse/TEIID-5143
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Jan Stastny
> Assignee: Ramesh Reddy
>
> When using an existing protobuf file to configure source metadata and providing full path via ProtoFilePath, there are issues with how the proto-schema is being registered in JDG.
> When using ProtoFilePath, the proto-schema is being registered under the exact value of this property. This is unexpected and source of many issues. Rather the proto-schema should be registered under the package context.
> This leads to a situation, when using the same file on various machines, or when the file is moved, then the proto-schema is being registered twice with different keys in JDG proto-schema cache, this leads to an exception on JDG's side and non-deployed VDB.
> I expect the ProtobufName to be a complementary to ProtoFilePath:
> # I register a proto-schema using ProtoFilePath property
> # I use that schema in different VDB using ProtobufName property
> Then the ProtobufName used in second case needs to contain exactly the same value as ProtoFilePath in the first case. This is hard to understand.
> Could the change be to:
> # Set both ProtoFilePath and ProtobufName properties (ProtobufName is our logical descriptor) when a user want to register new protobuf file. This way the contents of a file denoted by ProtoFilePath is registered in JDG under ProtobufName key.
> # Using only ProtobufName when an already registered proto-schema is to be used.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 4 months