[JBoss JIRA] (TEIID-4815) How could I debug JDV memory allocation properly
by Rafael Coutinho (JIRA)
[ https://issues.jboss.org/browse/TEIID-4815?page=com.atlassian.jira.plugin... ]
Rafael Coutinho commented on TEIID-4815:
----------------------------------------
Just added one more screenshot after 30 minutes.
> How could I debug JDV memory allocation properly
> ------------------------------------------------
>
> Key: TEIID-4815
> URL: https://issues.jboss.org/browse/TEIID-4815
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Rafael Coutinho
> Assignee: Steven Hawkins
> Attachments: Screenshot from 2017-03-17 17-54-24.png, Screenshot from 2017-03-17 18-14-50.png
>
>
> We hare having trouble with memory allocation on our JDV server (using Red Hat JBoss Data Virtualization - Version 6.3.0) for some reason memory gets allocated but never released.
> For simple queries memory increases just a little, however when we make complex joins etc, we are seeing scenarios with 20GB+ allocated heap.
> I wonder if there is any inspection tool for debugging what is consuming it in JDV.
> Our main datasource is an Oracle DS, but we do have a MariaDB being used too.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4815) How could I debug JDV memory allocation properly
by Rafael Coutinho (JIRA)
[ https://issues.jboss.org/browse/TEIID-4815?page=com.atlassian.jira.plugin... ]
Rafael Coutinho updated TEIID-4815:
-----------------------------------
Attachment: Screenshot from 2017-03-17 18-14-50.png
> How could I debug JDV memory allocation properly
> ------------------------------------------------
>
> Key: TEIID-4815
> URL: https://issues.jboss.org/browse/TEIID-4815
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Rafael Coutinho
> Assignee: Steven Hawkins
> Attachments: Screenshot from 2017-03-17 17-54-24.png, Screenshot from 2017-03-17 18-14-50.png
>
>
> We hare having trouble with memory allocation on our JDV server (using Red Hat JBoss Data Virtualization - Version 6.3.0) for some reason memory gets allocated but never released.
> For simple queries memory increases just a little, however when we make complex joins etc, we are seeing scenarios with 20GB+ allocated heap.
> I wonder if there is any inspection tool for debugging what is consuming it in JDV.
> Our main datasource is an Oracle DS, but we do have a MariaDB being used too.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4815) How could I debug JDV memory allocation properly
by Rafael Coutinho (JIRA)
[ https://issues.jboss.org/browse/TEIID-4815?page=com.atlassian.jira.plugin... ]
Rafael Coutinho updated TEIID-4815:
-----------------------------------
Description:
We hare having trouble with memory allocation on our JDV server (using Red Hat JBoss Data Virtualization - Version 6.3.0) for some reason memory gets allocated but never released.
For simple queries memory increases just a little, however when we make complex joins etc, we are seeing scenarios with 20GB+ allocated heap.
I wonder if there is any inspection tool for debugging what is consuming it in JDV.
Our main datasource is an Oracle DS, but we do have a MariaDB being used too.
Attachment: Screenshot from 2017-03-17 17-54-24.png
> How could I debug JDV memory allocation properly
> ------------------------------------------------
>
> Key: TEIID-4815
> URL: https://issues.jboss.org/browse/TEIID-4815
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Rafael Coutinho
> Assignee: Steven Hawkins
> Attachments: Screenshot from 2017-03-17 17-54-24.png
>
>
> We hare having trouble with memory allocation on our JDV server (using Red Hat JBoss Data Virtualization - Version 6.3.0) for some reason memory gets allocated but never released.
> For simple queries memory increases just a little, however when we make complex joins etc, we are seeing scenarios with 20GB+ allocated heap.
> I wonder if there is any inspection tool for debugging what is consuming it in JDV.
> Our main datasource is an Oracle DS, but we do have a MariaDB being used too.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4814) BatchedUpdateException should be provided for JDBC prepared statement batches
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-4814:
-------------------------------------
Summary: BatchedUpdateException should be provided for JDBC prepared statement batches
Key: TEIID-4814
URL: https://issues.jboss.org/browse/TEIID-4814
Project: Teiid
Issue Type: Bug
Components: JDBC Connector, Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 9.3, 9.2.2
TEIID-4243 only covered batches of non-prepared statements in the JDBC translator case - it did not implement the BatchUpdateException handling for prepared statement value batching.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4780) Clone TEIID-4734 (Backwards compatible solution)
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4780?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4780:
-------------------------------------
When they are invoking the SYSADMIN.loadMatView() directly that means that (or understanding that) they are not using teiid_rel:ALLOW_MATVIEW_MANAGEMENT true. So, they are not using the Status table. I do not think we know if the SYSADMIN.loadMatView() called directly vs as part of management, if we enable that then we can distinguish between the usages and not update/consult Status table at all. or simply we can override the Status table, but in that it will be an issue in clustered environment. So, it is easy to advocate to use a sensible TTL.
We can add a error or warning to define a TTL, either case different issue. You can open a enhancement/bug case on that.
> Clone TEIID-4734 (Backwards compatible solution)
> ------------------------------------------------
>
> Key: TEIID-4780
> URL: https://issues.jboss.org/browse/TEIID-4780
> Project: Teiid
> Issue Type: Sub-task
> Components: Common
> Affects Versions: 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.2.1, 9.1.4, 8.12.10.6_3
>
>
> This is clone of TEIID-4734, however the solution provided through it is a breaking change. This JIRA is logged to provide a non-breaking change for released Teiid versions based on comments from TEIID-4734
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4780) Clone TEIID-4734 (Backwards compatible solution)
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-4780?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-4780:
------------------------------------
Ah, consider scenario when user doesn't set the ttl and wants to invoke SYSADMIN.loadMatView() explicitly.
Property teiid_rel:MATVIEW_TTL defaults to 2^63 milliseconds according to docs. So we're waiting eternity and a half until we mark the load as failed.
So I tried a simple workaround, setting "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'FALSE' and "teiid_rel:MATVIEW_TTL" 2000. This resolves the issue.
Could this be targeted directly in code? Can you on SYSADMIN.loadMatView() invocation, when ALLOW_MATVIEW_MANAGEMENT is set to false, reset the pending job automatically?
> Clone TEIID-4734 (Backwards compatible solution)
> ------------------------------------------------
>
> Key: TEIID-4780
> URL: https://issues.jboss.org/browse/TEIID-4780
> Project: Teiid
> Issue Type: Sub-task
> Components: Common
> Affects Versions: 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.2.1, 9.1.4, 8.12.10.6_3
>
>
> This is clone of TEIID-4734, however the solution provided through it is a breaking change. This JIRA is logged to provide a non-breaking change for released Teiid versions based on comments from TEIID-4734
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months
[JBoss JIRA] (TEIID-4780) Clone TEIID-4734 (Backwards compatible solution)
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4780?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4780:
-------------------------------------
The way I fixed may be not optimal, as code reads like
{code}
if (elapsed > ttl*1.5) {
resetPendingJob(vdb.getVDB(), table);
}
{code}
so that, if the situation is being carried out in clustered environment, then "LOADING" state can be result from another node actively loading the matview load. I did not wanted to kill the on-going job. Instead, I opted to wait upto 1.5 times the configured TTL, if the job is not finished then I consider it as orphaned job. 1.5 is just a arbitrary number.
In upstream there is much more comprehensive solution that spans all nodes, and does not wait, however that requires a breaking change, thus this compromise. So, see if you wait enough time, this recovers?
> Clone TEIID-4734 (Backwards compatible solution)
> ------------------------------------------------
>
> Key: TEIID-4780
> URL: https://issues.jboss.org/browse/TEIID-4780
> Project: Teiid
> Issue Type: Sub-task
> Components: Common
> Affects Versions: 9.1.2
> Reporter: Pedro Inácio
> Assignee: Ramesh Reddy
> Fix For: 9.2.1, 9.1.4, 8.12.10.6_3
>
>
> This is clone of TEIID-4734, however the solution provided through it is a breaking change. This JIRA is logged to provide a non-breaking change for released Teiid versions based on comments from TEIID-4734
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 9 months