[JBoss JIRA] (TEIIDDES-2414) Large transformation SQL is not exported to INDEX file.
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2414?page=com.atlassian.jira.plu... ]
Steven Hawkins commented on TEIIDDES-2414:
------------------------------------------
> I think at this stage spanning blocks would be unnecessary
If that change could be made such that we don't require additional metadata, then it seems worth pursuing. Otherwise yes, something indicating the block size will have to be in the vdb in some capacity. Whether in the vdb.xml or in the index file, which would require a version bump.
> You are probably more aware than I but the algorithm works by adding a 'word entry' to an index block.
Not really. This is very old code initially copied/modified from eclipse I believe.
> Large transformation SQL is not exported to INDEX file.
> -------------------------------------------------------
>
> Key: TEIIDDES-2414
> URL: https://issues.jboss.org/browse/TEIIDDES-2414
> Project: Teiid Designer
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 8.3.4
> Reporter: Hisanobu Okuda
> Assignee: Paul Richardson
> Fix For: 9.2.1
>
> Attachments: reosurce-problem-marker.png, test.csv
>
>
> A large transformation SQL of a view model is not exported to INDEX file.
> I have a large transformation SQL > 8192 bytes in my view model. TDesigner fails to convert the xmi to XXXXXXXXXX.INDEX file. There is no transformation SQL in the INDEX file, therefore, deploying my vdb fails and the vdb is inactivated. When you deploy the vdb, you can see the following message in a log file:-
> {code}
> 17:16:11,632 WARN [org.teiid.PLANNER.RESOLVER] (teiid-async-threads - 2) TEIID31079 View ViewModel.new_table is not defined by a query expression.
> 17:16:11,636 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40073 The metadata for the VDB test.1 is loaded, however it is not valid. Check models for errors. Correct the metadata and re-deploy.
> 17:16:11,637 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40003 VDB test.1 is set to FAILED
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2734) Enable the tooling to derive the LDAP groups when configuring roles
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2734?page=com.atlassian.jira.plu... ]
Van Halbert commented on TEIIDDES-2734:
---------------------------------------
I'm saying, designer would create its own/separate logic, but could start by using the logic in the translator (cut-n-paste). It wouldn't directly rely on the translator.
To complete the whole picture, it would be nice if designer could assist in configuring EAP ldap. However, I would make that a long term goal. If we could just be able to get the list to select from when assigning roles, that would be a great help.
> Enable the tooling to derive the LDAP groups when configuring roles
> -------------------------------------------------------------------
>
> Key: TEIIDDES-2734
> URL: https://issues.jboss.org/browse/TEIIDDES-2734
> Project: Teiid Designer
> Issue Type: Feature Request
> Components: Diagrams, komodo-common
> Reporter: Van Halbert
>
> The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
> This has been requested before
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2414) Large transformation SQL is not exported to INDEX file.
by Paul Richardson (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2414?page=com.atlassian.jira.plu... ]
Paul Richardson commented on TEIIDDES-2414:
-------------------------------------------
[~shawkins]
I think at this stage spanning blocks would be unnecessary. Allowing management of the block size should be sufficient. My thoughts ....
The bug has emerged from the transformation SELECT query in the view model (generated from test.csv) being about 9K in size. You are probably more aware than I but the algorithm works by adding a 'word entry' to an index block. If that word entry fails to fit in the block then that block is written and a new block created to occupy the word entry. Since the SELECT query is 9K it fails to fit in an empty block so is lost from the index completely. Increasing the size of the index block allows for such an entry to be accomodated. The downside is more word entries per block but I presume that is the price to be paid for accommodating such a large query.
In this usecase, would a vdb metadata property be more useful over a system property? My thought is that the vdb is then in control of how its indexes were created and can tell teiid how they should be read on a pre-vdb basis rather than stipulating the property for all vdbs in the running teiid.
> Large transformation SQL is not exported to INDEX file.
> -------------------------------------------------------
>
> Key: TEIIDDES-2414
> URL: https://issues.jboss.org/browse/TEIIDDES-2414
> Project: Teiid Designer
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 8.3.4
> Reporter: Hisanobu Okuda
> Assignee: Paul Richardson
> Fix For: 9.2.1
>
> Attachments: reosurce-problem-marker.png, test.csv
>
>
> A large transformation SQL of a view model is not exported to INDEX file.
> I have a large transformation SQL > 8192 bytes in my view model. TDesigner fails to convert the xmi to XXXXXXXXXX.INDEX file. There is no transformation SQL in the INDEX file, therefore, deploying my vdb fails and the vdb is inactivated. When you deploy the vdb, you can see the following message in a log file:-
> {code}
> 17:16:11,632 WARN [org.teiid.PLANNER.RESOLVER] (teiid-async-threads - 2) TEIID31079 View ViewModel.new_table is not defined by a query expression.
> 17:16:11,636 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40073 The metadata for the VDB test.1 is loaded, however it is not valid. Check models for errors. Correct the metadata and re-deploy.
> 17:16:11,637 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40003 VDB test.1 is set to FAILED
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2734) Enable the tooling to derive the LDAP groups when configuring roles
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2734?page=com.atlassian.jira.plu... ]
Steven Hawkins commented on TEIIDDES-2734:
------------------------------------------
> using the LDAP translator logic, strip out what is needed only for designer to read the roles from an LDAP connection
Driving this through the translator/connector will be completely separate from the actual security configuration (the user / Designer would still be responsible for setting up the security-domain(s) in the EAP config) and it would need to be a distinct set of screens as it would get confusing to see this as a normal import or source definition scenario.
> Enable the tooling to derive the LDAP groups when configuring roles
> -------------------------------------------------------------------
>
> Key: TEIIDDES-2734
> URL: https://issues.jboss.org/browse/TEIIDDES-2734
> Project: Teiid Designer
> Issue Type: Feature Request
> Components: Diagrams, komodo-common
> Reporter: Van Halbert
>
> The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
> This has been requested before
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2414) Large transformation SQL is not exported to INDEX file.
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2414?page=com.atlassian.jira.plu... ]
Steven Hawkins commented on TEIIDDES-2414:
------------------------------------------
> Is it possible that the index block size in Teiid could be assigned by an external property?
A system property would be easy. Is there a full fix that is easy, such as allowing entries to span blocks?
> Large transformation SQL is not exported to INDEX file.
> -------------------------------------------------------
>
> Key: TEIIDDES-2414
> URL: https://issues.jboss.org/browse/TEIIDDES-2414
> Project: Teiid Designer
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 8.3.4
> Reporter: Hisanobu Okuda
> Assignee: Paul Richardson
> Fix For: 9.2.1
>
> Attachments: reosurce-problem-marker.png, test.csv
>
>
> A large transformation SQL of a view model is not exported to INDEX file.
> I have a large transformation SQL > 8192 bytes in my view model. TDesigner fails to convert the xmi to XXXXXXXXXX.INDEX file. There is no transformation SQL in the INDEX file, therefore, deploying my vdb fails and the vdb is inactivated. When you deploy the vdb, you can see the following message in a log file:-
> {code}
> 17:16:11,632 WARN [org.teiid.PLANNER.RESOLVER] (teiid-async-threads - 2) TEIID31079 View ViewModel.new_table is not defined by a query expression.
> 17:16:11,636 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40073 The metadata for the VDB test.1 is loaded, however it is not valid. Check models for errors. Correct the metadata and re-deploy.
> 17:16:11,637 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40003 VDB test.1 is set to FAILED
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2414) Large transformation SQL is not exported to INDEX file.
by Paul Richardson (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2414?page=com.atlassian.jira.plu... ]
Paul Richardson updated TEIIDDES-2414:
--------------------------------------
Attachment: reosurce-problem-marker.png
PR for master:
https://github.com/Teiid-Designer/teiid-designer/pull/528
9.0.x
https://github.com/Teiid-Designer/teiid-designer/pull/529
The PRs both causes an error/problem to be displayed against the model concerned, as shown in attached screenshot.
It is possible to change the block size from 8192 to a larger value and this 'fixes' the error in Designer. However, deployment to Teiid fails since the latter displays an input read error. This is unsurprising since Teiid uses the same index reading and writing framework as Designer with [8192 as the fixed block size|https://github.com/teiid/teiid/blob/master/metadata/src/main/java/or...].
[~shawkins]
Is it possible that the index block size in Teiid could be assigned by an external property?
> Large transformation SQL is not exported to INDEX file.
> -------------------------------------------------------
>
> Key: TEIIDDES-2414
> URL: https://issues.jboss.org/browse/TEIIDDES-2414
> Project: Teiid Designer
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 8.3.4
> Reporter: Hisanobu Okuda
> Assignee: Paul Richardson
> Fix For: 9.2.1
>
> Attachments: reosurce-problem-marker.png, test.csv
>
>
> A large transformation SQL of a view model is not exported to INDEX file.
> I have a large transformation SQL > 8192 bytes in my view model. TDesigner fails to convert the xmi to XXXXXXXXXX.INDEX file. There is no transformation SQL in the INDEX file, therefore, deploying my vdb fails and the vdb is inactivated. When you deploy the vdb, you can see the following message in a log file:-
> {code}
> 17:16:11,632 WARN [org.teiid.PLANNER.RESOLVER] (teiid-async-threads - 2) TEIID31079 View ViewModel.new_table is not defined by a query expression.
> 17:16:11,636 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40073 The metadata for the VDB test.1 is loaded, however it is not valid. Check models for errors. Correct the metadata and re-deploy.
> 17:16:11,637 INFO [org.teiid.RUNTIME] (teiid-async-threads - 2) TEIID40003 VDB test.1 is set to FAILED
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2734) Enable the tooling to derive the LDAP groups when configuring roles
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2734?page=com.atlassian.jira.plu... ]
Van Halbert commented on TEIIDDES-2734:
---------------------------------------
Since the LDAP logic is pretty static in the LDAP translator, maybe a better option would be for Designer to do the following:
- using the LDAP translator logic, strip out what is needed only for designer to read the roles from an LDAP connection
- the user creates an LDAP connection profile
- the user tells Designer which connection profile to use
- add a dropdown to the VDB editor to provide the user a list of roles to select from
> Enable the tooling to derive the LDAP groups when configuring roles
> -------------------------------------------------------------------
>
> Key: TEIIDDES-2734
> URL: https://issues.jboss.org/browse/TEIIDDES-2734
> Project: Teiid Designer
> Issue Type: Feature Request
> Components: Diagrams, komodo-common
> Reporter: Van Halbert
>
> The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
> This has been requested before
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2734) Enable the tooling to derive the LDAP groups when configuring roles
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2734?page=com.atlassian.jira.plu... ]
Steven Hawkins commented on TEIIDDES-2734:
------------------------------------------
The problem is a little indirect from just a vdb. Designer would have to know what security-domain is applicable for the vdb (either the security-domain vdb property or the default for the server, which prior to 9.0 could be different per transport). Then it would need to get the security configuration for that security domain(s) and infer how to get the groups from the LoginModules. Which is not simply a part of the LoginModule api. So it probably only makes sense to think of this from a top down scenario where the user has to explicitly configure an ldap browser or other logic in Designer, which could in turn configure the eap instance to have a security domain backed by ldap access.
> Enable the tooling to derive the LDAP groups when configuring roles
> -------------------------------------------------------------------
>
> Key: TEIIDDES-2734
> URL: https://issues.jboss.org/browse/TEIIDDES-2734
> Project: Teiid Designer
> Issue Type: Feature Request
> Components: Diagrams, komodo-common
> Reporter: Van Halbert
>
> The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
> This has been requested before
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (TEIIDDES-2734) Enable the tooling to derive the LDAP groups when configuring roles
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIIDDES-2734?page=com.atlassian.jira.plu... ]
Van Halbert updated TEIIDDES-2734:
----------------------------------
Description:
The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
This has been requested before
was:
The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
This has been requested before, and now Baker Hughes indicated it should.
> Enable the tooling to derive the LDAP groups when configuring roles
> -------------------------------------------------------------------
>
> Key: TEIIDDES-2734
> URL: https://issues.jboss.org/browse/TEIIDDES-2734
> Project: Teiid Designer
> Issue Type: Feature Request
> Components: Diagrams, komodo-common
> Reporter: Van Halbert
>
> The DV tooling should provide the ability to derive the roles from LDAP for when configuring roles.
> This has been requested before
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months