[JBoss JIRA] (TEIID-4532) Provide one-way or cryptographic hash functions
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4532?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4532:
----------------------------------
Component/s: Query Engine
(was: Common)
Fix Version/s: 9.2
We'll look at adding something that is consistent with pgcrypto and other db offerings.
> Provide one-way or cryptographic hash functions
> -----------------------------------------------
>
> Key: TEIID-4532
> URL: https://issues.jboss.org/browse/TEIID-4532
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Affects Versions: 9.2
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Fix For: 9.2
>
>
> Provide one-way or [cryptographic hash functions|https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions], such as any of the MD5, SHA-1, SHA-2, or SHA-3 functions, so that views can define columns that are hashes of other columns.
> The goal is to allow views to hide some columns (e.g., personally identifying information), but to expose a new "primary key" that is a hash of other existing columns. So, given this source table:
> {code:sql}
> CREATE TABLE person (
> id INT PRIMARY KEY,
> name VARCHAR(256) NOT NULL,
> age INT,
> height INT,
> weight DOUBLE
> );
> {code}
> a view could be created to hide the personally identifying information:
> {code:sql}
> CREATE VIEW anonymousPerson ()
> id VARCHAR(64) PRIMARY KEY,
> age INT,
> height INT,
> weight DOUBLE
> ) AS
> SELECT sha256(p.id, p.name) AS id,
> p.age AS age,
> p.height AS height,
> p.weight AS weight
> FROM person AS p;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-2578) add/remove schema elements
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2578?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2578:
---------------------------------------
> so if non-master node comes up it should be empty to start with.
> As soon as master node starts it should replicate the events
If the master is up and later a non-master comes up, that is the issue. The only way this metadata replication works currently is if everyone is up together or if the master joins later. A non-master joining later won't see the affect of the databasestore load - it will only become aware of vdbs as they are further modified and a reloadDatabase event is issued.
For example in the internal materialization logic there is handling for the state transfer that happens when someone joins the cluster. Something similar is needed here.
> but issue could be, what if master node went down in-between and comes back up, it is going to duplicate the state, that needs to be avoided.
Since the master would be issuing a reloadDatabase event, that would replace all of the state on the non-masters. But it does mean that all existing connections would be terminated.
> IMO we need to maintain a VDB wide version/lastmodified number that can be passed with all EventDistributor methods. aka. like a if-match-none HEADER in http, What do you think?
There is now only one event distributor method that causes a vdb restart, which is reloadDatabase. Having a more granular versioning could prevent an unnecessary restart of a vdb from the scenario above but that should be a rare occurrence.
> add/remove schema elements
> --------------------------
>
> Key: TEIID-2578
> URL: https://issues.jboss.org/browse/TEIID-2578
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 9.2
>
>
> Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-2578) add/remove schema elements
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2578?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2578:
-------------------------------------
>>It seems like if a non-master cluster member comes up after any changes have been made, it won't have those changes synched until after the vdb is reloaded by another change.
In this singleton service scheme, only master should have the persistent copy of VDBs, so if non-master node comes up it should be empty to start with. As soon as master node starts it should replicate the events. (atleast that is what I think :) ), but issue could be, what if master node went down in-between and comes back up, it is going to duplicate the state, that needs to be avoided. I did not add any provision for this, IMO we need to maintain a VDB wide version/lastmodified number that can be passed with *all* EventDistributor methods. aka. like a if-match-none HEADER in http, What do you think?
> add/remove schema elements
> --------------------------
>
> Key: TEIID-2578
> URL: https://issues.jboss.org/browse/TEIID-2578
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 9.2
>
>
> Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-2578) add/remove schema elements
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2578?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-2578:
---------------------------------------
Issues with keeping the connection alive or at least to suppress the cancellation exception should be addressed.
Before we promote to master we at least need to address authorization and cluster consistency more. It seems like if a non-master cluster member comes up after any changes have been made, it won't have those changes synched until after the vdb is reloaded by another change. We should also address any of the inconsistencies with the traditional metadata features - especially vdb import.
> add/remove schema elements
> --------------------------
>
> Key: TEIID-2578
> URL: https://issues.jboss.org/browse/TEIID-2578
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 9.2
>
>
> Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-4532) Provide one-way or cryptographic hash functions
by Randall Hauch (JIRA)
[ https://issues.jboss.org/browse/TEIID-4532?page=com.atlassian.jira.plugin... ]
Randall Hauch updated TEIID-4532:
---------------------------------
Description:
Provide one-way or [cryptographic hash functions|https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions], such as any of the MD5, SHA-1, SHA-2, or SHA-3 functions, so that views can define columns that are hashes of other columns.
The goal is to allow views to hide some columns (e.g., personally identifying information), but to expose a new "primary key" that is a hash of other existing columns. So, given this source table:
{code:sql}
CREATE TABLE person (
id INT PRIMARY KEY,
name VARCHAR(256) NOT NULL,
age INT,
height INT,
weight DOUBLE
);
{code}
a view could be created to hide the personally identifying information:
{code:sql}
CREATE VIEW anonymousPerson ()
id VARCHAR(64) PRIMARY KEY,
age INT,
height INT,
weight DOUBLE
) AS
SELECT sha256(p.id, p.name) AS id,
p.age AS age,
p.height AS height,
p.weight AS weight
FROM person AS p;
{code}
was:provide one-way or cryptographic hash functions, such as any of the MD5, SHA-1, SHA-2, or SHA-3 functions? https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions <https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions>
> Provide one-way or cryptographic hash functions
> -----------------------------------------------
>
> Key: TEIID-4532
> URL: https://issues.jboss.org/browse/TEIID-4532
> Project: Teiid
> Issue Type: Feature Request
> Components: Common
> Affects Versions: 9.2
> Reporter: Van Halbert
> Assignee: Steven Hawkins
>
> Provide one-way or [cryptographic hash functions|https://en.wikipedia.org/wiki/SHA-3#Comparison_of_SHA_functions], such as any of the MD5, SHA-1, SHA-2, or SHA-3 functions, so that views can define columns that are hashes of other columns.
> The goal is to allow views to hide some columns (e.g., personally identifying information), but to expose a new "primary key" that is a hash of other existing columns. So, given this source table:
> {code:sql}
> CREATE TABLE person (
> id INT PRIMARY KEY,
> name VARCHAR(256) NOT NULL,
> age INT,
> height INT,
> weight DOUBLE
> );
> {code}
> a view could be created to hide the personally identifying information:
> {code:sql}
> CREATE VIEW anonymousPerson ()
> id VARCHAR(64) PRIMARY KEY,
> age INT,
> height INT,
> weight DOUBLE
> ) AS
> SELECT sha256(p.id, p.name) AS id,
> p.age AS age,
> p.height AS height,
> p.weight AS weight
> FROM person AS p;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-4534) Querying 1-to-many relationship in JDG data source may return the wrong results
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4534?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-4534:
------------------------------------
The same filtering logic that is used when performing update/delete on a child object in order to delete/update the correct child can be used on select queries. This will mean the criteria for a child object (1-to-many) will be processed twice. Pushed down to ensure it filters the correct parent object for those children that's returned and then again when converting the children objects into tuples.
> Querying 1-to-many relationship in JDG data source may return the wrong results
> -------------------------------------------------------------------------------
>
> Key: TEIID-4534
> URL: https://issues.jboss.org/browse/TEIID-4534
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.12.x, 9.2
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Critical
>
> When there is a 1-to-many relationship defined in a JDG schema, the results from querying the "many" relationship may return the wrong results.
> Example:
> Person --> Phone Numbers
> INSERT INTO Person(id,name,email) VALUES (1,'name1','email1')
> INSERT INTO PhoneNumber(id, number) VALUES (1, '001234567');
> INSERT INTO PhoneNumber(id, number) VALUES (1, '001234568');
> SELECT p.name, p.id, pn.number FROM Person p, PhoneNumber pn WHERE p.id=pn.id AND p.id=1 and number = '001234568'
> or
> Select * from PhoneNumber where number = '001234568'
> return 2 rows, 1 for each phonenumber.
> Why? Because JDG returns the Person object that meets those conditions. It doesn't filter the internal data to the Person object. Therefore, all phonenumbers are returned for the same person, generating a row for each.
> This doesn't happen with 1-to-1 relationships because there's only ever 1 child object.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-4535) Non remote methods called twice on the EventDistributor
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-4535:
-------------------------------------
Summary: Non remote methods called twice on the EventDistributor
Key: TEIID-4535
URL: https://issues.jboss.org/browse/TEIID-4535
Project: Teiid
Issue Type: Bug
Components: Server
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 9.2
This is specifically an issue in clustered mode and with the EventListener methods. The proxy will call the local and the remoting handling, which again will call the local.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-4534) Querying 1-to-many relationship in JDG data source may return the wrong results
by Van Halbert (JIRA)
Van Halbert created TEIID-4534:
----------------------------------
Summary: Querying 1-to-many relationship in JDG data source may return the wrong results
Key: TEIID-4534
URL: https://issues.jboss.org/browse/TEIID-4534
Project: Teiid
Issue Type: Bug
Components: Misc. Connectors
Affects Versions: 8.12.x, 9.2
Reporter: Van Halbert
Assignee: Steven Hawkins
Priority: Critical
When there is a 1-to-many relationship defined in a JDG schema, the results from querying the "many" relationship may return the wrong results.
Example:
Person --> Phone Numbers
INSERT INTO Person(id,name,email) VALUES (1,'name1','email1')
INSERT INTO PhoneNumber(id, number) VALUES (1, '001234567');
INSERT INTO PhoneNumber(id, number) VALUES (1, '001234568');
SELECT p.name, p.id, pn.number FROM Person p, PhoneNumber pn WHERE p.id=pn.id AND p.id=1 and number = '001234568'
or
Select * from PhoneNumber where number = '001234568'
return 2 rows, 1 for each phonenumber.
Why? Because JDG returns the Person object that meets those conditions. It doesn't filter the internal data to the Person object. Therefore, all phonenumbers are returned for the same person, generating a row for each.
This doesn't happen with 1-to-1 relationships because there's only ever 1 child object.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months
[JBoss JIRA] (TEIID-4516) Add support for updating/deleting child objects
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4516?page=com.atlassian.jira.plugin... ]
Van Halbert updated TEIID-4516:
-------------------------------
Summary: Add support for updating/deleting child objects (was: Add support for updating child objects)
> Add support for updating/deleting child objects
> -----------------------------------------------
>
> Key: TEIID-4516
> URL: https://issues.jboss.org/browse/TEIID-4516
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 9.2, 8.12.7.6_3
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Priority: Critical
> Fix For: 9.2
>
>
> When using complex pojo with JDG 6.6, updating child tables doesn't do anything. It doesn't update the rows, but neither throws exception.
> The problem appears for both usages of JDG cache (annotated POJOs, protobuf+marshaller). I have following pojo structure:
> {code:plain}
> package org.jboss.qe.jdg.remote.protobuf.complex;
> /* @Indexed */
> message Person {
>
> /* @IndexedField(index=true, store=false) */
> required int32 id = 1;
>
> /* @IndexedField */
> required string name = 2;
>
> /* @IndexedField */
> optional string email = 3;
>
> /* @IndexedField(index=true, store=false) */
> repeated PhoneNumber phones = 4;
> /* @Indexed */
> message Address {
>
> /* @IndexedField */
> required string Address = 1;
>
> /* @IndexedField(index=true, store=false) */
> required string City = 2;
>
> /* @IndexedField(index=true, store=false) */
> required string State = 3;
> }
> /* @IndexedField(index=true, store=false) */
> optional Address address = 5;
> }
>
> /* @Indexed */
> message PhoneNumber {
>
> /* @IndexedField */
> required string number = 1;
> }
> {code}
> That is, one top-level POJO class Person, and two child POJO classes: Address(1-to-1) and PhoneNumber(1-to-N). Neither Address nor PhoneNumber tables could be updated. The DDL is following:
> {code:sql}
> CREATE FOREIGN TABLE Address (
> Address string NOT NULL PRIMARY KEY OPTIONS (NAMEINSOURCE 'address.Address', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> City string NOT NULL OPTIONS (NAMEINSOURCE 'address.City', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> State string NOT NULL OPTIONS (NAMEINSOURCE 'address.State', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> id integer NOT NULL OPTIONS (NAMEINSOURCE 'id', SELECTABLE FALSE, SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> CONSTRAINT FK_PERSON FOREIGN KEY(id) REFERENCES Person (id) OPTIONS (NAMEINSOURCE 'address')
> ) OPTIONS (UPDATABLE TRUE);
>
> CREATE FOREIGN TABLE Person (
> id integer NOT NULL OPTIONS (NAMEINSOURCE 'id', SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> name string NOT NULL OPTIONS (NAMEINSOURCE 'name', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> email string OPTIONS (NAMEINSOURCE 'email', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> CONSTRAINT PK_ID PRIMARY KEY(id)
> ) OPTIONS (UPDATABLE TRUE);
>
> CREATE FOREIGN TABLE PhoneNumber (
> number string NOT NULL PRIMARY KEY OPTIONS (NAMEINSOURCE 'phones.number', SEARCHABLE 'Searchable', NATIVE_TYPE 'java.lang.String'),
> id integer NOT NULL OPTIONS (NAMEINSOURCE 'id', SELECTABLE FALSE, SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> CONSTRAINT FK_PERSON FOREIGN KEY(id) REFERENCES Person (id) OPTIONS (NAMEINSOURCE 'phones')
> ) OPTIONS (UPDATABLE TRUE);
> {code}
> INSERT queries to prepare data:
> {code:sql}
> INSERT INTO Person(id,name,email) VALUES (1,'name1','email1')
> INSERT INTO Address(id,Address,City,State) VALUES (1,'address1','city1','state1')
> INSERT INTO PhoneNumber(id, number) VALUES (1, '001234567')
> INSERT INTO PhoneNumber(id, number) VALUES (1, '001234568')
> {code}
> UPDATE queries and queries for checking results:
> * 1-to-1
> {code:sql}UPDATE Address SET city='newCity' WHERE id=1{code}
> {code:sql}SELECT * FROM Person,Address{code}
> * 1-to-n
> {code:sql}UPDATE PhoneNumber SET number='newNumber' WHERE id=1{code}
> {code:sql}SELECT * FROM Person,PhoneNumber{code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
7 years, 8 months