[dna-issues] [JBoss JIRA] Commented: (DNA-40) Persistant storage for information not stored in other repository sources

[Randall Hauch (JIRA)]

    [ https://jira.jboss.org/jira/browse/DNA-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12439971#action_12439971 ] 

Randall Hauch commented on DNA-40:
----------------------------------

Deleting a subgraph in this connector could be expensive, since no records contain the path and thus deleting requires walking the subgraph and deleting the records for the individual nodes.

However, an approach that will probably work well is "mark and sweep", where any node(s) to be deleted are simply marked as "deleted" but left until a later request/process can "sweep" the deleted records from the database.  This could be pretty efficient, since it may be possible to only mark the nodes that are explicitly deleted, as long as all other operations exclude any records that are marked for deletion (or that are descendants of marked nodes).  

We may need to add a new request type that represents a "compaction" or "cleanup" or "maintenance" operation, which could be called periodically by a separate process/thread.  Ideally, each connector would implement this new request in such a way that it doesn't block other requests.

To implement the "mark and sweep" approach in this connector, I added a "deleted" attribute (boolean type, null if false) to ChildEntity and added to all the where clauses of the ChildEntity named queries a "and child.deleted is null" criteria.  Since these ChildEntity named queries are used to verify the path/UUID combination for Locations used in all requests, the descendant nodes don't need to be marked for deletion (if an ancestor node is marked for deletion, it will not be found as the path is verified, resulting in a PathNotFoundException).

Since we're using UUIDs as the primary key for each node (all the various records), a node that is marked for deletion can still exist in the database without the chance of a clash.

> Persistant storage for information not stored in other repository sources
> -------------------------------------------------------------------------
>
>                 Key: DNA-40
>                 URL: https://jira.jboss.org/jira/browse/DNA-40
>             Project: DNA
>          Issue Type: Feature Request
>          Components: Connectors
>            Reporter: Randall Hauch
>            Assignee: Randall Hauch
>             Fix For: 0.4
>
>
> Create a federation connector that is able to manage information in a relational database using a DNA-defined schema.  This would enable the persistant storage of information that isn't being managed by other connectors.
> Requirements include efficiently storing and accessing large numbers of nodes, large property values (e.g., Binary values), and large numbers of children on nodes.  

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira