]
Pedro Inácio commented on TEIID-4745:
-------------------------------------
Ok, I understand.
This is a nice to have.
Do you want me to close this?
Avoid loading the same file over again when using
'org.jboss.teiid.resource-adapter.file'
-----------------------------------------------------------------------------------------
Key: TEIID-4745
URL:
https://issues.jboss.org/browse/TEIID-4745
Project: Teiid
Issue Type: Enhancement
Environment: * WildFly 10
* Teiid Server 9.1.2
* CentOs 7
Reporter: Pedro Inácio
Assignee: Steven Hawkins
When using the file resource adapter _org.jboss.teiid.resource-adapter.file_, it should
be possible that the system per se, by the means of a new property in the
resource-adapter, or other option in the view definition, or other configuration
elsewhere, to not to unnecessary load a file that is already loaded. Theoretically it
should be possible, for example, to check the file date or the file checksum, to see if
the file is already loaded by _Teiid_. If it is, then there is no need to parse the file
again.
This is important due to, specially, performance improvements. If there are a lot of
files to be processed, and these files are big (200 MB or more), the system will be
processing over and over again the same files if they are not updated for quite some time.
If, for example, the user uses a Materialized View, then the view will be constantly
being updated with the same data, which brings another performance penalty.
*Note*: This could be extended to other resource-adapters with some sort of last data
date for example, if the user whant to use this sort of mechanism. For example, if using a
database like MySQL as source, then tthe user could define a last update table, and the
system could do a check first against this table before tryng to load the data.