Re: [hibernate-dev] HHH-6726 LONG and LONG RAW column types in Oracle
by Łukasz Antoniak
Currently Oracle supports database versions from 10.1 to 11.2 [1]. LONG
and LONG RAW data types are deprecated since version 8 and 8i (released
before September 2000) [2]. Oracle keeps those column types only for
backward compatibility [3].
I tried the following scenario (Oracle 10gR2):
1. Create schema with "hibernate.hbm2ddl.auto" set to "create". The LONG
column is created.
2. Insert some data.
3. Modify Oracle dialect as Gail suggested. Avoid setting
"hibernate.hbm2ddl.auto".
4. Insert some data.
To my surprise the test actually passed :). However, I think that we
cannot guaranty the proper behavior in every situation.
As for performance, ImageType is extracted by calling
ResultSet.getBytes() method, which fetches all data in one call [4]. I
don't suppose a major performance difference when data is streamed in
another call. oracle.jdbc.driver.LongRawAccessor.getBytes also fetches
data by reading the stream.
The bug reading LONG column affects JDBC drivers since version 10.2.0.4.
I think that we have to choose between:
- changing Oracle10gDialect. Make a not about it in migration guide to
4.0 and update "5.2.2. Basic value types" chapter in Hibernate
documentation.
- introducing Oracle11gDialect. It can sound weird to access Oracle 10g
database with Oracle 11g dialect.
- disabling execution of Hibernate tests that fail because of this issue
with @SkipForDialect (and maybe develop another version of them with
CLOBs and BLOBs, @RequiresDialect). Hibernate is written correctly
according to "Default Mappings Between SQL Types and Java Types"
(referenced earlier by Gail) and this is more Oracle's JDBC
implementation issue. This option came to my mind, but it's weird :P.
I would vote for the first option.
Regards,
Lukasz Antoniak
[1]
http://www.oracle.com/us/support/library/lifetime-support-technology-0691...
(page 4)
[2]
http://download.oracle.com/docs/cd/A91202_01/901_doc/server.901/a90120/ch...
[3]
http://download.oracle.com/docs/cd/B28359_01/server.111/b28318/datatype.htm
[4] "Getting a LONG RAW Data Column with getBytes"
http://download.oracle.com/docs/cd/B19306_01/java.102/b14355/jstreams.htm
Strong Liu pisze:
> I think oracle 11g is the only one supported DB version by oracle, can we just introduce a new oracle dialect with suggested changes, and deprecate all other existed oracle dialects? this won't affects users app
>
> -----------
> Strong Liu <stliu(a)hibernate.org>
> http://hibernate.org
> http://github.com/stliu
>
> On Oct 15, 2011, at 11:14 AM, Scott Marlow wrote:
>
>> How does this impact existing applications? Would they have to convert
>> LONGs to CLOBs (and LONGRAWs to BLOBs) to keep the application working?
>>
>> As far as the advantage of CLOB over TEXT, if you read every character,
>> which one is really faster? I would expect TEXT to be a little faster,
>> since the server side will send the characters before they are asked
>> for. By faster, I mean from the application performance point of view. :)
>>
>> Could this be changed in a custom Oracle dialect? So new
>> applications/databases could perhaps use that and existing applications
>> might use LONGs a bit longer via the existing Oracle dialect.
>>
>> On 10/14/2011 09:22 PM, Gail Badner wrote:
>>> In [1], I am seeing the following type mappings:
>>>
>>> Column type: LONG -> java.sql.Types.LONGVARCHAR -> java.lang.String
>>> Column type: LONGRAW -> java.sql.Types.LONGVARBINARY -> byte[]
>>>
>>> org.hibernate.type.TextType is consistent with the mapping for LONG.
>>>
>>> org.hibernate.type.ImageType is consistent with the mapping for LONGRAW.
>>>
>>> From this standpoint, the current settings are appropriate.
>>>
>>> I understand there are restrictions when LONG and LONGRAW are used and I see from your other message that there is Oracle documentation for migrating to CLOB and BLOB.
>>>
>>> I agree that changing column type registration as follows (for Oracle only) should fix this:
>>> registerColumnType( Types.VARBINARY, 2000, "raw($l)" );
>>> registerColumnType( Types.VARBINARY, "blob" );
>>>
>>> registerColumnType( Types.LONGVARCHAR, "clob" );
>>> registerColumnType( Types.LONGVARBINARY, "blob" );
>>>
>>> registerColumnType( Types.VARCHAR, 4000, "varchar2($l char)" );
>>> registerColumnType( Types.VARCHAR, "clob" );
>>>
>>> Steve, what do you think? Is it too late to make this change for 4.0.0?
>>>
>>> [1] Table 11-1 of Oracle® Database JDBC Developer's Guide and Reference, 11g Release 1 (11.1) (http://download.oracle.com/docs/cd/B28359_01/java.111/b31224/datacc.htm#g...)
>>> [2] Hibernate Core Migration Guide for 3.5 (http://community.jboss.org/wiki/HibernateCoreMigrationGuide35)
>>> [3] Table 2-10 of Oracle® Database SQL Language Reference
>>> 11g Release 1 (11.1) (http://download.oracle.com/docs/cd/B28359_01/server.111/b28286/sql_elemen...)
>>>
>>> ----- Original Message -----
>>>> From: "Łukasz Antoniak"<lukasz.antoniak(a)gmail.com>
>>>> To: hibernate-dev(a)lists.jboss.org
>>>> Sent: Thursday, October 13, 2011 12:50:13 PM
>>>> Subject: [hibernate-dev] HHH-6726 LONG and LONG RAW column types in Oracle
>>>>
>>>> Welcome Community!
>>>>
>>>> I have just subscribed to the list and wanted to discuss HHH-6726
>>>> JIRA
>>>> issue.
>>>>
>>>> Gail Badner wrote
>>>> (http://lists.jboss.org/pipermail/hibernate-dev/2011-October/007208.html):
>>>> HHH-6726 (Oracle : map TextType to clob and ImageType to blob)
>>>> https://hibernate.onjira.com/browse/HHH-6726
>>>> There have been a number of issues opened since the change was made
>>>> to
>>>> map TextType (LONGVARCHAR) 'long' and ImageType (LONGVARBINARY) to
>>>> 'long
>>>> raw'. This change was already documented in the migration notes.
>>>> Should
>>>> the mapping for Oracle (only) be changed back to clob and blob?
>>>>
>>>> HHH-6726 is caused by an issue in Oracle JDBC driver (version
>>>> 10.2.0.4
>>>> and later). This bug appears when LONG or LONG RAW columns are
>>>> accessed
>>>> not as first or last while processing SQL statement.
>>>>
>>>> I have discussed the topic of mapping TextType to CLOB and ImageType
>>>> to
>>>> BLOB (only in Oracle dialect) with Strong Liu. Reasons for doing so:
>>>> - Oracle allows only one LONG / LONG RAW column per table. This might
>>>> be
>>>> the most important from Hibernate's perspective.
>>>> - LONG / LONG RAW - up to 2 GB, BLOB / CLOB - up to 4 GB.
>>>> - In PL/SQL using LOBs is more efficient (random access to data).
>>>> LONG
>>>> only sequential.
>>>> - LONG and LONG RAW are deprecated.
>>>>
>>>> What is your opinion?
>>>>
>>>> Regards,
>>>> Lukasz Antoniak
>>>> _______________________________________________
>>>> hibernate-dev mailing list
>>>> hibernate-dev(a)lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>>
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
8 years, 9 months
Query handling : Antlr 3 versus Antlr 4
by Steve Ebersole
As most of you know already, we are planning to redesign the current
Antlr-based HQL/JPQL parser in ORM for a variety of reasons.
The current approach in the translator (Antlr 2 based, although Antlr 3
supports the same model) is that we actually define multiple
grammars/parsers which progressively re-write the tree adding more and more
semantic information; think of this as multiple passes or phases. The
current code has 3 phases:
1) parsing - we simply parse the HQL/JPQL query into an AST, although we do
do one interesting (and uber-important!) re-write here where we "hoist" the
from clause in front of all other clauses.
2) rough semantic analysis - the current code, to be honest, sucks here.
The end result of this phase is a tree that mixes normalized semantic
information with lots of SQL fragments. It is extremely fugly
3) rendering to SQL
The idea of phases is still the best way to attack this translation imo. I
just think we did not implement the phases very well before; we were just
learning Antlr at the time. So part of the redesign here is to leverage
our better understanding of Antlr and design some better trees. The other
big reason is to centralize the generation of SQL into one place rather
than the 3 different places we do it today (not to mention the many, many
places we render SQL fragments).
Part of the process here is to decide which parser to use. Antlr 2 is
ancient :) I used Antlr 3 in the initial prototyping of this redesign
because it was the most recent release at that time. In the interim Antlr
4 has been released.
I have been evaluating whether Antlr 4 is appropriate for our needs there.
Antlr 4 is a pretty big conceptual deviation from Antlr 2/3 in quite a few
ways. Generally speaking, Antlr 4 is geared more towards interpreting
rather than translating/transforming. It can handle "transformation" if
the transformation is the final step in the process. Transformations is
where tree re-writing comes in handy.
First lets step back and look at the "conceptual model" of Antlr 4. The
grammar is used to produce:
1) the parser - takes the input and builds a "parse tree" based on the
rules of the lexer and grammar.
2) listener/visitor for parse-tree traversal - can optionally generate
listeners or visitors (or both) for traversing the parse tree (output from
parser).
There are 2 highly-related changes that negatively impact us:
1) no tree grammars/parsers
2) no tree re-writing
Our existing translator is fundamentally built on the concepts of tree
parsers and tree re-writing. Even the initial prototypes for the redesign
(and the current state of hql-parser which Sanne and Gunnar picked up from
there) are built on those concepts. So moving to Antlr 4 in that regard
does represent a risk. How big of a risk, and whether that risk is worth
it, is what we need to determine.
What does all this mean in simple, practical terms? Let's look at a simple
query: "select c.headquarters.state.code from Company c". Simple syntactic
analysis will produce a tree something like:
[QUERY]
[SELECT]
[DOT]
[DOT]
[DOT]
[IDENT, "c"]
[IDENT, "headquarters"]
[IDENT, "state"]
[IDENT, "code"]
[FROM]
[SPACE]
[SPACE_ROOT]
[IDENT, "Customer"]
[IDENT, "c"]
There is not a lot of semantic (meaning) information here. A more semantic
representation of the query would look something like:
[QUERY]
[SELECT]
[ATTRIBUTE_REF]
[ALIAS_REF, "<gen:1>"]
[IDENT, "code"]
[FROM]
[SPACE]
[PERSISTER_REF]
[ENTITY_NAME, "com.acme.Customer"]
[ALIAS, "c"]
[JOIN]
[INNER]
[ATTRIBUTE_JOIN]
[IDENT, "headquarters"]
[ALIAS, "<gen:0>"]
[JOIN]
[INNER]
[ATTRIBUTE_JOIN]
[IDENT, "state"]
[ALIAS, "<gen:1>"]
Notice especially the difference in the tree rules. This is tree
re-writing, and is the major difference affecting us. Consider a specific
thing like the "c.headquarters.state.code" DOT-IDENT sequence. Essentially
Antlr 4 would make us deal with that as a DOT-IDENT sequence through all
the phases - even SQL generation. Quite fugly. The intent of Antlr 4 in
cases like this is to build up an external state table (external to the
tree itself) or what Antlr folks typically refer to as "iterative tree
decoration"[1]. So with Antlr 4, in generating the SQL, we would still be
handling calls in terms of "c.headquarters.state.code" in the SELECT clause
and resolving that through the external symbol tables. Again, with Antlr 4
we would always be walking that initial (non-semantic) tree. Unless I am
missing something. I would be happy to be corrected, if anyone knows Antlr
4 better. I have also asked as part of the antlr-discussion group[2].
In my opinion though, if it comes down to us needing to walk the tree in
that first form across all phases I just do not see the benefit to moving
to Antlr 4.
P.S. When I say SQL above I really just mean the target query language for
the back-end data store whether that be SQL targeting a RDBMS for ORM or a
NoSQL store for OGM.
[1] I still have not fully grokked this paradigm, so I may be missing
something, but... AFAICT even in this paradigm the listener/visitor rules
are defined in terms of the initial parse tree rules rather than more
[2] https://groups.google.com/forum/#!topic/antlr-discussion/hzF_YrzfDKo
9 years, 5 months
Changelog file in Hibernate ORM
by Sanne Grinovero
The file changelog.txt in the root ot the Hibernate ORM project seems outdated.
Is it not maintained anymore? I found it handy.
Sanne
9 years, 5 months
did hibernate.org crash?
by Scott Marlow
Trying to open http://hibernate.org/dtd/hibernate-configuration-3.0.dtd,
gives a:
"
Site temporarily disabled
This page has been temporarily disabled due to a misconfigured custom
domain.
Are you the site owner? Follow the instructions for setting up a custom
domain with GitHub pages to update your site's DNS records to point to
the proper IP address.
You can find more information in our GitHub Pages legacy IP deprecation
blog post. If you have any questions, please contact support.
"
The above causes a WildFly test failure http://pastebin.com/fm8VMCcb
(from org.hibernate.service.internal.JaxbProcessor.unmarshal).
Scott
9 years, 9 months
Re: [hibernate-dev] Closed pull requests
by Steve Ebersole
Well telling me/us which PRs got closed would be a good first step :)
Overall, checking that the PR still applies is a good idea too.
On Thu, Oct 30, 2014 at 5:34 PM, Andrej Golovnin <golovnin(a)gmx.net> wrote:
>
> > On 30.10.2014, at 23:15, Steve Ebersole <steve(a)hibernate.org> wrote:
> >
> > Well maybe rather than being hurt and sensitive maybe just understand
> that it was not intentional. GitHub automatically closed them. It was
> simply a mistake.
> >
> > Or be sensitive and threatening about it. Whichever you think helps you
> best ;)
>
> This does not answer my 1. question. :-) What is the best way to proceed?
> Are you going to reopen the pull requests or should we create new requests?
>
> I think, that at least one of my patches must be modified to be applied to
> the "new master".
>
> Any clear small guide, what we (contributors) should do to reopen the pull
> requests
> or a small info that you will care about the closed pull requests
> and we should sit down, relax and wait until you are done, would be really
> helpful.
>
> >
> > On Oct 30, 2014 11:05 PM, "Andrej Golovnin" <golovnin(a)gmx.net> wrote:
> > Hello Steve,
> >
> > you have closed my pull requests and
> > pull requests from others too without any comment.
> >
> > Should we create a new pull requests for the "new master" branch?
> > Or should we just forget all the time and the work we have invested
> > to create those pull requests and stop submitting any new pull requests
> > in the future?
> >
> > Best regards,
> > Andrej Golovnin
>
>
9 years, 11 months
org.hibernate.persister.spi.PersisterFactory and 5.0
by Steve Ebersole
Part of the goals for ORM 5.0 is moving from Configuration to the
ServiceRegistry+Metadata for building a SessionFactory.
One of the points I ran into that will have to change
is org.hibernate.persister.spi.PersisterFactory. The problems is that
PersisterFactory accepts a Configuration as part of building
CollectionPersisters. The need for Configuration in the standard
CollectionPersister impls is amazingly trivial; we literally use it to
locate the associated entity's PersistentClass to grab the classes dom4j
node name, and this is right after we have just resolved the corresponding
EntityPersister. The point being that the standard CollectionPersisters
really don't need access to the Configuration.
I am pretty sure OGM provides a custom PersisterFactory, or is it just
the PersisterClassResolver that OGM provides? Also, I would assume OGM is
providing custom CollectionPersister impls. This change would affect both
usages.
I wanted y'all to be aware of this upcoming change. But I also wanted to
start a discussion about what the signature(s) should become. Currently we
pass:
* Configuration
* Collection (the parsed mapping info)
* CollectionRegionAccessStrategy
* SessionFactoryImplementor
I suggest we pass:
* Collection
* CollectionRegionAccessStrategy
* SessionFactoryImplementor
* Mapping
(I changed order to align with the order for building EntityPersisters)
Mapping is org.hibernate.engine.spi.Mapping which is part of
Configuration. I decided to (at least temporarily) port this contract
forward to ease migration. Metadata implements it.
There is a similar discussion to be had wrt Integrators. I will follow up
with an email specific to them later.
9 years, 11 months
Maintenance on ci.hibernate.org
by Sanne Grinovero
All,
we needed to do some unplanned maintenance on ci.hibernate.org.
At some point it was clear that we needed do deploy some significant
upgrades so while Davide fixed Apache HTTPD, I went ahead and updated
all of Jenkins plugins as well.. which I usually avoid to not need
checking all builds and integrations with external services, but this
time all validations would be needed anyway.
So:
- I'm sorry if you received some false notifications from it.
- If some builds are now broken, it might need some reconfiguration.
- same for plugins configurations: I'm unable to double-check all
things you've all setup.
Please let me know what isn't working anymore, or if you have a moment
to volunteer looking yourself that would be great.
# Reminder
Anyone in the Hibernate organization on GitHub has configuration and
write permissions on this server; if you know what you're doing feel
free to take advantage of that but please always share a summary of
what you did / plan to do.
Sanne
9 years, 11 months
Making it pluggable how GenerationType.AUTO is mapped
by Gunnar Morling
Hi Steve, all,
For OGM it would be beneficial if there was an extension point in ORM which
allows us to customize how GenerationType.AUTO is mapped. AFAICS, that's
currently semi-hard coded in ORM. Depending on whether
"new_generator_mappings" is set, either SEQUENCE (true) or "native" (false)
will be used.
Now for the MongoDB grid dialect we'd ideally do the following: If the id
of an entity is declared as
org.bson.typesObjectId
or
@Type(type = "objectid") String
then map AUTO to IDENTITY (ObjectId is a MongoDB-specific type which
resembles an identity column). Otherwise map it to TABLE (or SEQUENCE,
which transparently falls back to TABLE).
I.e. this decision would not be a global one, but depend on the specific id
type.
Would this be feasible to do? or is it mandated by JPA somehow that AUTO is
always mapped in the same way? I cannot judge on the amount of changes
required in ORM as atm. AUTO always is mapped in the same way, it doesn't
depend on the specific entity and its id type.
Any thoughts?
Thanks,
--Gunnar
9 years, 11 months
Main feed on in.relation.to broken
by Gunnar Morling
Hi,
When going to in.relation.to, I only see a minority of posts, apparently
those authored by Hardy. The other posts are still there, if I select a
blogger in the menu, I see all their posts.
Someone knows how to make all posts show up again on the main feed?
Thanks,
--Gunnar
9 years, 12 months
[OGM] storing the column names in the entity keys for K/V stores
by Emmanuel Bernard
Hi,
With OGM-452 behind us which brings one cache per “table”, we now have
another decision in front of us.
Should we use a synthetic key for the cache key (say a
PersistentEntityKey class containing the array of column names and the
array of column values)?
Or should we use the natural object key?
== Natural entity key
In the latter, things gets complicated quickly, let me explain:
=== Simple case
For simple cases, the id is a simple property and the fit is very
natural
[source]
--
@Entity
class User {
@Id String name;
...
}
//corresponds to
cache.put(name, mapRepresentingUser);
--
=== Embedded id
If the identifier is an embedded id, you have several choices that all have
drawbacks.
1. use the embedded id class as key `cache.put( new Name("Emmanuel", "Bernard"), mapRepresentingUser );`
2. use an array of property values `cache.put( new Object[] {"Emmanuel", "Bernard"}, mapRepresentingUser );`
3. use a Map<String,Object> corresponding to the array `cache.put( new HashMap<String,Object>( {{ "firstname" -> "Emmanuel", "lastname"->"Bernard" } ), mapRepresentingUser );
4. use an synthetic key `cache.put( new PersistentEntityKey( new String[] {"firstname", "lastname" }, new String[] { "Emmanuel", "Bernard" } ), mapRepresentingUser);`
In 1, the problem is that we lose the proper data type abstraction
between the object model and the data stored. `Name` is a user class.
In 2, I think the model is somewhat acceptable but a bit arbitrary.
In 3, I suspect the map is pretty horrific to serialize - that could be
solved by a externalizer. But more importantly the order of the id
columns is lost - even though it might be recoverable with
EntityKeyMetadata?
In 4, we expose the person querying the grid to our OGM specific type.
Aside from this, it is essentially like 4.
=== Entity key approach
I really like the idea of the simple case be mapped directly, it makes
for *the* natural mapping one would have chosen. But as I explained, it
does not scale.
In the composite id case, I don't really know what to chose between 2, 3
and 4.
So, should we go for the simple case if we can? Or favor consistency
between the simple and complex case?
And which of the complex case do we favor?
== Association
In the case of associations, it becomes a bit trickier because the
"simple case" where the association key is made of a single column is
quite uncommon. Association keys are one of these combinations:
* the fk to the owning entity + the index or key of the List or Map
* the fk to the owning entity + the fk to the target entity (Set)
* the fk to the owning entity + the list of columns of the simple or
* embedded type (Set)
* the fk to the owning entity + the surrogate id of the Bag
* all columns in case of a non id backed bag
All that to say that we are most of the time in the complex case of
EntityKey with one of the 4 choices.
Any thoughts and preferences?
Emmanuel
9 years, 12 months
