[hibernate-dev] RE: Exposing legacy Lucene documents on searches.
John Griffin
jgriffin at thebluezone.net
Sun Jun 17 22:43:02 EDT 2007
Emmanuel,
You didn't misunderstand. Forget about the annotation I don't know what I
was thinking about. I was going to reply earlier to tell you that but
couldn't get to my email.
I do have an MSN IM account griffij at sosstaffing.com
Anyway, I'm looking at the FullTextQueryImpl class to implement this. The
list, scroll and iterate methods all getHits() so I could wedge in my
LegacyDocuments class to hold the top 20 hits and hits size and expose it
via a getter. It's immutable and has methods to return a List of the hits,
an Iterator over them and I'm working on a scroll. Of course these methods
could become part of FullTextQueryImpl and be defined in the FullTextQuery
Interface under names related to their other corresponding methods but I
thought it would better to keep them separate.
There are a lot of ways to go about this just IM me. I'm on mountain time so
I'm in the office from 8:30 to about 5:30 YOUR time. I'm on at home after
that LATE.
By the way could you please explain your 'projection' to me. I don't
understand the concept.
John Griffin
-----Original Message-----
From: Emmanuel Bernard [mailto:emmanuel at hibernate.org]
Sent: Sunday, June 17, 2007 6:56 PM
To: John Griffin
Cc: hibernate-dev at lists.jboss.org
Subject: Re: [hibernate-dev] RE: Exposing legacy Lucene documents on
searches.
hi John,
You mean having an @LegacyDocuments annotation on the application
method?
@LegacyDocuments
public void MyMethod() {
fullTextSession.scroll();
)
reading such an annotation is really expensive, the VM has to build
the stack trace. and access the caller method.
Besides, I'm more confirtable with an API for such a thing,
fullTextSession.setProjection(OBJECT, DOCUMENT).scroll() should do
the trick, no? Maybe I misunderstood you. What is your use case?
Emmanuel
On Jun 17, 2007, at 4:33 PM, John Griffin wrote:
> Emanuel,
>
> I finally have time to get this going. In agreement with you this
> is what I
> plan on doing. I would like to add a method level annotation on the
> iterate,
> list and scroll methods that indicates whether or not the user
> wants the
> legacy Lucene objects retained (probably @LegacyDocuments the
> absence of
> which means NO). That way no processing will be done if not
> necessary. I
> already have the immutable object LegacyDocuments written that
> contains a
> List of the top 20 Hit objects and a count of total hits returned.
> I just
> need to add the Annotation and LegacyDocuments processing
> statements to
> these methods in FullTextQueryImpl.
>
> John Griffin
>
>
> -----Original Message-----
> From: hibernate-dev-bounces at lists.jboss.org
> [mailto:hibernate-dev-bounces at lists.jboss.org] On Behalf Of
> hibernate-dev-request at lists.jboss.org
> Sent: Wednesday, June 13, 2007 7:46 AM
> To: hibernate-dev at lists.jboss.org
> Subject: hibernate-dev Digest, Vol 12, Issue 55
>
> Send hibernate-dev mailing list submissions to
> hibernate-dev at lists.jboss.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> or, via email, send a message with subject or body 'help' to
> hibernate-dev-request at lists.jboss.org
>
> You can reach the person managing the list at
> hibernate-dev-owner at lists.jboss.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of hibernate-dev digest..."
>
>
> Today's Topics:
>
> 1. Re: hsearch-6 serializable Hits object (Emmanuel Bernard)
> 2. Hibernate + Oracle RDB 7.1 (Rehnman Martin)
> 3. hibernate-hsqldb-testsuite build.57 Build Fixed
> (jboss-qa-internal at redhat.com)
> 4. hibernate-sqlserver-jtds-testsuite build.60 Build Fixed
> (jboss-qa-internal at redhat.com)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 12 Jun 2007 16:13:20 -0400
> From: Emmanuel Bernard <emmanuel at hibernate.org>
> Subject: Re: [hibernate-dev] hsearch-6 serializable Hits object
> To: John Griffin <jgriffin at thebluezone.net>,
> hibernate-dev at lists.jboss.org
> Message-ID: <F9CA3F8A-B468-46D5-AC6B-0993BD0A0F23 at hibernate.org>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> I see,
> Do you keep your IndexReader opened while the data is moved to your
> client?
> Because the document id is not guaranteed to stay constant.
>
> It seems to me that we can avoid such a feature for now, you can
> return getResultSize() + a list of the 20 top objects + score for
> example. Such a structure would be serializable.
> Let's see how it goes.
>
> On 11 juin 07, at 23:46, John Griffin wrote:
>
>>
>>
>> -----Original Message-----
>> From: Emmanuel Bernard [mailto:emmanuel at hibernate.org]
>> Sent: Monday, June 11, 2007 10:06 AM
>> To: John Griffin
>> Cc: hibernate-dev at lists.jboss.org
>> Subject: Re: [hibernate-dev] hsearch-6 serializable Hits object
>>
>> How do you deal with serialized Hits?
>> You raise a "LazyInitializationException" like when a user access an
>> unavailable document?
>> Same for the serializable HitIterator?
>>
>> --------------
>> I don't deal with lazy initialization. I use the low-level TopDocs
>> object.
>> This gives me a document number and score only. Hence it gives me
>> good
>> response times even across a 1-million document index. This way the
>> size of the document is not a consideration for memory constraints.
>> Once I have the TopDocs object I just retrieve the paged amount of
>> documents as the User asks for them. 10, 20 30, etc. at a time. I
>> also
>> maintain state with this TopDocs object by passing it back and forth
>> between client/server with pointers to current locations. We do limit
>> the size of the TopDocs object to 200. (somewhat arbitrary)
>>
>> TopDocs also contains a count of the number of actual hits that
>> resulted
>> from the search. I use this to warn the user that his search
>> should be
>> narrowed if necessary. Here's an ex. Our employeecomments table has 1
>> million records. A search for the word 'labor' returns a count of
>> 157,000+
>> hits but we only return the top 20 along with the TopDocs object
>> (hidden
>> for state) and a statement that 'maybe you should refine your
>> search a
>> little';>) (we do tell them the amount of hits though). Also we
>> limit the
>> actual size of the TopDocs Object to the top 200 records.
>> --------------
>>
>> Which approach are you following to expose the data? christian's
>> proposal?
>> In christian's proposal, what is really needed for serialization is
>> Hit, but then Hit is not thing but a Document and a score wich can be
>> projected as well.
>>
>> --------------
>> My SerializableHits object is really nothing more than a rebuild of
>> a Hit
>> and score into serializable components. I use the doc number from the
>> TopDocs to obtain the document and use the TopDocs score. The entire
>> reason behind this is to allow Lucene classes we had originally
>> located
>> locally to our application to be moved to a remote server that
>> deals solely
>> with Lucene. Now we have our application on clustered JBoss which
>> accesses
>> remote servers that solely deal with Lucene (add, update, delete,
>> search).
>> --------------
>>
>> I don't see the need for a lazy Hit in this scheme, hence no need for
>> a serializable version.
>>
>> BTW who is consuming Hit(s) aside for the user's application? does
>> Lucene has (public) APIs consuming Hits?
>>
>> --------------
>> In my case the application is the only consumer. I've never looked at
>> Lucene in that way i.e. if it were the consumer. I hope this helps.
>>
>> Let me know what you think.
>> --------------
>>
>> Emmanuel
>>
>> On 7 juin 07, at 22:47, John Griffin wrote:
>>
>>> Do we want to convert the Lucene Hits object to a serializable
>>> format so it could be accessed remotely? I'm have developed this at
>>> work and have a 'SerializablelHits' object for this purpose. Is
>>> this overkill right now? Change later? Thoughts?
>>>
>>>
>>>
>>> John G.
>>>
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 13 Jun 2007 15:12:32 +0200
> From: "Rehnman Martin" <Martin.Rehnman at lotterien.at>
> Subject: [hibernate-dev] Hibernate + Oracle RDB 7.1
> To: <hibernate-dev at lists.jboss.org>
> Cc: Boisits Franz <Franz.Boisits at lotterien.at>
> Message-ID:
>
> <11FC899FAE8B8748AB9E591B1454774401B8C8FA at EXCHCLU01.office.lottery.co.
> at>
>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi,
>
> We have discovered a minor problem within JDBCtransaction.java when
> using Oracle RDB 7.1,
> the database transaction is not automatically rollbacked by the
> database
> on commit failures,
> the client has to send a rollback over the JDBC-connection in that
> case.
> When no rollback is sent,
> the database transaction will still be open and associated with a
> JDBC-connection when the connection is returned
> to the connection pool. At another time some other client transaction
> will get this connection from the pool with
> an open database transaction which then will fail. We have solved this
> problem by changing JDBCTransaction in this way:
>
> private boolean rollbackAfterCommitFailed = false;
>
> public void setRollbackAfterCommitFailed(boolean
> rollbackAfterCommitFailed) {
> this.rollbackAfterCommitFailed =
> rollbackAfterCommitFailed;
> }
>
> private void commitAndResetAutoCommit() throws SQLException {
> try {
> jdbcContext.connection().commit();
>
> } catch (SQLException e) {
> // Extension for Oracle-RDB: Rollback after
> failed Commit
> if (rollbackAfterCommitFailed) {
> try {
> log.error("Rollback after
> Commit-Failure");
>
> jdbcContext.connection().rollback();
> } catch (SQLException s) {
> log.error("Exception during
> Rollback", s);
> }
> throw e;
> }
> finally {
> toggleAutoCommit();
> }
> }
>
> In the JDBCTransactionFactory we added following:
>
> private boolean rollbackCommitFailures; // Property
> hibernate.transaction.rollback_after_commit_failure
>
> public Transaction createTransaction(JDBCContext jdbcContext,
> Context transactionContext) throws HibernateException {
> JDBCTransaction trans = new JDBCTransaction(jdbcContext,
> transactionContext);
> trans.setRollbackAfterCommitFailed(rollbackCommitFailures);
> return trans;
> }
>
> /** {@inheritDoc} */
> public void configure(Properties props) throws
> HibernateException {
>
> rollbackCommitFailures = PropertiesHelper.getBoolean(
> Environment.TRANSACTION_ROLLBACK_COMMIT_FAILURES,
> props,false);
>
> }
>
> It would be nice to have this little problem solved in future
> Hibernate
> releases.
> I also really would like to see Oracle RDB in the list of supported
> databases, for every new
> Hibernate Release I have to compile/add RdbDialect.java (got it from
> HP/Oracle) to the jar.
>
>
> Regards
>
> Martin
>
>
> Oesterreichische Lotterien Gesellschaft m.b.H., Rennweg 44, A-1038
> Wien,
> FN 54472 g, Handelsgericht Wien, DVR-Nr: 0476706
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.jboss.org/pipermail/hibernate-dev/attachments/
> 20070613/971f488c
> /attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Wed, 13 Jun 2007 09:35:36 -0400 (EDT)
> From: jboss-qa-internal at redhat.com
> Subject: [hibernate-dev] hibernate-hsqldb-testsuite build.57 Build
> Fixed
> To: hibernate-dev at lists.jboss.org, jboss-qa-internal at redhat.com
> Message-ID:
>
> <20828680.1181741736211.JavaMail.cruisecontrol at dev01.qa.atl.jboss.com>
> Content-Type: text/plain; charset="us-ascii"
>
> An HTML attachment was scrubbed...
> URL:
> http://lists.jboss.org/pipermail/hibernate-dev/attachments/
> 20070613/6ca6921b
> /attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Wed, 13 Jun 2007 09:48:26 -0400 (EDT)
> From: jboss-qa-internal at redhat.com
> Subject: [hibernate-dev] hibernate-sqlserver-jtds-testsuite build.60
> Build Fixed
> To: hibernate-dev at lists.jboss.org, jboss-qa-internal at redhat.com
> Message-ID:
>
> <16703414.1181742506527.JavaMail.cruisecontrol at dev01.qa.atl.jboss.com>
> Content-Type: text/plain; charset="us-ascii"
>
> An HTML attachment was scrubbed...
> URL:
> http://lists.jboss.org/pipermail/hibernate-dev/attachments/20070613/
> f6fb43f2
> /attachment.html
>
> ------------------------------
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
>
> End of hibernate-dev Digest, Vol 12, Issue 55
> *********************************************
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
More information about the hibernate-dev
mailing list