[Hibernate-JIRA] Created: (HSEARCH-194) Inconsistent performance between hibernate search and pure lucene access
by Stephane Nicoll (JIRA)
Inconsistent performance between hibernate search and pure lucene access
------------------------------------------------------------------------
Key: HSEARCH-194
URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-194
Project: Hibernate Search
Issue Type: Bug
Components: query
Affects Versions: 3.0.1.GA
Environment: Linux - Hibernate 3.2.6, Hibernate Annotations 3..3.1 - Lucene 2.3.1
Reporter: Stephane Nicoll
Priority: Critical
Attachments: Monitor_Usage_Statistics.html
I have a simple index that contains:
* id (pk of the entity)
* keywords (a list of tokens)
The index contains 100.000 objects and the keywords field has 2 tokens from a list of 40 different values
What I want to do is retrieve all the IDs that matches a given lucene query on the keywords. So for that I'm doing something like:
FullTextSession fullTextSession = Search.createFullTextSession(session);
QueryParser parser = new QueryParser("keywords", luceneAnalyzer);
org.apache.lucene.search.Query hibernateQuery = parser.parse("foo AND bar");
FullTextQuery fullTextQuery = fullTextSession.createFullTextQuery(hibernateQuery, target);
fullTextQuery.setProjection("id");
fullTextQuery.setResultTransformer(resultTransformer);
Iterator it = fullTextQuery.iterate();
Where ResultTransformer is
private static class FirstObjectResultTransformer implements ResultTransformer {
public Object transformTuple(Object[] objects, String[] strings) {
return objects[0];
}
public List transformList(List list) {
return list;
}
}
If I do a load test with a single thread, the execution time of my lucene query is around 200 msec. If I do a load test with 10 threads, the execution time is 2 sec (per user!). If I run the profiler on the service, I see lots of deadocks on SegmentReader.
Switching to a "non-shared" strategy removes the deadlocks but it's still slow (1.5 sec).
Now, If I execute the same query on the same index and the same host with only the lucene API, the query takes around 100msec with 10 concurrent users. I tried to use the lucene API from Hibernate Search but it did not change anything.
What am I missing? Attached the profiling result.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[Hibernate-JIRA] Created: (HSEARCH-131) Make Filter caching more efficient
by Emmanuel Bernard (JIRA)
Make Filter caching more efficient
----------------------------------
Key: HSEARCH-131
URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-131
Project: Hibernate Search
Issue Type: New Feature
Components: engine
Reporter: Emmanuel Bernard
Priority: Minor
Fix For: 3.1.0
Emmanuel,
This is the contents of a message on the Lucene boards regarding cache and filter. I thought you moght be interested in it since you weren't completely happy with your bitset filtering.
---------------------
Check out the HashDocSet from Solr, this is the best way to cache small sets of search results. In general, the Solr BitSet/DocSet classes are more efficient than using the standard java.util.BitSet. You can use these independent of the rest of Solr (though I recommend checking out Solr if you want to do complex caching).
- Thom
John Patterson wrote:
> Hi,
>
> I am thinking about caching search results for common queries and just
> want to check that for small numbers of results it would be better to
> store the doc number as ints or shorts than to store a Filter with a
> BitSet. I guess if you results contain less than 1/32 or 1/16 of the
> number of documents then it would take less memory.
>
> Is there anything else to consider?
>
Hope this helps some.
John G.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[Hibernate-JIRA] Created: (HHH-3363) Invalid SQL: missing join and column
by Istvan Kovacs (JIRA)
Invalid SQL: missing join and column
------------------------------------
Key: HHH-3363
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3363
Project: Hibernate3
Issue Type: Bug
Affects Versions: 3.3.0.CR1, 3.2.6
Environment: hsqldb
Reporter: Istvan Kovacs
Using the same sources as in HHH-3362
select p.name, s.name, s.stock[p]
from Shop s, Product p
This produces SQL that does not contain the stock column, and does not
join tables PRODUCT and SHOP via SHOP_PRODUCT_STOCK:
select
product1_.name as col_0_0_,
shop0_.name as col_1_0_
from
Shop shop0_,
PRODUCT product1_
In another query, the join is performed when stock[p] is referenced:
select emp.name, shop.name
from
Employee emp join emp.shop as shop, Product p
where
shop.stock[p] > 5
and
p.name='Product_10'
SQL:
select
employee0_.name as col_0_0_,
shop1_.name as col_1_0_
from
Employee employee0_
inner join
Shop shop1_
on employee0_.SHOP_ID=shop1_.SHOP_ID,
PRODUCT product2_,
SHOP_PRODUCT_STOCK stock3_
where
shop1_.SHOP_ID=stock3_.SHOP_ID
and stock3_.PRODUCT_ID = product2_.PRODUCT_ID
and stock3_.AMOUNT>5
and product2_.name='Product_10'
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[Hibernate-JIRA] Commented: (HHH-1754) Sequence support for DB2400Dialect
by john daly (JIRA)
[ http://opensource.atlassian.com/projects/hibernate/browse/HHH-1754?page=c... ]
john daly commented on HHH-1754:
--------------------------------
The change:
public String getSequenceNextValString(String sequenceName) {
return "select nextval for " + sequenceName + " from sysibm.sysdummy1";
}
will work for both standard DB2 and DB2400 so it might be best to put it in DB2Dialect.java?
Select nextval for SEQUENCENAME from sysibm.sysdummy1
and
VALUES nextval for SEQUENCENAME
arer apparently equivalent in DB2 but the VALUES version doesn't work in DB2400
> Sequence support for DB2400Dialect
> ----------------------------------
>
> Key: HHH-1754
> URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-1754
> Project: Hibernate3
> Issue Type: Improvement
> Components: core
> Affects Versions: 3.1.3
> Environment: Hibernate 3.1.3, IBM DB2 UDB for iSeries on i5/OS V5R3
> Reporter: Carey Evans
> Priority: Minor
> Attachments: DB2400Dialect.java
>
>
> As of i5/OS V5R3 (the operating system previously known as OS/400), DB2 supports sequences, but DB2400Dialect doesn't include support for the new functionality. This support can be modelled on DB2Dialect, except that "values ..." must be written "select ... from sysibm.sysdummy1", and the syssequences view is in the qsys2 schema.
> I don't know what impact this change would have on older versions of OS/400 that don't support sequences.
> The attached DB2400Dialect source includes sequence support, with some additional minor corrections for timestamps on i5/OS.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[Hibernate-JIRA] Commented: (HBX-524) Reverse of one-to-one relationships
by Marcio Carvalho (JIRA)
[ http://opensource.atlassian.com/projects/hibernate/browse/HBX-524?page=co... ]
Marcio Carvalho commented on HBX-524:
-------------------------------------
Hi Max,
I am customizing the hibernate-tools reverse engineering process to generate code related to our home-made framework.
I would like to assume that the Primary Key (PK) - Foreign Key (FK) LAW #2 will always generate an annotated OneToOne relationship. But in this case, hibernate-tools generate an unidirectional OneToMany.
Which method could I override to accomplish this standard behavior?
Thanks in advance.
> Reverse of one-to-one relationships
> -----------------------------------
>
> Key: HBX-524
> URL: http://opensource.atlassian.com/projects/hibernate/browse/HBX-524
> Project: Hibernate Tools
> Issue Type: Bug
> Components: reverse-engineer
> Affects Versions: 3.1beta2
> Environment: HIbernate 3.1, Oracle 9i
> Reporter: Andrea Cattani
>
> Hi,
> I've posted this issue to the forum and got this response from Max, Hibernate Team:
> "the reveng tools does not detect this as a one-to-one. it probably could, so add a request/patch to jira."
> The problem I've faced is the following:
> I have two tables, let's say
> - table A with column ID (PK) and other fields
> - table B with column ID (PK) and other fields
> table B has a foreign key constraint against table A, from column ID to column ID (one-to-one)
> When I reverese the tables with the HibernateTools I have such a resultant mapping for table B:
> <class name="B" table="B" schema="SCHEMA">
> <id name="id" type="string">
> <column name="ID" length="12" />
> <generator class="assigned" />
> </id>
> <[b]many-to-one name[/b]="a" class="A" update="false" insert="false" fetch="select">
> <column name="ID" length="12" not-null="true" unique="true" />
> </many-to-one>
> ....
> And this one for table A:
> <class name="A" table="A" schema="SCHEMA">
> <id name="id" type="string">
> <column name="ID" length="12" />
> <generator class="assigned"/>
> </id>
> <set name="b" inverse="true">
> <key>
> <column name="ID" length="12" not-null="true" unique="true" />
> </key>
> <[b]one-to-many[/b] class="B" />
> </set>
> </class>
> while I was expecting something like:
> [i]<one-to-one name="a" class="A" constrained="true"/>[/i]
> in table B, and the same (or nothing) in table A
> Thank you
> Andi
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months
[Hibernate-JIRA] Created: (ANN-757) Multiple secondary tables give uncontrollable ordering of sql inserts and of joins in select statements.
by Pete Geraghty (JIRA)
Multiple secondary tables give uncontrollable ordering of sql inserts and of joins in select statements.
--------------------------------------------------------------------------------------------------------
Key: ANN-757
URL: http://opensource.atlassian.com/projects/hibernate/browse/ANN-757
Project: Hibernate Annotations
Issue Type: Bug
Affects Versions: 3.3.1.GA
Environment: Hibernate Annotations 3.3.1.GA
Hibernate 3.2.6.ga
Oracle 10g express edition
Reporter: Pete Geraghty
Priority: Minor
When an entity has multiple secondary tables, the order in which they are considered when constructing and executing sql is important in at least two cases:
(a) selection with LEFT OUTER JOIN
(b) the order of insertion of rows into the database when persisting
However the order in which Hibernate actually processes them is effectively random as it is using the values() collection of a HashMap.
I am using secondary tables predominantly to achieve a mixed inheritance strategy, as suggested in Gavin King's "Java Persistence With Hiberate" book, section 5.1.5. However there are also some situations where I need secondary tables for other reasons, and consequently I have two secondary tables declared on some entities.
Consider the following situation (table names for clarity of illustration):
@Entity
@Table(name = "a_main")
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
public abstract class A { ... }
@Entity
@SecondaryTables({
@SecondaryTable(name = "b_main"),
@SecondaryTable(name = "b_secondary")
})
public class B extends A { ...}
At a database level, there must be a row in b_main for every instance of B, and there may or may not be a row in b_secondary. I want foreign key constraints from b_main to a_main, and from b_secondary to b_main.
In fact the order in which inserts are processed depend on the hashed value of the actual table names corresponding to b_main and b_secondary, i.e., is effectively indeterminate, and in some cases I must ask the dba to drop the foreign key constraint which ought to be there.
Also, when an instance of B is being hydrated from the result set of a query on A, the select issued specifies a left outer join involving b_main and b_secondary. Which of these is considered left depends on the hash of the actual table names, and so for some table names the select is the wrong way round and will not retrieve the necessary values, i.e., it takes the following form:
select ... from b_secondary .... left outer join b_main ....
Whereas I want:
select ... from b_main .... left outer join b_secondary ....
The basic problem is that although EntityBinder processes SecondaryTable annotations in a defined order (the order in which they appear), it uses two HashMaps to store these , secondaryTables and secondaryTableJoins. Then the values collection iterator is used (for example) when adding joins to the PersistentClass object. HashMap offers no contract as to the order of the values collection. Strictly it doesn't even guarantee that the values collections of the two HashMaps in question will have corresponding entries in the same order, although in practice they will, as they have the same keys and same capacity.
When deciding on the tables relevant to a select, the table with the lowest "table number" is deemed to be the "driving table" e.g., in the AbstractEntityPersister renderSelect() method at line 2891. However, these table numbers are indirectly derived from the iterator on the values() collection of the secondaryTables map.
The map variables are assigned on lines 89 and 90 of EntityBinder
private java.util.Map<String, Join> secondaryTables = new HashMap<String, Join>();
private java.util.Map<String, Object> secondaryTableJoins = new HashMap<String, Object>();
If these are instead assigned to new LinkedHashMaps, the values collections reflect the order of insertion, and therefore the order in which the annotations appear determines the order in which left joins and inserts are processed, which gives more predictable and controllable behaviour.
private java.util.Map<String, Join> secondaryTables = new LinkedHashMap<String, Join>();
private java.util.Map<String, Object> secondaryTableJoins = new LinkedHashMap<String, Object>();
I think this represents a definite improvement, and it solves my problem.
I can imagine other situations where both secondary tables are "optional", and the use of a LEFT OUTER JOIN is wrong either way round I haven't looked further at what a solution to that would look like.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
17 years, 9 months