[Hibernate-JIRA] Created: (HHH-5300) Configurable strong and soft reference QueryPlanCache sizes
by Manuel Dominguez Sarmiento (JIRA)
Configurable strong and soft reference QueryPlanCache sizes
-----------------------------------------------------------
Key: HHH-5300
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-5300
Project: Hibernate Core
Issue Type: Patch
Components: core
Affects Versions: 3.5.2, 3.5.1, 3.5.0-Final
Environment: N/A
Reporter: Manuel Dominguez Sarmiento
Some of our production servers (Hibernate-based apps) have been hanging on full GC sporadically, most of the time after running normally for a few days, sometimes even over a week.
We suspected a memory leak. We used the Eclipse MAT tool to analyze a live heap dump, and found that most of the heap was being used by QueryPlanCache, most specifically, by the soft references held by SoftLimitMRUCache.
We use very large heaps (up to 30 GB in some cases). Since memory is plentiful and the SoftLimitMRUCache is unbounded, the heap eventually fills up until a major stop-the-world GC is necessary to cleanse the SoftLimitMRUCache soft references. We performed several live tests configuring the Concurrent-Mark-Sweep (CMS) GC collector in order to avoid the full GC caused by concurrent mode failures. We played around with the following settings available in the Sun JVM:
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-XX:CMSIncrementalDutyCycle=<PCT>
-XX:CMSIncrementalDutyCycleMin=<PCT>
-XX:CMSInitiatingOccupancyFraction=<PCT>
-XX:CMSMarkStackSize=<SIZE>
-XX:CMSMarkStackSizeMax=<SIZE>
-XX:SoftRefLRUPolicyMSPerMB=<MSECS>
-XX:+ParallelRefProcEnabled
Most of these options helped somewhat by allowing soft-reference GC to be performed in parallel, ahead of time before tipping the scales and requiring full GC. However, this did not avoid all problems, and the servers still periodically hang upon concurrent mode failures. These are high-load web servers which process hundreds of hits per second, so full GC is disastrous as garbage cannot be collected fast enough. Full GC would sometimes take over 15 minutes, sometimes it would not even finish doing its job requiring a manual app restart.
Before anyone cries out "well, it's probably the application's fault, why do you have so many different queries? Aren't you using parameterized queries / PreparedStatements?" - the application does in fact produce many, many different queries, but most of them are not reused. Even if the QueryPlanCache is highly effective, most of the absolute number of queries are issued only once. Some use cases are the following:
- Our system allows ad-hoc reporting and searching capabilities. Each query is typically issued once and never reused.
- Many of our parameterized queries use IN clauses with variable-length collection/array parameters. I'm unsure whether this affects the cache hit ratio for HQL query plans, but it sure affects native SQL queries, since multiple collection lengths means multiple "?" in the query string, and thus, multiple similar, if slightly different queries polluting the corresponding plan cache.
I'm sure there are other legitimate cases in which an unbounded query plan cache is also a problem, I'm just enumerating the ones we've run into.
This issue is not new. The same problem is described, in one way or another, by HHH-2431, HHH-3191, and HHH-4627. I created a new issue since we've produced a working patch and it would be made more visible, rather than as comments in the previous ones.
The solution involves giving up the unbounded soft-reference-based cache. We introduced two new configuration options:
- hibernate.query.plan_cache_max_strong_references -> defaults to 128
- hibernate.query.plan_cache_max_soft_references -> defaults to 2048
Entries are evicted using an LRU policy, or by memory pressure from the GC in the case of soft references. We used 2048 as a default for the soft size since it seems pretty reasonable, but of course it can be tuned to suit the user's needs. Users looking to emulate previous behaviour (we don't see the point, but who knows ...) can set this option to Integer.MAX_VALUE
The patch introduces no new dependencies. It uses Apache Commons Collections' LRUMap (just as the released version does), and does away with the ReferenceMap (which does not support LRU eviction) in order to manage soft references manually on top of an LRUMap.
We see this issue as a top priority and should be applied to the trunk ASAP. SoftLimitMRUCache has seen the most "radical" changes. SimpleMRUCache, QueryPlanCache and Environment only contain minor changes.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years
[Hibernate-JIRA] Created: (HV-345) Run automated diff between project model constraints and database model constraints
by Roman Arkadijovych Muntyanu (JIRA)
Run automated diff between project model constraints and database model constraints
-----------------------------------------------------------------------------------
Key: HV-345
URL: http://opensource.atlassian.com/projects/hibernate/browse/HV-345
Project: Hibernate Validator
Issue Type: New Feature
Components: engine
Affects Versions: 4.2.0
Reporter: Roman Arkadijovych Muntyanu
Assignee: Hardy Ferentschik
[This feature request has been added to this project due to relation to Java-level constraint validation. Please feel free moving it to appropriate project]
Since constraints validation is added at Java model level, it would be interesting to write a tool, that would do the following during application start-up:
* poll the database for schema;
* poll hibernate for XML/JPA Mappings;
* compare two of the above to constraints defined through validation framework and generate warnings in the logs.
For the initial version this could be simple checks like size of field defined in DB and in the model, field mandatority etc.
This would help a lot in keeping consistency between database and business constraints model.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years
[Hibernate-JIRA] Created: (EJB-276) Support for CLOB not working for DB2 when table per concrete class is being used
by breako (JIRA)
Support for CLOB not working for DB2 when table per concrete class is being used
--------------------------------------------------------------------------------
Key: EJB-276
URL: http://opensource.atlassian.com/projects/hibernate/browse/EJB-276
Project: Hibernate Entity Manager
Type: Bug
Versions: 3.2.1
Reporter: breako
Hi,
This came up in the forums:
http://forum.hibernate.org/viewtopic.php?p=2344008#2344008
Sample code: Two Pojos
@Entity
public class Person {
private int i;
@Identity
public int getI() {
return i;
}
public void setI(int i){
this.i = i;
}
}
@Entity
public class Employee extends Person{
@Basic(fetch=FetchType.LAZY)
@Lob
public String getClobAttr() {
return clobAttr;
}
public void setClobAttr(String clobAttr) {
this.clobAttr = clobAttr;
}
}
Simple test:
public static void main (String args[]) {
Query queryImpl = em.createQuery(" from Person");
List list = queryImpl.getResultList();
}
This will generate SQL
select person0_.clobAttr as clobAttr1 from (select nullif(0,0) as clobAttr from TPerson union all select clobAttr from TEmployee) person0_
which chucks the exception:
DB2 SQL error: SQLCODE: -415, SQLSTATE: 42825, SQLERRMC: null
I think the SQL hibernate should generate should be:
select person0_.clobAttr as clobAttr1 from (select cast(null as CLOB) as clobAttr from TPerson union all select clobAttr from TEmployee) person0_
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
13 years
[Hibernate-JIRA] Created: (HHH-5488) Hibernate fails with two kinds of exceptions on a compsite-key join table that references two composite keys (one column shared by both foreign keys)
by Karsten Wutzke (JIRA)
Hibernate fails with two kinds of exceptions on a compsite-key join table that references two composite keys (one column shared by both foreign keys)
-----------------------------------------------------------------------------------------------------------------------------------------------------
Key: HHH-5488
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-5488
Project: Hibernate Core
Issue Type: Bug
Components: annotations, core, entity-manager
Affects Versions: 3.6.0.Beta2, 3.5.4
Environment: Windows 7 64bit, localhost, Tomcat 6, embedded JBoss, Hibernate 3.5.4 and 3.6.0 Beta2, MySQL 5.1.46
Reporter: Karsten Wutzke
Priority: Blocker
Hibernate fails with two kinds of exceptions on a compsite-key join table that references two composite keys (one column shared by both foreign keys): see ZipArea.java and PostAddress.java inside the attached file
1. When having both entity classes use the annotations as they should be (ZipArea having @Entity and @Table, PostAddress association having @ManyToOne and @JoinColumns) then I get:
18.08.2010 18:22:51 org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet Geo Info Test Servlet as unavailable
18.08.2010 18:22:51 org.apache.catalina.core.StandardContext loadOnStartup
SCHWERWIEGEND: Servlet /geoinfotest threw load() exception
org.hibernate.MappingException: Unable to find column with logical name: state_code in ZipAreas
at org.hibernate.cfg.Ejb3JoinColumn.checkReferencedColumnsType(Ejb3JoinColumn.java:573)
at org.hibernate.cfg.BinderHelper.createSyntheticPropertyReference(BinderHelper.java:125)
at org.hibernate.cfg.ToOneFkSecondPass.doSecondPass(ToOneFkSecondPass.java:110)
at org.hibernate.cfg.Configuration.processEndOfQueue(Configuration.java:1618)
at org.hibernate.cfg.Configuration.processFkSecondPassInOrder(Configuration.java:1541)
at org.hibernate.cfg.Configuration.secondPassCompile(Configuration.java:1457)
at org.hibernate.cfg.Configuration.buildMappings(Configuration.java:1413)
at org.hibernate.ejb.Ejb3Configuration.buildMappings(Ejb3Configuration.java:1453)
at org.hibernate.ejb.EventListenerConfigurator.configure(EventListenerConfigurator.java:193)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:1081)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:275)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:359)
at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:56)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:48)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:32)
at geoinfotest.GeoInfoTestServlet.<init>(GeoInfoTestServlet.java:16)
...
This is the most constant and frequent error. Note though, when I removed all the INDEXes from the database it sometimes happens that Hibernate can't find the column in Cities:
18.08.2010 19:37:43 org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet Geo Info Test Servlet as unavailable
18.08.2010 19:37:43 org.apache.catalina.core.StandardContext loadOnStartup
SCHWERWIEGEND: Servlet /geoinfotest threw load() exception
org.hibernate.MappingException: Unable to find column with logical name: state_code in Cities
at org.hibernate.cfg.Ejb3JoinColumn.checkReferencedColumnsType(Ejb3JoinColumn.java:573)
at org.hibernate.cfg.BinderHelper.createSyntheticPropertyReference(BinderHelper.java:125)
at org.hibernate.cfg.ToOneFkSecondPass.doSecondPass(ToOneFkSecondPass.java:110)
at org.hibernate.cfg.Configuration.processEndOfQueue(Configuration.java:1618)
at org.hibernate.cfg.Configuration.processFkSecondPassInOrder(Configuration.java:1541)
at org.hibernate.cfg.Configuration.secondPassCompile(Configuration.java:1457)
at org.hibernate.cfg.Configuration.buildMappings(Configuration.java:1413)
at org.hibernate.ejb.Ejb3Configuration.buildMappings(Ejb3Configuration.java:1453)
at org.hibernate.ejb.EventListenerConfigurator.configure(EventListenerConfigurator.java:193)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:1081)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:275)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:359)
at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:56)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:48)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:32)
at geoinfotest.GeoInfoTestServlet.<init>(GeoInfoTestServlet.java:16)
...
2. When outcommenting PostAddress's @ManyToOne and @JoinColumns on zipArea and leaving @Entity and @Table on ZipArea class, I get the following exception:
18.08.2010 18:05:02 org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet Geo Info Test Servlet as unavailable
18.08.2010 18:05:02 org.apache.catalina.core.StandardContext loadOnStartup
SCHWERWIEGEND: Servlet /geoinfotest threw load() exception
org.hibernate.AnnotationException: referencedColumnNames(country_code, state_code, name) of geoinfotest.ZipArea.city referencing geoinfotest.City not mapped to a single property
at org.hibernate.cfg.BinderHelper.createSyntheticPropertyReference(BinderHelper.java:204)
at org.hibernate.cfg.ToOneFkSecondPass.doSecondPass(ToOneFkSecondPass.java:110)
at org.hibernate.cfg.Configuration.processEndOfQueue(Configuration.java:1618)
at org.hibernate.cfg.Configuration.processFkSecondPassInOrder(Configuration.java:1541)
at org.hibernate.cfg.Configuration.secondPassCompile(Configuration.java:1457)
at org.hibernate.cfg.Configuration.buildMappings(Configuration.java:1413)
at org.hibernate.ejb.Ejb3Configuration.buildMappings(Ejb3Configuration.java:1453)
at org.hibernate.ejb.EventListenerConfigurator.configure(EventListenerConfigurator.java:193)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:1081)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:275)
at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:359)
at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:56)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:48)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:32)
at geoinfotest.GeoInfoTestServlet.<init>(GeoInfoTestServlet.java:16)
...
I have no idea what's going on. The special situation of the ZipAreas join table is that it references country_code of both tables. As a result, I have only *one* country_code column, which is used twice by ZipArea's foreign keys. Maybe that's confusing Hibernate. I can only guess. The strange thing is that Hibernate fails with City not being mapped to a single property *or* not being able to map state_code. My assumption is that Hibernate gobbles on that special design in conjunction with indexes. Just guessing though.
Here's my original forum post: https://forum.hibernate.org/viewtopic.php?f=1&t=1006411
The DB design can be viewed here: http://www.kawoolutions.com/media/geoinfo.pdf
I have attached a ZIP of the test project including an Ant build script (including deploy-local, undeploy-local, and redeploy-local targets using the Tomcat deployment tasks). I outcommented the Tomcat deploy task setup and targets.
Use "ant war" to build the distribution WAR file. I also put a MySQL DDL script into the db dir, including the above PDF, and a MySQL Workbench model. The example doesn't need data to fail. The DB name is "geoinfo". Just set your MySQL DB username and password in xml/persistence.xml (I think it's not even needed).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years
[Hibernate-JIRA] Created: (HHH-3836) composite-element with null columns in map is ignored
by Benoit Goudreault-Emond (JIRA)
composite-element with null columns in map is ignored
-----------------------------------------------------
Key: HHH-3836
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-3836
Project: Hibernate Core
Issue Type: Bug
Components: core
Affects Versions: 3.3.0.SP1
Environment: 3.3.0 SP1, noticed with Oracle and H2 but probably happens with all databases
Reporter: Benoit Goudreault-Emond
Attachments: patate.zip
When a composite element with all properties set to NULL is inserted in a map of the said composite element, the data gets inserted to the database but is never retreived when reloading the entity.
For instance, with an entity
@Entity
public class TestEntity {
@Id
private int id;
@CollectionOfElements
private Map<String, AccessPeriod> roleAccesses = new HashMap<String, AccessPeriod>();
// boilerplate getters/setters/etc omitted
}
And AccessPeriod being
@Embeddable
public class AccessPeriod {
public Date startDate;
public Date endDate;
}
If you insert in the map an AccessPeriod with both startDate and endDate set to null and persist the TestEntity,
an entry is inserted in the auxiliary table, but when getting the TestEntity back, the map is empty.
You get the same problem with a mapping file.
This used to work in Hibernate 2.1--I found this out while porting some code to 3.3.
See the attached example. You'll need to include Hibernate and h2 (com.h2database:h2 1.0.20061217) on your classpath to run it with the given jdbc.properties, but feel free to change it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators....
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
13 years