| I'm getting an error executing batch inserts with a constraint violation exception, which to me makes no sense. I'm getting a list of users from a table and then looking for entries of those users in an auditing table. If not there, insert them. Because I'm dealing with a lot of users, I batch up the data to be inserted, creating a list of objects and then using *saveAll(list)* to batch them up. I'm using Spring Boot 2.1.4 and HikariCP 3.3.1. The primary key of the table getting the inserts is the ID field that is auto-generated. So how I could be getting duplicate entry errors is unclear, unless perhaps this is a timing issue. I'm batching 1000 objects/statements at a time. It works for at least 1 set of 1000 before it barfs. I'm assuming that the "managed flush" error is a side effect of the duplicate entry error. My entity is: @Data @Entity @Table(name = "event_state", schema = "subscription") public class EventState { @Id @GeneratedValue(generator = "generator") @GenericGenerator(name = "generator", strategy = "increment") private Long id; @Column(name = "customer_dimension_id") private Long customerDimensionId; @Column(name = "event_id") private Long eventId; @Column(name = "last_modified") @Temporal(TemporalType.TIMESTAMP) private Date lastModified; } Here is the application.properties file: spring.output.ansi.enabled=ALWAYS
- logging
logging.file=/tmp/activeusers.log
#HikariCP config subscription.datasource.url=jdbc:mysql://mysqlhost:3306/subscription subscription.datasource.username=USER subscription.datasource.password=PASS subscription.datasource.driverClassName=com.mysql.cj.jdbc.Driver subscription.datasource.hikari.connection-timeout=60000 subscription.datasource.hikari.maximum-pool-size=6 subscription.datasource.hikari.idle-timeout=300000 subscription.datasource.hikari.max-lifetime=1800000 subscription.datasource.hikari.minimum-idle=6 subscription.datasource.hikari.auto-commit=false subscription.datasource.hikari.validation-timeout=15000 subscription.datasource.cachePrepStmts=true subscription.datasource.prepStmtCacheSize=250 subscription.datasource.prepStmtCacheSqlLimit=2048 subscription.datasource.useServerPrepStmts=false subscription.datasource.useLocalSessionState=true subscription.datasource.rewriteBatchedStatements=true subscription.datasource.cacheResultSetMetadata=true subscription.datasource.cacheServerConfiguration=true subscription.datasource.elideSetAutoCommits=false subscription.datasource.maintainTimeStats=false subscription.datasource.hikari.pool-name=SubscriptionPool #HikariCP config activity.datasource.url=jdbc:mysql://mysqlhost:3306/activity activity.datasource.username=USER activity.datasource.password=PASS activity.datasource.driverClassName=com.mysql.cj.jdbc.Driver activity.datasource.hikari.connection-timeout=60000 activity.datasource.hikari.maximum-pool-size=6 activity.datasource.hikari.idle-timeout=300000 activity.datasource.hikari.max-lifetime=1800000 activity.datasource.hikari.minimum-idle=6 activity.datasource.hikari.auto-commit=true activity.datasource.hikari.validation-timeout=15000 activity.datasource.cachePrepStmts=true activity.datasource.prepStmtCacheSize=250 activity.datasource.prepStmtCacheSqlLimit=2048 activity.datasource.useServerPrepStmts=true activity.datasource.useLocalSessionState=true activity.datasource.rewriteBatchedStatements=true activity.datasource.cacheResultSetMetadata=true activity.datasource.cacheServerConfiguration=true activity.datasource.elideSetAutoCommits=true activity.datasource.maintainTimeStats=false activity.datasource.hikari.pool-name=ActivityPool spring.jpa.properties.hibernate.jdbc.batch_size=500 spring.jpa.properties.hibernate.order_inserts=true spring.jpa.properties.hibernate.order_updates=true I had batch_size set at 1000 originally, then changed it to 100. UPD 1: tried changing from using *saveAll* to *save* in a loop. Same problem. UPD 2: tried switching to the MariaDB driver, but had the same problem. I don't know if there is a config issue with HikariCP or a Hibernate issue or both. The problem is on the subscription DB, and only one thread does the *saveAll*. |