According to the JPA 2.0 API, invocations of lock() are only supposed to throw LockTimeoutException if only the lock statement is rolled back, rather than the entire transaction. (emphasis mine)

The PessimisticLockException is supposed to be thrown if the entire tx is rolled back.

The current implementation is working perfectly if you're not in a TX (See related issue,

The desired behavior:

• If in a TX and a query timeout is specified, return a LockTimeoutException and only rollback the lock() statement

Can be accomplished with PostgreSQL, but it takes multiple statements and the use of a Savepoint.

The code below demonstrates what doesn't work....

Map<String, Object> params = new HashMap<>();
params.put("javax.persistence.lock.timeout", 0);

List<Foo> foos = entityManager.createQuery("SELECT f FROM foo", Foo.class).getResultList();
Foo lockedFoo = null;
for (Foo foo : foos) {
    try {
         entityManager.lock(foo, LockModeType.PESSIMISTIC_WRITE, params);
        lockedFoo = foo;
        break;
    } catch (LockTimeoutException lte) {
        // This is what we want!
    } catch (PessimisticLockException ple) {
        // This is what we'd get if we didn't have a timeout set and we had an issue blocking for the lock.
        break;
    } catch (PersistenceException pe) {
        break;
    }
}

if (lockedFoo != null) {
   // do something with the locked foo.
}

When iterating attempting to grab a lock on the second item in the list (if the first item cannot be locked) you end up with:

ERROR: current transaction is aborted, commands ignored until end of transaction block

You can dance around this some creative use of createNativeQuery() and manually tweaking save points. It seems like since this goes against what the JPA2 javadocs describe, it would be more appropriate for Hibernate to manage the creation of and rollback to (if necessary) the savepoint if a SELECT FOR UPDATE NOWAIT cannot lock.