<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 08/03/16 09:41, Stian Thorgersen
wrote:<br>
</div>
<blockquote
cite="mid:CAJgngAeCGQW+09Bc8kyd3qRkhqh7Enzn-U-zzp93qor+ejhKAA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>I actually think the chance of someone killing it during
upgrade is relatively high. It could be they forgot to include
bind address or used wrong server config. It could be
migration takes longer than they expect. We shouldn't require
users to manually unlock.<br>
</div>
<div><br>
</div>
<div>The lock should be done in association with the
transaction. JPA provides pessimistic locks so you can do:</div>
<div><br>
</div>
<div>
<div>DatabaseLockEntity lock =
em.find(DatabaseLockEntity.class, "lock",
LockModeType.PESSIMISTIC_WRITE);</div>
</div>
</div>
</blockquote>
Ok, I will take a look at this possibility and if reliably works
with all databases. However it will take some time though...<br>
<br>
Marek<br>
<blockquote
cite="mid:CAJgngAeCGQW+09Bc8kyd3qRkhqh7Enzn-U-zzp93qor+ejhKAA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><br>
</div>
<div>That will work for all databases (except Mongo of course).
If the process dies the transaction will timeout and it's safe
to run again at that point because no chances would have been
committed to the db.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 8 March 2016 at 09:22, Marek Posolda
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:mposolda@redhat.com" target="_blank">mposolda@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<div>On 08/03/16 06:48, Stian Thorgersen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">What about obtaining a database lock on
a table/column? That would automatically be freed if
the transaction dies.</div>
</blockquote>
</span> You mean something like "Give me lock for table XY
until end of transaction" ? I doubt there is some
universal solution for something like this, which will
reliably work with all databases which we need to support
:/ Otherwise I guess liquibase would already use it too?<br>
<br>
Currently it works the way that lock is obtained by
updating the column in database. Something similar to
"UPDATE DATABASECHANGELOGLOCK set LOCKED=true where ID=1"
.<br>
Note there is always single record in this table with
ID=1. Something similar is done for Mongo too.<br>
<br>
The lock is released in "finally" block if something
fails. The only possibility how can DB remains locked is
if someone force to kill the process (like "kill -9"
command, then finally blocks are not called) or if network
connection between server and DB is lost. The chance of
this is very low IMO and we have option to manually
recover from this.<span class="HOEnZb"><font
color="#888888"><br>
<br>
Marek</font></span>
<div>
<div class="h5"><br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>-1 To having a timeout, I agree it's
dangerous and could leave the DB inconsistent so
we shouldn't do it</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 7 March 2016 at 21:59,
Marek Posolda <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mposolda@redhat.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mposolda@redhat.com">mposolda@redhat.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Then the record in DB will remain
locked and needs to be fixed manually.
Actually the same behaviour like
liquibase. The possibilities to repair
from this state is:<br>
- Run keycloak with system property
"-Dkeycloak.dblock.forceUnlock=true" .
Then Keycloak will release the existing
lock at startup and acquire new lock. The
warning is written to server.log that this
property should be used carefully just to
repair DB<br>
- Manually delete lock record from
DATABASECHANGELOGLOCK table (or "dblock"
collection in mongo)<br>
<br>
The other possibility is that after
timeout, node2 will assume the current
lock is timed-out and will forcefully
release existing lock and replace with
it's own lock. However I didn't it this
way as it's potentially dangerous though -
there is some chance that 2 nodes run
migration or import at the same time and
DB will end in inconsistent state. Or is
it acceptable risk?<span><font
color="#888888"><br>
<br>
Marek</font></span>
<div>
<div><br>
<br>
<br>
On 07/03/16 19:50, Stian Thorgersen
wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote type="cite">
<div dir="ltr">900 seconds is probably
ok, but what happens if the node
holding the lock dies?</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 7 March
2016 at 11:03, Marek Posolda <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mposolda@redhat.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mposolda@redhat.com">mposolda@redhat.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">Send PR
with added support for $subject
.<br>
<a moz-do-not-send="true"
href="https://github.com/keycloak/keycloak/pull/2332"
rel="noreferrer"
target="_blank">https://github.com/keycloak/keycloak/pull/2332</a>
.<br>
<br>
Few details:<br>
- Added DBLockProvider, which
handles acquire and release of
DB lock.<br>
When lock is acquired, the
cluster node2 needs to wait
until node1<br>
release the lock<br>
<br>
- The lock is acquired at
startup for the migrating model
(both model<br>
specific and generic migration),
importing realms and adding
initial<br>
admin user. So this can be done
always just by one node at a
time.<br>
<br>
- The lock is implemented at DB
level, so it works even if
infinispan<br>
cluster is not correctly
configured. For the JPA, I've
added<br>
implementation, which is reusing
liquibase DB locking with the
bugfix,<br>
which prevented builtin
liquibase lock to work
correctly. I've added<br>
implementation for Mongo too.<br>
<br>
- Added DBLockTest, which
simulates 20 threads racing for
acquire lock<br>
concurrently. It's passing with
all databases.<br>
<br>
- Default timeout for acquire
lock is 900 seconds and the time
for lock<br>
recheck is 2 seconds. So if
node2 is not able to acquire
lock within 900<br>
seconds, it fails to start.
There is possibility to change
in<br>
keycloak-server.json. Is 900
seconds too much? I was thinking
about the<br>
case when there is some large
realm file importing at startup.<br>
<br>
Marek<br>
_______________________________________________<br>
keycloak-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:keycloak-dev@lists.jboss.org"
target="_blank">keycloak-dev@lists.jboss.org</a><br>
<a moz-do-not-send="true"
href="https://lists.jboss.org/mailman/listinfo/keycloak-dev"
rel="noreferrer"
target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>