<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 9 October 2015 at 13:46, Stan Silvert <span dir="ltr">&lt;<a href="mailto:ssilvert@redhat.com" target="_blank">ssilvert@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF"><span class="">
    <div>On 10/9/2015 7:32 AM, Stian Thorgersen
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">#1 You&#39;re not going to catch all missing keys this
        way - as I said there&#39;s 2 types, custom defined as well which
        could be missing from default bundle</div>
    </blockquote></span>
    It catches it at load time.  As it loads each bundle, it checks
    against the previously loaded bundle.  That will indeed catch all
    missing keys in any bundle you try to test.<br>
    <br>
    I don&#39;t know exactly what you mean by &quot;custom defined&quot;.  Somehow a
    third-party bundle must be merged with our default bundle.  Unless I
    completely misunderstand, the code I wrote will still work.</div></blockquote><div><br></div><div>New keys can be defined by using keys in client descriptions, client names, etc, etc.. These won&#39;t be in our message bundle, but would be in the bundle in a customers theme. This is why message bundles inherit from the parent theme. Custom providers and themes can further introduce new keys.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class=""><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div>#2 Rendering the whole bundle useless just because you&#39;re
          missing one key is just daft</div>
      </div>
    </blockquote></span>
    It is the correct thing to do.  A missing key is like a null
    pointer.  It deserves a RuntimeException.</div></blockquote><div><br></div><div>No it&#39;s not the correct thing to do.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class=""><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div>#3 There will quite likely be separate teams that do
          translations to those that do development, which means stack
          traces and log output is not the solution</div>
      </div>
    </blockquote></span>
    I don&#39;t see what that has to do with anything.  You start with a set
    of bundles containing all the correct keys.  Then you translate each
    bundle.  If you accidentally delete a key then you want to know that
    right away.  But we should indeed ask the translation team what they
    want to see.</div></blockquote><div><br></div><div>Sure, go ahead and ask them if they want to look for RuntimeExceptions in the server log.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class=""><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div>#4 Doing a check each time you pull a message bundle to
          compare with the base bundle is probably not that expensive,
          but still pretty daft thing to do</div>
      </div>
    </blockquote></span>
    You only load each bundle once.  So the check only happens the first
    time you request the bundle.<span class=""><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div>#5 A proper util that&#39;s used to translate bundles is much
          better - we can implement a page in the admin console that
          allows you to validate a bundle and print out all missing
          bundles. This is something that would be more developer
          friendly and also would be usable by non-developers (aka
          people with other language skills than Java)</div>
      </div>
    </blockquote></span>
    We should ask the translation team what they want to see and how
    they do their work.  I&#39;m sure that they don&#39;t expect a tool to be
    built into the product.  None of our other products have that.</div></blockquote><div><br></div><div>I don&#39;t just care about our translation team, they will translate the built in keys, but they won&#39;t translate the ones introduced by users.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><div><div class="h5"><br>
    <blockquote type="cite">
      <div dir="ltr">
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On 9 October 2015 at 13:24, Stan
          Silvert <span dir="ltr">&lt;<a href="mailto:ssilvert@redhat.com" target="_blank">ssilvert@redhat.com</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF"><span>
                <div>On 10/9/2015 6:21 AM, Marko Strukelj wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">And we can always log the missing key
                    situation into server log - that should be enough
                    for developers to notice it, and fix it.</div>
                </blockquote>
              </span> This is basically what happens with the code I
              wrote for the fix:<br>
              <a href="https://github.com/keycloak/keycloak/pull/1690" target="_blank">https://github.com/keycloak/keycloak/pull/1690</a><br>
              <br>
              You get an error in the console and then a stack trace on
              the server.  The stack trace tells you exactly which key
              is missing.  But the console doesn&#39;t crash or anything
              like that.  You just switch back to your original language
              and everything works fine.
              <div>
                <div><br>
                  <blockquote type="cite">
                    <div class="gmail_extra"><br>
                      <div class="gmail_quote">On Fri, Oct 9, 2015 at
                        8:09 AM, Stian Thorgersen <span dir="ltr">&lt;<a href="mailto:sthorger@redhat.com" target="_blank">sthorger@redhat.com</a>&gt;</span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                          <div dir="ltr">There&#39;s two places where keys
                            can be missing:
                            <div><br>
                            </div>
                            <div>* In a translation - this can be an
                              honest mistake, or the translation wasn&#39;t
                              updated when KC was updated</div>
                            <div>* Custom keys added - for example when
                              keys are used for display names of
                              clients, roles, etc..</div>
                            <div><br>
                            </div>
                            <div>Manually having to go through all sorts
                              of pages to look for missing keys is very
                              error prone and time consuming, so will
                              not be the best option for developers. In
                              both cases above the correct way to do
                              this would be to have a way to verify a
                              message bundle. We need a tool that can
                              quickly identify if there are missing keys
                              and we could expose that through the admin
                              console. We currently have a student
                              looking at providing a UI for defining
                              locales and she is also going to look at
                              adding some way of identifying if a locale
                              is missing keys and also to easily list
                              only missing keys.</div>
                            <div><br>
                            </div>
                            <div>For end users as I&#39;ve said they will
                              have no clue what ???key??? is, and even
                              worse if we throw an exception/error just
                              because a missing key we&#39;ll actually break
                              the whole console just because of a
                              missing key. It&#39;s a much better option to
                              look for the key in another translation
                              and display that. Chances are they will be
                              able to interpret one or two English
                              words. Certainly higher chance of that
                              then them being able to interpret
                              ???key???.</div>
                            <div><br>
                            </div>
                          </div>
                          <div>
                            <div>
                              <div class="gmail_extra"><br>
                                <div class="gmail_quote">On 9 October
                                  2015 at 07:51, Thomas Raehalme <span dir="ltr">&lt;<a href="mailto:thomas.raehalme@aitiofinland.com" target="_blank">thomas.raehalme@aitiofinland.com</a>&gt;</span>
                                  wrote:<br>
                                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                    <p dir="ltr">How about returning
                                      something noticeable like
                                      ???key??? for example?</p>
                                    <div>
                                      <div>
                                        <div class="gmail_quote">On Oct
                                          9, 2015 8:10 AM, &quot;Stian
                                          Thorgersen&quot; &lt;<a href="mailto:sthorger@redhat.com" target="_blank">sthorger@redhat.com</a>&gt;

                                          wrote:<br type="attribution">
                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                            <div dir="ltr">That&#39;s not
                                              putting it to rest at all!
                                              Throwing a
                                              RuntimeException and
                                              rendering the whole admin
                                              console useless just
                                              because there&#39;s a missing
                                              key is a horrible idea.</div>
                                            <div class="gmail_extra"><br>
                                              <div class="gmail_quote">On
                                                8 October 2015 at 20:33,
                                                Stan Silvert <span dir="ltr">&lt;<a href="mailto:ssilvert@redhat.com" target="_blank">ssilvert@redhat.com</a>&gt;</span>
                                                wrote:<br>
                                                <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                  <div text="#000000" bgcolor="#FFFFFF">
                                                    <div>What if English
                                                      is the bundle that
                                                      has a missing key?<br>
                                                      <br>
                                                      Let&#39;s just put
                                                      this to rest and
                                                      solve it once and
                                                      for all.  The
                                                      simplest solution
                                                      I can think of is
                                                      to just compare
                                                      keys when a new
                                                      bundle is loaded. 
                                                      If any bundle has
                                                      a missing key or
                                                      it has key not
                                                      found in the
                                                      previous loaded
                                                      bundle, we throw a
                                                      RuntimeException. 
                                                      I can submit a
                                                      patch for that in
                                                      just a few
                                                      minutes.
                                                      <div>
                                                        <div><br>
                                                          <br>
                                                          On 10/8/2015
                                                          1:28 PM, Stian
                                                          Thorgersen
                                                          wrote:<br>
                                                        </div>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <blockquote type="cite">
                                                          <div dir="ltr">I&#39;m
                                                          not sure I&#39;m
                                                          buying into
                                                          the argument
                                                          that
                                                          displaying the
                                                          key is better
                                                          for
                                                          developers.
                                                          Having English
                                                          suddenly
                                                          pop-up in a
                                                          German
                                                          translation is
                                                          just as
                                                          obvious as a
                                                          key. Besides
                                                          as Stan points
                                                          out you catch
                                                          missing keys
                                                          by comparing
                                                          missing keys
                                                          between
                                                          English and
                                                          German.
                                                          <div><br>
                                                          </div>
                                                          <div>However,
                                                          if there is a
                                                          mistake in a
                                                          translation
                                                          then a user
                                                          may quite
                                                          likely be able
                                                          to interpret
                                                          English text,
                                                          while a user
                                                          will not be
                                                          able to
                                                          interpret a
                                                          key. So if a
                                                          key is missing
                                                          in a
                                                          translation
                                                          (which is
                                                          obviously a
                                                          &quot;bug&quot;) it&#39;s
                                                          better to
                                                          display
                                                          English than
                                                          to display the
                                                          key.</div>
                                                          </div>
                                                          <div class="gmail_extra"><br>
                                                          <div class="gmail_quote">On

                                                          8 October 2015
                                                          at 14:13, Stan
                                                          Silvert <span dir="ltr">&lt;<a href="mailto:ssilvert@redhat.com" target="_blank">ssilvert@redhat.com</a>&gt;</span>
                                                          wrote:<br>
                                                          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                                          <div text="#000000" bgcolor="#FFFFFF"><span>
                                                          <div>On
                                                          10/8/2015
                                                          12:48 AM,
                                                          Thomas
                                                          Raehalme
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote type="cite">
                                                          <p dir="ltr"><br>
                                                          On Oct 8, 2015
                                                          6:53 AM,
                                                          &quot;Stian
                                                          Thorgersen&quot;
                                                          &lt;<a href="mailto:sthorger@redhat.com" target="_blank">sthorger@redhat.com</a>&gt;



                                                          wrote:<br>
                                                          &gt;<br>
                                                          &gt; With
                                                          regards to
                                                          internationalization
                                                          I have two
                                                          questions:<br>
                                                          &gt;<br>
                                                          &gt; * Should
                                                          we fallback to
                                                          English
                                                          messages if a
                                                          key is missing
                                                          in a
                                                          translation?
                                                          Alternative is
                                                          to show key,
                                                          but that&#39;s not
                                                          going to help
                                                          anyone</p>
                                                          <p dir="ltr">A
                                                          missing key is
                                                          a bug and
                                                          showing the
                                                          message in the
                                                          default locale
                                                          may hide the
                                                          problem. </p>
                                                          <p dir="ltr">Even

                                                          though showing
                                                          the key does
                                                          not help the
                                                          end user it
                                                          helps the
                                                          developer and
                                                          identifies the
                                                          problem. For
                                                          this reason I
                                                          think showing
                                                          the key would
                                                          be a good
                                                          idea.</p>
                                                          </blockquote>
                                                          </span> For
                                                          our bundles,
                                                          we could catch
                                                          missing keys
                                                          at build
                                                          time.  <br>
                                                          <br>
                                                          Failing that,
                                                          I agree that
                                                          displaying the
                                                          key is better
                                                          than falling
                                                          back to
                                                          English.  This
                                                          is especially
                                                          true right now
                                                          while we
                                                          haven&#39;t
                                                          completed the
                                                          task of
                                                          converting
                                                          everything. 
                                                          If we fall
                                                          back to
                                                          English we
                                                          won&#39;t know if
                                                          the problem is
                                                          a missing key
                                                          or if the text
                                                          just hasn&#39;t
                                                          been converted
                                                          yet.<span><br>
                                                          <br>
                                                          <blockquote type="cite">
                                                          <p dir="ltr">&gt;

                                                          * Should we
                                                          change message
                                                          bundles to
                                                          UTF-8? Or
                                                          is ISO 8859-1
                                                          going to work
                                                          for all
                                                          languages?</p>
                                                          <p dir="ltr">Depends

                                                          what those all
                                                          languages are
                                                          :-)</p>
                                                          <p dir="ltr">I
                                                          think UTF-8 is
                                                          the best
                                                          choice as it
                                                          will handle
                                                          practically
                                                          any character.
                                                          </p>
                                                          <p dir="ltr">But

                                                          if you&#39;re
                                                          referring to
                                                          Java resource
                                                          bundles the
                                                          encoding for
                                                          .properties is
                                                          ISO-8859-1 but
                                                          there are
                                                          means to
                                                          handle any
                                                          UTF-8
                                                          character.</p>
                                                          </blockquote>
                                                          </span> Yes,
                                                          an UTF-8
                                                          character can
                                                          be encoded in
                                                          ISO-8859-1. 
                                                          Java provides
                                                          a native2ascii
                                                          tool for
                                                          converting
                                                          entire files. 
                                                          The resource
                                                          bundle tools
                                                          in most IDE&#39;s
                                                          do this for
                                                          you
                                                          automatically. 
                                                          So you just
                                                          edit as UTF-8
                                                          and it saves
                                                          the bundle as
                                                          ISO-8859-1.<br>
                                                          <br>
                                                          We can read
                                                          our bundles as
                                                          UTF-8 if we
                                                          want to do
                                                          that.  I&#39;d
                                                          rather not,
                                                          because I&#39;m
                                                          not sure what
                                                          we might run
                                                          into down the
                                                          road with Java
                                                          assuming
                                                          resource
                                                          bundles are
                                                          always
                                                          ISO-8859-1.<br>
                                                          <br>
                                                          But I&#39;d like
                                                          to get the
                                                          perspective of
                                                          people who
                                                          have handled
                                                          resource
                                                          bundles in
                                                          languages that
                                                          are not fully
                                                          supported by
                                                          ISO-8859-1. 
                                                          Is it too much
                                                          of a pain to
                                                          do a
                                                          conversion or
                                                          do the tools
                                                          make the
                                                          process
                                                          seamless?
                                                          <div>
                                                          <div><br>
                                                          <blockquote type="cite">
                                                          <p dir="ltr">Best

                                                          regards,<br>
                                                          Thomas<br>
                                                          </p>
                                                          <p dir="ltr">&gt;<br>
                                                          &gt; On 7
                                                          October 2015
                                                          at 18:42, Stan
                                                          Silvert &lt;<a href="mailto:ssilvert@redhat.com" target="_blank">ssilvert@redhat.com</a>&gt;


                                                          wrote:<br>
                                                          &gt;&gt;<br>
                                                          &gt;&gt; Marko
                                                          brought this
                                                          to my
                                                          attention
                                                          yesterday. 
                                                          For some
                                                          things, we<br>
                                                          &gt;&gt;
                                                          dynamically
                                                          create UI.  In
                                                          this case, the
                                                          java code
                                                          contains the
                                                          English<br>
                                                          &gt;&gt; text
                                                          and it needs
                                                          to be
                                                          localized. 
                                                          Luckily, the
                                                          solution was
                                                          pretty<br>
                                                          &gt;&gt;
                                                          straightforward. 
                                                          We just
                                                          replace the
                                                          English text
                                                          with a key
                                                          into the<br>
                                                          &gt;&gt;
                                                          message
                                                          bundle.  The
                                                          html template
                                                          that displays
                                                          this text
                                                          already pulls<br>
                                                          &gt;&gt; from
                                                          an Angular
                                                          scope so we
                                                          just leave
                                                          that alone and
                                                          pass it
                                                          through<br>
                                                          &gt;&gt; the
                                                          |translate
                                                          filter.  You
                                                          do need to
                                                          also add the
                                                          double-colon.<br>
                                                          &gt;&gt;<br>
                                                          &gt;&gt; One
                                                          nice side
                                                          effect is that
                                                          if the key is
                                                          not found in
                                                          the bundle
                                                          then<br>
                                                          &gt;&gt; the
                                                          output of the
                                                          translate
                                                          filter is the
                                                          unchanged
                                                          text.  This
                                                          means<br>
                                                          &gt;&gt; that
                                                          any code which
                                                          has not
                                                          converted to
                                                          using bundle
                                                          keys will
                                                          still<br>
                                                          &gt;&gt; work
                                                          as expected. 
                                                           And, any
                                                          third-party
                                                          providers can
                                                          just pass in<br>
                                                          &gt;&gt; plain
                                                          text if they
                                                          don&#39;t care
                                                          about l10n. 
                                                          If they ever
                                                          do care about<br>
                                                          &gt;&gt; l10n
                                                          we will just
                                                          need to
                                                          provide a
                                                          means for them
                                                          to add
                                                          key/value<br>
                                                          &gt;&gt; pairs
                                                          to the
                                                          resource
                                                          bundles.<br>
                                                          &gt;&gt;<br>
                                                          &gt;&gt; Here
                                                          is an example
                                                          for anyone who
                                                          needs to
                                                          localize
                                                          English text<br>
                                                          &gt;&gt;
                                                          embedded in
                                                          java:<br>
                                                          &gt;&gt; <a href="https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd8833c37be8549" target="_blank">https://github.com/ssilvert/keycloak/commit/c9437595b70810c4472325373dd8833c37be8549</a><br>
                                                          &gt;&gt;<br>
                                                          &gt;&gt; Stan<br>
                                                          &gt;&gt;
                                                          _______________________________________________<br>
                                                          &gt;&gt;
                                                          keycloak-dev
                                                          mailing list<br>
                                                          &gt;&gt; <a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a><br>
                                                          &gt;&gt; <a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br>
                                                          &gt;<br>
                                                          &gt;<br>
                                                          &gt;<br>
                                                          &gt;
                                                          _______________________________________________<br>
                                                          &gt;
                                                          keycloak-dev
                                                          mailing list<br>
                                                          &gt; <a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a><br>
                                                          &gt; <a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br>
                                                          </p>
                                                          </blockquote>
                                                          <br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br>
                                                          </div>
                                                        </blockquote>
                                                        <br>
                                                      </div>
                                                    </div>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br>
                                            </div>
                                          </blockquote>
                                        </div>
                                      </div>
                                    </div>
                                  </blockquote>
                                </div>
                                <br>
                              </div>
                            </div>
                          </div>
                          <br>
_______________________________________________<br>
                          keycloak-dev mailing list<br>
                          <a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a><br>
                          <a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                    <br>
                    <fieldset></fieldset>
                    <br>
                    <pre>_______________________________________________
keycloak-dev mailing list
<a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a>
<a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a></pre>
                  </blockquote>
                  <br>
                </div>
              </div>
            </div>
            <br>
            _______________________________________________<br>
            keycloak-dev mailing list<br>
            <a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a><br>
            <a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div></div>