[keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml
Christian Beikov
christian.beikov at gmail.com
Thu Nov 12 08:59:10 EST 2015
I just wanted to mention that you actually don't have to create the
EntityManagerFactory yourselves to do that. Hibernate Envers adds
entities to the persistence unit too. Maybe ask someone from the
hibernate team for details, but I think your entry point would be the
Integrator:
https://github.com/hibernate/hibernate-orm/blob/2964ecbf1b74003fcc337b0574487c724638fe94/hibernate-envers/src/main/java/org/hibernate/envers/boot/internal/EnversIntegrator.java
Mit freundlichen Grüßen,
------------------------------------------------------------------------
*Christian Beikov*
Am 12.11.2015 um 14:32 schrieb Stian Thorgersen:
>
>
> On 12 November 2015 at 13:58, Erik Mulder
> <erik.mulder at docdatapayments.com
> <mailto:erik.mulder at docdatapayments.com>> wrote:
>
> On 11/11/15 13:54, Stian Thorgersen wrote:
>> Would you be interested in contributing this feature? ATM we
>> don't have anyone available that could work on it. A contribution
>> would also need to include functional tests and documentation.
>
> Yes, I'd like to contribute this feature. I'm not sure about the
> timeline though. I hope to be able to do it as part of our current
> project, but I might have to use my spare time as well. Is there
> some kind of deadline to be included in a certain release version?
>
>
> We do a release every ~6 weeks. It's already a bit late for 1.7 (it's
> due end of the month) so would have to aim for 1.8 in either case
> (early January).
>
>
>
>> If so I'm happy with going down the route of using the Hibernate
>> specific classes. The remaining issue is figuring out how to deal
>> with classloading.
>>
>> Looks like the following should work:
>>
>> * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider
>
> I've done this and it works fine, successfully providing the extra
> classes to the EntityManagerFactory build process in
> DefaultJpaConnectionProviderFactory.
>
>> * JpaEntityProvider should have a single method "Class<?>
>> getEntities"
>
> Yes, only we need some kind of collection type, so you can provide
> multiple entity classes per provider. I guess you were intending
> this, considering the plural name 'getEntities'. I suggest either
> Collection<Class<?>> or Set<Class<?>> depending on what is most
> consistent with the rest of the system. Do you have a preference?
>
>
> Yup - List would be fine, that's what we tend to use as it's nicer to
> use than collection or set.
>
>
>
>> * Implement
>> org.hibernate.boot.registry.classloading.spi.ClassLoaderService -
>> looks like this can just return null for everything
>> except classForName where it would return the classes returned by
>> the JpaEntityProvider implementations
>
> I see no way to interfere in the creation of the
> ClassLoaderService. The official way is using the
> BootstrapServiceRegistryBuilder, but with the JPA /
> EntityManagerFactoryBuilderImpl route this happens 'out of reach'.
> I did find another way that works just as well: you can provide a
> 'custom' classloader to the
> Bootstrap.getEntityManagerFactoryBuilder. We can define a
> classloader that will return the extra JpaEntityProvider classes
> if requested. Only tricky part here is that Hibernate not only
> calls loadClass on a classloader, but before that also getResource
> to get a URL with an InputStream to the class bytes. It uses that
> to scan for annotations with Jandex. I fixed this by forwarding
> that request to the ClassLoader of the JpaEntityProvider provided
> class (through Class.getClassLoader()). This works fine and
> shouldn't be problem for any drop in jars. I can imagine though
> that if you use some exotic ProviderLoader(Factory), you might
> somehow get in trouble if the class byte[] is not available
> anymore after class loading. But this is a problem with the way
> Hibernate works, not with the way we extend Hibernate in this
> case. So I think it's fine to have a warning about this in the
> documentation, since it will probably never be a real problem. If
> you consider this as a no-go, please let me know.
>
>
> Sounds OK, but not sure what you mean about
> exotic ProviderLoader(Factory) is that a Hibernate thing?
>
>
>
> Last question I have considers the Hibernate version of KeyCloak.
> Currently it's 4.3.10, are there any plans to upgrade to 5? The
> code related to classloading etc is refactored considerably in
> Hibernate 5. So it would be a shame to fully get it working for
> 4.3.10, only to have to upgrade soon after that. I didn't look
> into the details of Hibernate 5 and I think the solution we came
> up with should remain more or less intact, but you never know, so
> that's why I ask.
>
>
> We are soon moving to WildFly 10 which includes Hibernate 5, but we
> still need to support EAP 6.4 which includes Hibernate 4. At some
> point next year we will drop support for EAP 6.4 and move on to EAP 7.
>
> We either have to support both Hibernate 4 and 5 for a while, or we
> make it use the old approach on Hibernate 4 (so now custom entity
> class support on EAP 6.4) and the new approach on Hibernate 5. That
> would probably require some magic reflection code though.
>
>
>
>
>> On 7 November 2015 at 23:39, Erik Mulder
>> <erik.mulder at docdatapayments.com
>> <mailto:erik.mulder at docdatapayments.com>> wrote:
>>
>> /On 06/11/15 14:46, Stian Thorgersen wrote://
>> //> We could use Hibernate directly to boostrap as long as it
>> can return an EntityManager. Do you know if that's possible?//
>> /
>> I was a little quick to state that with Hibernate you can add
>> extra entity class names besides the one in persistence.xml,
>> since I spotted a few answers on StackOverflow that said it
>> could be done. But they resolve around classpath scanning or
>> using a Spring managed Hibernate. Then I thought: 'if Spring
>> can do it, I can do it too' so I investigated the Hibernate
>> source code
>> 'behind'Persistence.createEntityManagerFactory(unitName,
>> properties). After some digging it turns out it's pretty
>> simple to get extra class names in the configuration. See
>> code sample below.
>>
>> The only problem is that Hibernate will only find classes
>> that are part of the 'main' KeyCloak application, because of
>> the way the Wildfly module system and ClassLoader strategy
>> work. The debugger showed me Hibernate has these 3 class
>> loaders available to look for classes:
>> 1. ModuleClassLoader for Module
>> "deployment.keycloak-server.war:main" from Service Module Loader
>> 2. ModuleClassLoader for Module "org.hibernate:main" from
>> local module loader
>> 3. sun.misc.Launcher$AppClassLoader
>>
>> Number 1 has all other KeyCloak modules in it, so the entity
>> classes from model-jpa will be found, but the
>> wildfly-extensions module is missing, so entities in classes
>> in a jar in the providers folder cannot be found. Now you
>> guys obviously know a lot more about these internals, but as
>> currently configured, it seems to me there is no way to let
>> Hibernate 'see' these extra classes, since only the KeyCloak
>> services module has a dependency on wildfly-extensions.
>>
>> So I think it boils down to these decisions:
>> A. Do you accept a non-pure-JPA way of building the
>> EntityManagerFactory that has some ties to the Hibernate
>> internals?
>> B. If A is no, than we're done. If yes, then you must find
>> some way to get the extra configured classes 'into'
>> Hibernate. You could get the wildfly-extensions module into
>> scope of the Hibernate classloading. There are serveral ways
>> to configure Hibernate classloading or you could flip some
>> switches / dependencies in the module configuration. Another
>> alternative is to create a separate 'dropfolder' besides
>> themes and providers for JPA extensions, like 'models' or so
>> and have that one be on the Hibernate classpath. But I don't
>> know the exact design principles behind KeyCloak or the
>> Wildfly module system. So maybe you have a better solution or
>> maybe you conclude that this is 'not done' in terms of the
>> architecture.
>>
>> Either way, I'd really appreciate some feedback on this and
>> some thoughts on whether this could be a possible addition to
>> KeyCloak in your eyes.
>>
>> Thanks, Erik
>>
>>
>> Current JPA way. No way to 'interfere':
>> emf = Persistence.createEntityManagerFactory(unitName,
>> properties);
>>
>> Alternative Hibernate only way with adding extra entity class
>> names:
>> // Let Hibernate find and parse all 'persistence.xml' files
>> found on the classpath.
>> List<ParsedPersistenceXmlDescriptor> persistenceUnits =
>> PersistenceXmlParser.locatePersistenceUnits(properties);
>> // Assume there is only one persistence unit found and that
>> is the one we need. This can be made more robust by checking
>> on the persistence unit name.
>> ParsedPersistenceXmlDescriptor persistenceUnitDescriptor =
>> persistenceUnits.get(0);
>> // Add extra class names. These could come from a 'JPA class
>> name provider' SPI or something alike.
>> persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity","org.keycloak.models.jpa.entities.MerchantEntity");
>> // Let Hibernate create an EntityManagerFactory out of the
>> (enriched) persistence unit configuration.
>> emf =
>> Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor,
>> properties).build();
>>
>>
>>
>>
>> On 6 November 2015 at 14:29, Erik Mulder
>> <erik.mulder at docdatapayments.com
>> <mailto:erik.mulder at docdatapayments.com>> wrote:
>>
>> Thanks for pointing me explicitly to the SPI
>> documentation. Of course that is exactly what I was
>> looking for in my original question. I don't know how I
>> overlooked this earlier! Probably I was not picking it
>> up, because of almost a decade of developing on Spring
>> projects, where this type of thing works differently. :-)
>>
>> I tried a quick test with a jar with one extra
>> ProtocolMapper configured, put it in the providers folder
>> and it worked like a charm!
>>
>> As for the JPA: We'll probably go with your suggestion of
>> the separate EntityManagerFactory. Indeed there seems to
>> be no way to 'programmatically extend' the list of entity
>> classes in JPA besides editing or overwriting the
>> persistence.xml. As you probably know it can be done in
>> Hibernate, but I guess KeyCloak wants to stick to a
>> generic JPA solution. That said, we might consider the
>> Hibernate specific solution for our case, since being
>> able to switch the JPA provider is not a requirement for
>> us. And keeping the same connection/transaction is a lot
>> easier in reasoning and debugging.
>>
>>
>> We could use Hibernate directly to boostrap as long as it can
>> return an EntityManager. Do you know if that's possible?
>>
>>
>>
>>
>> On 05/11/15 10:52, Stian Thorgersen wrote:
>>> The way to extend Keycloak is by implementing your own
>>> custom providers of the many SPIs we provide. Some SPIs
>>> are more stable (so marked as public) and others are not
>>> (so marked as private). If there are things that you
>>> want to customize that can't be done with an existing
>>> SPI then let us know and we may consider adding
>>> additional SPIs.
>>>
>>> On 4 November 2015 at 17:16, Erik Mulder
>>> <erik.mulder at docdatapayments.com
>>> <mailto:erik.mulder at docdatapayments.com>> wrote:
>>>
>>> Thanks for your response!
>>>
>>> Indeed we already did a proof of concept where we
>>> added a custom mapper
>>> the way you described (didn't know it was
>>> 'protected' territory :). The
>>> question is: do we have to override the file
>>> 'org.keycloak.protocol.ProtocolMapper' for this and
>>> add the new mapper
>>> in the original project or is there another way
>>> where we don't need to
>>> touch the original sources and keep all our changes
>>> in a separate
>>> project? And how can we do it such that it stays
>>> easy to upgrade to
>>> newer KeyCloak releases?
>>>
>>>
>>> Each jar has it's own
>>> org.keycloak.protocol.ProtocolMapper. Take a look at the
>>> docs
>>> (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html)
>>> and examples for other provider
>>> (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory)
>>>
>>>
>>> As for JPA: it would be easier to integrate with the
>>> existing JPA
>>> project. Again we are wondering whether to start
>>> modifying original
>>> sources (like persistence.xml) or try to
>>> 'externalize' our changes
>>> somehow and integrate them using existing 'hooks' in
>>> the system or maybe
>>> merge projects during build.
>>>
>>> Maybe there is no good answer to this and we'll
>>> always be having some
>>> manual merge pains when upgrading to new KeyCloak
>>> versions. We just
>>> wanted to check if there are preferred ways to add
>>> functionality with
>>> the least amount of impact on the original sources.
>>>
>>>
>>> I initially wanted the ability to add custom entities to
>>> the JpaConnectProvider, but couldn't find a way to
>>> define entities programatically with JPA. To add your
>>> own persistence.xml you would have to define your own
>>> implementation of JpaConnectionProvider and change what
>>> is loaded by default (connectionsJpa/provider attribute
>>> in keycloak-server.json).
>>>
>>> Alternative, which is cleaner, but you end up with
>>> separate connection/transaction, is to create your own
>>> EntityManagerFactory. If it's only used by one provider
>>> (for example a custom UserFederationProvider) there's no
>>> need to add a connect provider (that's just a way to
>>> share one EntityManagerFactory between multiple
>>> providers) and you can just create it in the
>>> MyUserFederationProviderFactory.
>>>
>>>
>>>
>>> On 04/11/15 15:30, Bill Burke wrote:
>>> > Custom mappers should be possible. I didn't
>>> document it as I wasn't
>>> > sure if we wanted to make the SPI public. Custom
>>> mappers should just
>>> > follow the Provider SPI and they will be picked
>>> up. If you see the
>>> > META-INF/services/... file in the resources
>>> directory of the "services"
>>> > or "broker" modules you'll see how to set this up.
>>> >
>>> > As for extending the JPA datamodel, what you could
>>> do is write a new JPA
>>> > Connections Provider and plug that in. See
>>> connections/jpa. I'm not
>>> > sure how you would handle the liquibase db migration.
>>> >
>>> > On 11/4/2015 6:03 AM, Erik Mulder wrote:
>>> >> Hi everybody,
>>> >>
>>> >> Quick intro: I’m part of a development team in
>>> The Netherlands that is
>>> >> building a company-wide SSO solution. We’ve
>>> chosen KeyCloak to realize
>>> >> this and will use OpenID Connect to secure our
>>> REST services. It’s a
>>> >> great product and seems to be the only one having
>>> both support for all
>>> >> kinds of security standards and a model and GUI
>>> for users and roles.
>>> >> Thanks for creating it! J
>>> >>
>>> >> (if this should be asked instead on the users
>>> mailing list, please
>>> >> correct me and I’ll post it there)
>>> >>
>>> >> So far, so good, but we have some extra
>>> requirements that do not fit
>>> >> into the base KeyCloak data model. See below for
>>> details if you’re
>>> >> interested. My question is: what is the preferred
>>> way / best practice to
>>> >> extend the functionality of KeyCloak while
>>> keeping the impact on the
>>> >> original sources to a minimum? Of course we could
>>> just fork the most
>>> >> recent version and start hacking away, but we’d
>>> like to be able to
>>> >> upgrade to newer versions of KeyCloak without too
>>> much hassle.
>>> >> Possibilities that we’ve come up with so far:
>>> >>
>>> >> 1.Create completely separate modules that will
>>> extend the functionality
>>> >> the way we need.
>>> >>
>>> >> 2.Fork on Github, apply custom changes, and try
>>> to merge in updates from
>>> >> the master / release branches / tags
>>> >>
>>> >> 3.Apply custom changes on KeyCloak artifacts
>>> using a Maven plugin, such
>>> >> as Truezip
>>> >>
>>> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html)
>>> -
>>> >> manipulate zip files by adding/removing/replacing
>>> or Shade
>>> >>
>>> (http://maven.apache.org/plugins/maven-shade-plugin/) -
>>> combine multiple
>>> >> jars to 1 'uber-jar' containing the contents of
>>> both and when
>>> >> overlapping decide on conflicts through
>>> configuration.
>>> >>
>>> >> Of course number 1 is preferred, but I do not see
>>> how to add custom
>>> >> mappers or JPA entities without making changes in
>>> the original module
>>> >> files. The other options seem like valid
>>> alternatives, but maybe there
>>> >> is better / standard way to do this. So any help
>>> / insight / shared
>>> >> experience on this is much appreciated, thanks!
>>> >>
>>> >> Kind regards,
>>> >>
>>> >> Erik Mulder
>>> >>
>>> >> Senior Software Engineer
>>> >>
>>> >> Docdata Payments – NL
>>> >>
>>> >> P.S. Details on why we want to extend the
>>> KeyCloak data model: (any
>>> >> feedback on the contents of this P.S. is also
>>> welcome!)
>>> >>
>>> >> Our clients are merchants that have several
>>> webshops. We manage their
>>> >> online payments (shopping cart checkout). We want
>>> to be able to let a
>>> >> merchant manage their own users and let a user
>>> have different roles for
>>> >> different webshops within the same merchant. The
>>> overall possible roles
>>> >> are fixed though, no specific roles per merchant.
>>> We could create a
>>> >> separate realm for every merchant, but then we
>>> need to duplicate all
>>> >> roles every time. Furthermore, in KeyCloak there
>>> is no concept of a role
>>> >> within a certain context. This is very
>>> understandable, since every
>>> >> situation has it’s own requirements. We did a
>>> proof of concept by adding
>>> >> tables and entities for Merchant, UserMerchant,
>>> UserMerchantRole etc.
>>> >> and adding a custom mapper that can put this
>>> information on the Access
>>> >> token. Worked like a charm! But it does need some
>>> changes in the
>>> >> KeyCloak modules and sources to work, hence the
>>> question above.
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> keycloak-dev mailing list
>>> >> keycloak-dev at lists.jboss.org
>>> <mailto:keycloak-dev at lists.jboss.org>
>>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >>
>>>
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> <mailto:keycloak-dev at lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>>
>>
>>
>>
>>
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/0cafa0f4/attachment-0001.html
More information about the keycloak-dev
mailing list