Some info on Jandex

2012-07-11T08:37:10 <emmanuel> I need your help to better understand your favorite toy: jandex
2012-07-11T08:37:11 <hardy_> you are alive
2012-07-11T08:37:14 <emmanuel> do you ahve a few minutes?
2012-07-11T08:37:15 <hardy_> he he
2012-07-11T08:37:20 <hardy_> sure
2012-07-11T08:37:43 <hardy_> how come you are interested in this now?
2012-07-11T08:37:44 <hardy_> OGM?
2012-07-11T08:37:58 <emmanuel> yes
2012-07-11T08:38:17 <emmanuel> So are we forcing people to add the build time jandex task to their persistence archives?
2012-07-11T08:38:39 <hardy_> no
2012-07-11T08:39:06 <hardy_> we are building our own index
2012-07-11T08:39:28 <hardy_> based on the classes, packages etc they provide to the configuration
2012-07-11T08:40:01 <emmanuel> So you always build it at runtime from scratch
2012-07-11T08:40:15 <emmanuel> do you "cache" them between application laods?
2012-07-11T08:40:34 <hardy_> atm it gets build every time
2012-07-11T08:40:45 <hardy_> but my plan would be to cache the jandex index
2012-07-11T08:41:04 <hardy_> but that's secondary atm
2012-07-11T08:42:00 <hardy_> of course it would be nice in the long run to be able to share an index between ORM, Search, etc
2012-07-11T08:42:07 <hardy_> that would safe parsing time
2012-07-11T08:44:43 <emmanuel> ok so you're not using Jandex to be faster, but rather for the API
2012-07-11T08:44:44 <emmanuel> and you don't share the same index as the app server, which I thought was the idea to speed up load time
2012-07-11T08:45:35 <emmanuel> What did you like about Jandex parsing wise that you did not like in our current parser?
2012-07-11T08:45:44 <emmanuel> s/current/old/
2012-07-11T08:45:53 <emmanuel> I am poking you to get an idea
2012-07-11T08:45:58 <emmanuel> not judging here
2012-07-11T08:46:09 <hardy_> right, I am mainly after the api
2012-07-11T08:46:18 <hardy_> but in the long run also about sharing the index
2012-07-11T08:46:33 <hardy_> the api has a couple of advantages imo
2012-07-11T08:46:51 <hardy_> #1 there is a clear separation of annotation discovery and annotation processing now
2012-07-11T08:47:09 <hardy_> in our current code this is always mixed since we do both things at the same time
2012-07-11T08:47:34 <hardy_> jandex is giving you the annotation discovery and build what I call a Annotation Repository
2012-07-11T08:48:00 <hardy_> having such a repository allows to to process data in different ways
2012-07-11T08:48:37 <hardy_> think about global annotations
2012-07-11T08:48:44 <hardy_> take AnalyzerDef from Search
2012-07-11T08:48:58 <hardy_> they are application scoped and I don't care where they are defined
2012-07-11T08:49:06 <hardy_> I just want to know if and how many exist
2012-07-11T08:49:31 <hardy_> in the "old" way I have to still iterate the configured entities and for each entity do the check
2012-07-11T08:49:59 <hardy_> with jandex I say: jandex.getAnnotation(AnalyzerDef.class) and I get all the annotations
2012-07-11T08:50:16 <emmanuel> Jandex gives you an API to look for a given annotation directly?
2012-07-11T08:50:17 <hardy_> I can do this processing outside my entity processing
2012-07-11T08:50:20 <emmanuel> I have not seen that
2012-07-11T08:50:21 <hardy_> yes
2012-07-11T08:50:36 <hardy_> one sec
2012-07-11T08:51:00 <emmanuel> https://github.com/jbossas/jandex/blob/master/src/main/java/org/jboss/jandex/Index.java
2012-07-11T08:52:01 <hardy_> List<AnnotationInstance> getAnnotations(DotName annotationName)
2012-07-11T08:52:06 <hardy_> in Index
2012-07-11T08:52:27 <hardy_> you just provide the dot name of the annotation you are after and you get a list of them
2012-07-11T08:52:55 <hardy_> the alternative is to go via the ClassInfo
2012-07-11T08:53:43 <emmanuel> CAn you say, give me all the classes hosting annotation A?
2012-07-11T08:53:45 <hardy_> that is more the reflection type of processing (which you also need of course)
2012-07-11T08:54:01 <emmanuel> Or give me all the classes hosting annotations which are anntated with @Stereotype?
2012-07-11T08:54:39 <emmanuel> Alternatively can you go from AnnotationInstance to the reflection info releated to where tha nnotationInstance is hosted?
2012-07-11T08:54:55 <hardy_> that's what getAnnotations does
2012-07-11T08:55:08 <hardy_> the AnnotationInstance contains AnnotationTarget
2012-07-11T08:55:32 <hardy_> which gives you access to the ClassInfo, MethodInfo or FieldInfo
2012-07-11T08:55:47 <emmanuel> got it, I missed that link
2012-07-11T08:55:58 <hardy_> np
2012-07-11T08:56:05 <emmanuel> (btw, I looked for doc but ended up scanning the code, did I miss something?)
2012-07-11T08:56:19 <hardy_> nope, the code is probably the best place to look
2012-07-11T08:56:34 <hardy_> I don't think they have a project site or something like this
2012-07-11T08:56:41 <hardy_> there is the code and the issue tracker
2012-07-11T08:56:54 <emmanuel> ok
2012-07-11T08:57:47 <hardy_> basically the API allows you to write at times cleaner code
2012-07-11T08:57:52 <hardy_> and less convoluted
2012-07-11T08:58:09 <hardy_> just because you are having multiple ways to read/process the annotations
2012-07-11T08:58:16 <emmanuel> Do you use getKnownDirectSubclasses
2012-07-11T08:58:33 <emmanuel> (yes I see that it has a couple of powerful APIs)
2012-07-11T08:59:05 <hardy_> I think the issue with this method is that only subclasses which are annotated appear there
2012-07-11T08:59:46 <hardy_> which might be enough in most cases
2012-07-11T09:00:02 <hardy_> to be honest I am not sure whether it is used atm or not
2012-07-11T09:00:06 <emmanuel> oh really
2012-07-11T09:00:17 <emmanuel> so they filter out non annotated types?
2012-07-11T09:01:12 <hardy_> wrong, it does to have to be annotated
2012-07-11T09:01:34 <emmanuel> you mena right then
2012-07-11T09:01:41 <hardy_> but it has to be found during the scanning process
2012-07-11T09:01:44 <emmanuel> ah hold on
2012-07-11T09:01:47 <hardy_> read the jaadoc
2012-07-11T09:01:52 <emmanuel> right ok got you
2012-07-11T09:02:05 <emmanuel> which one
2012-07-11T09:02:08 <emmanuel>
2012-07-11T09:03:09 <emmanuel> So with XML or say a programmatic API, how do you do it?
2012-07-11T09:03:20 <emmanuel> Do you have some kind of merger impl?
2012-07-11T09:03:20 <hardy_> that's another nice thing imo
2012-07-11T09:03:31 <hardy_> I never liked the Class vs XClass thing
2012-07-11T09:03:49 <emmanuel> hardy_: I'll come to that later
2012-07-11T09:04:33 <hardy_> the beauty of Jandex is that once you have the index you don't deal with java.lang.annotation.Annotation anymore
2012-07-11T09:04:56 <hardy_> you are dealing with ClassInfo and AnnotationInstance
2012-07-11T09:05:05 <emmanuel> so do you build the Jandex model offf the XML or the programmatc API?
2012-07-11T09:05:10 <hardy_> so all you have to do is to create AnnotationInstnaces
2012-07-11T09:05:13 <hardy_> right
2012-07-11T09:05:33 <emmanuel> so you ahve sort of a fork of IndexReader?
2012-07-11T09:05:34 <hardy_> Strong wrote the xml processing for ORM
2012-07-11T09:06:15 <emmanuel> I guess that would be more Indexer
2012-07-11T09:06:50 <hardy_> I think we just create the AnnotationInstances and then create a new Index by creating a merged annotations map
2012-07-11T09:06:57 <hardy_> check the constructor of Index
2012-07-11T09:07:39 *** sannegrinovero has joined #hibernate-dev
2012-07-11T09:08:30 <hardy_> Index.create in fact
2012-07-11T09:09:01 <emmanuel> k
2012-07-11T09:09:36 <emmanuel> If you ahve a pointer to stliu's code I'll check it out
2012-07-11T09:09:52 <hardy_> it is on the metamodel branch
2012-07-11T09:10:07 <hardy_> the package org.hibernate.metamodel.internal.source.annotations.xml
2012-07-11T09:10:21 <emmanuel> Other question, do you keep the index in memory? Or do you discard it after the init phase?
2012-07-11T09:10:47 <hardy_> it is not needed anymore once everything is processes
2012-07-11T09:10:52 <hardy_> no need to keep it around
2012-07-11T09:11:00 <emmanuel> And have you measured the overhead of the Index compared to having the class model only which we need anyways for Hibernate?
2012-07-11T09:11:11 <emmanuel> OK, well Search would need it longer
2012-07-11T09:11:24 <hardy_> why?
2012-07-11T09:11:36 <emmanuel> because the factory is now mutable
2012-07-11T09:11:44 <emmanuel> you can add entities
2012-07-11T09:11:51 <emmanuel> that was required for Infinispan
2012-07-11T09:12:11 <emmanuel> Ask sannegrinovero it was not a walk in the park
2012-07-11T09:12:28 <hardy_> do I need to keep everything just for that
2012-07-11T09:12:59 <emmanuel> AFAIR yes
2012-07-11T09:13:14 <emmanuel> So my final question is about the API and code maintainability
2012-07-11T09:13:34 <emmanuel> Since you are decorrelated from Java's reflection API and the annotations as well
2012-07-11T09:13:35 <sannegrinovero> we do add new entities at runtime, but of course that doesn't happen very often. We could consider reloading the disk-stored index if need arises?
2012-07-11T09:14:02 <emmanuel> how do you quickly find the Jandex related reading calls corresponding to a specific annotation say @Columns
2012-07-11T09:14:06 <hardy_> adding an unrelated entity seems to be something which should be possible w/o processing everything again
2012-07-11T09:14:09 <hardy_> anyways
2012-07-11T09:14:43 <hardy_> sannegrinovero: right
2012-07-11T09:14:56 <emmanuel> hardy_: because my concern is that with these DotNames thingy, we back in String code's stone age
2012-07-11T09:15:25 <emmanuel> Likewise, when the annotation structure changes are you warned in any way?
2012-07-11T09:15:46 <hardy_> right, the dot name thing is one of the issues I had w/ jandex initially as well
2012-07-11T09:15:56 <hardy_> in ORM we created a constant file
2012-07-11T09:16:19 <hardy_> i think that works ok and it is actually a nice place to get an overview over all annotations
2012-07-11T09:16:20 <hardy_> one sec
2012-07-11T09:17:10 <hardy_> https://github.com/hibernate/hibernate-orm/blob/metamodel/hibernate-core/src/main/java/org/hibernate/metamodel/internal/source/annotations/util/JPADotNames.java
2012-07-11T09:17:34 <hardy_> there is one other thing which sucks
2012-07-11T09:17:46 <hardy_> there is no access to the default values of an annotation
2012-07-11T09:18:28 <emmanuel> ok so when it reads it, it returns null or the default primitive value instead of the default value?
2012-07-11T09:18:44 <hardy_> https://issues.jboss.org/browse/JANDEX-5
2012-07-11T09:18:45 <jbossbot> jira [JANDEX-5] Default values for annotation attributes ignored [Resolved (Won't Fix) Bug, Major, Jason Greene] https://issues.jboss.org/browse/JANDEX-5
2012-07-11T09:19:02 <hardy_> in ORM we worked around this with a JandexHelper
2012-07-11T09:19:45 <hardy_> https://github.com/hibernate/hibernate-orm/blob/metamodel/hibernate-core/src/main/java/org/hibernate/metamodel/internal/source/annotations/util/JandexHelper.java
2012-07-11T09:20:04 <hardy_> for me it is the only bitter pill in the API
2012-07-11T09:21:11 <hardy_> but as you can see in the issue it is just not possible to get to the information by just looking the class file. a shame really
2012-07-11T09:23:50 <emmanuel> ok
2012-07-11T09:24:37 <emmanuel> It seems your logic won't work for
2012-07-11T09:24:38 <emmanuel> @interface Pool { 2012-07-11T09:24:38 <emmanuel> int size() default 10; 2012-07-11T09:24:38 <emmanuel> }
2012-07-11T09:24:43 <emmanuel> yu will get 0out of it
2012-07-11T09:25:09 <hardy_> ?
2012-07-11T09:25:26 <emmanuel> At least Jandex should have a isdefault() on AnnotationValue or whatever the type they use
2012-07-11T09:25:39 <emmanuel> If I use @Pool()
2012-07-11T09:25:44 <emmanuel> the value is not stored
2012-07-11T09:25:50 <emmanuel> so jandex don't see it
2012-07-11T09:25:57 <emmanuel> but since they return primitives
2012-07-11T09:26:02 <emmanuel> they will return the default value ie 0
2012-07-11T09:26:07 <emmanuel> instead of the expected 10
2012-07-11T09:26:34 <hardy_> you get a AnnotationValue back
2012-07-11T09:26:48 <hardy_> and if there is no value specified you get null
2012-07-11T09:27:02 <hardy_> in which case you have to go the the annotation and get the default value
2012-07-11T09:27:29 <emmanuel> ok
2012-07-11T09:27:35 <hardy_> https://github.com/hibernate/hibernate-orm/blob/metamodel/hibernate-core/src/main/java/org/hibernate/metamodel/internal/source/annotations/util/JandexHelper.java#L91
2012-07-11T09:27:50 <hardy_> that's what the helper tries to hide from you
2012-07-11T09:28:03 <emmanuel> I've seen your code but Iwasn't expecting jandex to return a ann value null
2012-07-11T09:28:12 <emmanuel> when you expect a primitive int
2012-07-11T09:28:16 <emmanuel> but that makes sense
2012-07-11T09:28:22 <emmanuel> in a twisted way
2012-07-11T09:28:29 <emmanuel> Ah another question
2012-07-11T09:28:30 <hardy_> in a twisted way indeed
2012-07-11T09:28:38 <emmanuel> you get the Type
2012-07-11T09:28:43 <emmanuel> but no info about generics right?
2012-07-11T09:29:01 <emmanuel> so for generics you guys go back th Java's reflection
2012-07-11T09:29:06 <hardy_> getting back to annotations commons I guess
2012-07-11T09:29:11 <hardy_> yes
2012-07-11T09:29:13 <emmanuel> ok
2012-07-11T09:29:39 <emmanuel> So you've changed the code to be visitor based
2012-07-11T09:29:42 <hardy_> there are libraries which do the reflection bit
2012-07-11T09:29:49 <emmanuel> or do you still walk imperatively through the structure
2012-07-11T09:30:11 <emmanuel> Say you do getAnnotations(Entity.class) ~
2012-07-11T09:30:19 <emmanuel> but form there you walk heach property
2012-07-11T09:30:24 <hardy_> right
2012-07-11T09:30:25 <emmanuel> imperatively right?
2012-07-11T09:30:26 <emmanuel> ok cool
2012-07-11T09:30:33 <emmanuel> Thanks for all that info
2012-07-11T09:30:36 <hardy_> np
2012-07-11T09:30:43 <emmanuel> I'll record it in OGM-208 for posterity
2012-07-11T09:30:44 <jbossbot> jira OGM-208 Create facility for OGM core and Dialects to receive custom metadata (annotation, programmatic) associated to entities, properties or associations [Open (Unresolved) Bug, Critical, Unassigned] https://hibernate.onjira.com/browse/OGM-208
2012-07-11T09:31:01 <emmanuel> Jandex is bitter sweet to me
2012-07-11T09:31:12 <emmanuel> the untype-safety is really hurting my feelings
2012-07-11T09:31:15 <hardy_> lol
2012-07-11T09:31:26 <hardy_> it is not too bad
2012-07-11T09:32:34 <hardy_> but I agree it is a little bitter sweet. imo the sweetness is definitely winning though