Hi, I think this email of Jason is worth forwarding to hibernate-dev. Jandex 2 sounds really interesting for all Hibernate projects. To a certain degree Jandex offers now a lot of the things Steve tried to do with the reflight stuff. I am not sure whether Jandex can completely replace it, but t can be for sure of help. Especially interesting to me is the fully resolved type information. In Search we have been talking for a long time to get rid of Commons Annotations. It is basically already possible now (using Jandex 1), but it got pushed back so far. Now with Jandex 2 we definitely should make the switch. Maybe Search 5.1? Last but not least Validator, there we have been talking about Jandex for a while as well. The new version would not have the benefit of being able to remove the dependency to classmate. I think we should have a look at this Beta and provide feedback to Jason if we find any problems. I am for sure to have a look at it in the context of Validator. --Hardy ----- Forwarded message from Jason Greene ----- I normally don’t spam you guys with Beta announcements, but since many of you are using Jandex, and this release has significant changes, in particular additional APIs, I wanted to make sure there was an opportunity for feedback before going final. For those unfamiliar with Jandex, it was originally created as an efficient bytecode based annotation indexer for AS7, but over time has evolved to also act as a reflection library that does not require class-loading nor complete linkage (you can analyze partial class graphs). For those interested in playing with runtime Java 8 Type annotations, which will likely end up in EE8 APIs, this release will likely be quite useful to you, since standard Java reflection does not currently expose this data. A brief guide for using Jandex is located here: https://github.com/wildfly/jandex Full notes are below. Thanks! Jandex 2.0 ————— - Binary and Source Compatibility All previous code *using* Jandex 1.1 should compile and run against 2.0. However, any code that extended “closed" contracts (e.g. AnnotationTarget) will require updating. - Offline reflective APIs and improvements All methods and fields, whether annotated or not are all accessible, and supporting APIs were created to enable access to the information. Generics information is provided in a familiar pattern similar to Java reflection but with usability improvements. Type information is fully resolved. Method and field access now supports binary searchable. Annotations are categorized and filtered at various levels. All need to cast has been eliminated from the framework. - Implemented offline index persistence The format is radically different to support the storage of generics, type annotations, and the aggressive interning used to reduce memory usage. Monster deployment (713MB of class data) only requires 26MB of storage. JDK, which has 128MB of class data is requires only 4.8 MB. Compressing the indexes by including them in a jar reduces the capacity even further to about 14MB and 2.6 MB respectively. Loading time is extremely efficient, at 1 second to load the 26MB index for the monster deployment. A portability layer is provided to convert the pervious offline index formats into the new internal model. - Full Java 8 type annotation support Java 8 type annotations all you to use annotations almost anywhere. For example, the following is now allowed: @A Map<@B Comparable<@F Object @C [] @D [] @E []>, @G List <@H D>> b; @H O1.@E O2<@F S, @G T>.@D O3.@A Nested<@B U, @C V> bar6; A new set of APIs were developed to represent all of this information in a useful way to the developer - Detailed class nesting information available CDI needed a way to determine the enclosing class information for all types of inner classes. This is all calculated and presented via a few easy methods - Memory optimization work The overhead required to store all offline reflection info + generics is significant. Implemented a variety of graph reduction strategies. End results of a monster deployment containing 713MB of class data (> 100k classes > 1 million methods) = 120MB of memory usage. The JDK has 128MB of class data, which Jandex uses only 20MB of heap to store. -- Jason T. Greene WildFly Lead / JBoss EAP Platform Architect JBoss, a division of Red Hat ----- End forwarded message ----- ----- End forwarded message ----- _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

Hi, I think this email of Jason is worth forwarding to hibernate-dev. Jandex 2 sounds really interesting for all Hibernate projects. To a certain degree Jandex offers now a lot of the things Steve tried to do with the reflight stuff. I am not sure whether Jandex can completely replace it, but t can be for sure of help. Especially interesting to me is the fully resolved type information. In Search we have been talking for a long time to get rid of Commons Annotations. It is basically already possible now (using Jandex 1), but it got pushed back so far. Now with Jandex 2 we definitely should make the switch. Maybe Search 5.1?

Last but not least Validator, there we have been talking about Jandex for a while as well. The new version would not have the benefit of being able to remove the dependency to classmate.

we find any problems. I am for sure to have a look at it in the context of Validator.

--Hardy ----- Forwarded message from Jason Greene ----- I normally don’t spam you guys with Beta announcements, but since many of you are using Jandex, and this release has significant changes, in particular additional APIs, I wanted to make sure there was an opportunity for feedback before going final. For those unfamiliar with Jandex, it was originally created as an efficient bytecode based annotation indexer for AS7, but over time has evolved to also act as a reflection library that does not require class-loading nor complete linkage (you can analyze partial class graphs). For those interested in playing with runtime Java 8 Type annotations, which will likely end up in EE8 APIs, this release will likely be quite useful to you, since standard Java reflection does not currently expose this data. A brief guide for using Jandex is located here: https://github.com/wildfly/jandex Full notes are below. Thanks! Jandex 2.0 ————— - Binary and Source Compatibility All previous code *using* Jandex 1.1 should compile and run against 2.0. However, any code that extended “closed" contracts (e.g. AnnotationTarget) will require updating. - Offline reflective APIs and improvements All methods and fields, whether annotated or not are all accessible, and supporting APIs were created to enable access to the information. Generics information is provided in a familiar pattern similar to Java reflection but with usability improvements. Type information is fully resolved. Method and field access now supports binary searchable. Annotations are categorized and filtered at various levels. All need to cast has been eliminated from the framework. - Implemented offline index persistence The format is radically different to support the storage of generics, type annotations, and the aggressive interning used to reduce memory usage. Monster deployment (713MB of class data) only requires 26MB of storage. JDK, which has 128MB of class data is requires only 4.8 MB. Compressing the indexes by including them in a jar reduces the capacity even further to about 14MB and 2.6 MB respectively. Loading time is extremely efficient, at 1 second to load the 26MB index for the monster deployment. A portability layer is provided to convert the pervious offline index formats into the new internal model. - Full Java 8 type annotation support Java 8 type annotations all you to use annotations almost anywhere. For example, the following is now allowed: @A Map<@B Comparable<@F Object @C [] @D [] @E []>, @G List <@H D>> b; @H O1.@E O2<@F S, @G T>.@D O3.@A Nested<@B U, @C V> bar6; A new set of APIs were developed to represent all of this information in a useful way to the developer - Detailed class nesting information available CDI needed a way to determine the enclosing class information for all types of inner classes. This is all calculated and presented via a few easy methods - Memory optimization work The overhead required to store all offline reflection info + generics is significant. Implemented a variety of graph reduction strategies. End results of a monster deployment containing 713MB of class data (> 100k classes > 1 million methods) = 120MB of memory usage. The JDK has 128MB of class data, which Jandex uses only 20MB of heap to store. -- Jason T. Greene WildFly Lead / JBoss EAP Platform Architect JBoss, a division of Red Hat ----- End forwarded message ----- ----- End forwarded message ----- _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

Hi, > > In Search we have been talking for a long time to get rid of Commons > > Annotations. It is basically already possible > > now (using Jandex 1), but it got pushed back so far. Now with Jandex 2 we > > definitely should make the switch. Maybe Search 5.1? > > > > Out of interest, what would be the benefit for Search? Getting rid of Commons Annotations. That is a long standing task. The annotation processing code could be simplified a fair bit in this case. The other advantage would be that in the case where Search is running in the container, we could just take an existing Jandex index at bootstrap (instead of building one ourselves). This would give a bit of boot speed-up. > > Last but not least Validator, there we have been talking about Jandex for > > a while as well. The new version would not have > > the benefit of being able to remove the dependency to classmate. > > I guess you mean it *would have* the benefit, right? sure :-) > Of course that comes at the price of adding another dependency, so that's > not really an advantage per se for me :) Not per se, but we already talked about consuming a Jandex index as part of the Bean Validation integration into Wildfly using CDI. In fact you seemed to be quite keen back then to give it a go. Now we could do it without increasing the amount of dependencies. > I think we should have a look at this Beta and provide feedback to Jason if > > we find any problems. I am for sure to have a look > > at it in the context of Validator. > > > > I find being able to re-use the index generated by the application > container very useful. It would give a nice performance boost out of the > box when using HV on WildFly/EAP. Exactly. > The type annotation stuff may be useful, but it's not that this information > cannot be accessed via reflection at all. There are known issues with > corner cases, though. So Jandex may improve the situation for us. My hope is that code would be simpler by using a single library for doing all reflection stuff including Java 8 type annotations, generic type resolution, etc > I'm asking to make sure we're on the same page regarding what we expect > from using Jandex in terms of user-visible features/improvements. Right, this is not about adding new features for the user. The user might get some speed-up and maybe less memory consumption, but otherwise we are talking about internal changes. --Hardy _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

I can't speak to Searchs or Validators use specifically, but Jandex is usable outside of WildFly. I know this through initially using Jandex on that metamodel work. Outside of WildFly, rather than being handed an index you would build an index. Pretty easy. See in-line wrt some of your specific requirements. On Tue, Dec 2, 2014 at 7:13 AM, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: > It looks awesome! but to be able to use it we also need to take these > requirements in consideration: > - users running in different containers than WildFly > See above. > - users not running in any container at all > See above. > - Hibernate Search integrators which are not based on ORM: running in > WildFly but not having the benefit of an EntityManager based bootstrap > where the container can "inject" a reference to the index; how do we > get the index from the application server there? > Actually this is "problem" for non-JPA ORM use in WildFly too. The passing of the Jandex index is part of the JPA bootstrap process. I do not know of a way for a component to ask for a Jandex index itself aside from some code in WildFly pushing that to you. - integrators running anywhere (WildFly but not only) which today > need to add new entity types "on the fly" at runtime (after the > bootstrap was completed). > I am not fully grokking what you are saying here. You mention it below too. I think you are saying something like what Envers does: after all ORM mappings are known, Envers wants to add additional mappings (its versioned entities). Of course Envers is a bit different in that the additional metadata it adds is not annotation based, but xml based. But "expanding" a Jandex index is pretty easy because they can be composed/aggregated. > I guess that's possible? > > We could also change one point dramatically: with Infinispan Query the > reason for it to need adding new types "on the fly" is just because > there is no reasonable solution for a full annotation scanning. But > having this we might be able to simplify some code in Search by > actually expecting to have a comprehensive list of types at bootstrap > through Jandex, then we can remove some of the code to do "on the fly" > type registration. > I'm saying that we could remove some of the code because with dynamic > types coming, we'll still want to make it possible to define new types > dynamically.. so we wouldn't be able to remove all complexity but it > still seems a good progress. > Actually, one of Jandex's huge strengths is exactly this use case. The ability to "scan" for annotations. For example, ORM couldask Jandex for all uses of the @Entity annotation as a means to discover all annotated entities. Maybe I am misinterpreting your "no reasonable solution for a full annotation scanning" comment. Perhaps you mean in the non-WildFly-handed index case? In that non-WildFly-handed index case there is a question as to how you know what files to add to the index. Ideally, Jandex would cover the entire available ClassLoader.

We could also change one point dramatically: with Infinispan Query the reason for it to need adding new types "on the fly" is just because there is no reasonable solution for a full annotation scanning. But having this we might be able to simplify some code in Search by actually expecting to have a comprehensive list of types at bootstrap through Jandex, then we can remove some of the code to do "on the fly" type registration.