Re: [hibernate-dev] [feature request][discuss] smoother serializers integration?

Well that is again exactly what a DTO is good for. If you as developer want the groups to be available, you add a list of groups to that special DTO type for that use case. In your data access layer you somehow populate that, which is normally done by using some mapper library like MapStruct or Dozer and then JAXB/JSONB can just work with the DTO type without any problems. Now if you forget to add a JOIN FETCH to your query and you end up with N+1 queries, that's a different problem, just like the amount of boilerplate code needed for having DTO types for every use case. That I try to solve with Blaze-Persistence Entity Views. Just a quick example to make my point here. If you have a REST endpoint /user/{id} and want to provide the list of group names along with the user information, you'd create a UserInfoDTO. @EntityView(User.class) interface UserInfoDTO { String getUsername(); @Mapping("groups.name") List<String> getGroups(); } Your repository returns an object of that type and you just pass that object through so JAXB/JSONB can do their work. The mapping information in the DTO is applied on a "source query" i.e. only doing the work absolutely necessary to satisfy the requested projection. Implementing this by hand is by no means impossible, but rather inconvenient I'd say, which is probably why you are seeking for other solutions. In the end, you can only try to create a minimal DTO that has exactly the fields you want to be serialized or annotate your existing entities with those "ignore" annotations and hope for the best. I don't see how hibernate could or should help in any of the two cases. Mit freundlichen Grüßen, ------------------------------------------------------------------------ *Christian Beikov* Am 04.05.2017 um 16:59 schrieb Romain Manni-Bucau: > Sure. If you add any conversion logic then you are clearly out of > hibernate scope and the problem doesnt appear anymore. Here is a > trivial example (hopefully trivial at least ;)) > > User 1 - n Group > > In json we would get something like {username:...,groups:[group1, > group2]}, no issue to know if group should be loaded or not since this > part of the logic is in the mapper layer. > > So yes you can say "not my problem" but next framework will > immediately ask "how do i know" and you likely end like all > spring-data-rest recommandation with a specific mapping and not a > framework solution which is the target of that thread - at least what > I tried to explain ;). > > 2017-05-04 16:41 GMT+02:00 Christian Beikov > <christian.beikov(a)gmail.com <mailto:christian.beikov@gmail.com>>: > > I don't understand what you mean by "you put that logic in the > conversion", could you elaborate? > > > Mit freundlichen Grüßen, > ------------------------------------------------------------------------ > *Christian Beikov* > Am 04.05.2017 um 16:32 schrieb Romain Manni-Bucau: >> Few more points: >> >> 1. Dto dont help at any moment - or you put that logic in the >> conversion and you are back to start >> 2. Making jaxb/jsonb easy to integrate is the goal IMO. No need >> to integrate with them but just provide some utility one level >> further than existing ones >> >> Le 4 mai 2017 16:13, "Steve Ebersole" <steve(a)hibernate.org >> <mailto:steve@hibernate.org>> a écrit : >> >> Oops, that (3) in previous reply should have read: >> 3. supporting each format creates a new "optional" library >> dependency >> >> Overall, I like Christian's approach as a potential >> generalized approach to >> this. Basically a combination of >> >> 1. a query used to provide the "view source values" >> 2. some indication of how to map those "source values" to >> your view model >> >> >> And again, I think 6.0's improved dynamic-instantiation >> queries are a >> simple, already-built-in way to achieve that for most cases. >> But I am open >> to discussing a way to supply that combination via API if we >> deem that >> would be good - although then I'd also question how the current >> TupleTransformer does not meet that need. >> >> On Thu, May 4, 2017 at 8:43 AM Steve Ebersole >> <steve(a)hibernate.org <mailto:steve@hibernate.org>> wrote: >> >> > Were there a standard "represent something in XML-ish >> format" contract >> > portable across a number of formats (XML, JAXB, JSON, etc) >> then I'd be more >> > inclined to agree with this. But as it is, supporting this >> would mean >> > Hibernate implementing multiple such contracts, one per >> format. However, >> > >> > 1. these formats are not our core competency >> > 2. maintaining a complete set of these transformers >> across all the >> > popular formats du-jour is a large undertaking >> > 3. I am not convinced that >> > >> > All of these increase the technical risk. >> > >> > Additionally, to properly support this we'd really need the >> ability to >> > then "map" multiple views for a given entity-graph-root. >> What I mean by >> > that, is that such DTO approaches often need multiple >> "views" of a given >> > entity, e.g. a CompanyListDTO, CompanyOverviewDTO, >> > CompanyDetailsGeneralDTO, etc for a Company entity. The >> point of this is >> > that >> > >> > 1. the transformers for these are specific to each DTO >> type and would >> > be applied per-transformation >> > 2. were Hibernate to "provide" this for applications >> > >> > IMO the use of queries to obtain views is logical. >> Populating each of >> > those specific DTOs (CompanyListDTO, etc) in the most >> efficient way is >> > going to require very different SQL for each DTO. This >> implies some kind >> > of "mapping" to be able associate each DTO with query. >> > >> > Given 6.0's improved dynamic-instantiation support, I even >> think that is a >> > great solution as well *for most cases*. >> > >> > So, while my objection has a "practical impact" component, >> I also just >> > question whether Hibernate integrating with each format's >> "serializer" is >> > the proper solution. >> > >> > >> > >> > On Thu, May 4, 2017 at 5:08 AM Christian Beikov < >> > christian.beikov(a)gmail.com >> <mailto:christian.beikov@gmail.com>> wrote: >> > >> >> This is exactly what I am trying to do with >> Blaze-Persistence Entity >> >> Views, making DTOs sexy and efficient :) >> >> >> >> Here a quick overview of how that looks like right now: >> >> >> >> >> https://persistence.blazebit.com/documentation/entity-view/manual/en_US/i... >> < https://persistence.blazebit.com/documentation/entity-view/manual/en_US/i... > >> >> >> >> One of my targets is to make it possible to do something >> like this >> >> >> >> entityManager.createQuery("FROM Order o", >> OrderDTO.class).getResultList() >> >> >> >> and get an optimal query, as well as objects with only the >> necessary >> >> contents. >> >> >> >> Maybe we can collaborate on that somehow? >> >> >> >> >> >> Mit freundlichen Grüßen, >> >> >> ------------------------------------------------------------------------ >> >> *Christian Beikov* >> >> Am 04.05.2017 um 10:20 schrieb Emmanuel Bernard: >> >> > Following up a bit on my previous email. >> >> > >> >> > While a core integration might be best I think, if there >> are too much >> >> > reluctance, we can start with a dedicated hibernate-dto >> or whatever >> >> > module or even separate project that makes life easier >> for these "pass >> >> > through" use cases. This could be in the form of a >> wrapper API of sort >> >> > and hence not affect existing Hibernate ORM APIs. >> >> > >> >> > Note that the ResultTransformer approach feels like it >> goes a long way >> >> > towards fixing the problem but as demonstrated in Vlad's >> article >> >> > >> >> >> https://vladmihalcea.com/2017/04/03/why-you-should-use-the-hibernate-resu... >> < https://vladmihalcea.com/2017/04/03/why-you-should-use-the-hibernate-resu... > >> >> > it still requires quite a bit of code and a special DTO >> constructor >> >> > object. That's what we need to get rid of I think. >> >> > >> >> > Emmanuel >> >> > >> >> > On Thu 17-05-04 10:04, Emmanuel Bernard wrote: >> >> >> I was very much in the Vlad, Steve, Christian camp >> until relatively >> >> >> recently. One of my main concern being that replacing a >> proxy by null >> >> >> was really sending the wrong message. So I was against >> having Hibernate >> >> >> ORM facilitate such a transformation. >> >> >> >> >> >> I am changing my mind because I am realizing that a lot >> of applications >> >> >> are less complex that my perceived median. A lot of >> apps really just >> >> >> want data to be fetched out and then passed to jackson >> (implicitly) and >> >> >> pushed out as a REST response in JSON or some other >> serialization >> >> >> protocol. >> >> >> >> >> >> So while we could try and keep the stance that such a >> solution should >> >> >> remain out of scope of Hibernate ORM core, we should >> have a very smooth >> >> >> integration with something like MapStruct to create >> such bounded DTO on >> >> >> the fly. Ideally with as close to zero code as possible >> from the user >> >> >> point of view. >> >> >> I can't really describe how that could look like >> because I am not >> >> >> familiar enough with MapStruct but I think it should >> have the following >> >> >> characteristics: >> >> >> >> >> >> 1. do an implicit binding between the mapped object >> graph and a >> >> detached >> >> >> object graph with a 1-1 mapping of type and >> replacing lazy objects >> >> and >> >> >> collections with null. That's the smoothest approach >> and the most >> >> >> common use case but also the one where an >> inexperienced person could >> >> >> shoot at someone else's foot >> >> >> 2. do a binding between the mapped object graph and a >> detached version >> >> of >> >> >> that object graph with a 1-1 mapping of type, but >> declaratively >> >> >> expressing the boundaries for the detached version. >> This enforces a >> >> >> clear thinking of the boundaries and will load lazy >> data in case the >> >> >> object graph loaded is missing a bit. I like the >> idea on principle >> >> but >> >> >> I think it overlaps a lot with the fetch graph. >> >> >> 3. offer a full integration between MapStruct and >> Hibernate ORM by >> >> >> letting people express a full fledge MapStruct >> transformation >> >> between >> >> >> the managed object graph and a different target >> structure >> >> >> >> >> >> I favored MapStruct over Dozer because we know the >> MapStruct lead >> >> quite well ;) >> >> >> >> >> >> Note however that the MapStruct approach requires an >> explicit object >> >> >> copy, it feels a bit sad to have to double memory >> consumption. But that >> >> >> might be a good enough approach and bypassing the >> managed object >> >> >> creation leads to questions around the Persistence >> Context contract >> >> >> where loading an object supposedly means it will be in >> the PC. >> >> >> Maybe a constructor like query syntax allowing to >> reference a MapStruct >> >> >> conversion logic might work? >> >> >> >> >> >> select mapStruct('order-and-items', o) from Order o >> left join >> >> fetch o.items >> >> >> >> >> >> Emmanuel >> >> >> >> >> >> >> >> >> On Wed 17-04-19 14:29, Vlad Mihalcea wrote: >> >> >>> Hi, >> >> >>> >> >> >>> Although I keep on seeing this request from time to >> time, I still >> >> think >> >> >>> it's more like a Code Smell. >> >> >>> Entities are useful for when you plan to modify them. >> Otherwise, a DTO >> >> >>> projection is much more efficient, and you don't >> suffer from >> >> >>> LazyInitializationException. >> >> >>> >> >> >>> With the ResultTransformer, you can even build graphs >> of entities, as >> >> >>> explained in this article; >> >> >>> >> >> >>> >> >> >> https://vladmihalcea.com/2017/04/03/why-you-should-use-the-hibernate-resu... >> < https://vladmihalcea.com/2017/04/03/why-you-should-use-the-hibernate-resu... > >> >> >>> >> >> >>> Due to how Hibernate Proxies are handled, without Bytecode >> >> Enhancement, >> >> >>> it's difficult to replace a Proxy with null after the >> Session is >> >> closed. If >> >> >>> we implemented this, we'd have to take into >> consideration both >> >> Javassist >> >> >>> and ByteBuddy as well as ByteCode Enhancements. >> >> >>> >> >> >>> all in all, the implementation effort might not >> justify the benefit, >> >> and >> >> >>> I'm skeptical of offering a feature that does not >> encourage data >> >> access >> >> >>> Best Practices. >> >> >>> >> >> >>> Vlad >> >> >>> >> >> >>> On Wed, Apr 19, 2017 at 2:18 PM, Christian Beikov < >> >> >>> christian.beikov(a)gmail.com >> <mailto:christian.beikov@gmail.com>> wrote: >> >> >>> >> >> >>>> Hey Romain, >> >> >>>> >> >> >>>> I don't think it is a good idea to expose entities >> directly if you >> >> >>>> really need a subset of the data. >> >> >>>> Reasons for that thinking are that it gets hard to >> define what needs >> >> to >> >> >>>> be fetched or is safe to be used for a particular use >> case. Obviously >> >> >>>> serialization is like a follow-up problem. >> >> >>>> I see 2 possible solutions to the problem and both >> boil down to the >> >> use >> >> >>>> of DTOs. >> >> >>>> >> >> >>>> 1. Use an object mapper(e.g. Dozer) that maps >> entity object graphs >> >> to >> >> >>>> custom DTO types. >> >> >>>> 2. Use specialized DTOs in queries. >> >> >>>> >> >> >>>> >> >> >>>> Implementing 1. does not help you with lazy loading >> issues and 2. >> >> might >> >> >>>> require very intrusive changes in queries which is >> why I implemented >> >> >>>> Blaze-Persistence Entity Views >> >> >>>> >> <https://github.com/beikov/blaze-persistence#entity-view-usage >> <https://github.com/beikov/blaze-persistence#entity-view-usage >>. >> >> >>>> This is a library that allows you to define DTOs with >> mappings to the >> >> >>>> entity. In a query you can define that you want >> results to be >> >> >>>> "materialized" as instances of the DTO type. >> >> >>>> This reduces the pain induced by properly separating the >> >> "presentation >> >> >>>> model" from the "persistence model" and at the same >> time will improve >> >> >>>> the performance by utilizing the mapping information. >> >> >>>> I don't want to advertise too much, just wanted to >> say that I had the >> >> >>>> same issues over and over which is why I started that >> project. >> >> >>>> >> >> >>>> Mit freundlichen Grüßen, >> >> >>>> >> >> >> ------------------------------------------------------------------------ >> >> >>>> *Christian Beikov* >> >> >>>> Am 19.04.2017 um 10:51 schrieb Romain Manni-Bucau: >> >> >>>>> Hi guys, >> >> >>>>> >> >> >>>>> Short sumarry: Wonder if hibernate could get a >> feature to kind of >> >> either >> >> >>>>> unproxy or freeze the entities once leaving the >> managed context to >> >> avoid >> >> >>>>> uncontrolled lazy loading on one side and >> serialization issues on >> >> another >> >> >>>>> side. >> >> >>>>> >> >> >>>>> Use case example: a common example is a REST service >> exposing >> >> directly >> >> >>>>> hibernate entities (which is more and more common >> with microservice >> >> >>>>> "movement"). >> >> >>>>> >> >> >>>>> Objective: the goal is to not need any step - or >> reduce them a lot - >> >> >>>>> between the hibernate interaction and a potential >> serialization to >> >> avoid >> >> >>>>> issues with lazy loading and unexpected loading. >> Today it requires >> >> some >> >> >>>>> custom and hibernate specific logic in the >> serializer which kind of >> >> >>>> breaks >> >> >>>>> the transversality of the two concerns >> (serialization and object >> >> >>>>> management/loading). >> >> >>>>> >> >> >>>>> >> >> >>>>> Implementation options I see: >> >> >>>>> >> >> >>>>> 1. a callback requesting if the lazy relationship >> should be fetched, >> >> >>>>> something like >> >> >>>>> >> >> >>>>> public interface GraphVisitor { >> >> >>>>> boolean shouldLoad(Object rootEntity, Property >> property); >> >> >>>>> } >> >> >>>>> >> >> >>>>> 2. An utility to remove any proxy potentially >> throwing an exception >> >> and >> >> >>>>> replacing the value by null or an empty collection, >> something like >> >> >>>>> >> >> >>>>> MyEntity e = Hibernate.deepUnproxy(entity); >> >> >>>>> >> >> >>>>> 3. A switch of the proxy implementation, this is >> close to 2 but >> >> wouldn't >> >> >>>>> require a call to any utility, just a configuration >> in the >> >> persistence >> >> >>>> unit. >> >> >>>>> Side note: of course all 3 options can be mixed to >> create a single >> >> >>>> solution >> >> >>>>> like having 3 implemented based on 1 for instance. >> >> >>>>> >> >> >>>>> Configuration proposal: this would be activated >> through a property >> >> in the >> >> >>>>> persistence unit (this shouldn't be only global IMHO >> cause >> >> otherwise you >> >> >>>>> can't mix 2 kind of units, like one for JSF and one >> for JAX-RS to be >> >> >>>>> concrete). This should also be activable as a query >> hint i think - >> >> but >> >> >>>> more >> >> >>>>> a nice to have. >> >> >>>>> >> >> >>>>> >> >> >>>>> What this feature wouldn't be responsible for: >> cycles. If >> >> relationships >> >> >>>> are >> >> >>>>> bidirectional then the unproxied entity would still >> "loop" if you >> >> browse >> >> >>>>> the object graph - this responsability would stay in >> the consumer >> >> since >> >> >>>> it >> >> >>>>> doesn't depend on hibernate directly but more on a >> plain object >> >> handling. >> >> >>>>> >> >> >>>>> What do you think? >> >> >>>>> >> >> >>>>> >> >> >>>>> Romain Manni-Bucau >> >> >>>>> @rmannibucau <https://twitter.com/rmannibucau >> <https://twitter.com/rmannibucau>> | Blog >> >> >>>>> <https://blog-rmannibucau.rhcloud.com >> <https://blog-rmannibucau.rhcloud.com>> | Old Blog >> >> >>>>> <http://rmannibucau.wordpress.com >> <http://rmannibucau.wordpress.com>> | Github < https://github.com/ >> >> >>>> rmannibucau> | >> >> >>>>> LinkedIn <https://www.linkedin.com/in/rmannibucau >> <https://www.linkedin.com/in/rmannibucau>> | JavaEE Factory >> >> >>>>> <https://javaeefactory-rmannibucau.rhcloud.com >> <https://javaeefactory-rmannibucau.rhcloud.com>> >> >> >>>>> _______________________________________________ >> >> >>>>> hibernate-dev mailing list >> >> >>>>> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> >>>>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> >>>> _______________________________________________ >> >> >>>> hibernate-dev mailing list >> >> >>>> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> >>>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> >>>> >> >> >>> _______________________________________________ >> >> >>> hibernate-dev mailing list >> >> >>> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> >> _______________________________________________ >> >> >> hibernate-dev mailing list >> >> >> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> > _______________________________________________ >> >> > hibernate-dev mailing list >> >> > hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> > https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> >> >> _______________________________________________ >> >> hibernate-dev mailing list >> >> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> >> >> > >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> <mailto:hibernate-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> <https://lists.jboss.org/mailman/listinfo/hibernate-dev> >> > > _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev