[hibernate-dev] [OGM] Keeping the order of the collection of elements in MongoDB

[Yoann Rodiere]

>
> I'm a bit lost here. If you don't want to expose an "unnecessary
> promise" like ordering, why would you need OGM to maintain ordering?


As explained below:

> You could add documentation on a getter stating what the expected ordering
> is (and, frankly, it may be relevant even with a List: how would you
> translate "the invoices are ordered by issuing date" in terms of a Java
> interface?). But what you cannot (or at least, shouldn't) do is
> contradicting the interfaces you expose ("yeah, I'm exposing a List because
> elements are ordered explicitly since we only ever add one at the end, but
> please be nice and don't try random access, it's a very long linked list").
>

In my example, ordering *was* necessary (since we state that invoices are
ordered by issuing date). What wasn't necessary was all the other contracts
exposed by List ("you can do random access", "you can manage ordering
explicitly", etc.), which is why someone may not want to use a List.
In the real world I guess that people who find such thing annoying will
have several layers of abstraction on top of their entities anyway, but
still.

ah, maybe I understand now. If you're proposing we do a similar thing
> for other specific collection types then sure we can discuss that. I
> had the impression you wanted to store ordering for properties
> literally using the *Collection* signature.


Well, I *also* wanted to store ordering for those, as long as "storing the
ordering" only means putting the elements in the correct order in the
resulting document.
But I'll be happy if we do it only for the sake of custom collection types,
since I suspect it means we will also do it for the default types :)

By the way, a property can have a Collection signature and *also* have a
custom type, it's not mutually exclusive. Actually, there's even an example
in Hibernate ORM tests (and documentation, apparently):

//tag::collections-custom-collection-mapping-example[]
@Entity(name = "Person")
public static class Person {

@Id
private Long id;

@OneToMany(cascade = CascadeType.ALL)
@CollectionType( type = "org.hibernate.userguide.
collections.type.QueueType")
private Collection<Phone> phones = new LinkedList<>();

//Getters and setters are omitted for brevity

//end::collections-custom-collection-mapping-example[]

public Person() {
}

public Person(Long id) {
this.id = id;
}

public Queue<Phone> getPhones() {
return (Queue<Phone>) phones;
}
//tag::collections-custom-collection-mapping-example[]
}

Keeping such ordering might come at a non-trivial cost; sometimes just
> computation wise, sometimes on query complexity and/or space when the
> order column needs to be stored. It just happens to be cheap and -
> apparently - "natural" on MongoDB.


Sure, we should only do it where it's possible efficiently.

BTW I'm not even 100% convinced about my previous suggestion about
> supporting this, as maybe it would set a bad precedent?
> The risk is that after developing on OGM/MongoDB you start expecting
> this to work fine on other systems, so from this perspective our API
> would not have a clear contract on the expectations one should have.

True... On the other hand, not implementing something that *is* efficient
for MongoDB would be annoying for users, too. And more rightly so.
Also, it would require a fair bit of bad faith from users to blame us for
not implementing an unspecified behavior... even if other dialects do
implement it.

If we really want to have a clear contract, it means we have to explicitly
state somewhere that ordering is undefined unless one is using @OrderColumn
/ @OrderBy. If we really want to do this, we may as well add "unless you're
using a document database" or something like this?

Yoann Rodière <yoann at hibernate.org>
Hibernate NoORM Team

On 13 January 2017 at 16:54, Sanne Grinovero <sanne at hibernate.org> wrote:

> On 13 January 2017 at 15:30, Yoann Rodiere <yrodiere at redhat.com> wrote:
> > Hi Sanne,
> >
> >>
> >> while you might be using a specific implementation when persisting
> >> your new entities, when you're loading the object back from the
> >> database how is Hibernate supposed to know
> >>  A) which implementation you want it to use
> >
> > Ok, that's true for most collection interfaces, but I'm not sure I
> > understand why what Hibernate can guess is relevant here. Users are free
> to
> > use custom collection types, at least in ORM. Is it not the case with
> OGM?
> > Will it never be possible?
>
> Sure, OGM should (eventually?) be able to do the same as ORM.
>
> But in this case we're not talking about the case in which you use a
> custom collection; my question is if you map a "Collection" with
> nothing else than a - for example - @OneToMany annotation, when you
> load it Hibernate core (OGM and/or ORM) will not use the same
> collection implementation which you used when you stored the object.
>
> >
> > I thought we were talking about making the collection persister
> > persist/retrieve the elements in the same order as they were retrieved
> from
> > the collection. If it's not the case, then yes, I'm missing something.
> >
> > However, even in the specific case of Collection, by default Hibernate
> > instantiates a List behind the scene for such properties (well, a
> > PersistentBag, but it's basically a List). Thus if the original
> collection
> > had some kind of "ordering memory", at least this would be preserved,
> > provided we persist elements in the same order we retrieved them from the
> > original collection.
> >
> >>  B) which order to use, if you didn't map e.g. an order column to
> >> store the ordering information (MongoDB not needing a strategy is an
> >> exception)
> >
> >
> > Well, we don't want to frustrate OGM users with limitations from the
> > relational world, do we? Wouldn't other document datastores also allow to
> > persist the elements' order?
>
> But why wouldn't you use a List if you need a list?
>
> >
> >> A List expresses the ordering requirement by contract; if you map a
> >> relation as "Collection" I wage you're not interested in the order,
> >> but you're rather more interested in accepting a wide range of
> >> implementations, some of which might ignore the ordering requirement.
> >> By doing so, you're explicitly telling both ORM and your other code
> >> interacting with the model that ignoring the order is acceptable.
> >> BTW I suspect mapping things as a generic Collection is a very unusual
> >> (or lazy) choice; I don't remember ever using it as it's quite
> >> ambiguous, and there's no way any specification diagram will lead you
> >> to recommend using a Collection (other than cleanly and explicitly
> >> wanting to express you don't care about ordering, nor other properties
> >> like uniqueness...)
> >
> >
> > By exposing a Collection, you're not telling the order isn't important,
> and
> > certainly not that it can change for no reason (which is our concern
> here).
> > In fact, you're not telling anything about the order.
> > By exposing a List, you're telling more than just "the order is
> important".
> > What you're telling is "the order can be managed explicitly by an
> external
> > actor". Also, you're telling "random access is possible" (not necessarily
> > efficient, but possible).
> >
> > So while I probably wouldn't bother myself, I can see why someone would
> > choose Collection over List: to avoid unnecessarily exposing promises to
> > users.
>
> I'm a bit lost here. If you don't want to expose an "unnecessary
> promise" like ordering, why would you need OGM to maintain ordering?
>
> Keeping such ordering might come at a non-trivial cost; sometimes just
> computation wise, sometimes on query complexity and/or space when the
> order column needs to be stored. It just happens to be cheap and -
> apparently - "natural" on MongoDB.
>
> BTW I'm not even 100% convinced about my previous suggestion about
> supporting this, as maybe it would set a bad precedent?
> The risk is that after developing on OGM/MongoDB you start expecting
> this to work fine on other systems, so from this perspective our API
> would not have a clear contract on the expectations one should have.
>
>
> > You could add documentation on a getter stating what the expected
> ordering
> > is (and, frankly, it may be relevant even with a List: how would you
> > translate "the invoices are ordered by issuing date" in terms of a Java
> > interface?). But what you cannot (or at least, shouldn't) do is
> > contradicting the interfaces you expose ("yeah, I'm exposing a List
> because
> > elements are ordered explicitly since we only ever add one at the end,
> but
> > please be nice and don't try random access, it's a very long linked
> list").
> > And no, I don't consider that "instanceof RandomAccess" on the client
> side
> > would be a clean solution...
> >
> > I'm not saying "it's super important, a critical use case, we can't ship
> a
> > new release without this, let's spend 3 months working on it". I'm just
> > saying it may be relevant for other collection types, so if we do it for
> > lists, and if it's not much more difficult to do it for other collection
> > types, why wouldn't we?
>
> ah, maybe I understand now. If you're proposing we do a similar thing
> for other specific collection types then sure we can discuss that. I
> had the impression you wanted to store ordering for properties
> literally using the *Collection* signature.
>
> >
> >
> > Yoann Rodière <yrodiere at redhat.com>
> > Software Engineer
> > Red Hat / Hibernate NoORM Team
> >
> > On 13 January 2017 at 15:26, Sanne Grinovero <sanne at hibernate.org>
> wrote:
> >>
> >> Hi Yoann,
> >>
> >> while you might be using a specific implementation when persisting
> >> your new entities, when you're loading the object back from the
> >> database how is Hibernate supposed to know
> >>  A) which implementation you want it to use
> >>  B) which order to use, if you didn't map e.g. an order column to
> >> store the ordering information (MongoDB not needing a strategy is an
> >> exception)
> >>
> >> The entity model expresses the intent of the end user, but also it
> >> expresses its requirements which will be taken into account by ORM
> >> when it needs to provide an implementation for the "Collection"
> >> interface.
> >>
> >> A List expresses the ordering requirement by contract; if you map a
> >> relation as "Collection" I wage you're not interested in the order,
> >> but you're rather more interested in accepting a wide range of
> >> implementations, some of which might ignore the ordering requirement.
> >> By doing so, you're explicitly telling both ORM and your other code
> >> interacting with the model that ignoring the order is acceptable.
> >>
> >> BTW I suspect mapping things as a generic Collection is a very unusual
> >> (or lazy) choice; I don't remember ever using it as it's quite
> >> ambiguous, and there's no way any specification diagram will lead you
> >> to recommend using a Collection (other than cleanly and explicitly
> >> wanting to express you don't care about ordering, nor other properties
> >> like uniqueness...)
> >>
> >> Thanks,
> >> Sanne
> >>
> >>
> >> On 13 January 2017 at 13:29, Yoann Rodiere <yrodiere at redhat.com> wrote:
> >> >> I think it's a reasonable expectation, as long as we're talking
> >> >> specifically about mapping a *List* and not just a generic
> Collection.
> >> >
> >> >
> >> > Ah, this topic again :) I know I'll be all lone, but I'll try anyway!
> >> >
> >> > If we do it for List, and unless there are technical issues that
> prevent
> >> > us
> >> > from doing so, I would be in favor of doing it for any kind of
> >> > collection.
> >> >
> >> > In Collections, the fact that iteration order is deterministic is
> mostly
> >> > up
> >> > to the implementation, which is different from saying it's not
> >> > deterministic. From the javadoc for Collection#iterator():
> >> >
> >> >> There are no guarantees concerning the order in which the elements
> are
> >> >> returned
> >> >> (unless this collection is an instance of some class that provides a
> >> >> guarantee).
> >> >
> >> >
> >> > Deterministic, and even predictable order is not exclusive to List,
> >> > either:
> >> > for instance, LinkedHashSet is not a List, it has a specific iteration
> >> > order, but there is nothing in its implemented interfaces (Collection,
> >> > Set)
> >> > that defines this order.
> >> >
> >> > My point is, we can't rely on the implemented interface to decide
> >> > whether
> >> > the iteration order is important or not, so we may as well decide it
> is
> >> > always important. Unless there are annoying technical challenges, of
> >> > course.
> >> >
> >> > Yoann Rodière <yrodiere at redhat.com>
> >> > Software Engineer
> >> > Red Hat / Hibernate NoORM Team
> >> >
> >> > On 13 January 2017 at 13:48, Sanne Grinovero <sanne at hibernate.org>
> >> > wrote:
> >> >>
> >> >> I think it's a reasonable expectation, as long as we're talking
> >> >> specifically about mapping a *List* and not just a generic
> Collection.
> >> >>
> >> >> On 13 January 2017 at 12:18, Davide D'Alto <davide at hibernate.org>
> >> >> wrote:
> >> >> > Hi,
> >> >> > it seems that when using Collection of elements in MongoDB, users
> >> >> > expect to have the elements in the same order as in the db. You can
> >> >> > see the question on the forum here:
> >> >> >
> >> >> >
> >> >> > https://forum.hibernate.org/viewtopic.php?f=31&t=1043903&
> p=2491218#p2491218
> >> >> >
> >> >> > I also found this StackOverflow question:
> >> >> >
> >> >> >
> >> >> > http://stackoverflow.com/questions/9013916/do-arrays-
> stored-in-mongodb-keep-their-order
> >> >> >
> >> >> > What do you think? Is it something that we should support?
> >> >> >
> >> >> > Cheers,
> >> >> > Davide
> >> >> > _______________________________________________
> >> >> > hibernate-dev mailing list
> >> >> > hibernate-dev at lists.jboss.org
> >> >> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >> >> _______________________________________________
> >> >> hibernate-dev mailing list
> >> >> hibernate-dev at lists.jboss.org
> >> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >> >
> >> >
> >
> >
>