[infinispan-dev] Query.getResultSize() to be available on the simplified DSL?

[Sanne Grinovero]

On 10 March 2014 17:09, Mircea Markus <mmarkus at redhat.com> wrote:
>
> On Mar 10, 2014, at 15:12, Sanne Grinovero <sanne at infinispan.org> wrote:
>
>> Ok you make some good points, and I've no doubts of it being useful.
>>
>> My only concern is that this could slow us down significantly in
>> providing other features which might be even more useful or pressing.
>> You have to pick your battles and be wise on where to spend energy
>> first.
>>
>> Considering that it's easier to add methods than to remove them, what
>> would you think of marking this as experimental for now?
>> I'd prefer to see the non-indexed query engine delivered first; this
>> sounds like being a stone on the critical path so it might be wise to
>> have the option to drop the requirement from a first implementation.
>> Definitely you're right that we should then implement "some" COUNT
>> strategy, I'm just not comfortable in committing on this one yet.
>
> I can imagine a lot of users emulating this by simply iterating over the entries in the result set. Even if we do just that and document it as slow, I think it's still worth exposing this somewhere.

I'm not questioning it to be useful. But the implementation is tricky,
for example simply iterating would require a global (and distributed)
lock to be accurate.
Otherwise we can only document it as an approximation, and worst is we
can't even estimate an error margin: under steady load it would
probably be a reasonable estimation, but there are corner cases in
which you can get off by several orders of magnitude.
Among others, your assumptions would need to include:
 - no nodes failing (no state transfers happening)
 - no write spikes (probably one of the best reasons to deploy
infinispan is to be able to absorb spikes)
 - no wild churning across datastore/cachestores
 - expirations happening in a homogeneus pattern (this entirely
depends on the use case)

Also there is no way to make this work in the context of a
transaction, as it would pretty much violate any promise of repeatable
read properties.

So the question is, if a user would still consider it useful after
(hopefully) understanding all the attached strings.

In other contexts we discussed the need for Infinispan to provide
something like a snapshot capability based on TOA. If we had that, we
could implement a count operation on top of it.

Sanne