Re: [infinispan-dev] grouping and GridFS
It doesn't make sense. :)
The reason grid file systems exist is to distribute the file around the
cluster.
(both for performance so the network interface of a single server isn't
a bottleneck,
and for disk space so the available space on a single server isn't a
bottlenect)
If you don't want to distribute the file, a grid filesystem probably
isn't the right choice.
-Dennis
On 03/05/2014 09:04 AM, Ales Justin wrote:
But yeah, the moment I start chunking, I would still like to have the
grouped -- same node.
Or that doesn't make sense?
(hence having this discussion ;-)
-Ales
On 05 Mar 2014, at 16:01, Sanne Grinovero <sanne(a)infinispan.org> wrote:
> On 5 March 2014 14:54, Ales Justin <ales.justin(a)gmail.com> wrote:
>> Why do you chunk at all if you want them stored together?
>>
>> I only use chunking if I can't avoid it, to spread large files.
>>
>> That's what's GridFS all about -- store very large files.
>> Hence chunking.
>>
>> So you're saying we should know the limit of what we can store on 1 node,
>> if bigger, spread, therefore no grouping.
> Yes, but a very conservative approximation would be good enough: you
> don't need hardware specifications to figure out a reasonable
> threshold.
> If I had to make up a number out of thin air, I'd pick something
> around 10MB: any file below that threshold would not use chunking and
> be nicely stored together to be retrieved efficiently; beyond that
> start distributing.
> (this figure could probably use some testing if you're looking into performance)
>
> Sanne
>
>> -Ales
>>
>> On 5 Mar 2014 11:22, "Ales Justin" <ales.justin(a)gmail.com>
wrote:
>>> Just having a discussion with Bela about this.
>>>
>>> I guess having "grouping" on GridFS' content would make sense.
>>> e.g. put all chunks on the same node
>>>
>>> Is this doable?
>>> Afaiu, we would need to have some sort of "similarity" function
for
>>> content's metadata?
>>>
>>> -Ales
>>>