[wildfly-dev] Map / reduce for management operations

[Brian Stansberry]

Great. Chatting on IRC will be fine. We should involve Emanuel 
Muckenhuber, as he did the single-process wildcard read support we have, 
and he likely remembers best the hassles in doing it across processes.

The use case is basically the same one Harald has -- trying to get a 
bunch of data in one request. Harald goes further by filtering out 
addresses and/or attributes, but it all starts with sucking in lots of data.

On 11/3/14, 12:01 PM, Heiko Braun wrote:
>
> Ok, let's discuss it further. Maybe on irc. I dont fully understand your use case, but if you explain it further i am confident we can contribute to it.
>
>> Am 31.10.2014 um 20:05 schrieb Brian Stansberry <brian.stansberry at redhat.com>:
>>
>>> On 10/31/14, 1:30 PM, Heiko Braun wrote:
>>>
>>>> On 31 Oct 2014, at 18:07, Brian Stansberry <brian.stansberry at redhat.com> wrote:
>>>>
>>>> This would definitely be useful.
>>>>
>>>> On the server side we already support a wildcard variant of
>>>> read-resource. But it only works within a single process, not across
>>>> processes in the domain. We've never had time to sort that issue. I'd
>>>> like to see this feature built on a foundation of a proper cross-domain
>>>> wildcard read-resource. That will kill two birds with one stone, plus
>>>> ensure consistent behavior.
>>>
>>> glad to hear. we specifically isolated it from the current codebase to explore the ideas, before merging with existing approaches. Bringing it into the main codebase would be a task for somebody with a better understanding of the management layer internals.
>>
>> Oh. I can't commit to when we'd be able to do multi-process wildcard read-resource. I was fired up by the possibility that a talented guy like Harald was interested. :) This thread certainly raises the visibility of it though. I'll be thinking about how to do it.
>>
>>>
>>>>
>>>> Any other ideas for a name for this? "Map / reduce" to me implies a much
>>>> broader set of functionality, and I wouldn't want to falsely advertise,
>>>> nor burn an operation name that we may want later.
>>>
>>> the name describes quiet well what's actually happening behind the scenes, but I agree that "map/reduce" is quiet overloaded these days and might lead to false expectations.
>>>
>>>> That also relates to
>>>> Stan's point in his response, re: this being an element in a broader
>>>> query feature. If it has a precise name that implies a narrow scope, if
>>>> later on there is a broader feature that broader feature can co-exist
>>>> with its ancestor.
>>>>
>>>>> On 10/31/14, 10:35 AM, Harald Pehl wrote:
>>>>> TL;DR
>>>>>
>>>>> For management clients it is tedious and awkward to read resources /
>>>>> attributes in big domains (e.g. return the state of all running servers
>>>>> across all hosts which are part of server group "foo"). Today this
>>>>> requires to setup multiple composite operations *on the client* which
>>>>> need to be executed in a specific order. This proposal suggests a new
>>>>> operation which collects all relevant information *on the
>>>>> server* and returns only the relevant data in one go to the client.
>>>>>
>>>>> --------------------
>>>>>
>>>>> # Background:
>>>>>
>>>>> In the admin console we often need to read specific attributes across a
>>>>> domain or across a deeply nested resource:
>>>>>
>>>>> - List all enabled data sources of the current profile
>>>>> - Show the port offset of all running servers across all hosts
>>>>> - Get all users which belong to role Operator
>>>>>
>>>>> Even for small domains with two server groups and a small number of
>>>>> servers this procedure is awkward and error prone. What makes it even
>>>>> more difficult is the asynchronous nature of the admin console. Soon you
>>>>> end up in a deeply nested callback-hell. Besides that these 'queries'
>>>>> lead to poor performance for big domains with tens of groups, hosts and
>>>>> servers.
>>>>>
>>>>> # Proposal / Prototype
>>>>>
>>>>> The GitHub repository at [1] contains a more detailed description of the
>>>>> problem statement and a working prototype. This includes details such as
>>>>> how to address the requested resources, how the information is retrieved
>>>>> and how to handle errors.
>>>>>
>>>>> The proposal also includes a way to filter and reduce the resources
>>>>> against a list of attributes and values *before* the response is sent to
>>>>> the client.
>>>>>
>>>>> [1] https://github.com/hpehl/map-reduce
>>>>>
>>>>> ---
>>>>> Harald Pehl
>>>>> JBoss by Red Hat
>>>>> http://hpehl.info
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> wildfly-dev mailing list
>>>>> wildfly-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>
>>>>
>>>> --
>>>> Brian Stansberry
>>>> Senior Principal Software Engineer
>>>> JBoss by Red Hat
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>> --
>> Brian Stansberry
>> Senior Principal Software Engineer
>> JBoss by Red Hat


-- 
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat