3:31 p.m.
For Map/Reduce tasks - does the Task, Mapper, Reducer implementations
classes need to be present in the classpath of all the nodes?
Thanks,
2:20 p.m.
Yes it does!
On 13-01-25 4:31 PM, Ray Tsang wrote:
For Map/Reduce tasks - does the Task, Mapper, Reducer implementations
classes need to be present in the classpath of all the nodes?
Thanks,
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
2:32 p.m.
I just wrote something quick that uses groovy scripts. my hope is to be
able to execute any arbitrary scripted mapper/reducer w/o needing to, say,
redeploy/restart the node/server.
The code is here:
https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
On Sat, Jan 26, 2013 at 12:20 PM, Vladimir Blagojevic
<vblagoje(a)redhat.com>wrote:
Yes it does!
On 13-01-25 4:31 PM, Ray Tsang wrote:
For Map/Reduce tasks - does the Task, Mapper, Reducer implementations
classes need to be present in the classpath of all the nodes?
Thanks,
_______________________________________________
infinispan-dev mailing
listinfinispan-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
2:58 p.m.
Sounds like an excellent idea!
Il giorno 26/gen/2013 20:33, "Ray Tsang" <saturnism(a)gmail.com> ha
scritto:
I just wrote something quick that uses groovy scripts. my hope is to
be
able to execute any arbitrary scripted mapper/reducer w/o needing to, say,
redeploy/restart the node/server.
The code is here:
https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
On Sat, Jan 26, 2013 at 12:20 PM, Vladimir Blagojevic <vblagoje(a)redhat.com
> wrote:
> Yes it does!
> On 13-01-25 4:31 PM, Ray Tsang wrote:
>
> For Map/Reduce tasks - does the Task, Mapper, Reducer implementations
> classes need to be present in the classpath of all the nodes?
>
> Thanks,
>
>
> _______________________________________________
> infinispan-dev mailing
listinfinispan-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:05 p.m.
The classloader used in Groovy classloader constructor is imo wrong.
Unless you would bundle this infinispan-groovy with your app.
But since most of Infinispan comes with AS already, I would also expect this to be a
module, hence no-go.
On Jan 26, 2013, at 21:58, Sanne Grinovero <sanne(a)infinispan.org> wrote:
Sounds like an excellent idea!
Il giorno 26/gen/2013 20:33, "Ray Tsang" <saturnism(a)gmail.com> ha
scritto:
> I just wrote something quick that uses groovy scripts. my hope is to be able to
execute any arbitrary scripted mapper/reducer w/o needing to, say, redeploy/restart the
node/server.
>
> The code is here: https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
>
>
>
> On Sat, Jan 26, 2013 at 12:20 PM, Vladimir Blagojevic <vblagoje(a)redhat.com>
wrote:
>> Yes it does!
>> On 13-01-25 4:31 PM, Ray Tsang wrote:
>>> For Map/Reduce tasks - does the Task, Mapper, Reducer implementations classes
need to be present in the classpath of all the nodes?
>>>
>>> Thanks,
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:20 p.m.
Ales,
Since it's a quick proof of concept - I'd love to see it getting improved.
Would the thread context classloader be the right thing to use? Or,
perhaps it can use ScriptEngine?
Thanks!
On Sat, Jan 26, 2013 at 1:05 PM, Ales Justin <ales.justin(a)gmail.com> wrote:
The classloader used in Groovy classloader constructor is imo wrong.
Unless you would bundle this infinispan-groovy with your app.
But since most of Infinispan comes with AS already, I would also expect
this to be a module, hence no-go.
On Jan 26, 2013, at 21:58, Sanne Grinovero <sanne(a)infinispan.org> wrote:
Sounds like an excellent idea!
Il giorno 26/gen/2013 20:33, "Ray Tsang" <saturnism(a)gmail.com> ha
scritto:
> I just wrote something quick that uses groovy scripts. my hope is to be
> able to execute any arbitrary scripted mapper/reducer w/o needing to, say,
> redeploy/restart the node/server.
>
> The code is here:
> https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
>
>
>
> On Sat, Jan 26, 2013 at 12:20 PM, Vladimir Blagojevic <
> vblagoje(a)redhat.com> wrote:
>
>> Yes it does!
>> On 13-01-25 4:31 PM, Ray Tsang wrote:
>>
>> For Map/Reduce tasks - does the Task, Mapper, Reducer implementations
>> classes need to be present in the classpath of all the nodes?
>>
>> Thanks,
>>
>>
>> _______________________________________________
>> infinispan-dev mailing
listinfinispan-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
3:55 p.m.
Would the thread context classloader be the right thing to use?
It depends how Infinispan's M/R code handles setting of TCCL.
Specially in remote invocations.
But I would expect it to work.
Not the best, but good enough. :-)
Or, perhaps it can use ScriptEngine?
What's ScriptEngine?
-Ales
6:35 a.m.
I do like this approach though. Ales - how could we bundle something like this as an AS
module and play nice with modular class loading there?
- M
On 26 Jan 2013, at 21:55, Ales Justin <ales.justin(a)gmail.com> wrote:
> Would the thread context classloader be the right thing to use?
It depends how Infinispan's M/R code handles setting of TCCL.
Specially in remote invocations.
But I would expect it to work.
Not the best, but good enough. :-)
> Or, perhaps it can use ScriptEngine?
What's ScriptEngine?
-Ales
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
8:01 a.m.
Ales - how could we bundle something like this as an AS module and
play nice with modular class loading there?
Well, I would say we're back to that discussion with AdvancedCache::withClassloader.
:-)
Same thing with M/R -- can it tap into app's CL?
Otherwise TCCL should do, in JEE env.
Or perhaps some init(Cache cache) on M/R classes, where it can get a hold of Cache,
which would then expose app' CL, via AC::withClassloader?
-Ales
On 26 Jan 2013, at 21:55, Ales Justin <ales.justin(a)gmail.com>
wrote:
>> Would the thread context classloader be the right thing to use?
>
> It depends how Infinispan's M/R code handles setting of TCCL.
> Specially in remote invocations.
> But I would expect it to work.
> Not the best, but good enough. :-)
>
>> Or, perhaps it can use ScriptEngine?
>
> What's ScriptEngine?
>
> -Ales
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
8:25 a.m.
M/R classes do have access to the cache so they can get a hold of the app's class
loader. But the tricky part is that the M/R tasks themselves need to be loaded from that
class loader first…
On 28 Jan 2013, at 14:01, Ales Justin <ales.justin(a)gmail.com> wrote:
> Ales - how could we bundle something like this as an AS module
and play nice with modular class loading there?
Well, I would say we're back to that discussion with AdvancedCache::withClassloader.
:-)
Same thing with M/R -- can it tap into app's CL?
Otherwise TCCL should do, in JEE env.
Or perhaps some init(Cache cache) on M/R classes, where it can get a hold of Cache,
which would then expose app' CL, via AC::withClassloader?
-Ales
> On 26 Jan 2013, at 21:55, Ales Justin <ales.justin(a)gmail.com> wrote:
>
>>> Would the thread context classloader be the right thing to use?
>>
>> It depends how Infinispan's M/R code handles setting of TCCL.
>> Specially in remote invocations.
>> But I would expect it to work.
>> Not the best, but good enough. :-)
>>
>>> Or, perhaps it can use ScriptEngine?
>>
>> What's ScriptEngine?
>>
>> -Ales
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik(a)jboss.org
> twitter.com/maniksurtani
>
> Platform Architect, JBoss Data Grid
> http://red.ht/data-grid
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
8:53 a.m.
M/R classes do have access to the cache so they can get a hold of the
app's class loader. But the tricky part is that the M/R tasks themselves need to be
loaded from that class loader first…
This already seems to work -- as we use tasks extensively in CapeDwarf,
and it seems to work OK; note: CapeDwarf runs in both ways - modular and embedded
-Ales
On 28 Jan 2013, at 14:01, Ales Justin <ales.justin(a)gmail.com>
wrote:
>> Ales - how could we bundle something like this as an AS module and play nice with
modular class loading there?
>
> Well, I would say we're back to that discussion with
AdvancedCache::withClassloader. :-)
> Same thing with M/R -- can it tap into app's CL?
> Otherwise TCCL should do, in JEE env.
>
> Or perhaps some init(Cache cache) on M/R classes, where it can get a hold of Cache,
> which would then expose app' CL, via AC::withClassloader?
>
> -Ales
>
>> On 26 Jan 2013, at 21:55, Ales Justin <ales.justin(a)gmail.com> wrote:
>>
>>>> Would the thread context classloader be the right thing to use?
>>>
>>> It depends how Infinispan's M/R code handles setting of TCCL.
>>> Specially in remote invocations.
>>> But I would expect it to work.
>>> Not the best, but good enough. :-)
>>>
>>>> Or, perhaps it can use ScriptEngine?
>>>
>>> What's ScriptEngine?
>>>
>>> -Ales
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik(a)jboss.org
>> twitter.com/maniksurtani
>>
>> Platform Architect, JBoss Data Grid
>> http://red.ht/data-grid
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
11:29 a.m.
Hi Guys,
I also implemented something w/ JSR 223 -
https://github.com/saturnism/infinispan/tree/jsr223-mr/core/src/main/java...
With tests that uses scripts like this:
https://github.com/saturnism/infinispan/tree/jsr223-mr/core/src/test/reso...
On Mon, Jan 28, 2013 at 6:53 AM, Ales Justin <ales.justin(a)gmail.com> wrote:
> M/R classes do have access to the cache so they can get a hold
of the
app's class loader. But the tricky part is that the M/R tasks themselves
need to be loaded from that class loader first…
This already seems to work -- as we use tasks extensively in CapeDwarf,
and it seems to work OK; note: CapeDwarf runs in both ways - modular and
embedded
-Ales
> On 28 Jan 2013, at 14:01, Ales Justin <ales.justin(a)gmail.com> wrote:
>
>>> Ales - how could we bundle something like this as an AS module and
play nice with modular class loading there?
>>
>> Well, I would say we're back to that discussion with
AdvancedCache::withClassloader. :-)
>> Same thing with M/R -- can it tap into app's CL?
>> Otherwise TCCL should do, in JEE env.
>>
>> Or perhaps some init(Cache cache) on M/R classes, where it can get a
hold of Cache,
>> which would then expose app' CL, via AC::withClassloader?
>>
>> -Ales
>>
>>> On 26 Jan 2013, at 21:55, Ales Justin <ales.justin(a)gmail.com> wrote:
>>>
>>>>> Would the thread context classloader be the right thing to use?
>>>>
>>>> It depends how Infinispan's M/R code handles setting of TCCL.
>>>> Specially in remote invocations.
>>>> But I would expect it to work.
>>>> Not the best, but good enough. :-)
>>>>
>>>>> Or, perhaps it can use ScriptEngine?
>>>>
>>>> What's ScriptEngine?
>>>>
>>>> -Ales
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev(a)lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Manik Surtani
>>> manik(a)jboss.org
>>> twitter.com/maniksurtani
>>>
>>> Platform Architect, JBoss Data Grid
>>> http://red.ht/data-grid
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik(a)jboss.org
> twitter.com/maniksurtani
>
> Platform Architect, JBoss Data Grid
> http://red.ht/data-grid
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
7:02 a.m.
On 01/26/2013 09:32 PM, Ray Tsang wrote:
I just wrote something quick that uses groovy scripts. my hope is to
be able to execute any arbitrary scripted mapper/reducer w/o needing
to, say, redeploy/restart the node/server.
The code is here:
https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
Ray, to avoid duplicate effort, I'm working on implementing "stored
scripts" for Infinispan based on ScriptEngine.
Scripts would be stored (compiled if possible) in a dedicated replicated
cache. The scripts can then be referred to by name in dist-exec and
map-reduce invocations. It will then be possible to invoke these scripts
both locally and remotely (over HotRod and REST).
Some Jiras:
https://issues.jboss.org/browse/ISPN-1094
https://issues.jboss.org/browse/ISPN-1192
Tristan
7:23 a.m.
You guys should collaborate on this. :)
On 28 Jan 2013, at 13:02, Tristan Tarrant <ttarrant(a)redhat.com> wrote:
On 01/26/2013 09:32 PM, Ray Tsang wrote:
> I just wrote something quick that uses groovy scripts. my hope is to
> be able to execute any arbitrary scripted mapper/reducer w/o needing
> to, say, redeploy/restart the node/server.
>
> The code is here:
> https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
>
Ray, to avoid duplicate effort, I'm working on implementing "stored
scripts" for Infinispan based on ScriptEngine.
Scripts would be stored (compiled if possible) in a dedicated replicated
cache. The scripts can then be referred to by name in dist-exec and
map-reduce invocations. It will then be possible to invoke these scripts
both locally and remotely (over HotRod and REST).
Some Jiras:
https://issues.jboss.org/browse/ISPN-1094
https://issues.jboss.org/browse/ISPN-1192
Tristan
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
11:42 a.m.
Tristan,
After the quick groovy poc, I discovered the design/plan doc for m/r
over hotrod that mentioned stored procedure approach.
Where will the script be stored and how will the user store it?
(Perhaps stored also in a cache?)
I just implemented something w/ jsr223 that might be generic enough
for hooking into the script store. :)
On Jan 28, 2013, at 5:23, Manik Surtani <msurtani(a)redhat.com> wrote:
You guys should collaborate on this. :)
On 28 Jan 2013, at 13:02, Tristan Tarrant <ttarrant(a)redhat.com> wrote:
> On 01/26/2013 09:32 PM, Ray Tsang wrote:
>> I just wrote something quick that uses groovy scripts. my hope is to
>> be able to execute any arbitrary scripted mapper/reducer w/o needing
>> to, say, redeploy/restart the node/server.
>>
>> The code is here:
>> https://github.com/saturnism/infinispan/tree/groovy-mr/groovy
> Ray, to avoid duplicate effort, I'm working on implementing "stored
> scripts" for Infinispan based on ScriptEngine.
> Scripts would be stored (compiled if possible) in a dedicated replicated
> cache. The scripts can then be referred to by name in dist-exec and
> map-reduce invocations. It will then be possible to invoke these scripts
> both locally and remotely (over HotRod and REST).
>
> Some Jiras:
>
> https://issues.jboss.org/browse/ISPN-1094
> https://issues.jboss.org/browse/ISPN-1192
>
> Tristan
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
twitter.com/maniksurtani
Platform Architect, JBoss Data Grid
http://red.ht/data-grid
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
4294
days inactive
4297
days old
infinispan-dev@lists.jboss.org
15 comments
6 participants
participants (6)
-
Ales Justin -
Manik Surtani -
Ray Tsang -
Sanne Grinovero -
Tristan Tarrant -
Vladimir Blagojevic