Hi Tim,
Thanks for looking at my question.
Yes, we have multiple consumers that might have the same selectors and or different
selectors.
The idea of using the producer and consumer model for us is this.
There will be many requests that will be made concurrently. These requests all go into a
persisent queue for guaranteed delivery. These requests are for a "job" to be
worked on. These jobs might be quick or long running jobs.
What we want is to have a variable number of consumers. Each consumer will sit on a
different box, since each job is very CPU intensive. Each consumer will look for certain
types of jobs and process them one-at-a-time.
We want to have multiple consumers to have the *same* selectors because we want to scale
side-ways; just add more boxes to be able to process more messages (jobs).
We also want to have multiple consumers to have different selectors, since certain types
of jobs are known to be much smaller than the other jobs. So we want to have dedicated
consumers to work on these types of jobs. This is so that the long running jobs do not
clog up all the consumers and queue up all the small jobs. In our use case, it's ok
to have the long running jobs take their time to do the work.. but it's mandatory to
get all the small jobs through as soon as possible.
The other caveat is that certain consumers should be able to config to look for small jobs
first, then if there are no small jobs in the queue, grab a big job. When do, start
looking for a small job again... and so on..
So, we have three types of consumers if you may. One dedicated to handling small jobs.
One dedicated to handling big jobs. One looks for small jobs, if non are there, look for
big jobs to work on.
We actually do have requirements to have multiple consumers of big jobs to work on same
and different types of of jobs..
We have all of these implemented and working with one problem that we didn't expect.
That is, the consumer that is applying a selector, seems to be holding hostage all the
messages that fit it's selector even though he's only called receive() for one
message.
What we need is for other consumers to be able to apply the same selector and be able to
get the messages that the other consumer didn't call receive() on yet. That way, we
can have many consumers work on many of the same types of jobs. We need this to scale the
processing of these jobs.
My other thought is if there is a way to specify for the consumer applying the selector
that he only wants 1 message to be returned that matches his criteria. He will work on
his job and then go ask for 1 more of the same.
Thanks,
Minh
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4056923#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...