[Design of Messaging on JBoss (Messaging/JBoss)] - Re: DB performances for large number of messages
by garu
Continuing from previuos append.
I'd like to follow your suggestion to write myself the patch for this, after all, after many year, coding is still giving me fun, but i have such a tight schedule in my job that i cannot commit or promise anything.
I tried to figure out how it's working now (as a colleague said many years ago when there were still 300 baud modems around, it is not written as i would have written it so ,obviously, it's not working :) (i'm kidding, it is instead very well written) and i've got some ideas.
- Queue and Topic mbeans should have a couple of attributes like StoreName and RefStoreName. They could be both read/write with defaults in Queue-xmbean.xml and Topic-xmbean.xml of jbm_msg and jbm_msg_ref, or perhaps only the first one could be r/w and RefStoreName could result from composition of StoreName + "_ref".
- those attribues should be available in a couple of columns into jbm_postoffice, so that when a queue is associated to a channel is also associated to specific StoreName and RefStoreName.
- those attributes should be available from Binding, Queue,and Topic so that they are available to persistence manager to get the table name to prime sql statements, that actually use jbm_msg and jbm_msg_ref, with the correct table name to read and write for that specific channel.
- whenever a binding is added to postoffice table the store tables must be created if not already present because other channels are already writing on them.
- whenever a binding is removed from postoffice table the store tables should be removed if empty (this could be conditioned by a a flag attribute in queue/topic mbean, it would be not worth removing them if the destination will be reused again)
Obviously this is the result of just a rapid overlook of the code, so i'm sure i missed many subtleties, but i'm interested to know what you think.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4055133#4055133
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4055133
18 years, 9 months
[Design of Messaging on JBoss (Messaging/JBoss)] - Re: DB performances for large number of messages
by garu
Hi Tim,
thanks for the info, as usual when all the rest has failed read the instructions :)
About the application, is not that simple to explain.
It's not a single application but a group of applications that handle many different data flows from sensors distributed in a geographical network.
These flows arrive with different protocols, pure udp, udp with application protocol, tcp, ftp, etc..., with different priorities and with rates that are not constant through the day but show very high peaks.
JMS is seen as the means by which we can decouple the network from the data usage and the JMS persistence db must guarantee that once the message has been aknowledged to the originator, it will remain available to subscribers for the time defined for that flow.
Those persistence times will vary from few hours to two or three days, depending on the particular flow and on the minimum time span we need for the data to become a significant measurement set.
Then there is another class of messages, alarm messages, from which can depend the safeness of people, so you can imagine how carefully those message must be handled.
Here i've just scratched the surface, but the unifying guidelines are that we must guarantee we are not loosing any message and the system must be able to handle traffic peaks much higher than average flow rate without slowing down the flows.
As i said in my first append, the legacy system i wrote several years ago in C is still working for production, but now we need to feed other channels and that legacy system is no longer suited for that and we devised JMS as the decoupling system that could allow us to feed as many channels as we need.
Obviously my objective is to have the minimum number of messages parked for the minimum time as possible in the messaging persistence db, but nonetheless i'll have to move hundredths millions messages a day in and out of persistence db.
I can partition the flows in different messaging instances so that they use different db, but i cannot reach the 1:1 granularity and only the possibility to associate a destination to specific msg and msg_ref tables can give the flexibility to handle the forecasted volumes.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4055123#4055123
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4055123
18 years, 9 months
[Design of Clustering on JBoss (Clusters/JBoss)] - Clustring HA-JMS on Internal Client (non-master node)
by Anandnatraj
I have this following scenario on my clustering of JBoss with JMS.
1. I have two ears placed on a two servers (App-A and App-B). The APP-A and APP-B are clustered for the HA-JNDI.
2. These two ears need a Queue myQueue which they look for during deployment
3. Since I am using HA-JNDI, i placed the queue definitions on the all/deploy-singleton/jms of App-A, considering that as my master node.
4. When I start App-A, the myQueue is Deployed and the ear is also able to pickup the queue during Deployment. No issues.
5. Now I start App-B, the myQueue is not deployed here, and there is no need since the App-A is already having it. But when my ear is getting deployed, it is not able to find the queue JNDI in this server App-B, and the ear deployment is failed: (exception : queue Not Found)
6. By any chance I shutdown the App-A, then the myQueue is replicated in the App-B and the ear is bound with it.
My question is : Is this the way the HA-JNDI of JMS is going to work when we have the client on one of the non-master node itself? Or am I missing something?
Moreover can I exclude the HA-JNDI deployment of JMS and handle two individual myQueue on both the server?
Some help / eye opener is really appreciated.
Anandh
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4055087#4055087
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4055087
18 years, 9 months