[J2EE Design Patterns] - Design for heavy processing
by yoggi2k5
Hi all experts.
I'm about to design a "price-finder" application that read prices from a website.
I have about 10 000 URLs to products in a database right now. Some URLs needs to be read every 5 minutes, others every 20 minutes and the rest every hour.
It takes up to 2 sec to download and process every page. It's the downloading thats the bottleneck right now, due to the "slow" webservers I read the data from.
I need to have around 10 "processes" working at the same time to be able to process it all.
I was thinking of having a scheduled MDB's that is triggered every 5, 20 and 60 minutes.
Every time it's triggered it will get rellevant URLs from the database (about 3500 each time), split it up into 10 parts and send each part to a stateless session bean for processing. The stateless session beans will then processes around 350 URLs each.
What do you experts think about that design?
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4101114#4101114
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4101114
18 years, 5 months
[Clustering/JBoss] - Re: XAConnectionFactory not found only in Clustering
by seoeng
Let me add one more thing, as anything can help at this point. I have overriden some settings in my jboss.xml in my ejb.jar...maybe this is causing the issue? (it works in the default server setting in a standalone configuration):
| <!DOCTYPE jboss PUBLIC
| "-//JBoss//DTD JBOSS 4.0//EN"
| "http://www.jboss.org/j2ee/dtd/jboss_4_0.dtd">
| <jboss>
| <enterprise-beans>
| <message-driven>
| <ejb-name>MDB1</ejb-name>
| <configuration-name>MDB1MDBConfig</configuration-name>
| <destination-jndi-name>queue/MDB1Queue</destination-jndi-name>
| </message-driven>
| <message-driven>
| <ejb-name>MDB2</ejb-name>
| <configuration-name>MDB2MDBConfig</configuration-name>
| <destination-jndi-name>queue/MDB2Queue</destination-jndi-name>
| </message-driven>
| <message-driven>
| <ejb-name>MDB3</ejb-name>
| <configuration-name>MDB3MDBConfig</configuration-name>
| <destination-jndi-name>queue/MDB3Queue</destination-jndi-name>
| </message-driven>
| <message-driven>
| <ejb-name>MDB4</ejb-name>
| <configuration-name>MDB4MDBConfig</configuration-name>
| <destination-jndi-name>queue/MDB4Queue</destination-jndi-name>
| </message-driven>
| </enterprise-beans>
|
| <container-configurations>
|
| <container-configuration>
| <container-name>MDB4MDBConfig</container-name>
| <call-logging>false</call-logging>
| <invoker-proxy-binding-name>MDB4-message-driven-bean</invoker-proxy-binding-name>
| <container-interceptors>
| <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
| <!-- CMT -->
| <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <!-- BMT -->
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
| </container-interceptors>
| <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
| <instance-cache></instance-cache>
| <persistence-manager></persistence-manager>
| <container-pool-conf>
| <MaximumSize>50</MaximumSize>
| </container-pool-conf>
| </container-configuration>
|
| <container-configuration>
| <container-name>MDB3MDBConfig</container-name>
| <call-logging>false</call-logging>
| <invoker-proxy-binding-name>MDB3-message-driven-bean</invoker-proxy-binding-name>
| <container-interceptors>
| <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
| <!-- CMT -->
| <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <!-- BMT -->
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
| </container-interceptors>
| <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
| <instance-cache></instance-cache>
| <persistence-manager></persistence-manager>
| <container-pool-conf>
| <MaximumSize>50</MaximumSize>
| </container-pool-conf>
| </container-configuration>
|
| <container-configuration>
| <container-name>MDB2MDBConfig</container-name>
| <call-logging>false</call-logging>
| <invoker-proxy-binding-name>MDB2-message-driven-bean</invoker-proxy-binding-name>
| <container-interceptors>
| <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
| <!-- CMT -->
| <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <!-- BMT -->
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
| </container-interceptors>
| <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
| <instance-cache></instance-cache>
| <persistence-manager></persistence-manager>
| <container-pool-conf>
| <MaximumSize>50</MaximumSize>
| </container-pool-conf>
| </container-configuration>
|
| <container-configuration>
| <container-name>MDB1MDBConfig</container-name>
| <call-logging>false</call-logging>
| <invoker-proxy-binding-name>MDB1-message-driven-bean</invoker-proxy-binding-name>
| <container-interceptors>
| <interceptor>org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.LogInterceptor</interceptor>
| <interceptor>org.jboss.ejb.plugins.RunAsSecurityInterceptor</interceptor>
| <!-- CMT -->
| <interceptor transaction="Container">org.jboss.ejb.plugins.TxInterceptorCMT</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor transaction="Container">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <!-- BMT -->
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.MessageDrivenTxInterceptorBMT</interceptor>
| <interceptor transaction="Bean">org.jboss.ejb.plugins.CallValidationInterceptor</interceptor>
| <interceptor>org.jboss.resource.connectionmanager.CachedConnectionInterceptor</interceptor>
| </container-interceptors>
| <instance-pool>org.jboss.ejb.plugins.MessageDrivenInstancePool</instance-pool>
| <instance-cache></instance-cache>
| <persistence-manager></persistence-manager>
| <container-pool-conf>
| <MaximumSize>50</MaximumSize>
| </container-pool-conf>
| </container-configuration>
| </container-configurations>
|
| <invoker-proxy-bindings>
|
| <invoker-proxy-binding>
| <name>MDB4-message-driven-bean</name>
| <invoker-mbean>default</invoker-mbean>
| <proxy-factory>org.jboss.ejb.plugins.jms.JMSContainerInvoker</proxy-factory>
| <proxy-factory-config>
| <JMSProviderAdapterJNDI>DefaultJMSProvider</JMSProviderAdapterJNDI>
| <ServerSessionPoolFactoryJNDI>StdJMSPool</ServerSessionPoolFactoryJNDI>
| <CreateJBossMQDestination>true</CreateJBossMQDestination>
| <!-- WARN: Don't set this to zero until a bug in the pooled executor is fixed -->
| <MinimumSize>1</MinimumSize>
| <MaximumSize>5</MaximumSize>
| <KeepAliveMillis>30000</KeepAliveMillis>
| <MaxMessages>1</MaxMessages>
| <MDBConfig>
| <ReconnectIntervalSec>10</ReconnectIntervalSec>
| <DLQConfig>
| <DestinationQueue>queue/DLQ</DestinationQueue>
| <MaxTimesRedelivered>10</MaxTimesRedelivered>
| <TimeToLive>0</TimeToLive>
| </DLQConfig>
| </MDBConfig>
| </proxy-factory-config>
| </invoker-proxy-binding>
|
| <invoker-proxy-binding>
| <name>MDB3-message-driven-bean</name>
| <invoker-mbean>default</invoker-mbean>
| <proxy-factory>org.jboss.ejb.plugins.jms.JMSContainerInvoker</proxy-factory>
| <proxy-factory-config>
| <JMSProviderAdapterJNDI>DefaultJMSProvider</JMSProviderAdapterJNDI>
| <ServerSessionPoolFactoryJNDI>StdJMSPool</ServerSessionPoolFactoryJNDI>
| <CreateJBossMQDestination>true</CreateJBossMQDestination>
| <!-- WARN: Don't set this to zero until a bug in the pooled executor is fixed -->
| <MinimumSize>1</MinimumSize>
| <MaximumSize>1</MaximumSize>
| <KeepAliveMillis>30000</KeepAliveMillis>
| <MaxMessages>1</MaxMessages>
| <MDBConfig>
| <ReconnectIntervalSec>10</ReconnectIntervalSec>
| <DLQConfig>
| <DestinationQueue>queue/DLQ</DestinationQueue>
| <MaxTimesRedelivered>10</MaxTimesRedelivered>
| <TimeToLive>0</TimeToLive>
| </DLQConfig>
| </MDBConfig>
| </proxy-factory-config>
| </invoker-proxy-binding>
|
| <invoker-proxy-binding>
| <name>MDB1-message-driven-bean</name>
| <invoker-mbean>default</invoker-mbean>
| <proxy-factory>org.jboss.ejb.plugins.jms.JMSContainerInvoker</proxy-factory>
| <proxy-factory-config>
| <JMSProviderAdapterJNDI>DefaultJMSProvider</JMSProviderAdapterJNDI>
| <ServerSessionPoolFactoryJNDI>StdJMSPool</ServerSessionPoolFactoryJNDI>
| <CreateJBossMQDestination>true</CreateJBossMQDestination>
| <!-- WARN: Don't set this to zero until a bug in the pooled executor is fixed -->
| <MinimumSize>1</MinimumSize>
| <MaximumSize>2</MaximumSize>
| <KeepAliveMillis>30000</KeepAliveMillis>
| <MaxMessages>1</MaxMessages>
| <MDBConfig>
| <ReconnectIntervalSec>10</ReconnectIntervalSec>
| <DLQConfig>
| <DestinationQueue>queue/DLQ</DestinationQueue>
| <MaxTimesRedelivered>10</MaxTimesRedelivered>
| <TimeToLive>0</TimeToLive>
| </DLQConfig>
| </MDBConfig>
| </proxy-factory-config>
| </invoker-proxy-binding>
|
| <invoker-proxy-binding>
| <name>MDB2-message-driven-bean</name>
| <invoker-mbean>default</invoker-mbean>
| <proxy-factory>org.jboss.ejb.plugins.jms.JMSContainerInvoker</proxy-factory>
| <proxy-factory-config>
| <JMSProviderAdapterJNDI>DefaultJMSProvider</JMSProviderAdapterJNDI>
| <ServerSessionPoolFactoryJNDI>StdJMSPool</ServerSessionPoolFactoryJNDI>
| <CreateJBossMQDestination>true</CreateJBossMQDestination>
| <!-- WARN: Don't set this to zero until a bug in the pooled executor is fixed -->
| <MinimumSize>1</MinimumSize>
| <MaximumSize>1</MaximumSize>
| <KeepAliveMillis>30000</KeepAliveMillis>
| <MaxMessages>1</MaxMessages>
| <MDBConfig>
| <ReconnectIntervalSec>10</ReconnectIntervalSec>
| <DLQConfig>
| <DestinationQueue>queue/DLQ</DestinationQueue>
| <MaxTimesRedelivered>10</MaxTimesRedelivered>
| <TimeToLive>0</TimeToLive>
| </DLQConfig>
| </MDBConfig>
| </proxy-factory-config>
| </invoker-proxy-binding>
| </invoker-proxy-bindings>
|
| </jboss>
|
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4101113#4101113
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4101113
18 years, 5 months