[JBoss JIRA] Created: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
3:09 p.m.
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in TestTopic
topic with default configuration where there are some consumers subscribed but not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g with
dedicated instance on a separate machine with no more than 1% cpu load during test and
physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load is
finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:51 p.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=comments#action_1... ]
Tim Fox commented on JBMESSAGING-1162:
--------------------------------------
How large are your messages?
JBoss Messaging stores persistent messages in a database. At startup it needs to load
messages from the database into the queue(s).
Fundamentally, the speed of this is determined by how fast your database is (along with
message size, message number, etc)
If you can suggest some other faster way of loading the messages from the Oracle database
into the queues then we are always open to suggestions.
If you don't want to load all the messages on startup, you can set the
"fullSize" attribute on the queue (see user guide).
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:12 p.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=comments#action_1... ]
Gabriele Garuglieri commented on JBMESSAGING-1162:
--------------------------------------------------
Hi Tim,
the messages are about 1kb each but i don't think this is the problem.
We experimented a bit with FullSize attribute and it looks there's a law that tie the
attribute value with the number of persisted messages, but we didn't find it
documented. We tried to extrapolate the values but we would like to be sure the relation
remains linear in the entire domain.
(i'm sorry i don't have the time to check it myself in the code right now)
Even if it should not be the normal operating condition we must be able to account, given
some particular conditions, for a db that could contain for short time tenth of million
messages without substantial performance penalty during usage and system restart. We can
lower the FullSize value for a single topic/queue but the db will contain several of them
so the cumulative effect at restart will be the same.
Anyway i wonder why we should condition the JBM startup and hence Jboss startup to the
memory load of the messages. IMO the system should be up and running in the faster way
possible and then delay the load once an application may need to access the topic/queue.
I think it is more tolerable a slow start for the single application than for the whole
system, even if i think that with the right parallelism the memory load could be executed
while the applications are already accessing the messages so without too much delay.
I don't know what experience you have of large accounts but i can assure you that if i
would deliver this configuration in production, with a test that shows a freeze of 10
minutes during startup i'd receive thumb down from our I/T people.
Thank you, Gabriele
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:33 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=comments#action_1... ]
Tim Fox commented on JBMESSAGING-1162:
--------------------------------------
There is a very clear link between fullSize and the amount of messages loaded at startup -
this is the max. number of messages it will load at startup. This is also documented in
the user guide.
90000 x 1kb = about 90MB of data. If your Oracle is taking 10 mins to load this, then tune
Oracle! Oracle is very tunable.
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:28 p.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=comments#action_1... ]
Gabriele Garuglieri commented on JBMESSAGING-1162:
--------------------------------------------------
Hi Tim,
perhaps i didn't explain myself well.
Depending on the number of messages persisted in db there's a minimum value for
FullSize under which JBM will refuse to start.
For instance from several runs we got the following results:
Messages min FullSize
180000 8000
300000 8188
600000 8880
900000 9500
1428188 26188
More or less linear until 900000 messages but then the line heads up. Even worst it
doesn't even have constant slope, with 1428188 msgs after a first good start with
26188 on the second run JBM refused to start unless FullSize was larger than 36000. (i
lost the log of that run and don't remember exact value, but it was slightly more than
36000).
Definitely there is a relation between minimum FullSize value and number of persisted
messages, but once the relation changed from linear to something else and i started
getting changing values for the same amount of messages it was impossible to determine
it.
Hence my request to document it.
And again max value for PageSize and DownCacheSize looks to have a relation with FullSize.
And IMO also this should be documented.
When you have to deal with very large number of messages these values are crucial for
optimal performances and deserves a better documentation.
My second point was that doesn't matter how well i tune Oracle, if i'm forced to
serially load in memory FullSize amount of messages for each queue/topic from a db that
contains tenth of millions of messages i'll always count startup times in tenth of
minutes.
Hence my second question, why JBM needs to serially load messages for each queue/topic
before startup instead of being able to parallel load messages at first queue/topic
access?
Thank you, Gabriele
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:25 p.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=comments#action_1... ]
Tim Fox commented on JBMESSAGING-1162:
--------------------------------------
Can you explain in more detail what you mean by "JBM refused to start.." - did
it give you an error message? If so please post it.
If you set fullSize, JBM should never try to load more messages than that on startup. It
should not force you to change fullSize either. If it does then that is a bug.
Maybe it is this bug http://jira.jboss.org/jira/browse/JBMESSAGING-1139 which I fixed
about a month ago and will be in the SP2 release I am doing today?
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:30 p.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=all ]
Tim Fox updated JBMESSAGING-1162:
---------------------------------
Fix Version/s: Unscheduled
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
Fix For: Unscheduled
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:50 a.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=all ]
Tim Fox updated JBMESSAGING-1162:
---------------------------------
Fix Version/s: 1.4.0.SP3.CP04
(was: Unscheduled)
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
Fix For: 1.4.0.SP3.CP04
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:52 a.m.
New subject: [JBoss JIRA] Closed: (JBMESSAGING-1162) JBM takes eons to startup with topic persisted messages within db
[ http://jira.jboss.com/jira/browse/JBMESSAGING-1162?page=all ]
Tim Fox closed JBMESSAGING-1162.
--------------------------------
Fix Version/s: Unscheduled
(was: 1.4.0.SP3.CP04)
Resolution: Won't Fix
Startup time is dependent on the database being used, and is a non issue anyway in JBM 2.0
where all new feature development is occurring.
JBM takes eons to startup with topic persisted messages within db
-----------------------------------------------------------------
Key: JBMESSAGING-1162
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-1162
Project: JBoss Messaging
Issue Type: Bug
Components: Messaging Core Persistence
Affects Versions: 1.4.0.SP1
Environment: Java 1.5.0_14
JBoss 4.2.2
Solaris 10, AMD Opteron biprocessor machine
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Critical
Fix For: Unscheduled
The problem is reproduced as follows:
1-Start server
2-start a producer that inserts persisted messages (we tried 90.000 messages) in
TestTopic topic with default configuration where there are some consumers subscribed but
not active
3-stop producer and orderly shutdown the server
4-start server
During server startup JBM executes a select to load the persisted messages id's then
executes 180 selects loading 500 messages at time.
With 90.000 messages those 180 selects takes about 10 minutes. The DBMS is Oracle 10g
with dedicated instance on a separate machine with no more than 1% cpu load during test
and physical db on a very fast NAS.
While JBM is executing those selects, obviously the server startup hangs until the load
is finished.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5916
days inactive
6142
days old
8 comments
2 participants
participants (2)
-
Gabriele Garuglieri (JIRA) -
Tim Fox (JIRA)