1:03 a.m.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:05 a.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=all ]
Gabriele Garuglieri updated JBMESSAGING-984:
--------------------------------------------
Attachment: JBMESSAGING-984.zip
Bug doc described above.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:20 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Gabriele Garuglieri commented on JBMESSAGING-984:
-------------------------------------------------
I forgot to mention that the deadlock problem could be not strictly related to
expiration.
I'm planning to use JBmessaging with an application that should handle tenth of
subscribers and hundredth millions messages a day so i'm trying to stress a bit it to
see its reliability.
It happens, way too often, that suddendly server and client freeze to the point that to be
able to receive or send any other message or open a new connection i have to kill both
client and server. And sometimes this is not enough, i even had cases where i had to clean
jbm_msg and jbm_msg_ref before restarting the server, to be able to open new connections.
The problem is that this seems to be happening randomly and i could never be able to find
any relation to time of activity, number of messages or an exact sequence of actions to
reproduce the deadlock until i tried to use expiring messages as described above.
This same problem was present in 1.2.0, 1.2.0.sp1, and now in 1.3.0.GA.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:25 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Gabriele Garuglieri commented on JBMESSAGING-984:
-------------------------------------------------
Pls, any comment/advice/bypass on this?
This problem has completely stopped our developments!
I'm sorry to bother you, but we need at least to understand if this problem will have
a reasonable resolution time that we can afford to wait for or we have to switch to a
different JMS implementation.
Thanks.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:13 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Tim Fox commented on JBMESSAGING-984:
-------------------------------------
Gabriele-
Can you see if the problem still exists with the current code in TRUNK.
It may be related to http://jira.jboss.com/jira/browse/JBMESSAGING-962 which was recently
fixed.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5:51 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Gabriele Garuglieri commented on JBMESSAGING-984:
-------------------------------------------------
Ckecked out jbmessaging/trunk, built jboss-messaging.jar and jboss-messaging-client.jar.
Substituted <server>/lib/jboss-messaging.jar with the new one, substituted
<project>/lib/jboss-messaging-client.jar with the new one.
If that is enough, i'm sorry but nothing has changed. All deadlocked as before.
Thanks.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:03 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Sergey Koshcheyev commented on JBMESSAGING-984:
-----------------------------------------------
I'm going to try running your test since I have the same software installed on my
laptop.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:05 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Sergey Koshcheyev commented on JBMESSAGING-984:
-----------------------------------------------
Gabriele, can you provide the source code of the test you are running so I have something
to start with? Thanks.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:23 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Gabriele Garuglieri commented on JBMESSAGING-984:
-------------------------------------------------
Sure, just give me a bit of time to extract the relevant part from the bundle.
When we hit this problem we were already started building the application around JMS code,
so there are many things related to application configuration and messages content
treatment that are mostly experimental, are not relevant to this problem and wouldn't
make any sense to you.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:52 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Tim Fox commented on JBMESSAGING-984:
-------------------------------------
Sergey- please test from TRUNK
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:31 a.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=all ]
Gabriele Garuglieri updated JBMESSAGING-984:
--------------------------------------------
Attachment: hanger.zip
Here's the code that reproduces the bug each time it is run.
It's not exactly an example of good programming, but forgive me, it's a cut and
paste of other pieces of code from which i've extracted enough to reproduce the bug
and show the skeleton of how the services will be run.
To run them just launch ProducerStarter, wait for messages to expire and launch
ConsumerStarter.
Here it always expire 1024-1025 messages then everything hang.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:20 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Sergey Koshcheyev commented on JBMESSAGING-984:
-----------------------------------------------
Thanks a lot, I'll look at it soon.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Tim Fox
Priority: Blocker
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:33 a.m.
New subject: [JBoss JIRA] Updated: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=all ]
Tim Fox updated JBMESSAGING-984:
--------------------------------
Fix Version/s: 1.4.0.CR1
Assignee: Sergey Koshcheyev (was: Tim Fox)
Priority: Critical (was: Blocker)
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Sergey Koshcheyev
Priority: Critical
Fix For: 1.4.0.CR1
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:57 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Sergey Koshcheyev commented on JBMESSAGING-984:
-----------------------------------------------
Gabriele, I just wanted to inform you that Tim and I found the problem and will fix it
shortly.
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Sergey Koshcheyev
Priority: Critical
Fix For: 1.4.0.CR1
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:16 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Gabriele Garuglieri commented on JBMESSAGING-984:
-------------------------------------------------
Thank you so much.
I was sure i was betting on the right horse!
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Sergey Koshcheyev
Priority: Critical
Fix For: 1.4.0.CR1
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:22 a.m.
New subject: [JBoss JIRA] Commented: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=comments#action_12... ]
Tim Fox commented on JBMESSAGING-984:
-------------------------------------
I can't take credit for finding it. Sergey did this :)
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Sergey Koshcheyev
Priority: Critical
Fix For: 1.4.0.CR1
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:58 a.m.
New subject: [JBoss JIRA] Closed: (JBMESSAGING-984) Expiring messages hang JBoss server and client
[ http://jira.jboss.com/jira/browse/JBMESSAGING-984?page=all ]
Sergey Koshcheyev closed JBMESSAGING-984.
-----------------------------------------
Resolution: Done
Expiring messages hang JBoss server and client
----------------------------------------------
Key: JBMESSAGING-984
URL: http://jira.jboss.com/jira/browse/JBMESSAGING-984
Project: JBoss Messaging
Issue Type: Bug
Affects Versions: 1.3.0.GA
Environment: JBoss 4.2.0.GA, JBmessaging 1.3.0.GA, Java 1.6.0_01, Windows XP Pro
sp2, Postgres 8.2.4
Reporter: Gabriele Garuglieri
Assigned To: Sergey Koshcheyev
Priority: Critical
Fix For: 1.4.0.CR1
Attachments: hanger.zip, JBMESSAGING-984.zip
I think i've found a serious bug in messaging behaviour.
The problem is always reproducible with the following steps:
1- send a couple thousands of messages with a short expiration time to a queue (i used 20
sec to shorten the test time, the problem does not depend on expiration time value). I
used the /queue/testQueue
2- wait enough for the messages to expire
3- start a client that read from /queue/testQueue
4- the client won't receive any message because as messaging is getting messages from
channel 2 it expires them and updates them to channel 1
5- the problem is that it never gets through, after expiring approximately 1025 messages
(not exactly the same number in each run) everything gets stuck
The client locks in Connection.close(), that never returns (even if i kill the
server!!!).
JBoss server locks who knows where, such that if i try to orderly shutdown the server, it
won't terminate until i kill it, or if i try to open a new connection from a client to
server mostly of the times it will hang in ConnectionFactory.createConnection(), that will
never return.
The run for which i'm sending doc was done whith a fresh db.
Included you'll find:
server boot.log and server.log,
client thread dump while it was locked in Connection.close(),
server thread dump while it was locked after shutdown request,
csv dump of jbm_counter, jbm_msg, jbm_msg_ref, jbm_postoffice taken after all those weird
things happened
Thread dumps of client and server were taken with jstack utility since none of them were
responding to ctrl-break.
The messaging configuration was based on default server and all the attributes were left
at their default.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6977
days inactive
6981
days old
16 comments
3 participants
participants (3)
-
Gabriele Garuglieri (JIRA) -
Sergey Koshcheyev (JIRA)
-
Tim Fox (JIRA)