"clebert.suconic(a)jboss.com" wrote : I did some work to make sure the LargeMessage sending will be replicated to the backup nodes. | | On my initial work, I was having a lot of problems on getting the LargeMessage to be replicated and sent on both nodes. I was aways getting an Out-of-credits on the backup when replicating the message, causing issues on replicating the ACKs and properly failing over to the backup. | | | Changes I had to make to fix the issues on LargeMessage and Failover: | | 1) I simple fix to sendLargeMessage | | | | | On ServerConsumerImpl:: | | | | private void sendLargeMessage(final MessageReference ref, final ServerMessage message) | | { | | // TODO: Should we block until the replication is done? | | channel.replicatePacket(new SessionReplicateDeliveryMessage(id, message.getMessageID(), message.getDestination())); | | | | // SendLargeMessage has to be done on the same thread used on the QueueImpl or we would have problems with ordering and flow control | | largeMessageSender = new LargeMessageSender((LargeServerMessage)message, ref); | | largeMessageSender.sendLargeMessage(); | | | | } | | | | | This above code used to wait the replication to finish before sending the LargeMessage, what would use another thread. The queue would continue its work asynchronously and the Consumer would eventually handle another message while sendLargeMessage was still processing, what would cause issues on the backup node (not enough credits). | When you say "send" I'm assuming you really mean "deliver". (Send means client->server, deliver means server->client). Your usage of send here is confusing. For non large messages, delivery to the client only occurs when the replication response has come back from the server. If you just do it the same as that, then should be ok? anonymous wrote : | 2) When taking credits on backup, we will eventually resume sending the largeMessage. | That process needs to be done synchronously while receiving the credit from the live node. If we play the commands on a different order we would eventually reject messages because of credit during the replication, or because we would still processing a largeMessage. | | | promptDelivery is now calling resumeLargeMessage if largeMessageSender != null | Sorry, didn't understand that explanation. anonymous wrote : | | 3) While receiving LargeMessages, the client will send credits back, as soon as the chunk is received on the client. | For that handleLargeMessageContinuation, will call flowControl. | | While flowcontrol is being called, handleLargeMessageContinuation caller will be holding a lock of the ClientRemoteConsumer. | What is ClientRemoteConsumer? anonymous wrote : | As a result, we will have a dead lock if failover happens while flowControl is being called within handleLargeMessageContinuation. | | | I am now using an executor for the flowControl, when receiving LargeMessages. This way I release the lock on the ClienteRemoteConsumers while sending the flowcontrol back, so Failover will be able to perform outside of the locks. | | | | | All these changes are only applied on https://svn.jboss.org/repos/messaging/branches/Branch_Failover_Page | View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4201702#... Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...