[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Howard Gao closed JBMESSAGING-1664. ----------------------------------- Resolution: Done done -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Howard Gao commented on JBMESSAGING-1664: ----------------------------------------- The paging can disrupt the order of message delivery in the following way: 1. Starting a queue with fullSize=20, pageSize=10, downCacheSize=10 2. Sending 35 persistent messages (m0 to m34) to the queue 3. The JBM_MSG_REF table will look like this: [code] +------------------+----------+ | MESSAGE_ID | PAGE_ORD | +------------------+----------+ | m0's id | NULL | | .... | NULL | | m19's id | NULL | | m20's id | 0 | | .... | 5 | | m29's id | 9 | | m30's id | NULL | | .... | NULL | | m34's id | NULL | +------------------+----------+ [/code] No the order of messages in the table is exactly the same order as they were added to the in-memory queue, only that m20 to m34 are paged and waiting to be loaded to the in-mem queue. Please note that messages m0 to m19 have NULL page_ord because they are in-memory messages, messages m20 to m29 have non-null page_ord values because they have been paged down to DB, messages m30 to m34 have NULL page_ord values because they are in the down cache and haven't yet been written to DB. If we at the moment shutdown and restart the server, the messages will be loaded from the DB in the same order as before. But if we cosume a message (m19) before shutdown, it's possible that the table may look like this [code] +------------------+----------+ | MESSAGE_ID | PAGE_ORD | +------------------+----------+ | m0's id | NULL | | .... | NULL | | m18's id | NULL | | m20's id | 0 | | .... | 5 | | m29's id | 9 | | m30's id | NULL | | .... | NULL | | m34's id | NULL | +------------------+----------+ [/code] Now if we start the server again, it will load m0 to m18, plus m30 into the in-memory queue and update the paging status. The key sql statement is this [code] LOAD_UNPAGED_REFS=SELECT MESSAGE_ID, DELIVERY_COUNT, SCHED_DELIVERY FROM JBM_MSG_REF WHERE STATE = 'C' AND CHANNEL_ID = ? AND PAGE_ORD IS NULL ORDER BY ORD [/code] This will cause m30 to be delivered before messages m20 to m29. So the order is broken. http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4244557#... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Tim Fox commented on JBMESSAGING-1664: -------------------------------------- Err, isn't the whole point of paging, that it's not possible (not enough RAM) to load *all* the messages into memory at once? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Tim Fox commented on JBMESSAGING-1664: -------------------------------------- That may be true for some databases but we can't assume they all support database cursors -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Tim Fox commented on JBMESSAGING-1664: -------------------------------------- Isn't the problem here because it's reloading *non persistent* messages on startup.. if you only loaded the persistent messages would the problem go away? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JBMESSAGING-1664?page=com.atlassian.ji... ] Howard Gao commented on JBMESSAGING-1664: ----------------------------------------- Currently I think I have two options. Option 1 is based on my current fix. To limit the memory when we fetch all the messages for a queue from DB, we can use Statement.setFetchSize(). Option 2 is that we split the query into two queries. First we query up to fullSize the messages for a Channel (queue), of which we can safely put those messages whose PAGE_ORD are null values into memory queue. Then we remember the number of messages (say n) having been loaded. Second we query all messages that have null PAGE_ORD values and, after dropping the first n messages, update their PAGE_ORD values. Option 1 is better. Option 2 is not so simple, as the first query may introduce new SQLs that have to use DB specific syntax. Here is the list of SQL for different DBs: SELECT * FROM T LIMIT 10 OFFSET 20 MySQL, PostgreSQL (also supports the standard, since version 8.4), SQLite, H2 SELECT * from T WHERE ROWNUM <= 10 Oracle (also supports the standard, since Oracle8i) SELECT FIRST 10 * from T Ingres SELECT FIRST 10 * FROM T order by a Informix SELECT SKIP 20 FIRST 10 * FROM T order by c, d Informix (row numbers are filtered after order by is evaluated. SKIP clause was introduced in a v10.00.xC4 fixpack) SELECT * FROM T FETCH FIRST 10 ROWS ONLY DB2 (also supports the standard, in Linux, Windows, and Unix since DB2 v8, z/OS support added in v9) SELECT TOP 10 * FROM T MS SQL Server (also supports the standard, since SQL Server 2005), Sybase ASE, MS Access SELECT TOP 10 START AT 20 * FROM T Sybase SQL Anywhere (also supports the standard, since version 9.0.1) SELECT FIRST 10 SKIP 20 * FROM T Interbase, Firebird SELECT * FROM T ROWS 20 TO 30 Firebird (since version 2.1) http://en.wikipedia.org/wiki/Select_(SQL) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira