[JBoss JIRA] Created: (JBMESSAGING-1456) Messages stuck in being-delivered state in cluster
by Justin Bertram (JIRA)
Messages stuck in being-delivered state in cluster
--------------------------------------------------
Key: JBMESSAGING-1456
URL: https://jira.jboss.org/jira/browse/JBMESSAGING-1456
Project: JBoss Messaging
Issue Type: Bug
Reporter: Justin Bertram
Assignee: Tim Fox
Priority: Critical
Messages become "stuck" in being-delivered state when clients use a clustered XA connection factory in a cluster of at least 2 nodes.
JBoss setup:
-2 nodes of JBoss EAP 4.3 CP02
-commented out "ClusterPullConnectionFactory" in messaging-service.xml to prevent message redistribution and eliminate the "message suckers" as the potential culprit
-MySQL backend using the default mysql-persistence-service.xml (from <JBOSS_HOME>/docs/examples/jms)
Client setup:
-both nodes have a client which is a separate process (i.e. not inside JBoss)
-clients are Spring based
-one client produces and consumes, the other client just consumes
-both clients use the ClusteredXAConnectionFactory from the default connection-factories-service.xml
-both clients publish to and consume from "queue/testDistributedQueue"
-clients are configured to send persistent messages, use AUTO_ACKNOWLEDGE, and transacted sessions
Symptoms of the issue:
-when running the clients I watch the JMX-Console for the "queue/testDistributedQueue"
-as the consumers pull messages off the queue I can see the MessageCount and DeliveringCount go to 0 every so often
-after a period of time (usually a few hours) the MessageCount and DeliveringCount never go back to 0
-I "kill" the clients and wait for the DeliveringCount to go to 0, but it never does
-after the clients are killed the ConsumerCount for the queue will drop, but never to 0 when messages are "stuck"
-a thread dump reveals at least one JBM server session that is apparently stuck (it never goes away) - ostensibly this is the consumer that is showing in the JMX-Console for "queue/testDistributedQueue"
-a "killall -3 java" doesn't produce anything from the clients so I know their dead
-nothing is in any DLQ or expiry queue
-the database contains as many rows in the JBM_MSG and JBM_MSG_REF tables as the DeliveringCount in the JMX-Console
-rebooting the node with the stuck messages frees the messages to be consumed (i.e. un-sticks them)
Other notes:
-nothing else is happening on either node but running the client and running JBoss
-this only appears to happen when a clustered connection factory is used. I tested using a normal connection factory and after 24 hours couldn't reproduce a stuck message.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] Created: (JBSER-119) Ensure compatibility with Trove
by Christian Effertz (JIRA)
Ensure compatibility with Trove
-------------------------------
Key: JBSER-119
URL: https://jira.jboss.org/jira/browse/JBSER-119
Project: JBoss Serialization
Issue Type: Thirdparty Change
Affects Versions: 1.0.3 GA
Reporter: Christian Effertz
Assignee: Clebert Suconic
Fix For: 1.1.0 Beta
The version of [Trove|http://trove4j.sourceforge.net] that is shipped with the latest version of JBoss Serialization is quite old. Trove has had a major release since it was bundled last time for Version 1.0.3 and seems to be facing a next major version 3.0.
As it is stated in their release notes that they had some performance improvements and Bug Fixes I would assume that JBoss Serialization would profit in terms of stability and performance if using the newer versions.
I will give it a try in our dev system, but it would be more comforting to get a statement if your testsuite turned green if using the newer version.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] Created: (EJBTHREE-1330) EJB timer service should use a thread pool to avoid OOM
by Galder Zamarreno (JIRA)
EJB timer service should use a thread pool to avoid OOM
-------------------------------------------------------
Key: EJBTHREE-1330
URL: http://jira.jboss.com/jira/browse/EJBTHREE-1330
Project: EJB 3.0
Issue Type: Bug
Components: pool
Affects Versions: AS 4.2.2.GA
Reporter: Galder Zamarreno
Assigned To: Galder Zamarreno
Priority: Minor
The default EJB timer service used by the EJB3 layer is based on
org.jboss.ejb3.timerservice.jboss.JBossTimerServiceFactory which delegates
to the standard org.jboss.ejb.txtimer.EJBTimerService.
For EJB3 beans using the EJB timer service the thread local pool should not be used.
Since the current EJB timer service creates a new thread for each timer being created, the
thread local pool will create a matching instance of the bean for that thread. Thus the number
of active instances in total can effectively grow unchecked and thus an OOM will occur.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[JBoss JIRA] Created: (JBSER-117) Problem with JBoss serializing of dynamic proxies
by Maksym Ivanchenko (JIRA)
Problem with JBoss serializing of dynamic proxies
-------------------------------------------------
Key: JBSER-117
URL: https://jira.jboss.org/jira/browse/JBSER-117
Project: JBoss Serialization
Issue Type: Bug
Affects Versions: 1.0.3 GA
Reporter: Maksym Ivanchenko
Assignee: Clebert Suconic
When we try to serialize a proxy that contains some child entity which points to proxy itself the following exception occurs:
org.jboss.serial.exception.SerializationException: Object reference 1 was not found
Simple test case for problem reproducing:
import java.io.Serializable;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import org.jboss.serial.io.JBossObjectOutputStream;
import org.junit.Test;
public class StandaloneProxySerializationTest {
@Test
public void testSerialization() throws Exception {
ISampleEntity entity = new SampleEntityImpl();
Class<?>[] interfaces = new Class<?>[] {ISampleEntity.class};
SampleEntityInvocationHandler handler = new SampleEntityInvocationHandler(entity);
ISampleEntity proxy = (ISampleEntity) Proxy.newProxyInstance(Thread.currentThread()
.getContextClassLoader(), interfaces, handler);
ISampleEntity childEntity = new SampleEntityImpl();
proxy.setChild(childEntity);
childEntity.setParent(proxy);
JBossObjectOutputStream jbossSerializer = new JBossObjectOutputStream(null);
jbossSerializer.smartClone(proxy);
}
private static interface ISampleEntity {
public ISampleEntity getParent();
public void setParent(ISampleEntity parent);
public ISampleEntity getChild();
public void setChild(ISampleEntity child);
}
private static class SampleEntityImpl implements ISampleEntity, Serializable {
private static final long serialVersionUID = 1L;
private ISampleEntity parent;
private ISampleEntity child;
@Override
public ISampleEntity getParent() {
return parent;
}
@Override
public void setParent(ISampleEntity parent) {
this.parent = parent;
}
@Override
public ISampleEntity getChild() {
return child;
}
@Override
public void setChild(ISampleEntity child) {
this.child = child;
}
}
private static class SampleEntityInvocationHandler implements InvocationHandler, Serializable {
private static final long serialVersionUID = 1L;
private ISampleEntity entity;
public SampleEntityInvocationHandler(ISampleEntity entity) {
this.entity = entity;
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
return method.invoke(entity, args);
}
}
}
Upon ivestigating this issue we found that the problem is in ProxyPersister.readData() method. It reads the handler, the proxy class, constructs a proxy and only then puts it to cache. But if a proxy data contains proxy itself then it tries to find it by reference (which is not in cache yet, because the handler is still being read). So it throws "Object reference not found" exception.
We worked around this problem by introducing an InvocationHandler wrapper in ProxyPersister:
Patched ProxyPersister.java:
/*
* JBoss, Home of Professional Open Source
* Copyright 2005, JBoss Inc., and individual contributors as indicated
* by the @authors tag. See the copyright.txt in the distribution for a
* full listing of individual contributors.
*
* This is free software; you can redistribute it and/or modify it
* under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation; either version 2.1 of
* the License, or (at your option) any later version.
*
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this software; if not, write to the Free
* Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
* 02110-1301 USA, or see the FSF site: http://www.fsf.org.
*/
package org.jboss.serial.persister;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import org.jboss.serial.classmetamodel.ClassMetaData;
import org.jboss.serial.classmetamodel.StreamingClass;
import org.jboss.serial.exception.SerializationException;
import org.jboss.serial.objectmetamodel.ObjectSubstitutionInterface;
import org.jboss.serial.objectmetamodel.ObjectsCache;
/**
* $Id: ProxyPersister.java,v 1.11 2006/04/24 23:49:40 csuconic Exp $
*
* @author <a href="mailto:clebert.suconic@jboss.com">Clebert Suconic</a>
*/
public class ProxyPersister implements Persister {
private byte id;
public byte getId() {
return id;
}
public void setId(byte id) {
this.id = id;
}
public void writeData(ClassMetaData metaData, ObjectOutput output, Object obj, ObjectSubstitutionInterface substitution) throws IOException{
Object handler = Proxy.getInvocationHandler(obj);
output.writeObject(obj.getClass());
output.writeObject(handler);
}
static class InvocationHandlerWrapper implements InvocationHandler {
private InvocationHandler wrappedHandler;
public void setWrappedHandler(InvocationHandler wrappedHandler) {
this.wrappedHandler = wrappedHandler;
}
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
return wrappedHandler.invoke(proxy, method, args);
}
}
public Object readData (ClassLoader loader, StreamingClass streaming, ClassMetaData metaData, int referenceId, ObjectsCache cache, ObjectInput input, ObjectSubstitutionInterface substitution) throws IOException{
try
{
Class proxy = (Class)input.readObject();
Constructor constructor = proxy.getConstructor(new Class[] { InvocationHandler.class });
InvocationHandlerWrapper wrapper = new InvocationHandlerWrapper();
Object obj = constructor.newInstance(new Object[]{wrapper});
cache.putObjectInCacheRead(referenceId,obj);
Object handler = input.readObject();
wrapper.setWrappedHandler((InvocationHandler) handler);
return obj;
}
catch (ClassNotFoundException e)
{
throw new SerializationException(e);
}
catch (NoSuchMethodException e)
{
throw new SerializationException(e);
}
catch (IllegalAccessException e)
{
throw new SerializationException(e);
}
catch (InstantiationException e)
{
throw new SerializationException(e);
}
catch (InvocationTargetException e)
{
throw new SerializationException(e);
}
}
public boolean canPersist(Object obj)
{
// not implemented
return false;
}
}
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months