[jboss-dev-forums] [JBoss Transactions Development] - dynamic reconfiguration

[Jonathan Halliday]

Jonathan Halliday [http://community.jboss.org/people/jhalliday] created the discussion

"dynamic reconfiguration"

To view the discussion, visit: http://community.jboss.org/message/623969#623969

--------------------------------------------------------------
The configuration mechanism in JBossTS has evolved over the years, most recently to use a set of javabeans to hold the myriad config values. There are a dozen or so EnvironmentBeans, roughly one per module, and a central repository (BeanPopulator) that holds a singleton instance of each. The singletons are instantiated on demand with initial values populated from an embedded properties file or system properties. These defaults can then be overridden programatically, either by a bean wiring framework using xml (as in older JBossAS transaction-jboss-beans.xml) or in java code using the bean's setter methods (as in AS6 ArjunaTransactionManagerService etc). So far so good - this approach provides type safety for the values as well as compatibility with modern DI frameworks and is a big improvement over what we has before.

It still has one significant shortcoming though: once the transaction system is initialized, any subsequent config changes are ignored. That would not be so bad, except that the initialization can happen very early. Specifically we have a lot of static initializers that read and cache config values at class load time. e.g.

{code}
    public class SomeThing
    {
        private static final boolean XA_TRANSACTION_TIMEOUT_ENABLED;

        static
        {
            XA_TRANSACTION_TIMEOUT_ENABLED = jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled();
        }

        public void doThing()
        {
            XA_TRANSACTION_TIMEOUT_ENABLED ? thing() : otherThing();
        }
    }

    public class jtaPropertyManager
    {
        public static JTAEnvironmentBean getJTAEnvironmentBean()
        {
            return BeanPopulator.getDefaultInstance(JTAEnvironmentBean.class);
        }
    }

    public class BeanPopulator
    {
        private static final ConcurrentMap<String, Object> beanInstances = new ConcurrentHashMap<String, Object>();

        public static <T> T getDefaultInstance(Class<T> beanClass) throws RuntimeException {
            return (T) beanInstances.get( beanClass.getName() );
        }
    }
{code}

This provides very fast runtime execution, but a call to e.g.

{code}jtaPropertyManager.getJTAEnvironmentBean().setXaTransactionTimeoutEnabled(true);{code}

must come before the classload of SomeThing if it's going to have any effect.

So what are the alternatives?

Option A:

{code}
    public class SomeThing
    {
        public void doThing()
        {
            if(jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled())
            {
                thing()
            }
            else
            {
                otherThing();
            }
        }
    }
{code}

looks attractive at first glance. However the system is now so highly tuned that the runtime overhead of the underlying hashmap lookup for
{code}beanInstances.get( beanClass.getName() );{code} is going to cause significant performance impact, see e.g. JBTM-853.

Additionally, usage patterns of the form

{code}
        public class SomeThing
        {
            public void doThing()
            {
                if(jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled())
                {
                    thing()
                }

                unrelatedStuff();

                if(jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled())
                {
                    relatedThing();
                }
            }
        }
{code}

will need to be hunted down and replaced with

{code}
        public class SomeThing
        {
            public void doThing()
            {
                boolean cachedConfig = jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled();

                if(cachedConfig)
                {
                    thing()
                }

                unrelatedStuff();

                if(cachedConfig)
                {
                    relatedThing();
                }
            }
        }
{code}

including all the more complex cases where the usages are in the same call chain but not necessarily the same method, class or even package.

Option B:

{code}
        public class SomeThing
        {
            private final ConfigBeanHolder config;

            public SomeThing(ConfigBeanHolder config)
            {
                this.config = config;
            }

            public void doThing()
            {
                if(config.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled())
                {
                    thing()
                }

                unrelatedStuff();

                if(config.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled())
                {
                    relatedThing();
                }
            }
        }
{code}

which eliminates the expensive hashmap operation and potentially allows us to do nifty stuff like running multiple different configurations in parallel e.g.

{code}
        jtaThing = new SomeThing(jtaConfig)
        jtsThing = new SomeThing(jtsConfig)

        if( needDistributedTx ) {
            useThing( jtsThing );
        } else {
            useThing( jtaThing );
        }
{code}

However, we would need to retrofit the entire codebase with suitable constructors as well as hunting down the repeated usages of the same parameter as mentioned for the previous option.

Option C:

{code}
    public class SomeThing
    {
        private static boolean XA_TRANSACTION_TIMEOUT_ENABLED;

        static
        {
            XA_TRANSACTION_TIMEOUT_ENABLED = jtaPropertyManager.getJTAEnvironmentBean().isXaTransactionTimeoutEnabled();

            jtaPropertyManager.getJTAEnvironmentBean().registerPropertyChangeListener(
                new PropertyChangeListener() {
                    public void propertyUpdated(String name, Object value) {
                        if("XA_TRANSACTION_TIMEOUT_ENABLED".equals(name) {
                            XA_TRANSACTION_TIMEOUT_ENABLED = value;
                        }
                    }
                }
        }

        public void doThing()
        {
            XA_TRANSACTION_TIMEOUT_ENABLED ? thing() : otherThing();
        }
    }
{code}

which should be fairly quick at runtime but introduces a lot of boilerplate code and still suffers from some of the problems identified for other options.


There are additional concerns that affect all options. For example, some properties cannot be considered in isolation as they interact with others:

{code}
    public class SomeThing
    {
        private Object someProperty;
        private Object relatedProperty;

        public boolean isConfigValid()
        {
            return someProperty.equals(relatedProperty);
        }
    }
{code}

maintaining validity in the face of mutability would require e.g.

{code}
    public class SomeThing
    {
        private final ImmutableConfigHolder config;

        public void updateConfig(ImmutableConfigHolder replacementConfig) {
            config = replacementConfig;
        }

        public boolean isConfigValid()
        {
            return config.getSomeProperty().equals(config.getRelatedProperty());
        }
    }
{code}

which given the complex interrelationships between properties may bean we can't effectively decouple individual module's configs from one another any more and wind up with a single huge environment bean containing all the properties.

Finally, we need to consider the problems that arise in trying to support a system that is reconfigurable. The content of the properties file is no longer sufficient to determine the runtime config. To diagnose or reproduce issues we'd additionally require either a comprehensive log of runtime property changes or individual error log messages to be retrofitted to contain a dump of any relevent config option values.

The final option is a bit of a wildcard: don't change anything in the JBossTS codebase at all. Instead, use the JBossMC module system to effectively isolate multiple copies of the transaction system with different configs and hot swap between them to effect a config change. That's not without its problems, particularly where recovery is concerned, but they may turn out to be more tractable or at least less labor intensive than solutions that require wholesale audit and refactoring of the entire JBossTS codebase.  On the other hand if we plan to substantially refactor the codebase anyhow then making changes to the config system along the way may not be so bad.

We can also potentially tackle the issue on a property by property basis, giving priority to those options for which dynamic reconfiguration has most benefit. Or go for a more limited objective of keeping the config static but delaying its effective freeze beyond classloading time, for example by requiring an explicit lifecycle with an instantiate/start step that snapshots the config values at that point and builds the system using them.

Thoughts?
--------------------------------------------------------------

Reply to this message by going to Community
[http://community.jboss.org/message/623969#623969]

Start a new discussion in JBoss Transactions Development at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2041]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-dev-forums/attachments/20110831/dbada944/attachment.html