[Fwd: Re: Fwd: [jboss-dev] Project ByteMan]
by Andrew Dinn
Sorry replied to the forward before I saw the post on jboss-dev
-------- Original Message --------
Subject: Re: Fwd: [jboss-dev] Project ByteMan
Date: Tue, 23 Jun 2009 12:29:13 +0100
From: Andrew Dinn <adinn(a)redhat.com>
To: Kabir Khan <kabir.khan(a)jboss.com>
Hi Kabir,
I am in the middle of setting up the byteman project at the moment
(Jazoon talk tomorrow, so plenty of tie yet ;-). There should be a
project page at jboss.org/byteman by the end of today and a forum plus
JIRA very soon.
If you want to obtain the latest release then you can check it out from
the current temporary svn home at
http://anonsvn.jboss.org/repos/labs/labs/jbosstm/workspace/adinn/byteman/...
There is a binary zip containing the byteman jar and README/user guide
and a zip containing the full sources and test code.
regards,
Andrew Dinn
-----------
On 06/22/2009 07:00 PM, Kabir Khan wrote:
> dev list seems broken
>
> Begin forwarded message:
>
>> From: Kabir Khan <kabir.khan(a)jboss.com>
>> Date: 22 June 2009 18:19:00 BST
>> To: "JBoss.org development list" <jboss-development(a)lists.jboss.org>
>> Subject: Re: [jboss-dev] Project ByteMan
>>
>> Is ByteMan in the maven repository somewhere? I'd like to try it out
>> to test parallel deployments in MC
>> On 11 Jun 2009, at 15:08, Andrew Dinn wrote:
>>
>>> Galder Zamarreno wrote:
>>>> Andrew, did you get around to compare ByteMan with JBoss AOP?
>>>
>>> Yes, I had a closer look at AOP.
>>>
>>>> IMO, both seem to do the same, instrument classes to add some
>>>> behaviour here or there. ByteMan simply seems to use load time
>>>> weaving style whereas JBoss AOP had two modes.
>>>
>>> Yes, I think you can use AOP to inject code in all the same places that
>>> ByteMan can inject code. ByteMan does make local variables available
>>> for use in the injected code which I don't think AOP can do but
>>> that's not the most important feature of either.
>>>
>>>> So, why should I use ByteMan over JBoss AOP if JBoss AOP is already
>>>> integrated with AS? Easy of use? Memory consumption? Speed?
>>>> Functionality?
>>>
>>> Well, firstly, when it comes to testing via fault injection ByteMan
>>> is a lot simpler and easier to use than AOP not that I am thereby
>>> knocking AOP). You don't have to define any classes or compile any
>>> code in order to use Byteman. The injected side effects can simply be
>>> written into the rule script as Java code or built-in helper method
>>> calls. Mostly when you are testing code you want to tweak the current
>>> behaviour and that doesn't normally need you to write a lot of code.
>>> Usually, most of the behaviour you need to change is either simple to
>>> script using a few basic Java operations and/or the public API of the
>>> classes you are testing.
>>>
>>> However, sometimes the desired side-effects include operations which
>>> are not simple to script and ByteMan provides help in that area too.
>>> The extra features (beyond calls to Java operations or application
>>> APIs) that ByteMan provides out of the box are exposed via the Helper
>>> class. This class implements the default built-in methods available
>>> for use in scripts. These built-ins allow you to write simple, clear
>>> rules which do many of the things that are needed during testing.
>>>
>>> The default Helper provides 3 distinct sets of operations for:
>>> coordinating the timing of independent threads; creating and managing
>>> rule state from one rule triggering to the next; and generating
>>> output to trace progress of a test. I specifically implemented the
>>> first set to help enforce normal and abnormal timings during XTS
>>> crash recovery testing and they have been very useful when it comes
>>> to ensuring that tests run with both expected and unexpected
>>> interleavings. The second set allows scripts to succinctly express
>>> quite complex scenarios. The third set is just a basic way of dumping
>>> trace output to files.
>>>
>>> Now you _could_ implement the same functionality as a library to be
>>> called from AOP injected code. In fact its easy to do so because it's
>>> just the public API of a single class that I defined. But I don't
>>> think the resulting AOP-based code would be as quick to write, test
>>> and change, nor as clear and easy for others to read and follow. The
>>> rules I used in the TS tests are few in number, concise and express
>>> directly how they modify the behaviour of the application code.
>>> Anyone who understands the application can easily follow how these
>>> rules implement the desired test scenario.
>>>
>>> The same concern for clarity, simplicity and flexibility led me to
>>> provide support for redefinition of the helper class. I envisage that
>>> when testing a specific application there will be the need to perform
>>> common operations not contained in the set I provided. As with XTS
>>> these operations will be required in order to to set up, maintain and
>>> monitor test conditions across multiple triggerings and in different
>>> tests. By defining a helper class to encapsulate those operations you
>>> can still employ small, simple and clear rule sets to define the test
>>> scenarios.
>>>
>>> As regards performance, I don't yet know how well Byteman performs. I
>>> have not yet had a chance to test performance of either the trigger
>>> injection code or the rule type checker/compiler. The former is very
>>> efficient in that it avoids _any_ work if a loaded class does not
>>> match a rule in the current rule set i.e. the target class and target
>>> method do not hash to values mentioned in the rule base or. If they
>>> do match, then a very simple bytecode scan filters out methods which
>>> do not contain a location matching the rule specification (e.g. there
>>> is no read of a field call name).
>>>
>>> If a method does match a rule's class, method and lcocation then the
>>> transformer code still only performs a relatively simple modification
>>> of the bytecode. It injects a single call to the rule engine at each
>>> trigger location and provides a catch block for the potential
>>> exceptions thrown by that call. This involves a single walk through
>>> the bytecode for each matched rule. It's not quite so simple as just
>>> throwing in a few LOADS and an INVOKE_STATIC call since doing the
>>> exception handling correctly requires identifying enough of the
>>> control flow to ensure that synchronized blocks are exited cleanly.
>>> But I still expect this transformation to be very fast (if it is not
>>> then of course we could probably switch to using AOP to do the job,
>>> but I don't suppose that is likely to do a lot better ;-)
>>>
>>> The type checker is pretty simple also, especially if the rules are
>>> kept compact. The overhead of type checking (and compilation to
>>> bytecode) are determined by the complexity of the rule body since
>>> both involve little beyond a simple walk of the rule parse tree
>>> (another reason to define custom helpers for test-specific
>>> operations). Once again I expect them to be very fast. Type checking
>>> is done at first execute so any cost is not incurred unless and until
>>> a rule is triggered. By default rules are run interpreted. I have
>>> found with my tests that rules are triggered only once or twice and I
>>> envisage that this will be common since setting up and tarcing test
>>> scenarios usually requires a small number of tweaks at very specific
>>> points in the code. However, rule compilation i.e. translation of the
>>> rule body to invokable bytecode has now been implemented so it should
>>> help in cases where rules are triggered frequuemtly. The initial crop
>>> of obvious bugs have been fixed and I'll be happy to get my teeth
>>> into the harder ones as they turn up (so please break it :-).
>>>
>>> regards,
>>>
>>>
>>> Andrew Dinn
>>> -----------
>>> _______________________________________________
>>> jboss-development mailing list
>>> jboss-development(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jboss-development
>>
>
--
JBoss, a Division of Red Hat
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire,
SI4 1TE, United Kingdom.
Registered in UK and Wales under Company Registration No. 3798903
Directors: Michael Cunningham (USA), Charlie Peters (USA), Matt Parsons
(USA) and Brendan Lane (Ireland)
--
JBoss, a Division of Red Hat
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire,
SI4 1TE, United Kingdom.
Registered in UK and Wales under Company Registration No. 3798903
Directors: Michael Cunningham (USA), Charlie Peters (USA), Matt Parsons
(USA) and Brendan Lane (Ireland)
16 years, 4 months
Scanning jars for package capabilities
by Jaikiran Pai
Resending with a zipped snapshot. Did not realize that the earlier html
attachment was large.
I was just looking at the profiler output against AS trunk today and
noticed that considerable amount of time is being spent in the
AbstractClassLoaderDeployer.deploy which internally creates a
classloader for deployers/deployments. Internally a
VFSDeploymentClassLoaderPolicyModule goes on to determine the
"capabilities" of each module. This include traversing the jar file(s)
to find out what "packages" are contained. Attached is the profiler
snapshot.
I was thinking whether we could reduce this time by having a INDEX.LIST
file http://java.sun.com/j2se/1.5.0/docs/tooldocs/solaris/jar.html#i (or
something similar) which would list the packages available/exposed in a
jar file (sample generated file attached for jboss-common-core.jar). We
cannot enforce this on end user deployments but atleast the jars shipped
by the server probably could include this? This probably will have its
own issues in terms of maintaining that list so that it does not become
outdated. We could use the jar -i command to create the index files
everytime a jar is generated, but again needs to be followed by each
project.
Is this something that we should be looking into? Or are there any
better ways of dealing with this?
-Jaikiran
16 years, 4 months
Project ByteMan
by Andrew Dinn
Hi Foax,
After several false starts and concomitant rethinks legal have finally
agreed that I can use the name ByteMan (aka JBoss ByteMan) for the tool
formerly known as TOAST. The name abbreviates 'Byte Manipulation' rather
than identifying some nerdy superhero -- although I am still toying with
the idea of presenting at Jazoon in mask, tights and cape. Mark Newton
is currently in the process of setting up a ByteMan project with
repository, forum etc. In the meantime I would like to claim the package
space org.jboss.byteman for all code products associated with the
project. I believe this does not clash with any existing code. If you
know otherwise please inform me ASAP.
regards,
Andrew Dinn
-----------
16 years, 4 months
Interesting JBAS profiling results
by David M. Lloyd
I was talking to Jason G on the phone, and he had an interesting idea: what
happens when we profile the AS and look at the results by package rather
than method? This should give us a rough idea, by project, of where our
startup time is being spent. So I took a couple minutes and did just that,
with some interesting results.
First, results for a minimal startup (results under 2% omitted):
9.7% - 2,418 ms - 41,306 hot spot inv. org.jboss.aop.pointcut
8.3% - 2,063 ms - 38,480 hot spot inv. org.jboss.reflect.plugins.introspection
5.9% - 1,461 ms - 373,799 hot spot inv. org.jboss.aop
5.7% - 1,415 ms - 52,528 hot spot inv. org.jboss.virtual.plugins.context.zip
5.7% - 1,411 ms - 294,922 hot spot inv. org.jboss.virtual.plugins.context
5.0% - 1,245 ms - 672,024 hot spot inv. org.jboss.aop.pointcut.ast
3.6% - 893 ms - 7,928 hot spot inv. org.jboss.classloader.spi.base
3.0% - 735 ms - 17,241 hot spot inv. org.jboss.dependency.plugins
2.9% - 720 ms - 20,215 hot spot inv. org.jboss.aop.util
2.8% - 697 ms - 150,242 hot spot inv. org.apache.log4j
2.6% - 636 ms - 272,879 hot spot inv. org.jboss.reflect.plugins
2.2% - 550 ms - 72,341 hot spot inv. org.jboss.metadata.plugins.context
As you can see, a minimal startup is dominated by AOP with VFS being a
not-so-close second.
Next, a default startup, which shows somewhat different behavior:
10.5% - 25,156 ms - 5,827,279 hot spot inv. org.jboss.virtual.plugins.context
8.5% - 20,320 ms - 1,232,357 hot spot inv.
org.jboss.virtual.plugins.context.zip
6.6% - 15,822 ms - 1,054,169 hot spot inv.
org.jboss.virtual.plugins.vfs.helpers
6.4% - 15,319 ms - 297,465 hot spot inv. org.jboss.aop.pointcut
5.9% - 14,256 ms - 237,457 hot spot inv. org.jboss.dependency.plugins
5.1% - 12,148 ms - 6,769,258 hot spot inv. org.jboss.metadata.spi.scope
4.7% - 11,254 ms - 72,717 hot spot inv. javassist.bytecode
4.2% - 10,132 ms - 8,205,516 hot spot inv. org.jboss.dependency.spi
3.5% - 8,426 ms - 2,308,354 hot spot inv. org.jboss.aop
3.2% - 7,713 ms - 4,109,176 hot spot inv. org.jboss.aop.pointcut.ast
2.7% - 6,575 ms - 182,581 hot spot inv. org.jboss.reflect.plugins.introspection
2.7% - 6,366 ms - 52,818 hot spot inv. org.jboss.classloader.spi.base
2.2% - 5,284 ms - 333,613 hot spot inv. org.jboss.mx.server
VFS, AOP, metadata, classloading. If I separate out filtered classes, it's
basically the same story:
17.4% - 42,348 ms - 53,786,066 hot spot inv. java.lang
8.4% - 20,408 ms - 5,827,335 hot spot inv. org.jboss.virtual.plugins.context
8.2% - 19,943 ms - 21,005,518 hot spot inv. java.util
5.6% - 13,706 ms - 1,232,359 hot spot inv.
org.jboss.virtual.plugins.context.zip
4.5% - 11,068 ms - 297,465 hot spot inv. org.jboss.aop.pointcut
3.5% - 8,531 ms - 72,717 hot spot inv. javassist.bytecode
3.4% - 8,266 ms - 6,769,258 hot spot inv. org.jboss.metadata.spi.scope
3.3% - 8,106 ms - 1,054,287 hot spot inv. org.jboss.virtual.plugins.vfs.helpers
3.2% - 7,754 ms - 237,461 hot spot inv. org.jboss.dependency.plugins
2.8% - 6,787 ms - 8,205,588 hot spot inv. org.jboss.dependency.spi
2.4% - 5,964 ms - 2,308,354 hot spot inv. org.jboss.aop
2.3% - 5,524 ms - 52,818 hot spot inv. org.jboss.classloader.spi.base
2.2% - 5,316 ms - 4,109,176 hot spot inv. org.jboss.aop.pointcut.ast
So it looks to me like we ought to be focusing our optimization efforts on
VFS and AOP, maybe classloading as well. I encourage folks to play around
with profiling this way; it's fairly illuminating.
- DML
16 years, 4 months
Recording build environment information
by Paul Gier
I put some information on the wiki about recording information about the build
environment in the jar manifest files [1]. The wiki page shows how to add
manifest entries for information like svn revision, build timestamp, java
version, etc. I would like to encourage everyone who is using Maven for their
builds to take a look.
All the jars generated by the AS6 build have this information included, and I'm
considering adding this to the jboss parent pom so that sub projects
automatically get this configuration.
[1]http://www.jboss.org/community/wiki/MavenRecordingBuildInformation
16 years, 4 months
Pass @Parameter value to a TestNG test from Eclipse?
by Galder Zamarreno
Hi,
I've got a question regarding TestNG and Eclipse. Does anyone know how
to pass the value of a @Parameter to a test from Eclipse? I've got a
test that has:
@Parameters({"basedir"})
protected void setUpTempDir(String basedir) {
...
But I can't execute it from Eclipse cos it didn't get the basedir
parameter. So, I was wondering if there's any way to pass such parameter
from Eclipse.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years, 4 months
Improving scanning performance
by Bill Burke
I suggested this a year or two ago:
* You should have a AnnotationScanner deployer that scans jars for
annotations and puts an annotation database somewhere in memory that
other deployers can reference. (This might already exist, I don't know).
* If a file META-INF/no.scan exists in a jar, then don't scan the file.
This allows libraries to exclude themselves from scanning.
* If a file META-INF/scan.only exists it will contain a newline
delimited list of class names to scan. Only scan those class names.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
16 years, 4 months
Re: [jboss-dev] AS Embedded
by Joshua Davis
Andrew & Pete,
Thanks for the info. The two issues I encounter on a regular basis are: 1) ConcurrentModificationException during deployment, and 2) Weird 'delayed delivery' with JBM.
Guess I'd better start reading if I want to unit test my app. :)
- Josh
----- Original Message ----
From: Pete Muir <pmuir(a)redhat.com>
To: JBoss.org development list <jboss-development(a)lists.jboss.org>
Cc: Joshua Davis <pgmjsd(a)yahoo.com>
Sent: Tuesday, June 16, 2009 11:29:39 AM
Subject: Re: [jboss-dev] AS Embedded
Josh,
As Andy says, what you are looking at is legacy Embedded, which is just maintained for old apps - we've seen this JBM problem too. There are no plans to fix it.
Pete
On 16 Jun 2009, at 17:11, Andrew Lee Rubinger wrote:
> Hi Josh:
>
> Embedded is undergoing an overhaul at the moment, @see this Thread:
>
> http://lists.jboss.org/pipermail/jboss-development/2009-April/thread.html...
>
> Most recently we've been bringing Application Server into a new bootstrap component from which the embedded launcher may extend. This month we're seeing some prototypes, but I still need to align these with trunk.
>
> If you're interested in being part of the development, I've got a few JIRA tasks in line. :)
>
> http://www.jboss.org/index.html?module=bb&op=viewforum&f=266
>
> S,
> ALR
>
> On 06/16/2009 09:08 AM, Joshua Davis wrote:
>> Hello everyone.
>>
>> I'd like to help out by testing AS Embedded if that's possible. I'm currently using Embedded beta3.SP5 and I'm experiencing some intermittent deployment failures and JBoss Messaging problems. How do I get started?
>>
>> - Josh Davis
>> _______________________________________________
>> jboss-development mailing list
>> jboss-development(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jboss-development
>
> --Andrew Lee Rubinger
> Sr. Software Engineer
> JBoss by Red Hat
> http://exitcondition.alrubinger.com
> _______________________________________________
> jboss-development mailing list
> jboss-development(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-development
16 years, 4 months
AS Embedded
by Joshua Davis
Hello everyone.
I'd like to help out by testing AS Embedded if that's possible. I'm currently using Embedded beta3.SP5 and I'm experiencing some intermittent deployment failures and JBoss Messaging problems. How do I get started?
- Josh Davis
16 years, 4 months
