Srcdeps in WildFly and WildFly Core
by Peter Palaga
Hi *,
this is not new to those of you who attended my talk on the F2F 2016 in
Brno. Let me explain the idea here again for all others who did not have
a chance to be there.
Srcdeps [1] is a tool to build Maven dependencies from their sources.
With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
undertow:
<version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
where aabbccd is the git commit id to build when any undertow artifact
is requested during the build of wildfly-core.
[1] describes in detail, how it works.
The main advantage of srcdeps is that changes in components can be
integrated and tested in wildfly-core immediately after they are
committed to a public component branch. There is no need to wait for the
component release.
Here in the WildFly family of projects, it is often the case that
something needs to be fixed in a component, but the verification (using
bug reproducer, or integration test) is possible only at the level of
wildfly or wildfly-core. Engineers typically work with snapshots
locally, but when their changes need to get shared (CI, reviews) in a
reproducible manner, snapshots cannot be used anymore.
In such situations a source dependency come in handy: it is very easy to
share and it is as reproducible as a Maven build from a specific commit
can be. All CIs and reviewers can work with it, because all source
dependency compilation is done under the hood by Maven.
Developers working on changes that span over multiple interdependent git
repos can thus get feedback (i-tests, reviews) quickly without waiting
for releases of components.
Srcdeps emerged in the Hawkular family of projects to solve exactly this
kind of situation and is in use there since around October 2015.
When I said there is no need to wait for releases of components, I did
not mean that we can get rid of component releases altogether. Clearly,
we cannot, because i.a. for any tooling uninformed about how srcdeps
work, those source dependencies would simply be non-resolvable from
public Maven repositories. So, before releasing the dependent component
(such as wildfly-core) all its dependencies need to be released. To
enforce this, srcdeps is by default configured to make the release fail,
as long as there are source dependencies.
I have sent a PR introducing srcdeps to wildfly-core:
https://github.com/wildfly/wildfly-core/pull/2122
To get a feeling how it works, checkout the branch, switch to e.g.
<version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
(that happens to be the commit id of the 1.4.8.Final tag)
and build wildfly-core as usual with "mvn clean install". You'll see in
the build log that undertow is being cloned to ~/.m2/srcdeps/io/undertow
and that it is built there. After the build, check that the
1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
version of Undertow got installed to your local Maven repo (usually
~/m2/repository/io/undertow/undertow-core )
Are there any questions or comments?
[1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
Thanks,
Peter
P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at 14:30 at
DevConf Brno.
3 years, 3 months
Apache Lucene modules for WildFly : evolution
by Sanne Grinovero
# Context
historically the Hibernate Search project released modules for
WildFly, so that people could use the latest version of it w/o having
to wait for WildFly to upgrade it and release.
This zip we distribute would also include updated modules for Apache Lucene.
We have now separated the Lucene modules into their own repository, so
that other projects (e.g. Infinispan) can use the same modules.
In case you want to have a look, the POC is hosted at:
- https://github.com/hibernate/lucene-modules
We planned to deploy this zip file as a Maven artifact under
coordinates "org.hibernate.lucene-modules:lucene-modules:lucene-modules".
# I have three questions:
## Would it be useful to make this a "Feature Pack" ?
Ideally we would like future versions of WildFly to use the same
structure (and same content) of these modules, when it will come to
upgrade the Lucene version within WildFly.
That's because we will have already tested these in (at least) the
Hibernate and Infinispan projects, while replicating the same
structure within the WildFly build would not guarantee the same
output, and would not be covered by the same amount and quality of
integration tests.
## Would you prefer this to be hosted as github.com/wildfly ?
We'd be happy to transfer ownership and still maintain it: having this
under "Hibernate" probably sends the wrong message.
It contains no Hibernate code and we'd like to encourage other people
possibly using Lucene on WildFly to use the same modules, again for
consistency reasons.
## Should we use a WildFly flavoured organization id?
For the same reasons as in the previous question, yet I ask this
separately as I guess we could still keep it on github/hibernate if
you prefer, yet distribute it as "org.wildfly:lucene-modules" or
something else you might want to suggest.
Thanks,
Sanne
3 years, 6 months
IPv6 related utility classes in Wildfly / Elytron / ...
by Tomas Hofman
Hello,
I'm working on an issue in Elytron, which requires validation of IPv6
addresses. Potentially also normalizing IPv6 addresses and matching
addresses by masks could be needed in future. For now I implemented my
own validation code which may be good enough, but still, better would be
to rely on proven utility class.
Do we use some standard library for such purposes, or is there such
functionality in JBoss codebase somewhere? I found NetworkUtils class in
wildfly-network package, which does some of that, but that can't be used
in Elytron.
Are there some rules when considering new dependencies? For instance
would Apache Commons Validator/Net represent conceivable dependencies
for Elytron?
Thanks,
Tomas
3 years, 6 months
Latest WildFly Elytron / Invocation Integration Test Failures 2017-02-27
by Darran Lofthouse
Here are the latest results for Elytron / Invocation related test
failures: -
https://ci.wildfly.org/viewLog.html?buildId=47710&buildTypeId=WF_MasterNe...
We currently stand at 19 failures.
org.jboss.as.test.integration.ejb.client.descriptor.EJBClientDescriptorTestCase.testClientInvocationTimeout
org.jboss.as.test.integration.jpa.epcpropagation.EPCPropagationTestCase.testNoTxEPCPropagation
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testDir
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testLdap
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleTestCase.testNotAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleTestCase.testAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleUseNewClientCertTestCase.testNotAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleUseNewClientCertTestCase.testAuthorizedClient
org.jboss.as.test.clustering.cluster.dispatcher.CommandDispatcherTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.provider.ServiceProviderRegistrationTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.registry.RegistryTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testGracefulShutdownConcurrentFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatefulFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testSecureStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testClientException
org.jboss.as.test.clustering.single.provider.ServiceProviderRegistrationTestCase.test
org.jboss.as.test.clustering.single.dispatcher.CommandDispatcherTestCase.test
org.jboss.as.test.clustering.single.registry.RegistryTestCase.test
3 years, 6 months
Preventing duplicate registration of capabilities
by Brian Stansberry
tl;dr;
Currently on a server (let’s ignore an HC as it’s a tangent) we allow the same capability to be registered by more than one resource. In the large majority but not all cases a user doing this has made a mistake we should catch. I plan to make whether this is allowed configurable by the capability author. I *propose* making the default behavior ‘false’ which which is a behavior change and will require some cleanup.
Only user impact is if some capability needs to allow multiple registrations and doesn’t pick this up and configure it, and that isn’t caught before release.
Long version
In most cases a server configuration should only have a single resource that registers a given capability. If more than one registers it, which really provides it? More than one will likely fail due to conflicts over things like MSC service names.
But, the capability registry actually allows more than one registration point (i.e. resource address) for the same capability.
In the large bulk of cases this doesn’t matter. Only one type of resource provides a capability, and if the user tries to configure two of that type of resource with the same name it will fail due to resource duplication (you can’t add socket-binding=http if there already is a resource at that address.)
Elytron changes all this a bit as that subsystem has different resources with different types that all provide the same capability, but each with different implementation deals (*analogous* to an API providing Set but you can configure use of HashSet or LinkedHashSet or ConcurrentSkipListSet). Here we want to enforce that the user doesn’t configure two resources that provide the same capability. If they do, right now this will fail in runtime (due to MSC service name conflicts) but we want to detect the mistake earlier.
OTOH there are some specialized cases where it is valid for two resources to register the same capability. Clustering has one (see WFLY-8040); there are others on a Host Controller. Basically the capability implementations know about each other, check for duplication at runtime and if present deal with it in a correct manner.
I plan to make whether a capability allows multiple registration points to be configurable. See https://github.com/wildfly/wildfly-core/pull/2154
Question is whether the default behavior should remain ‘true’ (allow multiple registrations).
Pros:
1) Any existing capabilties that really need multiple registration can remain unchanged. This is more problematic for other projects based on WildFly Core that use capabilities as they may not know about this until they switch to core 3.0. But are there such projects? (I specifically copied a few folks to directly check).
2) Some subsystem unit test setups that don’t need this but inadvertently rely on it still work. (I’ve already fixed these in WildFly and WildFly Core so this only relates to other projects based on WildFly Core that use capabilities.)
Cons:
1) The Elytron use case is more standard, IMO, while the use cases like the clustering one that allow multiple registrations already require special logic to make that work. So I’d rather isolate any configuration to allow it to work to the cases that already are taking special steps.
2) The elytron team is really busy and doesn’t need more things to do.
My thought is the Cons outweigh the Pros, so I plan to change the default behavior. But your thoughts are welcome.
--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat
3 years, 7 months
JPADependencyProcessor "infecting" classpath with the wrong Javassist version
by Sanne Grinovero
Hi all,
Scott sent a nice PR to Wildfly a while back to fix the problem:
- https://github.com/wildfly/wildfly/pull/9305
It wasn't merged, I guess it's not a priority for WildFly but let me
clarify that without such fixes it's impossible for people to use
newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
as well for other libraries using Javassist.
There's quite some people in the Hibernate community who expressed
interest in using a not-so-stale version as the one which is typically
available in the latest stable release of WildFly.
To make this happen, all Hibernate projects are now publishing
"WildFly modules" which can be easily downloaded as additional drop-in
layer.
Granted these are not for everyone but we get good feedback from the
power users interested, and not least this allows us to develop all
our projects while regularly testing integration with WildFly, making
sure that the eventual integration goes smoother.
The current problem is that the WildFly JPADependencyProcessor adds
the wrong version of javassist to the deployments, and there's no way
for us to prevent this or override this, making it impossible to use a
recent version of Hibernate ORM as it requires a newer version.
- https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb7af...
We'd highly appreciate if that PR could be merged? Including on
product branches please, as enforcing a dependency which is neither
needed nor desired is going to break a long list of other frameworks
as well.
Thanks,
Sanne
3 years, 7 months
Current Invocation Integration / Elytron Integration Related Failures 2017-02-16
by Darran Lofthouse
The latest test results are available at: -
https://ci.wildfly.org/viewLog.html?buildId=46451&buildTypeId=WF_MasterNe...
We currently stand at 42 failures, at the last count we were at 45 failures.
However included in this latest we have 6 last minute failures related
to graceful shut down related to EJB and transactions which is a known
issue currently being worked on and one additional transnational failure
to be worked on.
The current failures are: -
org.jboss.as.test.integration.ejb.mdb.MDBTestCase.testSuspendResumeWithMDB
org.jboss.as.test.integration.ejb.mdb.MDBTestCase.testAnnoBasedMDB
org.jboss.as.test.integration.ejb.mdb.MDBTestCase.testDDBasedMDB
org.jboss.as.test.integration.ejb.client.descriptor.EJBClientDescriptorTestCase.testClientInvocationTimeout
org.jboss.as.test.integration.ejb.container.interceptor.security.SwitchIdentityTestCase.testSecurityContextAssociation
org.jboss.as.test.integration.ejb.container.interceptor.security.SwitchIdentityTestCase.testClientLoginModule
org.jboss.as.test.integration.ejb.container.interceptor.security.api.SwitchIdentityTestCase.testSecurityContextAssociation
org.jboss.as.test.integration.ejb.container.interceptor.security.api.SwitchIdentityTestCase.testClientLoginModule
org.jboss.as.test.integration.ejb.mapbased.MapBasedInitialContextEjbClientTestCase.testScopedEJBClientContexts
org.jboss.as.test.integration.ejb.remote.client.api.tx.EJBClientUserTransactionTestCase.testBatchOperationsInTx
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testNotSwitched
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testUnsecured
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testSwitched
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testDirect
org.jboss.as.test.integration.ejb.remote.suspend.EjbRemoteSuspendTestCase.testSuspendedCallRejected
org.jboss.as.test.integration.ejb.timerservice.suspend.TimerServiceSuspendTestCase.testSuspendWithIntervalTimer
org.jboss.as.test.integration.ejb.timerservice.suspend.TimerServiceSuspendTestCase.testSingleActionTimerWhenSuspended
org.jboss.as.test.integration.ejb.timerservice.suspend.TimerServiceSuspendTestCase.testIntervalTimersDoNotBackUp
org.jboss.as.test.integration.jpa.epcpropagation.EPCPropagationTestCase.testNoTxEPCPropagation
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testDir
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testLdap
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleTestCase.testNotAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleTestCase.testAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleUseNewClientCertTestCase.testNotAuthorizedClient
org.jboss.as.test.integration.security.loginmodules.RemotingLoginModuleUseNewClientCertTestCase.testAuthorizedClient
org.jboss.as.test.clustering.cluster.dispatcher.CommandDispatcherTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.provider.ServiceProviderRegistrationTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.registry.RegistryTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testGracefulShutdownConcurrentFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatefulFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testSecureStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testClientException
org.jboss.as.test.clustering.single.provider.ServiceProviderRegistrationTestCase.test
org.jboss.as.test.clustering.single.dispatcher.CommandDispatcherTestCase.test
org.jboss.as.test.clustering.single.registry.RegistryTestCase.test
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testServerToServerSFSBInvocation
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testServerToServerSFSBUsingEJB2xHomeView
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testSFSBPassivationWithScopedEJBProxyMemberInstances
org.jboss.as.test.manualmode.ejb.client.outbound.connection.RemoteOutboundConnectionReconnectTestCase.testRemoteServerRestarts
org.jboss.as.test.manualmode.ejb.client.outbound.connection.RemoteOutboundConnectionReconnectTestCase.testRemoteServerStartsLate
org.jboss.as.test.manualmode.ejb.client.reconnect.EJBClientReconnectionTestCase.testReconnection
3 years, 7 months
Current Invocation / Elytron Integration Related Failures 2017-02-08
by Darran Lofthouse
Another round of merges have just been made against master for the
latest Elytron / Invocation integration work - from the latest test run
we are now down to 45 failures.
https://ci.wildfly.org/viewLog.html?buildId=44766&buildTypeId=WF_MasterNe...
org.jboss.as.test.integration.ejb.client.descriptor.EJBClientDescriptorTestCase.testClientInvocationTimeout
org.jboss.as.test.integration.ejb.container.interceptor.security.SwitchIdentityTestCase.testSecurityContextAssociation
org.jboss.as.test.integration.ejb.container.interceptor.security.SwitchIdentityTestCase.testClientLoginModule
org.jboss.as.test.integration.ejb.container.interceptor.security.api.SwitchIdentityTestCase.testSecurityContextAssociation
org.jboss.as.test.integration.ejb.container.interceptor.security.api.SwitchIdentityTestCase.testClientLoginModule
org.jboss.as.test.integration.ejb.mapbased.MapBasedInitialContextEjbClientTestCase.testScopedEJBClientContexts
org.jboss.as.test.integration.ejb.remote.client.api.EJBClientAPIUsageTestCase.testNonExistentViewForEJB
org.jboss.as.test.integration.ejb.remote.client.api.tx.EJBClientUserTransactionTestCase.testBatchOperationsInTx
org.jboss.as.test.integration.ejb.remote.client.api.tx.EJBClientXidTransactionTestCase.testSLSBMandatoryTx
org.jboss.as.test.integration.ejb.remote.client.api.tx.EJBClientXidTransactionTestCase.testClientTransactionManagement
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testNotSwitched
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testUnsecured
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testSwitched
org.jboss.as.test.integration.ejb.remote.security.RemoteIdentityTestCase.testDirect
org.jboss.as.test.integration.ejb.remote.suspend.EjbRemoteSuspendTestCase.testSuspendedCallRejected
org.jboss.as.test.integration.ejb.remote.view.LocalViewRemoteInvocationTestCase.testServerInVMInvocationOnLocalViewOfSingleton
org.jboss.as.test.integration.ejb.remote.view.LocalViewRemoteInvocationTestCase.testServerInVMInvocationOnLocalViewOfSFSB
org.jboss.as.test.integration.ejb.remote.view.LocalViewRemoteInvocationTestCase.testServerInVMInvocationOnLocalViewOfSLSB
org.jboss.as.test.integration.ejb.transaction.bmt.lazyenlist.LazyTransactionEnlistmentUnitTestCase.testRawSQL
org.jboss.as.test.integration.jpa.datasourcedefinition.DataSourceDefinitionJPATestCase.testUserTxRollbackDiscardsChanges
org.jboss.as.test.integration.jpa.transaction.TransactionTestCase.testUserTxRollbackDiscardsChanges
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testDir
org.jboss.as.test.integration.naming.ldap.LdapUrlInSearchBaseTestCase.testLdap
org.jboss.as.test.clustering.cluster.dispatcher.CommandDispatcherTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.provider.ServiceProviderRegistrationTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.registry.RegistryTestCase(SYNC-tcp).test
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testGracefulShutdownConcurrentFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatefulFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testSecureStatelessFailover
org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testClientException
org.jboss.as.test.clustering.single.provider.ServiceProviderRegistrationTestCase.test
org.jboss.as.test.clustering.single.dispatcher.CommandDispatcherTestCase.test
org.jboss.as.test.clustering.single.registry.RegistryTestCase.test
org.jboss.as.test.multinode.transaction.TransactionInvocationTestCase.testRollbackOnlyBeforeCompletion
org.jboss.as.test.multinode.transaction.TransactionInvocationTestCase.testSynchronizationSucceeded
org.jboss.as.test.multinode.transaction.TransactionInvocationTestCase.testRollbackOnly
org.jboss.as.test.multinode.transaction.TransactionInvocationTestCase.testSynchronizationFailed
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testServerToServerSFSBInvocation
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testServerToServerSFSBUsingEJB2xHomeView
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testEJBClientContextIdentiferIsNotSerialized
org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase.testSFSBPassivationWithScopedEJBProxyMemberInstances
org.jboss.as.test.manualmode.ejb.client.outbound.connection.RemoteOutboundConnectionReconnectTestCase.testRemoteServerRestarts
org.jboss.as.test.manualmode.ejb.client.outbound.connection.RemoteOutboundConnectionReconnectTestCase.testRemoteServerStartsLate
org.jboss.as.test.manualmode.ejb.client.reconnect.EJBClientReconnectionTestCase.testReconnection
3 years, 7 months
Analyse the management model
by Harald Pehl
TL;DR
A tool to analyse the WildFly management model tree using a graph database.
# Longer version
As a heavy consumer of the WildFly management model I've always been
looking for a way to analyse the management model. So I decided to
start a little side project over the weekend. The result is a tool [1]
which reads a (sub)tree of the WildFly management model and stores the
results into a graph database using Neo4j [2]. To get started, you
need a running WildFly and Neo4j instance.
The tool writes nodes for each resource, attribute, operation, request
property and capability. In addition it creates relationships between
these nodes. You can use the data to
- get alternatives and requires relations between attributes
- get deprecated attributes and request parameters for one or all resources
- find attributes which might miss a capability reference
- find inconsistent attributes
- see differences between resources (using external diff tools)
See [3] for more use cases. I hope this is useful for others as well.
Feedback, suggestions and critics are welcome!
[1] https://github.com/hal/model-graph
[2] https://neo4j.com/
[3] https://github.com/hal/model-graph#examples
--
Harald Pehl
JBoss by Red Hat
http://hpehl.info
3 years, 7 months
Domain mode "Node" status?
by Ramesh Reddy
Hi,
Is there is anyway in WF to register to listen lifecycle events from all the registered nodes in domain mode, without going to the JGroup's MembershipListener level?
Thank you.
Ramesh..
3 years, 7 months
