[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen commented on ARQ-918: ------------------------------- Here are more details about the run modes and how they are affected by the current testable attribute. https://docs.jboss.org/author/display/ARQ/Test+run+modes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen commented on ARQ-918: ------------------------------- I agree we shouldn't break backward compatibility. I'm open to not deprecating the testable attribute either if that's the trade-off for introducing a better attribute name. (It's possible to honor both with a mutually exclusive validation). I don't agree we can just let it go. New users are having trouble with this attribute and it's a trouble spot in Arquillian we need to smooth out. I came up with a few more choices, which I think get us even closer: * attachTestRunner - Indicates whether the test runner infrastructure should be attached (or somehow combined) with the deployment...attaching onto it so that it gets sent to the container w/ the deployment * testExecutor - Indicates that this deployment will become the test executor inside the container. * testInContainer - Indicates the tests should be run in container, specifically inside this deployment (hence the location of the attribute), which implies that the test infrastructure must be there. * supportsInContainerTesting - Although long, this is the heart of what we are saying in all previous passages. I'd be prepared to move on "supportsInContainerTesting" since it's the clearest point. I'm also feeling strongly about "attachTestRunner" and "testExecutor". If you want a better name, please let us know which of these names you think are most intuitive...and of course suggest another one if you have one that fits better. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen commented on ARQ-918: ------------------------------- If "Testing" is to long, we could go with "Tests" or "Test". Btw, thanks to Jim Crossley for pointing me in the direction of "InContainer" in the name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] jaikiran pai commented on ARQ-918: ---------------------------------- FWIW, some months back there was a doodle poll for this one here http://www.doodle.com/rqes7hpsgzwn3k2p. Looks like it's no longer accessible. At that time I had come up with a name for this attribute which wasn't one of the suggested ones and it's not among the ones in this JIRA as well. I wish I remembered what it was or maybe that poll data can be fetched somehow? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Aslak Knutsen commented on ARQ-918: ----------------------------------- We might be looking at 3 different packaging modes - Client, as today, don't touch the deployment - InContainer, as today, do what ever is needed to get it running in container - Half and half mix, No InContainer testing directly, but support for CodeCoverage, JRebel, Byteman (jsfunit.ng) The new mode and how they should work together is still a bit fuzzy.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Samuel Santos commented on ARQ-918: ----------------------------------- Or something like: {code} @Deployment(runMode = ArquillianRunMode.IN_CONTAINER|CLIENT|CLIENT_ENRICHED) {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen edited comment on ARQ-918 at 5/10/12 2:47 AM: -------------------------------------------------------- Several suggestions have come up that indicate to me I need to clarify what behavior we're attempting to control. * testMode = TestMode.CLIENT etc... * runMode = ArquillianRunMode.IN_CONTAINER|CLIENT|CLIENT_ENRICHED * Do the 'supports' and 'testing' bookends reinforce the meaning of 'inContainer' that much? A deployment is "in container" by definition. We aren't trying to control whether or not its deployed to the container. Otherwise, we wouldn't be defining a deployment in the first place. What we are trying to control is whether Arquillian adds its own infrastructure to the deployment (effectively repackaging it). That infrastructure adds extra services to the deployment that become available once the deployment is deployed. Those services are (as of now): * a service to run tests in-container * a service to monitor, analyze or execute callbacks on the server-side code We are not trying to control how the tests are run. That behavior is controlled on the test class or test method using the @RunAsClient annotation. When absent, the default is to run the test "in-container" if the deployment supports in-container testing or "as client" if the deployment does not. (If there are multiple deployments, @OperateOnDeployment tells Arquillian which deployment to use for that test). I'll summarize the three ways a deployment is packaged and the behavior it provides: * Add the test infrastructure to the deployment - capable of running tests in-container (white box testing) * Add instrumentation to the deployment - supports client testing w/ knowledge of internal state (gray box testing) * Use the deployment exactly as it's defined - supports purely client testing (black box testing) With that in mind, I have another proposal for how to control the packaging of these services: {code} @Deployment(testingServices = ALL | INSTRUMENTATION_ONLY | NONE) {code} or as a separate annotation {code} @Deployment @TestingServices(NONE) {code} ALL could be IN_CONTAINER instead. was (Author: dan.j.allen): Several suggestions came up that indicate to me I need to clarify what behavior we're attempting to control. * testMode = TestMode.CLIENT etc... * runMode = ArquillianRunMode.IN_CONTAINER|CLIENT|CLIENT_ENRICHED * Do the 'supports' and 'testing' bookends reinforce the meaning of 'inContainer' that much? A deployment is "in container" by definition. We aren't trying to control whether or not its deployed to the container. Otherwise, we wouldn't be defining a deployment in the first place. What we are trying to control is whether Arquillian adds its own infrastructure to the deployment (effectively repackaging it). That infrastructure adds extra services to the deployment that become available once the deployment is deployed. Those services are (as of now): * a service to run tests in-container * a service to monitor, analyze or execute callbacks on the server-side code We are not trying to control how the tests are run. That behavior is controlled on the test class or test method using the @RunAsClient annotation. When absent, the default is to run the test "in-container" if the deployment supports in-container testing or "as client" if the deployment does not. (If there are multiple deployments, @OperateOnDeployment tells Arquillian which deployment to use for that test). I'll summarize the three ways a deployment is packaged and the behavior it provides: * Add the test infrastructure to the deployment - capable of running tests in-container (white box testing) * Add instrumentation to the deployment - supports client testing w/ knowledge of internal state (gray box testing) * Use the deployment exactly as it's defined - supports purely client testing (black box testing) With that in mind, I have another proposal for how to control the packaging of these services: {code} @Deployment(testingServices = ALL | INSTRUMENTATION_ONLY | NONE) {code} or as a separate annotation {code} @Deployment @TestingServices(NONE) {code} ALL could be IN_CONTAINER instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen edited comment on ARQ-918 at 5/10/12 2:47 AM: -------------------------------------------------------- The following suggestions indicate to me I need to clarify what behavior we're attempting to control. * testMode = TestMode.CLIENT etc... * runMode = ArquillianRunMode.IN_CONTAINER|CLIENT|CLIENT_ENRICHED * Do the 'supports' and 'testing' bookends reinforce the meaning of 'inContainer' that much? A deployment is "in container" by definition. We aren't trying to control whether or not its deployed to the container. Otherwise, we wouldn't be defining a deployment in the first place. What we are trying to control is whether Arquillian adds its own infrastructure to the deployment (effectively repackaging it). That infrastructure adds extra services to the deployment that become available once the deployment is deployed. Those services are (as of now): * a service to run tests in-container * a service to monitor, analyze or execute callbacks on the server-side code We are not trying to control how the tests are run. That behavior is controlled on the test class or test method using the @RunAsClient annotation. When absent, the default is to run the test "in-container" if the deployment supports in-container testing or "as client" if the deployment does not. (If there are multiple deployments, @OperateOnDeployment tells Arquillian which deployment to use for that test). I'll summarize the three ways a deployment is packaged and the behavior it provides: * Add the test infrastructure to the deployment - capable of running tests in-container (white box testing) * Add instrumentation to the deployment - supports client testing w/ knowledge of internal state (gray box testing) * Use the deployment exactly as it's defined - supports purely client testing (black box testing) With that in mind, I have another proposal for how to control the packaging of these services: {code} @Deployment(testingServices = ALL | INSTRUMENTATION_ONLY | NONE) {code} or as a separate annotation {code} @Deployment @TestingServices(NONE) {code} ALL could be IN_CONTAINER instead. was (Author: dan.j.allen): Several suggestions have come up that indicate to me I need to clarify what behavior we're attempting to control. * testMode = TestMode.CLIENT etc... * runMode = ArquillianRunMode.IN_CONTAINER|CLIENT|CLIENT_ENRICHED * Do the 'supports' and 'testing' bookends reinforce the meaning of 'inContainer' that much? A deployment is "in container" by definition. We aren't trying to control whether or not its deployed to the container. Otherwise, we wouldn't be defining a deployment in the first place. What we are trying to control is whether Arquillian adds its own infrastructure to the deployment (effectively repackaging it). That infrastructure adds extra services to the deployment that become available once the deployment is deployed. Those services are (as of now): * a service to run tests in-container * a service to monitor, analyze or execute callbacks on the server-side code We are not trying to control how the tests are run. That behavior is controlled on the test class or test method using the @RunAsClient annotation. When absent, the default is to run the test "in-container" if the deployment supports in-container testing or "as client" if the deployment does not. (If there are multiple deployments, @OperateOnDeployment tells Arquillian which deployment to use for that test). I'll summarize the three ways a deployment is packaged and the behavior it provides: * Add the test infrastructure to the deployment - capable of running tests in-container (white box testing) * Add instrumentation to the deployment - supports client testing w/ knowledge of internal state (gray box testing) * Use the deployment exactly as it's defined - supports purely client testing (black box testing) With that in mind, I have another proposal for how to control the packaging of these services: {code} @Deployment(testingServices = ALL | INSTRUMENTATION_ONLY | NONE) {code} or as a separate annotation {code} @Deployment @TestingServices(NONE) {code} ALL could be IN_CONTAINER instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Jim Crossley commented on ARQ-918: ---------------------------------- I agree any mention of "in-container" seems weird after understanding that the attribute only controls whether the deployment will be slathered in "arquillian juice" or not. Of course, any deployment will ultimately end up in a container, duh. To me, any use of the term "testing" is completely redundant within a TESTING framework. :) Maybe something like 'extraServicesAllowed' or 'frozen' or 'arqJuiceOk'. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen edited comment on ARQ-918 at 5/10/12 3:29 PM: -------------------------------------------------------- Right on, Jim. I consulted my resident advisor, and she recommended a very similar approach. Here's the resulting proposal. Therefore, it seems to make the most sense to explicitly specify which services you want to disable or disallow. That would suggest an "exclude" annotation. Here's how you might exclude all services: {code} @Deployment @ExcludeServices {code} I realized that with the Spring extension, we have a new type of Arquillian service, a service deployer. This is an extension that adds the Spring JAR files for you automatically, even when you are doing black-box testing. In that case, you might use: {code} @Deployment @ExcludeServices({TESTING, INSTRUMENTATION}) {code} Rather than an enum, we may want the value of @ExcludeServices to be a String (with a set of built-in constants) so that additional categories can be added w/o changes to Arquillian Core. Here's the proposed annotation: {code} public @interface ExcludeServices { String[] value() default {ALL}; } {code} Possible values to exclude are: * ALL * TESTING (in-container testing, meaning test runner, enrichers, test case, etc) * INSTRUMENTATION (monitoring, code coverage, callbacks) * DEPLOYERS (packages extra JAR files for convenience, like Spring or Weld Servlet) This proposal: a) implies that Arquillian defaults to adding all services (according to what is on the classpath) b) makes disabling all services simple, just a raw annotation c) gives you fine grained control, if needed d) is consistent w/ existing behavior (the absense of the annotation means add all services) was (Author: dan.j.allen): Right on, Jim. I consulted my resident advisor, and she recommended a very similar approach. Here's the resulting proposal. Therefore, it seems to make the most sense to explicitly specify which services you want to disable or disallow. That would suggest an "exclude" annotation. {code} @Deployment @ExcludeServices {code} I realized that with the Spring extension, we have a new type of Arquillian service, a service deployer. This is an extension that adds the Spring JAR files for you automatically, even when you are doing black-box testing. In that case, you might use: {code} @Deployment @ExcludeServices({TESTING, INSTRUMENTATION}) {code} Rather than an enum, we may want the value of @ExcludeServices to be a String (with a set of built-in constants) so that additional categories can be added w/o changes to Arquillian Core. Here's the proposed annotation: {code} public @interface ExcludeServices { String[] value() default {ALL}; } {code} Possible values to exclude are: * ALL * TESTING (in-container testing, meaning test runner, enrichers, test case, etc) * INSTRUMENTATION (monitoring, code coverage, callbacks) * DEPLOYERS (packages extra JAR files for convenience, like Spring or Weld Servlet) This proposal: a) implies that Arquillian defaults to adding all services (according to what is on the classpath) b) makes disabling all services simple, just a raw annotation c) gives you fine grained control, if needed d) is consistent w/ existing behavior (the absense of the annotation means add all services) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Samuel Santos edited comment on ARQ-918 at 5/10/12 8:37 PM: ------------------------------------------------------------ Do we want to continue supporting "mixed-mode"? Currently, this is possible doing something like this: {code} @RunWith(Arquillian.class) public class ExampleIT { @Deployment public static WebArchive createDeployment() { // ... } @Test public void inContainerTest() { // ... } @Test @RunAsClient public void clientTest(@ArquillianResource URL baseURL) { // ... } } {code} was (Author: Silenius): Do we want to continue supporting "mixed-mode"? Currently, this is possible doing something like this: {code} @RunWith(Arquillian.class) public class ExampleIT { @Deployment public static WebArchive createDeployment() { // ... } @Test public void InContainerTest() { // ... } @Test @RunAsClient public void ClientTest(@ArquillianResource URL baseURL) { // ... } } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Samuel Santos edited comment on ARQ-918 at 5/10/12 8:36 PM: ------------------------------------------------------------ Do we want to continue supporting "mixed-mode"? Currently, this is possible doing something like this: {code} @RunWith(Arquillian.class) public class ExampleIT { @Deployment public static WebArchive createDeployment() { // ... } @Test public void InContainerTest() { // ... } @Test @RunAsClient public void ClientTest(@ArquillianResource URL baseURL) { // ... } } {code} was (Author: Silenius): Do we want to continue supporting "mixed-mode"? Currently, this is possible doing something like this: {code} @RunWith(Arquillian.class) public class ExampleIT { @Deployment public static WebArchive createDeployment() { // ... } @Test public void InContainerTest() { // ... } @Test @RunAsClient public void InContainerTest(@ArquillianResource URL baseURL) { // ... } } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen edited comment on ARQ-918 at 5/10/12 11:30 PM: --------------------------------------------------------- Samuel, The mixed-mode use case is definitely still supported. Your example code will work exactly as is. The first test method will run in container and the second will run as a client. That combination supported in the case that you have a plain old @Deployment annotation on the deployment producer method. The service that provides the in-container part of mixed mode is the one I named "TESTING" in my last proposal. It means that the deployment will be augmented with the service necessary to execute a test (in-container). was (Author: dan.j.allen): Samuel, The mixed-mode use case is definitely still supported. Your example code will work exactly as is. The first test method will run in container and the second will run as a client. That combination supported in the case that you have a plain old @Deployment annotation on the deployment producer method. The service that provides the in-container part of mixed mode is the one I named "TESTING" in my last proposal. It means that the deployment will have the service necessary to execute a test (in-container). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Karel Piwko edited comment on ARQ-918 at 5/11/12 2:54 AM: ---------------------------------------------------------- +1 on Dan's proposal: making this flag a separate annotation was (Author: kpiwko): +1 on Dan's proposal with making this flag a separate annotation -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Jim Crossley commented on ARQ-918: ---------------------------------- I like it! :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/ARQ-918?page=com.atlassian.jira.plugin.sy... ] Dan Allen commented on ARQ-918: ------------------------------- I had a thought that perhaps @ExcludeServices could also handled the role of the managed attribute as well. The managed attribute tells Arquillian whether to deploy the archive to the container. This is yet another service, though in this case not a packaged (in-container) service. {code} @ExcludeServices(MANAGED_DEPLOYMENT) {code} There seems to be a lot of flexibility here in how we can control the behavior of Arquillian. On the one hand, it can seem tedious to have to exclude various services, but if that's what you are trying to do, well, then it seems pretty nice to have that control. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira