Hello,

We're planning to create a tool to be used inside our AeroGear Cordova plugin projects, in order to automate the Jasmine test suite execution inside emulator & devices. Initially we're targeting on Android & iOS platforms and the tool will be available to the public through NPM registry as a Grunt plugin.

Here is how we're thinking to implement it:

Step 1. Configuration in Grunt file will include:
- jasmine:
    - project configuration per platform:
        - target directory: directory where the Cordova project will be created
        - name: project's name
        - package: project's package
        - version: the Cordova version to be used for creating the project
    -  plugin_test:
        - path: the path of an existing Cordova plugin in the filesystem
        - git: will be used in the case where the plugin does not exist in the filesystem and there is a GitHub repository for it
            - url: the GitHub URL of a Cordova plugin
            - target: the folder in which the GitHub project will be cloned
        - testsuite: the Jasmine test suite to execute inside the emulators/devices
    - platforms: Android and/or iOS
    - device ids: the device ids to be used in order to install the cordova application (e.g APK in Android). If this is empty then the Cordova apps will be installed to all the devices/emulators which are present during the test execution

- cordova_libs: the cordova libraries (cordova-android, cordova-ios) which are required to create a clean Android, iOS Cordova project
    - path: the filesystem path which contains the Cordova library
    - git:
        - url: the GitHub URL of the Cordova library in case it does not exist in the filesystem
        - target: the folder in which the Cordova librray will be cloned

Sample Grunt file: https://gist.github.com/tolis-e/aba6b002c276bcf60a4a

Step 2. Setup the environment. If the cordova libraries are not present in filesystem, they will be cloned from GitHub.
Step 3. We're planning to use our https://npmjs.org/package/grunt-android-emulator to automatically start Android emulators. iOS emulators automatic startup is under investigation.
Step 4. if a specific Cordova version e.g 3.0.0 is defined in Grunt file then we will perform a git checkout of the specific tag inside the cordova-android and cordova-ios. This way, the projects which will be created on the next step, are compatible with a specific Cordova version.
Step 5. Clean Android, iOS projects will be created by using the cordova-android, cordova-ios libraries which exist due to step 2
Step 6. Modify the Jasmine suites by adding a custom reporter which will log the results on stdout (Android logcat in Android case). Investigation is needed on whether something similar to Android logcat exists for iOS emulators/devices.
Step 7. The plugin will be installed on the clean Android/iOS projects by using https://npmjs.org/package/plugman I haven't tried plugman to see how it works but in the worst case we will install the plugins by modifying and copying files (config.xml).
Step 8. Build APK using ant for Android, execute the build script for iOS
Step 9. Deploy APKs on devices using adb on Android. For iOS, fruitstrap https://github.com/ghughes/fruitstrap might do the job. However, the fruitstrap project is no longer maintained and it would make sense to use a different tool. any idea?
Step 10. Attach a listener to logcat output and wait for test results. Parse the results, aggregate them and exit the Grunt process either with a success or failure status. It is under investigation how to transfer the results from the iOS emulators/devices (see step 6)

This above idea is inspired from the Medic tool https://github.com/filmaj/medic (Continuous integration setup for Apache Cordova). Apache has recently created a new GitHub repo: https://github.com/apache/cordova-medic and they are using a Couch DB in which the Jasmine results are posted. Moreover afaik, the tool assumes that the emulators are already started during the test execution and their lifecycle is not managed by the tool.

sorry for the long e-mail :)

Let us know your thoughts