After presenting a long workshop about testing Android apps last Sunday at AnDevCon 2, I bounced into a lot of people interested in the topic and also had some very interesting conversations that show there might be a bright future ahead for testing, TDD and BDD on Android so I thought I share what I found out..
In my presentation I talked about Robotium and compared it to Selenium, which of course triggered the question if there is some sort of recording tool. Robotium itself has none although it is on their wishlist of future features, but I heard about BitBar and TestDroid before and knew they had something but never actually saw it. So it was happy to meet Jauko at AnDevCon and see the tool in action. I was quite impressed seeing it and its integration in their cloud based offering, which runs the tests on actually devices taking full resolution screenshots straight via adb and not some crummy pictures taken by a digital camera looking at the devices (yes I am looking at you deviceanywhere). I was glad to hear they are helping Robotium upstream as well and are considering open sourcing the recorder. Now send them an email if you want that. I sure do!
Another tool that got quite a bit of attention is the monkey tool. Amazon mentioned in their keynote as one of the tools they are using although very high level and apkudo is basing their initial launch of their test tool solely on it. Knowing the limitations of monkey as a pure exerciser without any scriptable behaviour I have my doubts about the usefulness of such a tool. Especially considering that it is easy to script yourself and that it can not invoke a specific activity, but can only be restricted by application package. So if your application is complex the chance that all activities are exercised is slim to non-existant for a reasonable test run time. And if you lock anything behind e.g. a login screen.. well the monkey won’t get past that. But the apkudo guys seem to think that some testing is better than none. I agree with that, but I would not start with monkey..
But maybe with a gorilla, which brings us to the next tool I saw in action. FoneMonkey from GorillaLogic offers a recording tool as well, but from a technical perspective it works quite differently. It does not use standard Android instrumentation or adb. Instead you build your app together with some aspectj weaving/magic. Then you can record tests with the testing tool, which will in turn produce plain junit test. Those test will however run on your development machine (and not in the emulator/phone) and connect to the device with a custom protocol and exercise your application. Like the bitbar recorder you can simulate an actual user providing testing functionality that can replicated actual usage patterns for your app. And talking to Shaun Terry from GorillaLogic I found out that they are going to open source the whole tool chain too. Contact them to make sure it happens!
Last but not least I met Jonas and Erik from LessPainful. They are working on bringing the elegance of BDD with Cucumber to Android, Any (Ruby) coder that has used Cucumber before will love to see that happening. Apart from the finer details this all seems to be sorted so writing really expressive tests for Android seems to be near to reality as well. And it will all be open source too!
All in all it seems like testing Android applications is a hot topic and interest is on a steep rise. Good luck to all the businesses that try to capture a piece of the market! Contact me if you would like to see it working with the Android Maven Plugin.. or better send a pull request!
This was my testing related recap of AnDevCon 2 … stay tuned for a more general one later.