Sunday, November 23, 2008

JUnit in Android

Now that we have the instrumentation framework in our basket and we are able to connect instrumentation objects with a test runner, it is time to put a bit of structure into out tests. The number of test cases grows quicky. If there is no way to organize them or they depend on each other in a subtle way, test case maintenance soon becomes a nightmare. Experience shows that in case of a resource-constrained project, test effort is the first to suffer cutbacks and if the test cases are not maintained properly, the team or the developer may soon feel that the test set is too much hassle compared to the value the tests provide. The result is a fallback to the default, "chaotic" way of development with the predictable drop of quality.

The popular JUnit test framework is integrated into Android. JUnit, if used properly, brings two major benefits to the test implementation.
  • JUnit enforces the hierarchical organization of the test cases
  • The pattern JUnit is based on guarantees the independence of the test cases and minimizes their interference.
Entire books have been written about JUnit and unit testing therefore I will restrict myself to the minimum in explaining the test framework. JUnit's main building block is the TestCase class. Actual test case classes are inherited from TestCase. One TestCase child class contains one or more test methods, these test methods are the real test cases and the TestCase descendants are really already test groups. By convention, the test method names are prefixed with "test" (like testThis(), testThat(), etc.). JUnit can discover the test methods by itself if this convention is respected, otherwise the test method names have to be declared one by one.

The most important impact of using JUnit is the way one test method is executed. For each test method, the TestCase descendant class is instantiated, its setUp() method is invoked, then the test method in question is invoked, then the tearDown() method is invoked then the instance becomes garbage. This is repeated for each test method. This guarantees that test methods are independent of each other, their interference is minimized. Of course, "clever" programmers can work around this pattern and implement test cases that depend on each other but this will bring us back to the mess that was to be eliminated in the first place. JUnit also provides ways to collect test classes into test suites and organize these suites hierarchically.

JUnit has been included into Android and Android-specific extensions were provided in the android.test package. Android test cases classes should inherit from android.test.AndroidTestCase instead of junit.framework.TestCase. The main difference between the two is that AndroidTestCase knows about the Context object and provides method to obtain the current Context. As many Android API methods need the Context object, this makes coding of Android test cases much easier. AndroidTestCase already contains one test method that tests the correct setup of the test case class, the testAndroidTestCaseSetupProperly method. This adds one test method to every AndroidTestCase descendant but this fact rarely disturbs the programmer.

Those familiar with the "big", PC-based JUnit implementations will miss a UI-oriented TestRunner. The android.test.AndroidTestRunner does not have any user interface (not even console-based UI). If you want to get any readable feedback out of it, you have to handle callbacks from the test runner.

You can download the example program from here.

Our example program contains a very simple test runner UI, implemented as an Activity. The UI provides just the main progress indicators, details are written into the log so this implementation is nowhere near to the flexible test runners of the "big" JUnit. The test suite contains two test case classes. One of them (the MathTest) is really trivial. It demonstrates, however, the hidden testAndroidTestCaseSetupProperly() test method that explains the two extra test cases (the test runner will display 5 test cases when we have really only 3). The main lesson from the other test case class (ContactTest) is that individual test cases must be made independent from each other. This test case class inserts and deletes the test contact for each test method execution, cleaning up its garbage after each test execution. This is the effort that must be made to minimize interference. ExampleSuite organizes these two test classes into a suite. As we respected the JUnit test method naming conventions, we can leave the discovery of test methods to JUnit.



At this point, we have everything to write structured, easy to maintain tests for Android, using the Android test instrumentation mechanism to drive the UI. This will be my next step.

26 comments:

Anonymous said...

Hi, i'm the newbie of the android test, i'm learning the android.test. But i could find the current way to create the testcase and run it. So, would you tell me the step to create a testcase and run it. and my email is hfm1215@sina.com. Beside, the example you provided cann't download. Thank you.

Oleksandr Kucherenko said...

public class CoreTests extends TestSuite {

/**
* To run these tests:
* $ mmm java/tests && adb sync
* $ adb shell am instrument -w \
* -e class android.core.CoreTests \
* android.core/android.test.InstrumentationTestRunner
*/
public static TestSuite suite() {
TestSuite suite = new TestSuite(CoreTests.class.getName());

// Re-enable StateListDrawableTest when we are running in the
// framework-test directory which allows access to package private
// access for MockView
// suite.addTestSuite(StateListDrawableTest.class);
suite.addTestSuite(DayOfMonthCursorTest.class);
suite.addTestSuite(MonthDisplayHelperTest.class);
suite.addTestSuite(StateSetTest.class);
suite.addTestSuite(ColorStateListTest.class);
suite.addTestSuite(FocusFinderTest.class);
suite.addTestSuite(ViewGroupAttributesTest.class);
suite.addTest(TelephonyTests.suite());
suite.addTestSuite(FloatMathTest.class);
suite.addTest(JavaTests.suite());
suite.addTestSuite(LocationTest.class);
suite.addTestSuite(LocationManagerProximityTest.class);
suite.addTestSuite(AndroidTestRunnerTest.class);
suite.addTestSuite(InstrumentationTestRunnerTest.class);
suite.addTestSuite(CookieTest.class);

return suite;
}
}


Take a close look on first comment. That part of code was found in ANDROID Platform SRC.

Raj said...

Hi Oleksandr Kucherenko,

Can you exeplain me your coding.. what it means for what your using that coding....

Korab said...

Hello,
I tried your junit app but there si one problem, after i click the run test button an java.lang.VerifyError occurs! Have you any idea?

thx,
regards,
nik

Anonymous said...

Nice example. Thanks.

Anonymous said...

can you please tell me the exact steps I have to fallow in order to try the example you gave? thank you very much

Anonymous said...

Hey , does anybody know how to create xml reports of Android test cases like the similar one we create for junit test cases.

Gabor Paller said...

I am not aware of XML reports but results can be logged into database.

Check out:

android.test.TestRecorder

asif said...

Hi I am a newbie to java field, Would anyone let me know the steps to run the example junitexample as I have to test android applications like browser, calculator etc. My mail ID is asifanwar59@gmail.com

Gabor Paller said...

Asif, you can start with this blog entry It is a bit outdated but you will get the idea.

asif said...

Gabor, thank you for your input and time you gave me! The blog is worth.

asif said...

When I compile OneWayActivity.java example, my compiler pops two errors:
1: Illegal character: \64
@override with the pointer being below @.
2: expected at line 88 i.e., empty line just above "classCounterServiceConnection implements ServiceConnection {" this line.
Gabor: any solution to this.

asif said...

correction in error 2: it is "identifier expected" at line 88...

Hammer said...

regarding testSuites:
In standard jUnit you could create testSuites likt this:
-------------
public static void suite(){
TestSuite suite = new TestSuite();
suite.addTest(new MyTest("testEquals"));
suite.addTest(new MyTest("testAdd"));
return suite;
}
------------
It seams like this is not possible when using an AndriodTestCase (since AndriodTestCase doesnt override the constructor taking a single String as argument).
Is there any other way of creating testSuites containing specific test-methods?

asif said...

Gabor, it worked.

asif said...

The example program runs with couple of warnings. I mean, it says the type contacts.people is deprecated. Can anyone let me know how can I get this work without this 'deprecated" error.
Secondly, I need to add a contact including name, phone number and email ID to my android sdk emulator using this program. What modifications should i do to the program in order to make this work?

Anonymous said...

Hi All,

I have question on the Android API. Android API provides a class called "Instrumentation" class. What is the use of this class? Is the Instrumentation class be used only together with Junit for unit testing.

Can Junit framework can be used to test the methods of the Android API without using the Instrumentation class.

Since Junit package has been included in the Android package, I hope we dont need to use install separately for unit testing.

I would appreciate if you could provide me the information as i can't find these clear information anywhere on the Web.

If we use Junit test framework to test the Android API, can we have test results in the UI format rather than test format.?

Thanks a lot. Apprecite your time.

Regards,
Riyas

Gabor Paller said...

Riyas, the instrumentation framework and JUnit are independent of each other. The example program of this post uses only Instrumentation, without JUnit. The example program of this post above uses only JUnit, no instrumentation. And the example program of this post presents how to use the two together.

Anonymous said...

Hi Gabor,

Thanks for your swift reply. BTW, what is the advantage of using instrumention class with junit test framework?. Can the junit alone fulfill the unit testing requirements for the entire Android API. Thanks again.

Gabor Paller said...

Instrumentation is just a special way of starting an Activity. Instrumentation starts the Activity in test mode so your test script can start and stop the Activity under test again and again.

You can use only JUnit to test elements of the API as long as your test scripts don't influence the Android lifecycle management. For example you may have a test script that stops the currently running Activity. If your test script is implemented as an Activity, this will stop testing.

Gubatron said...

Here's a screencast I made on how to do Unit Testing for Android on Eclipse, watch this and save a lot of time and headaches if you need to start unit testing your android project.

http://www.gubatron.com/blog/2010/05/02/how-to-do-unit-testing-on-android-with-eclipse/

Cheers

Ajit said...

Thanks Gabor for all sharing your experiences with JUnit testing on Android.

Saverio Labella said...

Hi all,

is there a way in Android to do a one-time initialization fro the whole test suite?

In JUnit you could use TestSetup but junit.extensions is not part of the android API.

Thanks

Android said...

I followed the instructions, but my X51v can't read the SD card after formatting. I get "Do you want to format this storage card to make it readable? This will permanently delete any files on the card."

Anonymous said...

" The result is a fallback to the default, "chaotic" way of development with the predictable drop of quality."

Could you please provide any references to the claim that tests results in better software quality? I have for a long time tried to find any research about the subject but cant seem to find any results? Have you?

Gabor Paller said...

Anonymous, you mean that you need reference to justify the claim that automated unit testing improves software quality?

I think the fact that automated unit tests detect regressions (e.g. faults in existing functionality that are affected by code changes) is quite a strong indicator about the value of the unit tests in improving code quality.

But if you need a reference, here is one: Roy Osherove: the Art of Unit Testing (Manning, 2009), page 4:

Unit testing has been floating around since the early days of Smalltalk programming language in the 1970s and it proves time to time again as one of the best ways a developer can improve code quality.