Diego Torres Milano's blog

monkeyrunner: testing views properties

noreply@blogger.com (Diego Torres Milano) — Sat, 28 Jan 2012 20:31:00 +0000

There are several questions floating around, like this one in stackoverflow, about how some view properties could be obtained from a monkeyrunner script or putting the question in more general terms:
how a test that verifies some properties can be created using monkeyrunner ?

This is a required feature if we are going to create tests in monkeyrunner otherwise our alternatives to verify some state in the views is limited to visual comparison. These cases were treated in previous articles like:

but now we will be using a logical approach rather than visual comparison. To be able to do it we need a mechanism of obtaining view properties values.
Lastest versions of monkeyrunner provides an extension to MonkeyDevice called EasyMonkeyDevice and this class has methods to get some properties like MonkeyDevice.getText().
This API is not documented so expect changes in the future.
TemperatureConverter, a sample application that has been used in other articles before, will be our application under test. The source code can be obtained from github.

The idea behind this test is, using monkeyrunner to connect to the device, enter 123 in the Celsius field and expect to find 253.40 in the Fahrenheit field.

#! /usr/bin/env monkeyrunner

import sys
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
from com.android.monkeyrunner.easy import EasyMonkeyDevice
from com.android.monkeyrunner.easy import By

# connect to the device
device = MonkeyRunner.waitForConnection()

# start TemperatureConverter
device.startActivity(component="com.example.i2at.tc/.TemperatureConverterActivity")

# use the EasyMonkey API
easyDevice = EasyMonkeyDevice(device)

celsiusId = By.id('id/celsius')
if not celsiusId:
   raise Exception("View with id/celsius not found")

fahrenheitId = By.id('id/fahrenheit')
if not fahrenheitId:
   raise Exception("View with id/fahrenheit not found")

MonkeyRunner.sleep(3)
easyDevice.type(celsiusId, '123')
MonkeyRunner.sleep(3)

celsius = easyDevice.getText(celsiusId)
fahrenheit = easyDevice.getText(fahrenheitId)
expected = '253.40'

if fahrenheit == expected:
   print 'PASS'
else:
   print 'FAIL: expected %s, actual %s' % (expected, fahrenheit)

Unfortunately, it won't work in most of the cases. You are likely to receive this exception:

java.lang.RuntimeException: java.lang.RuntimeException: No text property on node

I felt frustrated at first, but what the advantage of an Open Source project is other than going to the source code and find out why it's not working.

I dug into the Chimpchat, HierarchyView and ViewServer code and found out that for some reason EasyMonkeyDevice is looking for the text:mText property when in most of the cases it should be only mText.

So here is the patch, that I will be uploading to android soon:

diff --git a/chimpchat/src/com/android/chimpchat/hierarchyviewer/HierarchyViewer.java b/chimpchat/src/com/android/chimpchat/hierarchyviewer/HierarchyViewer.java
index 6ad98ad..6c34d71 100644
--- a/chimpchat/src/com/android/chimpchat/hierarchyviewer/HierarchyViewer.java
+++ b/chimpchat/src/com/android/chimpchat/hierarchyviewer/HierarchyViewer.java
@@ -170,7 +170,11 @@ public class HierarchyViewer {
         }
         ViewNode.Property textProperty = node.namedProperties.get("text:mText");
         if (textProperty == null) {
-            throw new RuntimeException("No text property on node");
+            // dtmilano: give it another chance, ICS ViewServer returns mText
+            textProperty = node.namedProperties.get("mText");
+            if ( textProperty == null ) {
+                throw new RuntimeException("No text property on node");
+            }
         }
         return textProperty.value;
     }

Once this patch is applied and you rebuild monkeyrunner or the entire SDK if you prefer, you will be presented with the expected result: PASS

Automated Android testing using Sikuli

noreply@blogger.com (Diego Torres Milano) — Sat, 21 Jan 2012 06:28:00 +0000

Sikuli is a tool that can supplement you testing toolbox.
It is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). The scripts you create with this tool are in the Sikuli Script language, which is a Python (Jython) extension. It also features Sikuli IDE which is an integrated development environment for writing visual scripts with screenshots easily.
There are plenty of examples and tutorials in its web site but most of them are intended for desktop operating systems. As usual here we will be focusing on its Android praxis.

Unlocking the emulator
Out example will be unlocking the emulator screen using some screenshots. That is, instead of guessing or finding out a-priori the coordinates of the touch events that are needed to achieve the goal of unlocking the screen we will use images.
The Sikuli IDE provides the means of obtaining the screenshots to complete the arguments of the specific methods.
For example, if you select the click() method to click over something, you are prompted to take a screenshot of an area of the screen that will be the target and the the IDE shows this thumbnail.
Instead of trying to describe it, it's worth showing you the IDE window to understand what I mean.

The idea is to click on the Android text, just to gain windows focus in case it was lost. The specific method to do this is App.focus("5554") assuming that "5554" appears in the emulator's window title, work on Windows and Linux but it doesn't work on Mac OSX.
Then we search the screen for the image of the lock button, when we find it we touch with the mouse, wait a little, and drag till the position of the unlock dot, where we move up.

Unlocking a pattern lock
The previous example was very illustrative, but sincerely it was a bit simple and could have been done with other tools too.
So, let's get things a bit more complicated as it is the phone locked by a pattern.
As before we take the screenshot of the pattern but this time we double click on it in the IDE to get the Pattern Settings window and there we set the Target Offset to follow the shape of the pattern. This is then identified by a red cross over the screenshot as follows:

When you run this script you can see how the pattern is completed and consequently the phone unlocked.

Pretty simple, right ?
Definitely something you should consider when automating GUI tests.

Android Continuous Integration Guide

noreply@blogger.com (Diego Torres Milano) — Thu, 22 Dec 2011 20:10:00 +0000

Android Application Testing Guide features a whole chapter about Continuous Integration (Chapter 8), however some latest changes and additions to the tools available may require a more in-depth coverage of the subject.
Consequently, I'm preparing an Android Continuous Integration Guide to compile all the information laying around on the subject and to provide concise, working examples that you could use to base your own projects on.
In the creation of these examples using Jenkins and EMMA for code coverage one of the most annoying things is the R class affecting the results of the coverage report, as most probably you are not creating tests for such auto-generated class.

This screenshot show how the coverage report for classes reaches 100% once we filter out the R class.

The problem is that in current Android SDK Tools (Rev 16), there's no way to filter classes from EMMA coverage unless you modify the file /tools/ant/build.xml, changing the emma target to include the filters as showed in this code snippet:



            <!-- It only instruments class files, not any external libs -->
            <emma enabled="true">
               <instr verbosity="${verbosity}"
                               mode="overwrite"
                               instrpath="${out.absolute.dir}/classes"
                               outdir="${out.absolute.dir}/classes">
                    <!-- DTM: 2011-12-23: added filter for R -->
                     <filter excludes="*.R" />
                     <filter excludes="*.R$*" />
                </instr>
                <!-- TODO: exclusion filters on R*.class and allowing custom exclusion from
                             user defined file -->
            </emma>

I hope this help you getting started and stay tuned, Android Continuous Integration Guide is scheduled to be released by the end of January 2012.

Any comments, suggestions and requests are welcome and can be entered using the Google+ pages.

Obtaining code coverage of a running Android application

noreply@blogger.com (Diego Torres Milano) — Wed, 16 Nov 2011 04:13:00 +0000

How can we obtain the code coverage of a running application, not just its tests ?
I have been asked this question many times. Recently, Jonas posted a similar question as comment to Eclipse, Android and EMMA code coverage. So we will elaborate the solution to this problem.
But firstly, let's do a brief introduction of the concepts.

EMMA: a free Java code coverage tool

EMMA is an open-source toolkit for measuring and reporting Java code coverage. EMMA distinguishes itself from other tools by going after a unique feature combination: support for large-scale enterprise software development while keeping individual developer's work fast and iterative.

Android includes EMMA v2.0, build 5312, which includes some minor changes introduced by Android to adapt it for the platform specifics.

Android Instrumentation

The instrumentation framework is the foundation of the testing framework. Instrumentation controls the application under test and permits the injection of mock components required by the application to run.

Usually, an InstrumentationTestRunner, a special class the extends Instrumentation, is used to run various types of TestCases, against an android application.

Typically, this Instrumentation is declared in the test project's AndroidManifest.xml and then run from Eclipse or from the command line using am instrument.

Also, to generate EMMA code coverage -e coverage true option is added to the command line.

Basically, we have all the components but in different places because we want to obtain the code coverage from the running application not from its tests.

EmmaInstrumentation
The first thing we need to do is to create a new Instrumentation that starts the Activity Under Test using EMMA instrumentation and when this Activity is finished the coverage data is saved to a file.
To be notified of this Activity finish we need a listener that we can set extending the AUT because one of our objectives is to keep it unchanged.

To illustrate this technique we will be using the Temperature Converter application that we have used many times in other posts. The source code is as usual available through github.


package com.example.instrumentation;

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

import com.example.i2at.tc.TemperatureConverterActivity;
//import com.vladium.emma.rt.RT;

import android.app.Activity;
import android.app.Instrumentation;
import android.content.Intent;
import android.os.Bundle;
import android.os.Looper;
import android.util.Log;

public class EmmaInstrumentation extends Instrumentation implements FinishListener {

    private static final String TAG = "EmmaInstrumentation";

    private static final boolean LOGD = true;

    private static final String DEFAULT_COVERAGE_FILE_PATH = "/mnt/sdcard/coverage.ec";

    private final Bundle mResults = new Bundle();

    private Intent mIntent;

    private boolean mCoverage = true;

    private String mCoverageFilePath;

    /**
     * Extends the AUT to provide the necessary behavior to invoke the
     * {@link FinishListener} that may have been provided using
     * {@link #setFinishListener(FinishListener)}.
     * 
     * It's important to note that the original Activity has not been modified.
     * Also, the Activity must be declared in the
     * <code>AndroidManifest.xml</code> because it is started by an intent in
     * {@link EmmaInstrumentation#onStart()}. This turns more difficult to use
     * other methods like using template classes. This latter method could be
     * viable, but all Activity methods should be re-written to invoke the
     * template parameter class corresponding methods.
     * 
     * @author diego
     * 
     */
    public static class InstrumentedActivity extends
    TemperatureConverterActivity {
        private FinishListener mListener;

        public void setFinishListener(FinishListener listener) {
            mListener = listener;
        }

        @Override
        public void finish() {
            if (LOGD)
                Log.d(TAG + ".InstrumentedActivity", "finish()");
            super.finish();
            if (mListener != null) {
                mListener.onActivityFinished();
            }
        }

    }

    /**
     * Constructor
     */
    public EmmaInstrumentation() {

    }

    @Override
    public void onCreate(Bundle arguments) {
        if (LOGD)
            Log.d(TAG, "onCreate(" + arguments + ")");
        super.onCreate(arguments);

        if (arguments != null) {
            mCoverage = getBooleanArgument(arguments, "coverage");
            mCoverageFilePath = arguments.getString("coverageFile");
        }

        mIntent = new Intent(getTargetContext(), InstrumentedActivity.class);
        mIntent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
        start();
    }

    @Override
    public void onStart() {
        if (LOGD)
            Log.d(TAG, "onStart()");
        super.onStart();

        Looper.prepare();
        InstrumentedActivity activity = (InstrumentedActivity) startActivitySync(mIntent);
        activity.setFinishListener(this);
    }

    private boolean getBooleanArgument(Bundle arguments, String tag) {
        String tagString = arguments.getString(tag);
        return tagString != null && Boolean.parseBoolean(tagString);
    }

    private void generateCoverageReport() {
        if (LOGD)
            Log.d(TAG, "generateCoverageReport()");

        java.io.File coverageFile = new java.io.File(getCoverageFilePath());

        // We may use this if we want to avoid refecltion and we include
        // emma.jar
        // RT.dumpCoverageData(coverageFile, false, false);

        // Use reflection to call emma dump coverage method, to avoid
        // always statically compiling against emma jar
        try {
            Class<?> emmaRTClass = Class.forName("com.vladium.emma.rt.RT");
            Method dumpCoverageMethod = emmaRTClass.getMethod(
                    "dumpCoverageData", coverageFile.getClass(), boolean.class,
                    boolean.class);
            dumpCoverageMethod.invoke(null, coverageFile, false, false);
        } catch (ClassNotFoundException e) {
            reportEmmaError("Is emma jar on classpath?", e);
        } catch (SecurityException e) {
            reportEmmaError(e);
        } catch (NoSuchMethodException e) {
            reportEmmaError(e);
        } catch (IllegalArgumentException e) {
            reportEmmaError(e);
        } catch (IllegalAccessException e) {
            reportEmmaError(e);
        } catch (InvocationTargetException e) {
            reportEmmaError(e);
        }
    }

    private String getCoverageFilePath() {
        if (mCoverageFilePath == null) {
            return DEFAULT_COVERAGE_FILE_PATH;
        } else {
            return mCoverageFilePath;
        }
    }

    private void reportEmmaError(Exception e) {
        reportEmmaError("", e);
    }

    private void reportEmmaError(String hint, Exception e) {
        String msg = "Failed to generate emma coverage. " + hint;
        Log.e(TAG, msg, e);
        mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, "\nError: "
                + msg);
    }

    /* (non-Javadoc)
     * @see com.example.instrumentation.FinishListener#onActivityFinished()
     */
    @Override
    public void onActivityFinished() {
        if (LOGD)
            Log.d(TAG, "onActivityFinished()");
        if (mCoverage) {
            generateCoverageReport();
        }
        finish(Activity.RESULT_OK, mResults);
    }

}

We are also implementing the FinishListener interface, which is defined as


package com.example.instrumentation;

/**
 * Listen for an Activity to finish and invokes {@link #onActivityFinished()} when this happens.
 * 
 * @author diego
 *
 */
public interface FinishListener {

        /**
         * Invoked when the Activity finishes.
         */
        void onActivityFinished();

}

Running the instrumented application
Once we have the EmmaInstrumentation class in place we need a few more adjustments to be able to get the coverage report of the running application.

Firstly, we need to add the new Activity to the manifest. Secondly, we should allow our application to write to the sdcard if this is where we decided to generate the coverage report. To do it you should grant the android.permission.WRITE_EXTERNAL_STORAGE permission.

Then, it's time to build and install the instrumented apk:

$ ant clean

$ ant instrument

$ ant installi

Everything is ready to start the instrumented application

$ adb shell am instrument -e coverage true \

-w com.example.i2at.tc/\

com.example.instrumentation.EmmaInstrumentation

If everything went well, the Temperature Converter application will be running and we can use it for a while

when we exit by pressing the BACK button we can see that the coverage data was written to the file and reflected in the logcat

I/System.out(2453): EMMA: runtime coverage data written to [/mnt/sdcard/coverage.ec] {in 975 ms}

this file can then be moved to the host computer using adb pull.

Hope this helps you obtaining the code coverage for your application to help you understand its usage patterns. As always, comments and questions are always welcome.

Android: Using monkey from Java

noreply@blogger.com (Diego Torres Milano) — Fri, 11 Nov 2011 04:31:00 +0000

The latest version of the Android SDK and tools include chimpchat, a library that facilitates the use of monkey from Java. This is equivalent to monkeyrunner, which is the bridge between monkey and the Python scripting language.
While Python is an incredibly powerful and expressive scripting language and will permit you creating tests with just a few statements, there are some occasions when you don't want to introduce a new language to the project leaving your Java confort zone or you prefer to leverage the use of previously created libraries instead of writing new ones.
In such cases, you can now have the same access to monkey running on the device with the help of chimpchat, as we are going to demonstrate.

Creating a Java project
Our first step will be to create a new Java project and we will add the required libraries to the Java Build Path as External Jars.
We are naming the project JavaMonkey, for obvious reasons.

We are adding these libraries from Android SDK, which are used directly or indirectly by our project, to the Java Build Path:

chimpchat.jar
ddmlib.jar
guavalib.jar
sdklib.jar

JavaMonkey.java
Our intention is to create a simple class, serving the purpose of a simple example to get as started. We will be simply:

Creating a JavaMonkey object
initializing it, this implies creating the connection with any emulator or device found or throwing an exception is not connection was made before the timeout expires
listing all the properties in the device or emulator
shutting down the connection

Following, is the JavaMonkey class:



/**
 * Copyright (C) 2011  Diego Torres Milano
 */
package com.example.javamonkey;

import java.util.TreeMap;

import com.android.chimpchat.ChimpChat;
import com.android.chimpchat.core.IChimpDevice;

/**
 * @author diego
 *
 */
public class JavaMonkey {

        private static final String ADB = "/Users/diego/opt/android-sdk/platform-tools/adb";
        private static final long TIMEOUT = 5000;
        private ChimpChat mChimpchat;
        private IChimpDevice mDevice;

        /**
         * Constructor
         */
        public JavaMonkey() {
                super();
        TreeMap<String, String> options = new TreeMap<String, String>();
        options.put("backend", "adb");
        options.put("adbLocation", ADB);
        mChimpchat = ChimpChat.getInstance(options);
        }

        /**
         * Initializes the JavaMonkey.
         */
        private void init() {
                mDevice = mChimpchat.waitForConnection(TIMEOUT, ".*");
                if ( mDevice == null ) {
                        throw new RuntimeException("Couldn't connect.");
                }
                mDevice.wake();
        }

        /**
         * List all properties.
         */
        private void listProperties() {
                if ( mDevice == null ) {
                        throw new IllegalStateException("init() must be called first.");
                }
                for (String prop: mDevice.getPropertyList()) {
                        System.out.println(prop + ": " + mDevice.getProperty(prop));
                }
        }

        /**
         * Terminates this JavaMonkey.
         */
        private void shutdown() {
                mChimpchat.shutdown();
                mDevice = null;
        }

        /**
         * @param args
         */
        public static void main(String[] args) {
                final JavaMonkey javaMonkey = new JavaMonkey();
                javaMonkey.init();
                javaMonkey.listProperties();
                javaMonkey.shutdown();
        }

}

Configuration
One of the important things you have to adapt to your environment is the location of the adb command. Otherwise if you don't set it you will receive:

E/adb: Failed to get the adb version: Cannot run program "adb": error=2, No such file or directory

Hope this helps you get started with chimpchat. As always, comments and questions are always welcome.

Android Application Testing Guide: Q&A

noreply@blogger.com (Diego Torres Milano) — Thu, 03 Nov 2011 18:25:00 +0000

Q: Hi Diego, I wanted to ask that can i write a monkey runner script which controls a web based apk ?
Eg; I install youtube.apk which is nothing but a browser with hardcoded youtube url.
Now my monkeyrunner script shall install this apk and then pass events such as a search string etc on this web based application.
All this i want to do and control externally through the monkey runner script. Is this possible? If yes, then could you please guide me by some pseudo code?

Comment on Using Android monkeyrunner from Eclipse

Posted by latha

A: This is an interesting question and a good monkeyrunner example, so here we go. monkeyrunner has the ability of installing APKs after obtaining the connection with the device. Then we start Youtube main activity, sleep for a bit to let things settle down.
Once we have the activity running is time to start our search. To do it, we touch the Search icon, enter the desired search string, 'android' in this particular case and the we touch the Search button again to actually start the action.
Following, is the script that translates our plan to monkeyrunner:


#! /usr/bin/env monkeyrunner

import sys
import os
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice

YOUTUBE = 'com.google.android.youtube-2.1.6.apk'
prog = os.path.basename(sys.argv[0])

def usage():
        print >>sys.stderr, "usage: %s" % prog
        sys.exit(1)

def main():
        if len(sys.argv) != 1:
                usage()

        print "waiting for connection..."
        device = MonkeyRunner.waitForConnection()

        print "installing youtube"
        device.installPackage(YOUTUBE)

        device.startActivity(component="com.google.android.youtube/.HomeActivity")
        MonkeyRunner.sleep(3)
        # search
        device.touch(450, 80, MonkeyDevice.DOWN_AND_UP)
        MonkeyRunner.sleep(5)
        device.type('android')
        # done
        device.touch(450, 740, MonkeyDevice.DOWN_AND_UP)



if __name__ == '__main__':
    main()

This script covers the case described in the question but it could be easily adapted for other cases and application.

I hope this is the answer you were looking for.

Android Application Testing Guide: Q&A

noreply@blogger.com (Diego Torres Milano) — Sat, 01 Oct 2011 04:09:00 +0000

Q: i am working on android on automation i have setup all the environment required for it like(eclipse,jdk, Android SDk) and i am having monkeyrunner installed.

Now i want to automate the process to do this task--The monkeyrunner API can apply one or more test
suites across multiple devices or emulators. You can physically attach all the
devices or start up all the emulators (or both) at once, connect to each one in
turn programmatically, and then run one or more tests

i thought of doing this through monkeyrunner extending plugins concept, this concept doesn't help me becoz communicaton between two or more plugins creating a new problem. so ,please help me how to get my task done through other approach

Posted by vijju

A: Probably the best solution to your problem is based on the divide and conquer philosophy. Create your test scripts in a way that they receive a serial number argument to determine which device to connect to. The following is a very simple example of this concept. We will call it getprop.mr


#! /usr/bin/env monkeyrunner

import sys, os
from com.android.monkeyrunner import MonkeyRunner

prog = os.path.basename(sys.argv[0])

def usage():
        print >>sys.stderr, "usage: %s serial-no" % prog
        sys.exit(1)

def main():
        if len(sys.argv) != 2:
                usage()

        serialno = sys.argv[1]
        print "waiting for connection to %s..." % serialno
        device = MonkeyRunner.waitForConnection(30, serialno)

        s = ""
        for p in ['build.manufacturer', 'build.device', 'build.model']:
                s += " " + device.getProperty(p)
        print s

if __name__ == '__main__':
    main()

This script expects the first argument to be the serial number of the emulator or device where the specific tests will be run. In this simple case we are just obtaining some properties to identify it.

Once we have our test script we need a driver to run it in every device we specify. In this example we will be using a python script but a bash script would be good enough.


#! /usr/bin/env python

import os

devices = [ 'XXX00000001', 'emulator-5554' ]
cmd = 'getprop.mr'

for d in devices:
        os.system(cmd + " " + d)

This script will run the test, getprop.mr in this case, for each of the devices which serial numbers are in the list.
I hope this is the answer you were looking for.

Android Testing: Running tests from code

noreply@blogger.com (Diego Torres Milano) — Thu, 08 Sep 2011 04:04:00 +0000

I was answering some questions at StackOverflow today and one caught my attention. It was asking, well, not directly but I understood this was the intention, how to run the tests not from another computer using adb or Eclipse but from an android application itself. Then, here we are presenting a solution to run instrumentation from code.



private void runTests() {
   final String packageName = getPackageName();
   final List<InstrumentationInfo> list = 
         getPackageManager().queryInstrumentation(packageName, 0);
   if ( list.isEmpty() ) {
      Toast.makeText(this, "Cannot find instrumentation for " + packageName,
         Toast.LENGTH_SHORT).show();
      return;
   }
   final InstrumentationInfo instrumentationInfo = list.get(0);
   final ComponentName componentName = 
         new ComponentName(instrumentationInfo.packageName,
         instrumentationInfo.name);
   if ( !startInstrumentation(componentName, null, null) ) {
      Toast.makeText(this, "Cannot run instrumentation for " + packageName,
         Toast.LENGTH_SHORT).show();
   }
}

You need a valid context to call startInstrumentation() so this is probably added to an Activity.

I can imagine only a valid use case for this, which is running the tests when you don't have the device connected to a computer.
Do you have another use case ?
Speak out.

Hope this helps.

Android Application Testing Guide: Q&A

noreply@blogger.com (Diego Torres Milano) — Sat, 27 Aug 2011 01:22:00 +0000

Q:I followed your example. I set up the test project in a similar way.
But try to write a TestCase for the utility class from the original
project.
Eclipse says "Class under test does not exist in the current project."

It's kind of reasonable to me, since that class is indeed in another
project.
I only have experienced using JUnit to test normal Java Project where
the test directory is inside the project.

Also, I checked out the two example projects and found there is a
build.properties and build.xml. Is that the reason that you can import
the original class:
"import com.example.i2at.tc.TemperatureConverter;"?

Thanks

Best wishes,
Ryan

Posted by Ryan Huang.

A:If you have imported both projects (main & test) into Eclipse you should have no problems because the versions available at github have the required properties set.
However, if for some reason they were not set properly, this is what you should verify in your test project's Java Build Path -> Libraries

The other files you mentioned are used when you build with ant.

LinuxCon 2011 North America: Introduction to Android Testing

noreply@blogger.com (Diego Torres Milano) — Thu, 18 Aug 2011 22:04:00 +0000

It was terrific having such a great audience showing a big interest in the topic of my tutorial, asking questions and starting discussions that made the presentation more enjoyable. I really want to thank you all.

For those who missed it, here are the slides that should be available at the Linux Foundation web site any time soon too.

Introduction to android testing

View more presentations from dtmilano.

Android Application Testing Guide: Q&A

noreply@blogger.com (Diego Torres Milano) — Fri, 05 Aug 2011 06:17:00 +0000

Q:Diego, if we were to test something that is asynchronous, like in my case I'm wanting to test if a webview loads an URL how would I go about waiting for the webpage to finish loading in my webview ?

Posted by Pedro Veloso.

A:Actually this is a very interesting question as there are many ways and you should be cautious about WebView semantics and how some of the WebViewClient methods are called.

For example onPageFinished may be invoked wether the page was successfully loaded or there was an error. So, you may need a different approach if your intention is to test if an url was successfully loaded.

In this code snippet I'm using a MockWebViewClient to detect error conditions and simply waiting some time for the page to load. We could also iterate over a period of time checking if the value has changed instead of just waiting but we are keeping this as simple as possible. This is also assuming you have an Activity holding the WebView and it has the required getters.


/**
 * 
 */
package com.example.aatg.webview.test;

import android.test.ActivityInstrumentationTestCase2;
import android.test.suitebuilder.annotation.Suppress;
import android.webkit.WebView;
import android.webkit.WebViewClient;

import com.example.aatg.webview.AndroidHelloWebViewActivity;

/**
 * @author diego
 *
 */
public class AndroidHelloWebViewActivityTests extends
        ActivityInstrumentationTestCase2<AndroidHelloWebViewActivity> {

    private static final String VALID_URL = "http://developer.android.com";
    private static final String INVALID_URL = "http://invalid.url.doesnotexist987.com";

    private static final long TIMEOUT = 5000;

    private AndroidHelloWebViewActivity mActivity;
    private WebView mWebView;
    private MockWebViewClient mMockWebViewClient;

    /**
     * @param name
     */
    public AndroidHelloWebViewActivityTests() {
        super(AndroidHelloWebViewActivity.class);
    }

    /* (non-Javadoc)
     * @see android.test.ActivityInstrumentationTestCase2#setUp()
     */
    protected void setUp() throws Exception {
        super.setUp();
        mActivity = getActivity();
        mWebView = mActivity.getWebView();
        mMockWebViewClient = new MockWebViewClient();
        mWebView.setWebViewClient(mMockWebViewClient);
    }

    /* (non-Javadoc)
     * @see android.test.ActivityInstrumentationTestCase2#tearDown()
     */
    protected void tearDown() throws Exception {
        super.tearDown();
    }

    public final void testLoadValidUrl() {
        assertLoadUrl(VALID_URL);
        assertFalse(mMockWebViewClient.mError);
    }

    public final void testLoadInvalidUrl() {
        assertLoadUrl(INVALID_URL);
        assertTrue(mMockWebViewClient.mError);
    }

    private void assertLoadUrl(String url) {
        mWebView.loadUrl(url);
        sleep();
        assertTrue(!(mWebView.getProgress() < 100));
    }

    private void sleep() {
        try {
            Thread.sleep(TIMEOUT);
        } catch (InterruptedException e) {
            fail("Unexpected timeout");
        }
    }

    private class MockWebViewClient extends WebViewClient {
        boolean mError;

        @Override
        public void onReceivedError(WebView view, int errorCode,
                String description, String failingUrl) {
            mError = true;
        }
    }
}

Hope this helps.

Android Application Testing Guide: Q&A

noreply@blogger.com (Diego Torres Milano) — Wed, 03 Aug 2011 03:59:00 +0000

Lately I've been receiving some comments or questions about some of the book's subjects in my email. To help the community the best I think that is preferable that you share your questions here, as comments to this post if you cannot find a post dealing with the same topic. If possible I will answer also here for the benefit of all.

Eclipse, Android and EMMA code coverage

noreply@blogger.com (Diego Torres Milano) — Fri, 08 Jul 2011 20:57:00 +0000

On of the examples I'm including in the tutorial I will be presenting at LinuxCon North America 2011 is a step by step use of EMMA code coverage from Eclipse to help you navigate through the Android application source code while applying Test Driven Development techniques.

This screenshot, taken from the tutorial examples, shows:

The code coverage results directly summarized and highlighted in the Activity source code inside Eclipse editor
The coverage view lists coverage summaries for the Android project, allowing drill-down to method level

If you are seriously developing Android applications you shouldn't miss this tutorial.

Hope to see you there.

Using Android monkeyrunner to automate test steps

noreply@blogger.com (Diego Torres Milano) — Fri, 24 Jun 2011 05:07:00 +0000

It is not unlikely that in writing your tests you discover that some simple steps are required by several tests.

monkeyrunner is a great tool in this respect because it offers a tremendously powerful and complete language like python.
You can write your tests using python but also you can build libraries containing primitives belonging to the test domain.
This example shows how you can lock and unlock the device screen. Usually this methods should be in a different class or module but for the sake of simplicity we are including lockDevice() and unlockDevice() here in the main script.

#! /usr/bin/env monkeyrunner
'''
Created on Jun 22, 2011

@author: diego
'''

from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice


def lockDevice(device):
    return device.press("POWER", MonkeyDevice.DOWN_AND_UP)


def unlockDevice(device):
    device.wake()
    device.drag((130, 620), (350, 620), 1.0, 120)

def main():
    device = MonkeyRunner.waitForConnection()
    if device:
        lockDevice(device)
        MonkeyRunner.sleep(5.0)
        unlockDevice(device)

if __name__ == '__main__':
    main()

Now, a demonstration of this script running on a Nexus One, locking, waking up, and unlocking the screen.

Hope this helps you start creating you monkeyrunner libraries.
Comments are welcome.

Save the date: Android Testing at LinuxCon North America 2011

noreply@blogger.com (Diego Torres Milano) — Mon, 20 Jun 2011 00:17:00 +0000

Mark your calendars. On Wednesday August 17th, 2011, 10:30 (schedule) I will be presenting the tutorial Introduction to Android Testing. It will give you an overview of current methodologies and tools available on Android. This tutorial will also introduce Test Driven Development, Behaviour Driven Development and Continuous Integration, techniques that every serious development project should at least consider.

More information at LinuxCon North America 2011.
Hope to see you there.

Android Application Testing Guide

noreply@blogger.com (Diego Torres Milano) — Fri, 03 Jun 2011 04:04:00 +0000

The wait is almost over and after a year of hard work the book is finished and is expected to be published by PACKT this month (June 2011).

You can Pre-order now !

Approach
Adroid Application Testing Guide is a highly detailed book which gives step-by-step examples for a great variety of real-world cases, providing professional guidelines and recommendations that will be extremely valuable for optimizing your development time and resources. In the chapters you will find an introduction to specific testing techniques, and tools for specific situations.

Overview of Android Application Testing Guide

The first and only book that focuses on testing Android applications
Step-by-step approach clearly explaining the most efficient testing methodologies
Real world examples with practical test cases that you can reuse
eBook available as PDF and ePub downloads and also on PacktLib

More detailed information can be obtained from its web page at PACKT.

Running instrumentation from monkeyrunner

noreply@blogger.com (Diego Torres Milano) — Sat, 23 Apr 2011 20:54:00 +0000

Sometimes you may want to run the instrumentation for a package from monkeyrunner, and the following script does just this. It has the advantage of using the package manager to find the correct instrumentation for the specified package simplifying the parameters you must provide.

#! /usr/bin/env monkeyrunner

import sys
import os
from com.android.monkeyrunner import MonkeyRunner

PLI = 'pm list instrumentation'
prog = os.path.basename(sys.argv[0])

def usage():
   print >>sys.stderr, \
       "usage: %s target-package-name" % prog
   sys.exit(1)

def main():
   if len(sys.argv) != 2:
      usage()

   pkg = sys.argv[1]

   print "waiting for connection..."
   device = MonkeyRunner.waitForConnection()

   print "running istrumentation for %s" % pkg
   for (i, t) in map(lambda l: l.split(), device.shell(PLI).splitlines()):
      if t == '(target=%s)' % pkg:
         print device.instrument(i.split(':')[1], { 'wait':True })['stream']
         return
   
   print >>sys.stderr, "ERROR: instrumentation for %s not found" % pkg


if __name__ == '__main__':
    main()

You can invoke this script as in the following command line:

$ instrumentation.mr com.example.package

compare this against the command line you may need to invoke the same instrumentation using plain am instrument.
Once you invoke the script with the correct package name, the instrumentation for that package is run and the results are presented, like in this example:

waiting for connection...
running istrumentation for com.example.package

Test results for InstrumentationTestRunner=..................................
Time: 37.844

OK (34 tests)

monkeyrunner: visual image comparison

noreply@blogger.com (Diego Torres Milano) — Sun, 10 Apr 2011 22:38:00 +0000

We have discussed taking screenshots with monkeyrunner in previous posts, but now we are taking a step further by comparing the screenshots obtained during the test run against some reference image.

Firstly, we need these reference images that you can obtain by running specific cases of the tests specially designed to do it. It's not recommended to use DDMS to take the reference image as in some cases the image format and compression may slightly differ from the ones taken by monkeyrunner leading to false positive identification of the dissimilarity.

As a fairly simple example we will be creating a test to verify the correct drag and drop to a new position of a screen widget , in this case the Home screen tips. We will be sending touch events to move it from its original position in row 2:

to row 4:

This second image, showing the Home screen tips widget in row 4 will be our reference image or the final state we want to verify in our test.

There are some items we can anticipate will be different, like the time in the status bar, the battery level, the connectivity, etc. Thus why we expect some degree of flexibility in our test not to fail under these circumstances.

We also use compare from the great ImageMagick package that you should have installed for this script to work.

In Debian/Ubuntu and derivatives

$ sudo apt-get install imagemagick

Having this in mind, our monkeyrunner test would look something like this.


#! /usr/bin/env monkeyrunner

import sys
import subprocess
from com.android.monkeyrunner
import MonkeyRunner, MonkeyDevice, MonkeyImage

TIMEOUT = 30
SERIALNO = 'emulator-5554'
REF = 'reference.png'
SCR = 'screenshot.png'
CMP = 'comparison.png'
ACCEPTANCE = 0.9

device = None


def testDropWidgetIntoFourthRow():
    reference = MonkeyImage.loadFromFile(REF)
    print "moving widget..."
    device.drag((230, 300), (230, 600), 2.0, 125)
    MonkeyRunner.sleep(3)
    screenshot = device.takeSnapshot()
    screenshot.writeToFile(SCR)
    if not screenshot.sameAs(reference, ACCEPTANCE):
       print "comparison failed, getting visual comparison..."
       subprocess.call(["/usr/bin/compare", REF, SCR, CMP])

def main():
    global device
    print "waiting for connection..."
    device = MonkeyRunner.waitForConnection(TIMEOUT, SERIALNO)
    if device:
       testDropWidgetIntoFourthRow()


if __name__ == '__main__':
     main()

We are using MonkeyImage.loadFromFile() to load the reference image from a file.

This method was not available in monkeyrunner so I implemented it myself and I decided to do it in MonkeyImage, but it is now included in latest verions of monkeyrunner but as MonkeyRunner.loadImageFromFile().

In case you don't have it in your version you can build it from source (being sure that this patch is included).
monkeyrunner is a SDK component and thus is included in this project that can be downloaded from android SDK source.
Anyway, if you can't do it you may just avoid this step and replace the condition in the if by True.

A brief explanation of the script is:

in the main method we obtain the connection with the device using the serial number specified in SERIALNO. A more sophisticated script should use parameters.
if the connection was successful we run the test
in the test method we load the reference image from the specified file
we send a simulated drag event to move the widget from the second row to the fourth row, using screen coordinates
sleep for a while
get the screenshot
if the images are not the same, considering 90% acceptance value, we use the visual comparison to get a clue of the difference

If you run the script you can verify that the screenshot is taken and the test succeed because they are the same. They only have minor differences in the time and battery level but they lie above the acceptance value.
Now, if we run the test again but this time using (230, 400) instead of (230, 600) in the drag command the widget will be dropped in the third line instead of the fourth and we expect this difference be detected as our reference image for the final state contains the widget in the fourth row, and thus precisely what happens.
This is the comparison image highlighting in red the differences:

Check that as we mentioned before the battery level and time in the status bar are also detected as differences but they are usually under the threshold.

This is not a real repeatable test as we are not leaving the system in the same state that was found but the idea is to present the subject in the simplest possible way to avoid deviating from the main goal which is demonstrating how you can add visual comparison to your tests using monkeyrunner and ImageMagick.
I'm sure this will give you lots of ideas to implement you own tests.
As always, comments are gladly welcome.

Using Android monkeyrunner from Eclipse

noreply@blogger.com (Diego Torres Milano) — Sun, 13 Mar 2011 01:40:00 +0000

You may want to edit scripts and run them from Eclipse.
To be able to do this you must first install PyDev from Eclipse Marketplace.

Latest versions of PyDev don't work with monkeyrunner because it is not detected as a valid interpreter. I'm sure this will be finxed in the future. In the meantime you should stick to PyDev 1.6.5.

Then you have to define a new python interpreter

but before you can do it you need to do some changes to the Android installation. We need to replace the monkeyrunner interpreter because Eclipse invokes it using the -u command line option, unbuffer stdin, stdout and stderr, which is not supported by monkeyrunner. To solve it we should rename the original interpreter to monkeyrunner-original and then use this script as a replacement for monkeyrunner.

# /bin/bash
if [ "$1" = '-u' ]
then
 shift
fi

exec /opt/android-sdk-linux_86/tools/monkeyrunner-original "$@"

if you are using Microsoft Windows, you should use something like this (script contributed by tagmaster)

REM
@echo off
if("%1")==("-u") shift
"C:\Program Files\Android\android-sdk\tools"/monkeyrunner-original %1 %2 %3 %4 %5 %6 %7 %8

Once the new interpreter is defined and assigned to a PyDev project you will be able to edit and run monkeyrunner scripts from Eclipse

Don't forget to set the previously defined interpreter to the project properties.

Update: July 2011
This patch was added to monkeyrunner to ignore the -u option sent by PyDev:
http://android.git.kernel.org/?p=platform/sdk.git;a=commitdiff;h=f07d8c2026606633f1de7e1ab9d985c74ad8b340
However, the usage string does not include the -u option yet. BTW, credit would be nice.
So, the wrapper is not needed any more.

Update: September 2011
monkeyrunner included in Android SDK Tools revision 12 supports -u option, however as it was mentioned before it is not described in the help text.
Come on, we are only one step behind...

Obtaining Android screenshots in tests with monkeyrunner

noreply@blogger.com (Diego Torres Milano) — Tue, 01 Feb 2011 03:44:00 +0000

Today I helped somebody that was asking why an extremely simple monkeyrunner script fails to obtain the screenshot.
Interestingly enough, this script was almost a verbatim copy of what is specified in monkeyrunner documentation. So you guessed, the documentation is wrong and unfortunately there is no way of correcting it, Android documentation is not a wiki and the only way is reporting a bug that will be probably ignored sunk in a sea of thousands more important bugs.

So here you have the problem and the solution.

The problem
The documentation includes this line to take the screenshot:

# Takes a screenshot
result = device.takeSnapShot

if you run this, you soon realize that something is wrong. You will receive an error similar to this one:

AttributeError: 'com.android.monkeyrunner.adb.AdbMonkeyDevice' object has no attribute 'takeSnapShot'

The problem, or better I should say the problems, because there are two, are:

missing parenthesis, so instead of using the call operator on the callable object we are getting a reference to the object itself and then try to access one attribute
if there were parenthesis, the call won't succeed anyway because the second 'S' is lowercase 's'

The solution
Then this is the complete solution to obtain the screenshot:

# Imports the monkeyrunner modules used by this program
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
# Connects to the current device, returning a MonkeyDevice object
device = MonkeyRunner.waitForConnection()
# Takes a screenshot
result = device.takeSnapshot()
# Writes the screenshot to a file
result.writeToFile('/tmp/device.png','png')

Extra tip
You can add the corresponding she-bang to the script and set the execution permission to be able to run it from the command line just invoking the script name:

#! /opt/android-sdk-linux_86/tools/monkeyrunner
...

This should include the path to your monkeyrunner script, and because it uses Jython, an implementation of Python that uses the Java programming language, thus allowing the use of Python syntax to access the constants, classes, and methods of the API.
Having added this she-bang line you can simply run your monkeyrunner script by invoking its name:

$ monkeyrunner-take-screenshot

Alternative logcat viewer

noreply@blogger.com (Diego Torres Milano) — Fri, 14 Jan 2011 23:48:00 +0000

If you are developing Android with Eclipse and have several devices and emulators connected you may have been annoyed by the constant disconnection that the DDMS view suffers, in such cases you could try to reconnect to the desired device by clicking on it, but this is not always successful.

I've found as a workaround that sometimes is easier to reconnect if you reach another device or emulator first and then the desired one, so as a rule of thumb is probably better to always have more than one.

Anyway, here you are a very simple trick that might turn you life a little easier just using a text logcat from adb in a gnome-terminal that will remain open no matter what have happened to the connection.

First, you need to create a gnome-terminal profile, logcat in this case, an set the following values as dispalyed:

The important things here are:

Run a custom command instead of my shell
Custom command: sh -c '/opt/android-sdk/platform-tools/adb -e logcat | coloredlogcat.py'
When command exits: Hold the terminal open

As a bonus, we are including coloredlogcat.py, a filter that will colorize our logcat output.
What's nice about this simple setup is that even if you are disconnected because you closed the emulator or unplugged the device, just clicking on the Relaunch button will reconnect you again.

Hope you enjoy it and improve your development not having to insistently click on DDMS' device list.

Problems updating Android to 2.3 in Ubuntu

noreply@blogger.com (Diego Torres Milano) — Fri, 31 Dec 2010 19:38:00 +0000

If you were using Ubuntu 9.10 (karmic) because you wanted to avoid some issues with Eclipse and Android ADT plugin that happened on newer versions on Ubuntu, well now is time to upgrade to the latest Ubuntu 10.10 (maverick) if you want to update to the latest Android 2.3 (gingerbread).

You may have discovered this after the update finished using Android SDK and AVD Manager tool and you sadly realize that your Android development environment was turned useless without a single word of warning. Latest Android SDK version depends on GLIBC_2.11 but it's no available for previous versions of Ubuntu, so unless you want to take the risk of using a different repository trying to find an update with unknown consequences to other components, upgrading is your only possible path.
If this is not a serious bug we should reconsider the definition of the word.

The origin of the problem is that Android SDK distribution is not package based and thus no dependencies are considered before updating the software. This is clearly a nonsense. Google has taken a completely different approach with Chrome which is cleverly distributed from a package repository as we can see in this screenshot of Ubuntu Software Center.

Android SDK must be distributed as packages as Chrome does, otherwise there is no way to prevent this problems happening again and again in the future. We are only scaring people away of Android development with this poor policies. It's true that the requirements page was updated and GLIBC 2.11 is listed now as a requirement, but when you discover it, it's too late. You expect some verifications done from software doing an update that cannot easily be reverted.

This is perhaps the most important bug, but there are others after upgrading to Ubuntu 10.10 (maverick) and Android SDK 2.3 (gingerbread) some other things are still broken, for example hierarchyviewer does not run out of the box and you have to manually copy some libraries,
Android SDK and AVD Manager updates some packages again an again ignoring they are already installed and some deficiencies of the Android ADT Layout editor, to name a few.

I complained about many things about Android in this blog and it seems that in one way or another they were fixed in following releases, so I hopefully think this would be the case.

Android 2.2 changes

noreply@blogger.com (Diego Torres Milano) — Mon, 24 May 2010 23:43:00 +0000

Android 2.2, a minor platform release, arrived last week with a bunch of changes ranging from Dalvik JIT and Kernel Memory Management Boost to a new Launcher.

You can check the highlights to get a bigger picture.
But there's one change that's almost not announced anywhere, and I consider it very important too.

With the advent of Android 2.2 you can install or upgrade the SDK and tools in Linux as root and the permissions are (almost) correct and you don't need further changes in this situation, or what is almost the same we can declare android-fixperms-updated defunct and no longer needed.

This is very good news if you are using continuos integration on Android as was described by various posts in this blog.

Great job Google.

Android course to be held at SkillsMater in April

noreply@blogger.com (Diego Torres Milano) — Sun, 28 Mar 2010 19:53:00 +0000

The Android course authored by myself and published by Ricston is to be held at SkillsMater (London, UK) again on April 14-16 2010.

There are still some seats available for those that want to attend.

Ross Mason, founder and CTO at MuleSoft who attended this Android course said:

"it provided a great introduction to developing on the Android platform. You can see that the author has crafted a well-structured course that covers the core knowledge required by developers to understand the potential of the platform and how build apps on it. Delivery was smooth, the instructor very well informed. This course provides an excellent way to get proficient in Android in a few days."

Android: Generate javadoc for your project

noreply@blogger.com (Diego Torres Milano) — Wed, 24 Feb 2010 23:48:00 +0000

So you want to follow the best practices and generate the javadoc for your android project from Eclipse and as soon as you start you hit the first wall, messages like "package android.app does not exist" completely fill your output console.

What's wrong ?

And most important, how can you generate complete javadoc with links to android documentation ?

Find out how reading the complete article in Google Docs: http://docs.google.com/View?id=ddwc44gs_240hkc84xfd