Direct service invocation considered harmful

I have a more than 2 years old - and quite popular - blog post about service binding and I thought I knew enough of Android services. Then I ran into a service binding pattern that I was not aware of before. I would like to clarify before I start that every alternative in this post "works" - meaning it does what it is intended to do on the Android versions released so far. Still, it seems that this hack I am going to present got into tutorials and books. I think there is a risk here that a good number of coders think about it as the right thing to do and not as a hack.

Click here to download the example program.

This is a version of the good old DualService presented in this blog entry. The main difference that the AIDL file has been removed and instead you will find this strange construct in DualService.java:

public class DualServiceBinder extends Binder implements ICounterService {
public int getCounterValue() {
return counter;
}
}

and in onBind:

public IBinder onBind( Intent intent ) {
return dualServiceBinder;
}

Now this DualServiceBinder "Binder derivative" is a strange thing. Binders are serialization-deserialization constructs that turn RPC invocations into byte streams and back. That byte stream can be sent through the Binder kernel driver and that's how interprocess communication works in Android. Binder derivatives are normally generated by the aidl tool. You can do it by hand but it is not for the faint of heart. Basically you have to implement transact/onTransact methods and serialize/deserialize your data by hand. If you follow my advice, you leave it to the aidl tool.

DualServiceBinder does none of it. It implements a plain Java interface (ICounterService) and exposes a plain Java method. On the other side, in DualServiceClient it justs casts the IBinder to ICounterService in the onServiceConnected method and invokes the method on it.

As you might have guessed, this is a hack. The author sidestepped entirely the Android IPC mechanism and abused two properties of the Android application model.

• By default, the Android application manager loads activities/services/providers into the Dalvik VM running in the same Linux process.
• Classes from the same DEX file (there is one DEX file per APK) are loaded by the same class loader.

These guys here do not use Binder at all. They need an IBinder implementation because of the onBind/onServiceConnected methods' contracts and Binder comes handy as it is an implementation of the IBinder interface. Other than that, the DualServiceBinder instance is just passed from the Service to the Activity without any IPC taking place. That is why it can be recasted to a simple Java interface and the method can be invoked without any IPC mechanism. The hack is actually faster than the AIDL-based implementation (as there is no serialization/deserialization, just in-memory parameter passing) but this approach is not supported in any way by the Android API contracts.

Do you think I found the hack in a shady, unsupported freeware? No, I found it in a book (actually, my attention was turned to this book by a fellow Android programmer). The book proposes another trick: the "service interface" of their Binder descendant exposes just a getService() method that returns a reference to the Service instance and later the Activity invokes public methods of the service directly, circumventing even their own tricky service interface.

I definitely think it is a bad idea to propagate this hack even though it works on the existing Android implementations and is somewhat faster than the regular, AIDL-based invocation.

Plugins

I blogged 2 years ago that Android is a surprisingly modern application platform, I even said that it was the most modern application platform. This is partly due to Android's advanced component features. These features are not always evident to the naked eye so I present here a standard exercise to demonstrate them.

Click here to download the example program.

In this exercise we will implement a plugin framework for an example program. Plugins are components that connect to the main application over a uniform interface and are dynamically deployable. The download package contains two Android applications. The first, pluginapp is the example application that will use the plugins. The second, plugin1 is a plugin package that contains two plugins for pluginapp. Plugin1 package is not visible among the executable applications, it does not expose any activity that may be launched by the end user. If deployed, however, its plugins appear in pluginapp. Plugin deployment/undeployment is dynamic, if you uninstall plugin1 package while pluginapp is running, pluginapp notices the changes and the plugins disappear from the plugin list.




In order to implement these functionalities, the following features of the Android framework were used.

• Each plugin is a service. Plugin1 exposes two of them. In order to identify our plugins, I defined a specific Intent (aexp.intent.action.PICK_PLUGIN) and I attached an intent filter listening to this intent in plugin1's AndroidManifest.xml. In order to select a particular plugin, I abused the category field again. One plugin can be accessed therefore by binding a service with an intent whose action is aexp.intent.action.PICK_PLUGIN and whose category equals to the category listed in AndroidManifest.xml of the package that exposes the plugin service.
• Plugins are discovered using the Android PackageManager. Pluginapp asks the PackageManager to return the list of plugins that are bound to aexp.intent.action.PICK_PLUGIN action. Pluginapp then retrieves the category from this list and can produce an intent that is able to bindthe selected plugin.
• Pluginapp updates the plugin list dynamically by listening to ACTION_PACKAGE_ADDED, ACTION_PACKAGE_REMOVED and ACTION_PACKAGE_REPLACED intents. Whenever any of these intents are detected, it refreshes the plugin list.

Even though Android is not as sophisticated as OSGi when it comes to component framework, it has every necessary element. It has a searchable service registry, packages can be queried for the services they expose and there are deployment events.

Shake-walk-run

Well, anybody can draw up frightening equations like I did in the acceleration signal analysis report in my previous post and claim that they work just because of some diagrams made by a mathematical analysis program. You'd better not believe this sort of bullshit before you experience it yourself. To help you along that path, I implemented a simple application that let you try out, how these methods work in real life.

Click here to download the example program.

If you open this simple application, you can enable the sampling with a checkbox. Once the sampling is switched on, the service behind the app constantly samples the acceleration sensor and tries to detect motion patterns. It is able to distinguish 3 patterns now: shake, walk and run. Not only it distinguishes these patterns, it also tries to extract shake, step and run count. Unfortunately I tested it only on myself so the rule base probably needs some fine tuning but it works for me. The walk detector uses the w3 wavelet (read the report if you are curious, what w3 wavelet can be) and is therefore a bit slow, it takes about 2-3 seconds before it detects walking and starts counting the steps but that delay is consistent - if you stop walking, it continues counting the steps it has not counted yet.

The moral for this application is that the tough part in detecting motion patterns is the rule base behind the signal processing algorithms. It is OK that wavelets separate frequency bands and signal power calculation allows to produce "gate" signals that enable/disable counting for certain motion patterns. But where are the signal power limits for e.g. shaking and how to synchronize "gate" signals with the signals to be counted? This example program is a good exercise as you can observe, how it uses delayers and peak counters synchronized with signal power calculators to achieve the desired functionality.

Some notes on the implementation. The machinery behind the background service that sends events to an activity is described in this and this blog posts. Also, observe the debug flags in SamplingService, if you set one of these, the service spits out signal data that you can analyse with Sage later. I did not include the keep-awake hack into this implementation because it was not necessary on my Nexus One with the leaked Android 2.2. I put a public domain license on the download page because somebody was interested in the license of the example programs.

I have to think a bit, where I want to go from here. The most attractive target for me is to formalize all these results into a context framework for Android applications but I may fiddle a bit more on signal processing.

Analysing acceleration sensor data with wavelets

Before we get back to Android programming, we need some theoretical background on signal analysis.The document is somewhat heavy on math. To quote Dr. Kaufman's Fortran Coloring Book: if you don't like it, skip it. But if your teacher likes it, you failed.

Click here to read the report.

If you are too impatient to read, here is the essence.

There is no single perfect algorithm when analysing acceleration signals. The analysis framework should provide a toolbox of different algorithms, some working in the time-domain, some operating in the frequency domain. The decision engine that classifies the movements may use a number of algorithms, a characteristic set for each movement type.

It has been concluded in the medical research community that wavelet transformation is the most optimal algorithm for frequency-domain analysis of acceleration signals. This report presented concrete cases, how wavelet transformation can be used to classify three common movements: walking, running and shake. In addition, the wavelet transformation provided data series that can be used to extract other interesting information, e.g. step count.

For those who would like to repeat my experiments, I uploaded the prototype. First
you need Sage (I used version 4.3.3). Download and unpack the prototype package and enter the proto directory. Then launch Sage (with the "sage" command) and issue the following commands:

import accel
accel.movements(5)

now you will be able to look at the different waveforms, e.g.

list_plot(accel.shake_w5)

Sage is scriptable in Python. If you know Python, you will find everything familiar, if not - bad luck, you won't get far in Sage.

Movement patterns

This blog entry does not have associated example program, the samples of this post were taken using the sensor monitoring application presented here. This time I will give a taste what can be achieved by using the acceleration sensor for identifying movement patterns. Previously we have seen that the acceleration sensor is pretty handy at measuring the orientation of the device relative to the Earth's gravity. At that discussion we were faced with the problem what if the device is subject to acceleration other than the Earth's gravity. In that measurement, it caused noise. For example if you move the device swiftly, you can force change between landscape and portrait mode even if the device's relative position to the Earth's surface is constant. That's because the acceleration of the device's movement is added to the Earth's gravity acceleration, distorting the acceleration vector.

In this post, we will be less concerned about the device's orientation. We will focus on these added accelerations. Everyday movements have characteristic patterns and by analysing the samples produced by the acceleration sensor, cool context information can be extracted from the acceleration data.

The acceleration sensor measures the acceleration along three axes. This 3D vector is handy when one wants to measure orientation of e.g. the gravity acceleration vector. When identifying movements, it is more useful to work with the absolute value of the acceleration because the device may change its orientation during the movement. Therefore we calculate

ampl=sqrt( x^2 + y^2 + z^2 )

where x,y,z are elements of the measured acceleration vector along the three axes. Our diagrams will show this ampl value.

Let's start with something simple.



This is actually me walking at Waterloo station. The diagram shows the beginning of the walking. It starts with stationary position then I start to walk. In order to understand the pattern, we must remember that acceleration is change of the velocity vector. This means that if we move quickly but our velocity does not change, we have 0 acceleration. On the other hand, if we move with constant speed but our direction changes, that's quite a significant change in the velocity vector. That is the source of the gravity acceleration, the Earth's surface (and we along with it) rotates with constant speed but as the speed is tangent to the Earth's surface, its direction constantly changes as the Earth rotates.

The same goes for walking. Whenever the foot hits the ground, its velocity vector changes (moved down, starts to move up). Similarly, when the foot reaches the highest point, it moved up and then it starts to move down. The maximums and the minimums of the acceleration amplitude are at these points. As these accelerations are added to the Earth's acceleration (9.81 m/s^2), you will see an offset of about 10. The sine wave-like oscillation of the acceleration vector's absolute value is unmistakable. The signal is not exactly sine wave but the upper harmonics attenuate quickly. You can see that my walking caused about 1.2g fluctuation (-0.6 g to 0.6 g).

Running is very similar except that the maximum acceleration is much higher - about 2 g peak-to peak.



And now, everybody's favourite, the shaking. Shaking is very easy to identify. First, the peak acceleration is very high (about 3 g in our case). Also, the peaks are spike-like (very high upper harmonic content).



In the following, I will present some algorithms that produce handy values when identifying these movements.

Monitoring sensors in the background

Once I got the taste of making all sorts of measurements with sensors, I happily started to collect samples. Then I quickly ran into the background sensor monitoring problem. The use case is very simple: the phone is in idle state, keyguard is locked but it collects and processes sensor data. For example I wanted to record acceleration sensor data while cycling. I started the sensor application, started the measurement and locked the keyguard. On the Nexus One it means that there will be no further sensor data delivered to the application until the screen is on.

The source of the problem is that the sensor is powered down when the screen is off. This is implemented below the Android framework (either in the kernel driver or in hardware, I would be curious if anyone knows the answer) because if you recompile the latest sources from android.git.kernel.org, sensor data will be delivered nicely to the application in the emulator even if the keyguard is locked (the stock 2.1 emulator's Goldfish sensor does not emit any event, that is why you have to recompile from source) . The fact remains: if you want acceleration sensor data on the Nexus One, the screen must be on. This pretty much kills all the user-friendly applications that want to analyze sensor data in the background while the phone is idle. In the worst case, the screen must be constantly on (e.g. for a pedometer that needs to measure constantly because you never know when the user makes a step) but the situation for a simple context reasoner service is not much better. Such a service (read the vision paper for background, you need free registration to access) may want to sample the sensors periodically (e.g. collecting 5 sec. of samples in every minute). In order to perform the sampling, the screen should be switched on for this duration which would result in a very annoying flashing screen and increased power consumption.

You can observe these effects in the improved Sensors program that is able to capture sensor data in the background.

Click here to download the example program.

Start and deploy this program and you will see the familiar sensor list. If you click on any list item, the old interactive display comes up. The improvement here is that the application measures the sensor sampling frequency. The rate can be set in the settings menu, from the main activity. If, however, you long-press on the list item, the sampling will be started in the background. In this case there is no interactive display of the samples, they are always saved into the /sdcard/capture.csv file. The background sampler does respect the sampling rate setting, however. The background sampler is implemented as a service.

So far there is nothing extraordinary. You may observe the ScreenOffBroadcastReceiver class in SamplingService that intercepts ACTION_SCREEN_OFF intent broadcasts and does something weird in response. In order to understand its operation, you must be aware that the power manager deactivates all the wake locks that are stronger than PARTIAL_WAKE_LOCK when the keyguard powers the device down. Obtaining for example a SCREEN_BRIGHT_WAKE_LOCK in the broadcast receiver would be of no use because at this point the screen is still on and the wake lock would be deactivated when the screen is turned off. Instead, the broadcast receiver starts a thread, waits for 2 sec (experimental value) and then it activates the wake lock with wakeup option. Try to push the power button when the background sampling is on, the screen will go off for 2 seconds but then it is turned on again and you will see the keyguard screen. If you check the log, you will see that the sensor sampling stops for that 2 seconds then it comes back on. Don't worry, when the background sampling is stopped (by long-pressing the sensor's name in the list again) the screen will turn off correctly.

Ugly? You bet. Not only the solution is a bad hack but it also prevents the device from switching off the screen and those AMOLED displays plus the high-powered processors eat battery power like pigs. The decision that the sensors are powered down when the screen is off prevents the implementation of some of the most exciting mobile applications and significantly decreases the value of the platform.

But enough of the grunting. This is the state of the art in background sensor sampling in Android (as of version 2.1 of the platform), I will continue with signal processing algorithms.

Android training in video format

I received an e-mail asking me to advertise another Android training material. And I do it because I have never seen an Android training in video format before.

Here it is, lo and behold.

I have to admit, I am a bit skeptical. If there is a text material and video (e.g. on a news site), I always choose the text because I read pretty quickly. But I leave the decision for you. Try the free sample (increase the resolution and make it full screen, then it becomes readable) and if it works for you, consider subscribing to the course for an - IMHO - quite modest fee.