Saturday, June 13, 2009

Controlling application separation

I thought I knew how application separation in Android works. Then I had a little project and I realized that there is more than meets the eye. Here are my experiences.

The basic - and very efficient - separation method in Android is the multiprocess capability of the Dalvik VM. Dalvik is able to fork many instances of itself and can run applications in different Linux processes. By default, each application (the application tag in AndroidManifest.xml) has its own process. There is however another, often overlooked separation mechanism inside each VM process that provides further separation. The good old ClassLoader separation - originally designed to separate applets, used heavily in multi-module frameworks like OSGi - is also at work. Inside each VM process, each task - acitivity or service - uses in its own ClassLoader. This means that object instances created in different tasks are not visible to each other. The standard setup for Android applications is such that the classes.dex in the application's APK package is on the path of the application's own classloader. The application's own classloader delegates the loading of system classes to the boot classloader instance which is common for all the tasks in one particular VM process.

Class loading implementation in Dalvik is tied heavily to DEX files. This has visible side effects. Classes loaded by different class loaders that don't delegate to each other should be completely separated. The implementation differentiates the "initiating" and "defining" classloader (the latter is the one that managed to call defineClass() for a particular class from a particular DEX file in a particular VM process). This differentiation does not work very well and results in a glitch: once a class from a particular DEX file is defined, its static fields are visible and shared from all the class loaders that access the DEX file, even if they don't delegate to each other. This means that tasks of the same application (usually packed into the same DEX file) see each other's static fields.

As if this was not enough, Android provides control of task-VM process mapping. As said previously, usually each application goes into its own VM process. It is possible to declare, however, that applications share a process. This works like the following:
  • First we declare that applications wishing to share the same VM process belong to the same user ID. This is done by adding the sharedUserId attribute to the manifest tag in AndroidManifest.xml file like the following: android:sharedUserId="aexp.share.sharedapp". The value of the attribute must be a unique string shared among the applications wishing to use the same VM process. This is a powerful mechanism, hence Android enforces strong check of the APKs containing the declaration. Only the first APK for a particular sharedUserId can be installed without limitation. The issuer of the digital signature of that APK is saved and further APKs with the same sharedUserId must have digital signature from the same issuer, else their installation is rejected.
  • We have to declare the alias of the process a certain task or the entire application goes into with the process attribute. This attribute can be added to application, activity or service tags in the AndroidManifest.xml file. For example: android:process="aexp.share.sharedappprocess". VM processes are proper Linux processes identified by PIDs but Android does an additional transformation: whenever a process is started which is tagged by the process alias, the Android framework saves the Linux PID and the process alias. If some other application refers to that process alias and the VM process is still running, that process will be used.
How can we exploit this mechanism? We can create ordinary, well-behaving Android applications and place them into the same process. Android IPC is such that the usual binder-based communication is much faster in this case. The other approach is that we exploit our knowledge about class loaders. If we place two applications into the same process, they not only have different class loaders but also come from different DEX files. This means that the Android class loading glitch about the static fields cannot be exploited (I don't recommend it anyway as it is a clear bug and I believe, it will be corrected soon). We know, however, that each class loader in the same VM process delegates the resolution of the system classes to the boot class loader. If we find a system class that allows us to store and retrieve data, we win.

Click here to download the example program.

Our two example programs demonstrate the techniques described below. SharedApp1 allows the user to enter a string and it shares that string in different ways. With its own popup, it shares the string by means of a static field of SharedApp1 activity class. SharedApp2 is another application, having its own APK file but is placed into the same process as SharedApp1. SharedApp1 and SharedApp2 communicate through the good old System.getProperty/setProperty mechanism. Observe, that this works only if the sharedUserId/process mechanism is properly used, if you remove e.g. the process attribute from the manifest file of any of the apps, the property set by SharedApp1 is not visible anymore to SharedApp2 because they run in different VM processes. Also note that I compiled these applications with the debug option which means that they are both signed with the debug key. That key is common for both applications therefore they can have the same sharedUserIds. If you use a proper signing key, take care of using the same key when you sign applications that depend on the sharedUserId mechanism.



And now for something completely different. I was one of the reviewers of Roy Osherove's book titled The Art of Unit Testing (Manning, about 27 USD). Even though the book uses .NET examples, it is easily readable for Java programmers too. Unit testing is one of the most useful techniques I am aware of in software engineering, if you feel that your knowledge is not up to date, you might as well read this book. :-)

15 comments:

Lior said...

Maybe you can help me out:

I'm trying to replace the default class loader. I don't want to change the existing functionality, just add new functionality. My code has the following lines:

ClassLoader classLoader = new CustomClassLoader("/data/data/com.test/com.test.ClassLoaderTest.apk", ClassLoaderTestActivity.class.getClassLoader());
Thread.currentThread().setContextClassLoader(classLoader);
TestClass test = (TestClass)Class.forName("com.test.TestClass", true, classLoader).newInstance();

I'd expect that after this code runs and I check the value of test.getClass().getClassLoader() I'd see an instance of CustomClassLoader, but I always get the default PathClassLoader, on matter what I do.

Gabor Paller said...

What if you call ClassLoader.loadClass()? There are examples of this call in Android platform code e.g. here.

Anonymous said...

Hi,

I need some help. I am trying to figure out a way in which my native service (started through init.rc) and its interface exposed through a android service (in a apk) can both run in the same process. Is it possbile? If yes, how?

Thanks

Gabor Paller said...

Anonymous, Dalvik is itself a process so if you insist on having a native service and started from init.rc then they cannot be in the same process.

If, however, you build an Android service and start it at boot time (see this for details), you can start an Android service from it which can then call into the native code using JNI. This way your native code and the Dalvik instance that started the Android service will be in the same Linux process.

Anonymous said...

The issuer of the digital signature of that APK is saved and further APKs with the same sharedUserId must have digital signature from the same issuer, else their installation is rejected.

How do I ensure this happens.? How do I set the digital signature to the other application that has the same shareduserID.? Can u pls help me out with this.

Thanks in advance... :)

Gabor Paller said...

Anonymous, you have to have access to the signing key that signed the first APK.

I know that this is not the answer you want to hear.

If you want to place your Activity into the process of some system function, e.g. Phone, you have to know the signing key that the Phone.apk was signed with. That is possible only if you manufactured the phone or you built the firmware image.

Anonymous said...

Hi Thanks a lot for the input. Now it is working fine. 1 more doubt if u can pls answer it would be really great.

Now I have my two application installed in my device using the SharedUserId, now i want to access the database of the 1st application when I run my 2nd application and store them into the database of my 2nd application. Is it possible? if so can u pls pls pls help me how to do it. Its really very important to me. Please... I would be really grateful to u if it works.

Thanks inadvance. Looking forward to u r reply. :) Cheers

Anonymous said...

Hi Gabor,

Thanks for this post! Its really helpful.
I am looking at the sample application you have inlcuded in the post. You are using System.get and setproperty to access data across the apps.

Wanted to ask you your opinion on the following:

Is this possible to actually load another class from the shared app?

I am doing this -

1. Create 2 apks. (1 with an activity - lets call it F1App and 2nd
with a service - F1Service)
2. Both APKs have same sharedUserId and same android.process and are
signed with the same key - I have been able to confirm that the 2 are
now running in the same process.
3. From App 1 I bind to F1Service
4. Bind returns successfully but I am unable to access the classes
using the binder. The same code works fine in the sample code but the 2 there are in the same apk.

For my code you can look at this link -
http://stackoverflow.com/questions/3162538/2-apks-running-in-1-process-sharing-code-and-data

would appreciate any help.

Thanks!

Gabor Paller said...

IMHO, it is impossible to load a class from another APK even if the applications run in the same process. This is due to the DEX class loader. The DEX class loader is able to access classes only from the DEX in the APK or from the system class path.

Anonymous said...

hi,
suppose i have an application "App1" which has a button and when this is clicked it has to start another app say "App2" but this "App2" should not be visible to any other application except "App1" in simple i should be able to start "App2" only from "App1" not even from managed activities or long press of home button
pls help me with this

Gabor Paller said...

Anonymous, remove the familiar intent-filter tag from the activity (action=android.intent.action.MAIN, category="android.intent.category.LAUNCHER") then you can start the activity from App1 with explicit intent invocation.

Charles said...

Hi Gabor,

Thanks for this post! It's really helpful.
I've one more question about this mechanism.

With this mechanism we can share data in the same process (using system.getProperty and system.setProperty).

I'd like to make this data persistent ( across a reboot for example).

Is there a way to do the following thing :

The app A is launched in the process P.
The app A sets a system property D.
The app A is definitely killed.
The app B is launched in the process p.
The app B gets the system property D set by the app A.

In other words, is there a way to maintain shared and persistent data for a same process ?

BR,

Charles.

Gabor Paller said...

"The app A sets a system property D.
The app A is definitely killed.
The app B is launched in the process p."

Is p==P? If yes, there is a chance that the attraction succeeds. "Empty" processes (that do not host activities or services) do not die immediately but they have the lowest priority if resources need to be freed. So if P is still alive then B will be launched into it and would get D. This behaviour, however, is probabilistic and must not be relied upon.

chihwahli said...

Hello, I wonder if it's possible to make the zip file available again.


Manu thanks.

Betty said...

This post helped me a lot. I implemented the same functionality but was facing so many issues. The program that you have posted helped me a lot and I would like to thank you for sharing this info.
electronic signatures