Wednesday, January 26, 2011

Plugins with user interface

A while ago I wrote about plugins and now I received a mail from somebody asking about plugins that extend the UI of an application. From one side, these "plugins" are the cornerstone of Android application design: activity invocation can be considered a "UI plugin mechanism". But what if somebody wants to plug in UI elements into an application's screen so that the UI elements provided by the plugin appear on the application's own screen? Limited screen estate makes these sorts of plugins less relevant than in case of PC applications but there is a precedent: widgets plug into the Launcher's screen this way. I decided therefore to play with the idea and created a prototype.

Click here to download the application.

You will find 3 subdirectories in the package (respluginapp, resplugin1, resplugin2), each is a separate Android project. Compile and install respluginapp and launch the application. You will see something like this:
Now compile and install resplugin1 and resplugin2. You will notice that the screen of respluginapp dynamically changes as you install the plugin packages. Eventually it will look like this:



The prototype idea is simple. We have 4 rows on the screen. Each row has a default content (some text) but if a plugin is installed, the default row content is replaced by the UI elements provided by the plugin.

Sounds simple but it isn't.
  • Getting the layout from the newly installed plugin package and recognizing the plugin package is the easy part. The plugin package is recognized because it exposes a service with aexp.intent.action.PICK_RESPLUGIN intent action. That service will be used to handle plugin events. There is already a hack here: the name of a layout is available only to the application in the package because the resource compiler generates R.java that maps layout names to layout IDs. As the plugin host application cannot access this R.java, it has no access to layout names. Our plugin host application accesses the plugin's row layout by ID (0x7f030000 is hardcoded). This ID is allocated to the layout whose name is the first alphabetically among the layouts. As our plugins have only one layout in the package, it is not really an issue but this hack can cause quite a nightmare in larger projects.
  • The next hack happens with widget IDs. The resource compiler assigns IDs sequentially. Unfortunately it uses the same ID base for the plugin host application and the plugins so chances are that the IDs in the plugin host application and in the layouts of the plugins overlap. The plugin host application solves it by adding a row-specific offset to the IDs of the UI elements when they are loaded from the layout provided by the plugin package. But the downside is that from this moment, IDs used by the plugin host application and IDs used by the plugin will be different and the plugin host application has to constantly convert back and forth when it sends events to the plugin event handler service.
  • While the UI elements reside in the plugin host application's screen, the event handling is provided by a service exposed by the plugin. The event handling looks like the following: the event is intercepted in the plugin host application, the state of the plugin UI elements is serialized, the plugin event handler service is invoked that processes the event using the serialized plugin UI element state and spits out actions, how the plugin UI elements should be updated. Only some basic functionality is implemented in this prototype. Only button click events are intercepted and the state capture/update only works for TextViews and descendants. But you get the idea how complex it would be to do it properly. While fiddling with this part, I ran into a dead end street because I thought RemoteViews could be used to implement the serialization/deserialization. RemoteViews, however, does not support EditText so this approach had to be abandoned.
Is this something you should do? Well, I am not sure. Pulling UI events between host applications and services are not trivial and also it can be slow because by default the plugins are in a different package therefore in a different Linux process (see further explanation here). Our button click handlers probably do not consume so much CPU time but it can be worse with complex and frequent events. Also, the serialization mechanism presented here is kind of naive, check out the RemoteViews source to check, how deep you can go.

On the other hand, applications that change their screen layout when you install plugins are kind of cool as the Android widgets demonstrate. Decide yourself.

Update: this issue caused compilation errors in the projects. As raw Map is not supported anymore on the AIDL interface, I decided to follow the advice and replaced Maps with Bundles. I also converted the projects into Eclipse projects.

Click here to download the updated code.

19 comments:

Anonymous said...

I face this issue http://code.google.com/p/android/issues/detail?id=26841 when compiling your code.

Gabor Paller said...

Replace the Map with Bundle as Dianne Hackborn proposed.

Nam Pham said...

Thank you, this is a cool article, this is what I looking for. But I have problem with your code, after I import it into Eclipse, I got an error "cl cannot be resolved to a variable". Could you please help me to fix it?

Unknown said...

me too,"cl cannot be resolved to a variable".

Gabor Paller said...

Gemini Yellow, check out the update to the post. It is caused by a glitch in the SDK.

Anonymous said...

@Gabor Paller thank you for your update. but i got another error."the method asBinder() of type IResPlugin.Stub must override a superclass method". something miss?

Gabor Paller said...

Anonymous, you ran into this issue. Right-click on the project, select Properties/Java compiler and set the compiler compliance level to 1.6.

Jagadeesh.C.B said...

I use images under drawable folder as background src of my widgets in plugin row layout.So when my plugin host application accesses the plugin's row layout containing those widgets its throwing "Resource not found Exception". Could anyone see the cause

Jagadeesh.C.B said...

plugin host application could access the text view and buttons of plugin layout as shown in the example but could not access the images (from drawable folder) if used any.

Jagadeesh.C.B said...

@Gabor Paller any suggestions ?

Jagadeesh.C.B said...

@Gabor Paller any suggestions ?

Gabor Paller said...

Jagadeesh, could you upload your example somewhere?

M.Stahl said...

Really thanks for the example. Have you a way to use imageview, with onClick(), for the plug-in?

Gabor Paller said...

M. Stahl, look at registerButtonListener in ResPluginApp.java and extend the list of supported widgets. E.g.
if( v instanceof ImageView )
...

M.Stahl said...

Thank you very much for your quick reply. The onclicklistener isn't my problem though, it's that the app can't interpret the layout's xml code with ImageView in the mix. Am I simply too stupid or is it just not possible with this method to code a self created ImageView with this layout in xml in a way that makes the PluginApp recognise it? Here's the exact error message I get:

java.lang.RuntimeException: Unable to start activity ComponentInfo{...}: android.view.InflateException: Binary XML file line #24: Error inflating class

The main Problem i think is the resource not found exeption.
Can you me help?

Gabor Paller said...

M. Stahl, could you send me an example project? gaborpaller at gmail.com

Gabor Paller said...

M. Stahl, I looked into your project. The source of the problem is that your plugin contained another resource than the row layout (a drawable). This application does not scan the layout file and fetches dependent resources. I have no idea about a *simple* solution of this problem.

Unknown said...

Ok! that's awesome!
I need some information more about the UX. Does the plugin installation is totally invisible for the user?
Thx ;)

Anonymous said...

How to write Language Plugin in Android? Thanks