So for a little while now I’ve been wanting to try an experiment, using Zero Install as a Plugin Manager. Some background:

• Zero install is designed to be used for distributing applications and libraries.
• One of its greatest strengths is that it has no global state, and doesn’t require elevated privileges (i.e root) to run a program (and as the name implies, there is no installation to speak of).

Since there’s no global state, it seems possible to use this as a dependency manager within a program - also known as a plugin system.

Happily, it turns out it’s not so difficult. There are a few pieces required to set it up, but for the most part they are reusable - and much simpler to implement (and more flexible) than most home-rolled plugin systems.

(note: this article makes use of features in zero install version 0.52. If you are trying to run the examples, you will need that version or later. Fortunately, 0install can be easily bootstrapped by running 0launch http://0install.net/2007/interfaces/ZeroInstall.xml <interface>)

Note to program authors: This article is an explanation and rationale of the techniques I’ve used to manage plugins with zeroinstall. If you simply want to enable plugins for your own zeroinstall feed, you may want to skip to the documentation for zeroinstall-plugin-manager.

Background:

nosetests is an extensible test runner for python. I think it’s great, and have written a number of plugins for it. It has a built-in plugin system that relies on setuptools entry points, which annoys me because it means I have to package and install my plugins using setuptools, which is not a very good dependency manager for at least these reasons:

• it only works for python
• packages are installed globally, and there can be only one “active” version¹
• it can’t uninstall anything (this is pretty rubbish)

That’s why I want to use 0install to publish and manage nosetests plugins.

The basics - every user gets their own feed:

A zero install feed has two important parts: The implementation, and the dependencies. For nosetests itself, it doesn’t really have dependencies. But if you consider a user’s own set of preferred plugins, that’s really just a list of packages that you want nosetets to know about. Nosetests isn’t directly dependent upon them, but the conceptual package of “Tim’s preferred nosetests plugins” does.

So we’ll start with a local 0install feed to represent the user’s preferred set of plugins. This allows us to list nosetests as the “runner”, while depending on whatever additional plugins we want nosetests to know about. This will be specific to the user, and will never be published (it doesn’t even need a URI).

Here’s the relevant excerpt from my own such feed:

<group>
  <command name="run">
    <runner interface="http://gfxmonk.net/dist/0install/nosetests-plugin-resolver.xml"/>
  </command>
  <requires interface="http://gfxmonk.net/dist/0install/rednose.xml"/>
</group>

That just says “use the nosetests-plugin-resolver to run this feed, and make sure the rednose plugin is made available”. Observant readers will notice that I originally said we could use nosetests itself as a runner, so what’s this plugin resolver thing about?

Making plugins known to nosetests at runtime

Setuptools provides an entry point mechanism for python programs to discover plugins that have registered an implementation for certain well-known entry points. This is done using a setuptools-specific API, and mandates that all plugins must also use setuptools. In other words, we can’t re-use that. But it’s not too hard to make our own.

Zero Install allows packages one main way of allowing packages to know about other packages: environment variables. It’s a simple mechanism, and works well with existing systems - for example, if you write a python library, you just have to add it to $PYTHONPATH. If you want something to be runnable, you add it to $PATH.

So all we need to do is:

1. Put our plugin’s name into an environment variable
2. Make a plugin loader that can use this environment variable to load plugins (this takes the place of setuptools’ entry point loader)

The first one is trivial. In the feed for rednose, we just include:

<environment name="NOSETESTS_PLUGINS" value="rednose/RedNose"/>

That will tell our plugin resolver to perform the equivalent of:

from rednose import RedNose
plugins.append(RedNose())

The implementation of the plugin loader (#2) is reasonably unexciting. It simply parses a PATH-like $NOSETESTS_PLUGINS variable, imports each plugin (using __import__), and hands control over to nosetests itself. For the sake of reusability I’ve pulled the entry-point-like mechanism of instantiating objects from an environment variable into a helper library.

Note that if support for loading plugins from environment variables can be added to nosetests proper, there will be no need for this nosetests-plugin-resolver wrapper. And indeed if you are using this mechanism for your own program, you shouldn’t need one either.

Making it useable

So now we have all of the functional pieces, but it’s not very user-friendly. We don’t want the user to have to be an expert at editing zero-install xml feed files, so we’ll need a simple runner that puts together the feed for the user. This program must:

• load a persisted list of plugins (implemented as a file in ~/.config with one URI per line)
• add or remove plugins from this set (if –plugin-add or –plugin-remove options are given)
• compose a local XML feed, using a template and adding in a <requires> element for each requested plugin
• ask 0install to launch the feed

The runner could modify a 0install file directly, but I think managing a simple file with one URI per line is a much simpler proposition.

In order to help with this, I’ve created zeroinstall-plugin-manager. If your requirements aren’t terribly demanding, you can simply use it as a <runner> for your own feed and move your actual functionality to the core command of your feed (instead of the default run). If you require more control, it’s an importable python module you can use to do the heavy lifting of maintaining the configuration and generating the user-specific feed file.

Summary: the final setup

So after all this, we have three pieces to our puzzle:

1. nosetests-runner.xml - this is the feed that is actually run by the user, and contains the code to load plugins from environment variables and then pass control to nosetests proper.

2. zeroinstall-plugin-manager.xml - this is the <runner> for nosetests-runner.xml, and it processes the plugin options before passing control back to nosetests-runner (with user-specific plugins as additional dependencies).

3. nose.xml - this is just a zero-install feed for the official nosetests package

Again, if you can add the plugin-loading features from nosetests-runner to nosetests itself, there would be no need to publish two separate feeds - it’s just needed in the case where the original program can’t load plugins from environment variables.

Benefits

So what do we actually gain from using zero install?

Specifically for nosetests, we gain a lot of freedom:

• You can use and publish nosetests plugins without having to use setuptools.
  • You don’t have to use zero-install either, as long as your preferred system can add to $NOSETESTS_PLUGINS.
• Plugins are not installed globally, and are only loaded when required by either the program under test or the user’s preference.
  • This reduces the chance of conflicting plugins.
• You can easily stop using a plugin temporarily to diagnose problems.

And in general, for any program that uses 0install to manage plugins, you get:

• A robust dependency manager that can deal with not just inter-dependent plugins, but also plugins that require native libraries or libraries written in another language.
• Developers can use a feed that runs your application with the required plugins for their project as dependencies, saving end-users from having to manually install / remove plugins.
• 0test can test multiple versions of your app automatically, even if different versions have conflicting plugin dependencies.

Results

I may not be able to convince the author of nosetests to use this new system instead of the setuptools-based approach (since setuptools is so widely used in the python community), but nonetheless as an experiment it’s shown that zero install can work quite well as a plugin manager. No knowledge of zero install is required to use it, and developers get the benefits of a general-purpose dependency manager instead of the quirks of rolling your own plugin system. I strongly suggest that anyone looking to add a plugin system to an application take a look at zero install.

Extending to other software: Gnome Shell

To show that this is really not a hard thing to apply to other projects, I’ll give another example using Gnome Shell, the new user interface for Gnome 3. I have a vested interest in using zero install for gnome shell, as I have published an extension (shellshape) that relies upon a custom fork of mutter (which the existing extension system cannot support).

So, how hard was it to add zeroinstall extensions to gnome shell? Pretty easy - since gnome-shell is already a good citizen using $XDG_DATA_DIRS to find extensions, there was no code modification needed. It’s also an excellent example of the benefits of zero install, since the manual (user-level) installation process is downright awful for extensions that need their own gsettings schemas (it’s possible, though I’ve never seen anything but discouragement for those who would like to).

To use this system without any modifications to gnome-shell, you can simply run:

0launch \
	http://gfxmonk.net/dist/0install/zeroinstall-plugin-manager.xml \
	--plugin-command run \
	http://gfxmonk.net/dist/0install/gnome-shell.xml

Obviously, this is better used as a <runner>² - the above script is just to demonstrate that this mechanism requires no code modifications. I’ve also started using this runner for shellshape, because that’s become my gnome-shell replacement.

Two examples that use this mechanism already are my own shellshape.xml, and the much simpler gnome-shell-updateindicator.xml feed I created for updateindicator.