Over the last few days I've been at the GNOME Developer Experience hackfest
in Brussels, looking into
xdg-app
and how best to use it in Debian and Debian derivatives.
xdg-app is basically a way to run "non-core" software on Linux distributions,
analogous to apps on Android and iOS. It doesn't replace distributions
like Debian or packaging systems, but it adds a layer above them. It's
mostly aimed towards third-party apps obtained from somewhere that isn't
your distribution vendor, aiming to address a few long-standing problems in
that space:
There's no single ABI that can be called "a standard Linux system" in
the same way there would be for Windows or OS X or Android or whatever,
apart from LSB which is rather limited. Testing that a third-party app
"works on Linux", or even "works on stable Linux releases from 2015",
involves a combinatorial explosion of different distributions, desktop
environments and local configurations. Steam uses
the Steam Runtime,
a chroot environment closely resembling Ubuntu 12.04 LTS; other vendors
tend to test on a vaguely recent Ubuntu LTS and leave it at that.
There's no widely-supported mechanism for installing third-party
applications as an ordinary user. gog.com
used to distribute Ubuntu- and Debian-compatible .deb files,
but installing a .deb involves running arbitrary vendor-supplied
scripts as root, which should worry anyone who wants any sort of
privilege-separation. (They have now switched to executable self-extracting
installers, which involve running arbitrary vendor-supplied scripts
as an ordinary user... better, but not perfect.)
Relatedly, the third-party application itself runs with the user's
full privileges: a malicious or security-buggy third-party application
can do more or less anything, unless you
either switch to a different uid to run third-party apps, or
use a carefully-written, app-specific AppArmor profile or equivalent.
To address the first point, each application uses a specified "runtime",
which is available as /usr inside its sandbox. This can be used to run
application bundles with multiple, potentially incompatible sets of
dependencies within the same desktop environment. A runtime can be
updated within its branch - for instance, if an application uses the
"GNOME 3.18" runtime (consisting of a basic Linux system, the GNOME 3.18
libraries, other related libraries like Mesa, and their recursive dependencies
like libjpeg), it can expect to see minor-version updates from GNOME 3.18.x
(including any security updates that might be necessary for the bundled
libraries), but not a jump to GNOME 3.20.
To address the second issue, the plan is for application bundles
to be available as a single file, containing metadata (such as
the runtime to use), the app itself, and any dependencies that
are not available in the runtime (which the app vendor is responsible for
updating if necessary). However, the primary way to distribute
and upgrade runtimes and applications is to package them as
OSTree repositories, which
provide a git-like content-addressed filesystem, with efficient updates
using binary deltas. The resulting files are hard-linked into place.
To address the last point, application bundles run partially isolated
from the wider system, using containerization techniques such as namespaces
to prevent direct access to system resources. Resources from outside the
sandbox can be accessed via "portal" services, which are responsible for
access control; for example, the Documents portal (the only one, so far)
displays an "Open" dialog outside the sandbox, then allows the application to
access only the selected file.
xdg-app for Debian
One thing I've been doing at this hackfest is improving the existing
Debian/Ubuntu packaging
for xdg-app (and its dependencies ostree and libgsystem), aiming to get it
into a state where I can upload it to Debian experimental. Because xdg-app
aims to be a general freedesktop project, I'm currently intending to make
it part of the "Utopia" packaging team alongside projects like D-Bus
and polkit, but I'm open to suggestions if people want to co-maintain
it elsewhere.
In the process of updating xdg-app, I sent various patches to Alex,
mostly fixing build and test issues, which are in the new 0.4.8 release.
I'd appreciate co-maintainers and further testing for this stuff,
particularly ostree: ostree is primarily a whole-OS deployment technology,
which isn't a use-case that I've tested, and in particular ostree-grub2
probably doesn't work yet.
Source code:
https://anonscm.debian.org/cgit/users/smcv/libgsystem.git
https://anonscm.debian.org/cgit/users/smcv/ostree.git
https://anonscm.debian.org/cgit/users/smcv/xdg-app.git
Binaries (no trust path, so only use these if you have a test VM):
deb https://people.debian.org/~smcv/xdg-app xdg-app main
The "Hello, World" of xdg-apps
Another thing I set out to do here was to make a runtime and an app
out of Debian packages. Most of the test applications in and around GNOME
use the "freedesktop" or "GNOME" runtimes, which consist of a Yocto base
system and lots of RPMs, are rebuilt from first principles on-demand,
and are extensive and capable enough that they make it somewhat non-obvious
what's in an app or a runtime.
So, here's a step-by-step route through xdg-app, first using typical
GNOME instructions, but then using the simplest GUI
app I could find - xvt, a small xterm clone. I'm using a Debian testing
(stretch) x86_64 virtual machine for all this. xdg-app currently requires
systemd-logind to put users and apps in cgroups, either with systemd as
pid 1 (systemd-sysv) or systemd-shim and cgmanager; I used the default
systemd-sysv. In principle it could work with plain cgmanager, but nobody
has contributed that support yet.
Demonstrating an existing xdg-app
Debian's kernel is currently patched to be able to allow unprivileged users to
create user namespaces, but make it runtime-configurable, because there have
been various security issues in that feature, making it a security risk for a
typical machine (and particularly a server). Hopefully unprivileged user
namespaces will soon be secure enough that we can enable them by default,
but for now, we have to do one of three things to let xdg-app use them:
enable unprivileged user namespaces via sysctl:
make xdg-app root-privileged (it will keep CAP_SYS_ADMIN and drop the rest):
make xdg-app slightly less privileged:
First, we'll need a runtime. The standard xdg-app tutorial would tell you to
download the "GNOME Platform" version 3.18. To do that, you'd add a remote,
which is a bit like a git remote, and a bit like an apt repository:
(I'm ignoring considerations like trust paths and security here, for
brevity; in real life, you'd want to obtain the signing key via https
and/or have a trust path to it, just like you would for a secure-apt signing
key.)
You can list what's available in a remote:
The Platform runtimes are what we want here: they are collections of
runtime libraries with which you can run an application. The Sdk runtimes
add development tools, header files, etc. to be able to compile apps that
will be compatible with the Platform.
For now, all we want is the GNOME 3.18 platform:
Next, we can install an app that uses it, from Alex Larsson's nightly
builds of a subset of GNOME. The server they're on doesn't have a great deal
of bandwidth, so be nice :-)
We now have one app, and the runtime it needs:
Digression: what's in a runtime?
Behind the scenes, xdg-app runtimes and apps are both OSTree trees.
This means the ostree tool, from the package of the same name, can be
used to inspect them.
A "ref" has roughly the same meaning as in git: something like a branch or
a tag. ostree can list the directory tree that it represents:
You can see that /files in a runtime is basically a copy of /usr.
This is not coincidental: the runtime's /files gets mounted at /usr
inside the xdg-app container. There is also some metadata, which is in
the ini-like syntax seen in .desktop files:
Looking at an app, the situation is fairly similar:
This time, /files maps to what will become /app for the application, which
was compiled with --prefix=/app:
There is also a /export directory, which is made visible to the host system
so that the contained app can appear as a "first-class citizen" in menus:
Again, there's some metadata:
Building a runtime, probably the wrong way
The way in which the reference/demo runtimes and containers are generated is...
involved. As far as I can tell, there's a base OS built using Yocto, and the
actual GNOME bits come from RPMs. However, we don't need to go that far
to get a working runtime.
In preparing this runtime I'm probably completely ignoring some best-practices
and tools - but it works, so it's good enough.
First we'll need a repository:
I'm just keeping this local for this demonstration, but you could rsync it
to a web server's exported directory or something - a lot like a git
repository, it's just a collection of files. We want everything in /usr
because that's what xdg-app expects, hence usrmerge:
This obviously has a lot of stuff in it that we don't need - most obviously
init, apt and dpkg - but it's Good Enough™.
We will also need some metadata. This is sufficient:
That's a runtime. We can commit it to ostree, and generate xdg-app metadata:
(I'm not sure why ostree and xdg-app report "Operation not permitted" when
we aren't root or fakeroot - feedback welcome.)
build-update-repo would presumably also be the right place to GPG-sign your
repository, if you were doing that.
We can add that as another xdg-app remote:
Building an app, probably the wrong way
The right way to build an app is to build a "SDK" runtime - similar to that
platform runtime, but with development files and tools - and recompile the app
and any missing libraries with
./configure --prefix=/app && make && make install. I'm not going to do that,
because simplicity is nice, and I'm reasonably sure xvt doesn't actually
hard-code /usr into the binary:
Again, we'll need metadata, and it's much simpler than the more
production-quality GNOME nightly builds:
The obligatory screenshot
OK, good, now we can install it:
and you can play around with the shell in the xvt and see what you
can and can't do in the container.
I'm sure there were better ways to do most of this, but I think there's value
in having such a simplistic demo to go alongside the various GNOMEish apps.
Acknowledgements:
Betacowork Coworking Brussels and
ICAB Business & Technology Incubator
hosted the hackfest, with Collabora
providing snacks, and
the GNOME Foundation supporting the
hackfest in general;
Collabora also sponsored my travel, accommodation and time;
my colleague Philip Withnall organised the
hackfest.
Thanks to all those!