Welcome to my personal web page
Ubuntu System Image is the name of the client/server infrastructure used for Ubuntu Touch and (currently also) Ubuntu Core for image-based upgrades. In certain scenarios the standard apt/dpkg package-based upgrade mechanism is simply not good enough, so the whole Ubuntu System Image initiative was started a few years back. The purpose of an s-i server is to generate and export new images for selected use-cases whenever needed, allowing the client to notice and do the upgrade. Anyone can setup their own system-image server, in which case it's good to know what is what and how to prepare your configuration - along with some useful tricks.
2015-12-23 20:34
I have been working on the system-image server for a while now, nowadays becoming one of the main maintainers. My current tasks involve making the server scripts properly unit-testable and adding new options related to image phased percentage, both of which should be available soon. But even without those two so far system-image is stable and driving the whole Ubuntu for Phones OTA mechanism.
But there comes a time when one needs to create their own system-image server locally, either for testing some custom touch images (like for new device ports and/or image fine-tuning) or simply for testing changes in s-i itself. Generally the documentation outlines all the most important parts with only some specifics missing, but in the nearest future I will try to fill in all the missing gaps in the all the wiki pages. Be sure to take a look at ServerOperation for all the saucy details - no need to repeat all the useful information you can find there.
For now, consider the slightly modified example config taken from the official wiki page ServerSetup:
(...) [channel_test] type = auto versionbase = 1 fullcount = 15 deltabase = files = ubuntu, device, custom-savilerow, keyring, version file_ubuntu = remote-system-image;https://system-image.ubuntu.com;trusty-proposed;ubuntu file_device = remote-system-image;https://system-image.ubuntu.com;trusty-proposed;device;keyring=archive-master file_custom-savilerow = http;https://jenkins.qa.ubuntu.com/job/savilerow-watch/lastSuccessfulBuild/artifact/custom.tar.xz;name=custom-savilerow,monitor=https://jenkins.qa.ubuntu.com/job/savilerow-watch/lastSuccessfulBuild/artifact/build_number file_keyring = keyring;archive-master file_version = version
As per the ServerOperation page we know that most of the channel operation is defined by so-called generators - in the example those are ubuntu, device, custom-savilerow, keyring and version. Anyone with their own system-image server can define their own generators - all that's needed is adding a new conditional line to the generate_file() dispatcher function in the lib/systemimage/generators.py file and doing whatever we want there. Generally the only requirement is that you return the path of the newly generated file at the end of the function. The available parameters, besides the global s-i configuration tree, are arguments and environment. The first one is an arguments list generated from the generator configuration line, simply by splitting the semi-colon separated string. The environment is a global dictionary holding all the local 'environment', allowing the access and modification of options shared between generators.
The list of default available generators is:
A thing worth noting: whenever we copy an image manually, all files besides the version file are copied between channels - even if the copy-image --keep-version option is used. The main reason is that the version tarball always includes the end-channel-name information, so it needs to be re-packaged fresh every time. Recently we also use the version tarball to include so-called tags. Tags are special string labels that can be attached to an image that will be visible both in the version tarball and the version_details string on system-image. The notion of tags is slightly different from descriptions (already available since long), the main difference being: tags are short-strings that can be read from the image itself without accessing the image server, so useful for better identifying an image even in no-network situations. Those can be either modified during image copy with the --tag parameter or by using the separate tag-image script. Image tags are overwriten whenever a completely new image is imported, with images not being tagged at all by default.
Another small detail worth noticing in the example config above is: each channel can define its own deltabase list - i.e. a list of channels from which it should generate its delta from. If no deltabase string is defined at all then system-image only generates the delta from the last image in the current channel (that is the default). If, on the other hand, we define an empty deltabase (as in the example above) this basically means: do not generate any deltas at all. This will basically turn all new images in the channel into full-only images, in which each update will require downloading the whole new image from scratch.
That being said... cross-channel delta creation is not properly supported anyway (see LP: #1516602), so basically only deltas from the same channel matter. But from the user perspective only those are really important, as we only care about users upgrading from image to image in the same channel.
At the end, going away from the config, one more interesting part in system-image that deserves some mention is phased-percentage concept. Each image has a so-called phased-percentage value - if the image json file does not define it, the default is 100. This value defines a statistical percentage value of how many of the server's clients should see the update notification of the new image. Administrators can use the set-phased-percentage script to set the value or inspect it through si-shell by using the device method get_phased_percentage(). Most of the phased-percentage handling is done on the client side, but it's a very useful addition during official update releases, potentially decreasing the number of faulty upgrades in case the newly published image breaks users in any serious ways. Every Ubuntu Touch update is phased out over the period of one day just as a precaution.
All in all Ubuntu System Image server is a decent piece of code, although sometimes written in a hurry - which can be understood considering the tight timeframes it was implemented on. From other news: I finally became an Ubuntu Core Developer! Yay. This means I now have Ubuntu archive power over all packages ranging from multiverse to main. I'm really happy about finally making it in as it wasn't easy that's for sure (I might write a post about my journey there in the nearest future). Also, the article I was writing about the Ubuntu Touch release process got published on the latest Ubuntu User (issue #27) - big thanks to Paul Brown for all the awesome editing he did.
Oh, and while we're at it: happy holidays everyone!
