Welcome to my personal web page
Those of you that know a thing or two about the Ubuntu archives also most probably know about the proposed pocket for every distribution series. In a quick overview, every upload made to the main archives first goes to -proposed and then migrates (in case of the development series) to the release pocket once the so called proposed migration is happy with it. Most of the time it just migrates fine on its own, but sometimes a package can fail to "move on". And this is where update_excuses.html and update_output.txt come in handy.
2015-02-02 12:47
There's a lot going on in the Ubuntu world right now. We are constantly working on Ubuntu Touch with out first commercial devices almost ready for shipping and we have recently announced the Ubuntu Core on Internet Things. But even with all those great things happening the main Ubuntu repositories stay the same - with the core internals resolving around debian packages. For normal users proposed migration is something compeltely transparent, but for developers publishing their packages (when being MOTU/core-devs or through sponsors) it is crucial to know why a selected component is not moving forward to the release pocket. And even though parsing update_excuses.html output is rather straightforward, sometimes the data there is not enough and a deeper dive into update_output.txt is required. And oh my, how non-trivial to parse update_output.txt can be on first look.
For those that are completely new to this topic and do not know what release and proposed pockets are, in short, the former is basically the main package repository for the given distribution and series (like ubuntu/utopic, ubuntu/vivid etc.) that the Ubuntu package manager (apt etc.) uses all the time. The proposed pocket on the other hand is a place where all newly released packages land before their 'approval' to be made available to all users. More information regarding those can be found in the help pages on Ubuntu Repositories.
The software performing proposed-migration is called britney. The Ubuntu wiki pages contain some limited documentation of the whole process and, as well, some aspects of both files outputted during migration processing (see ProposedMigration for more information). Sadly it is missing many of the common cases that can be seen in update_output.txt - which knowledge can be crucial when trying to handle bigger library transitions. These situations happen when a package changes its ABI/API and has many reverse-dependencies. During my landing team lead work on the Ubuntu-RTM branch (used for stable Ubuntu Touch images) and during my short +1 Maintenance duty I had contact with transitions like those and it's a really bumpy ride everytime problems pop up.
As a quick explaination: packages can get stuck in -proposed for various reasons. Some can be at hold because of autopkgtests failing, some because FTBFS reasons for selected architectures or - the worst kind, package installation causing conflicts with other existing packages. This is most frequently the case during package transitions, when we release a new library with an SONAME bump. But first of all, some links.
For now let's skip explaining update_excuses.html as it's rather trivial in the long term. This is the place where all autopkgtest failures are listed, same for build faiures and other obvious reasons for failure. Once the given package has Not considered written down with no other explaination or Valid candidate for a longer period of time, it is time to dive into update_output.txt which has always all the information needed.
The update_output.txt's first section is always filled with entries starting with the trying: [package name] clause. This is always our first clue in further investigation - if the package of interest in this section has a skipped: entry, this usually means we have a conflict somewhere which needs to be resolved. Besides providing some diagnostics, the skipped entry also mentions briefly which packages for which architectures would become uninstallable after the selected component gets released. I will try to overview some of them later on. But anyway, once that happens, we should continue examining the output file looking for the package name in the autohinter section.
Trying easy from autohinter: network-manager/0.9.10.0-4ubuntu2 network-manager-applet/0.9.10.1-0ubuntu1 gnome-control-center/1:3.14.2-2ubuntu2
leading: network-manager,network-manager-applet,gnome-control-center
start: 192+0: a-52:a-28:a-29:i-26:p-29:p-28
orig: 192+0: a-52:a-28:a-29:i-26:p-29:p-28
easy: 196+0: a-53:a-28:a-30:i-27:p-30:p-28
* amd64: lubuntu-desktop
* armhf: lubuntu-desktop
* i386: lubuntu-desktop
* powerpc: lubuntu-desktop
FAILED
Above is an example output of a package failing to migrate and generating an autohinter entry. Proposed-migration is test-upgrading the archive and noting down which packages for which architecture is uninstallable after the selected autohint goes in. In our example it seems that after trying to install a newer network-manager along with packages from the network-manager-applet and gnome-control-center sources, the lubuntu-desktop package becomes no longer installable for all the selected architectures. Whenever such a thing happens it is best to prepare a chroot for the selected architecture we want to analyze and try upgrading/installing the selected packages. Example below. Remember that britney lists source packages in the autohinter line, not binary packages. Also, remember to enable proposed for the selected series before proceeding.
# In the arch-series chroot with proposed enabled # network-manager-gnome comes from network-manager-applet - the autohinter tries different combinations, so we just need # to try the most logical ones until we see the root cause of the conflict sudo apt-get -s install network-manager network-manager-gnome lubuntu-desktop # In case the migrating package provides a library that broke ABI/API and now the SONAME needed bumping # Example: new libav, with libav11 deprecating libav10 # In this case we also need to make sure we're removing the old library, as this is what happens during autohinting sudo apt-get -s install uninstallable-package1 uninstallable-package2 libav11 -libav10
Test-upgrading with -proposed enabled helps out a lot when trying to find the root-cause. In case of library transitions, this usually means that in the list of uninstallable packages (the list next to the arch, e.g. * amd64: ) there probably is a package that did not yet get rebuilt against the new version of the library and the dependencies are wrong. But the number of overall possibile problems is much bigger, from invalid dependencies to removed architectures.
The update_output.txt file has also some other interesting information before the autohinter sections. For this let me quote the ProposedMigration documentation:
The "got" line includes the number of problems in the release pocket on the different architectures (until the first architecture where a problem is found — see above). The "i-45" means that if cln would go into the release pocket, there would be 45 uninstallable packages on i386. Some of the entries above and below cln show there were 43 uninstallable packages in the release pocket on i386 at the time.
The "skipped: cln (0) (150+4)" line means that there are still 150 packages to go through after this package until this check of all packages is completed, and that 4 packages have been found already that won't be planned to be upgraded because they would break dependencies. The "(0)" is irrelevant; you can safely ignore it.
This quick overview should be enough to be able to identify most of the archive problems during proposed migration. There are some edge cases which can happen, but those are rare enough cases to be skipped in this post. Remember: if your package is stuck in -proposed and you still don't know what's going on, feel free to join #ubuntu-release on Freenode IRC and ask for help. The available archive admins or core-developers should be able to resolve your problems or at least point you in the right direction.
As a side-note: if you're interested in the landing process and Ubuntu Touch in overall, be sure to watch and join us during Ubuntu Engineering Live! meetings on Ubuntu On Air. Those are fortnightly live hangout sessions where we overview of what new is happening in the world of Ubuntu. I'm usually there acting as the representative of the release and landing teams. I am also performing frequent patch pilot duties as a MOTU, so feel free to poke me on IRC if you need any of your universe packages reviewed and published. Cheers.