sil2100//vx developer log

Welcome to my personal web page

Launchpad API

Binary to source name. The Launchpad API

It's been a while since I wrote a programming-related post. Today I'd like to share with you a very simple, but useful, thing in the 'devel' version of the Launchpad API. When using LP or writing Python tools that need to deal with Ubuntu repositories, packages and their versions, frequently the need appears to get the source package name from its resulting binary package name as published in the selected archive (usually the main archive). It's a rather new addition, but really really useful.

2015-06-06 17:22

As always, first a quick overview for those less oriented in the topic at hand. Launchpad exports a REST API for almost all operations needed in case you want to develop some custom tools that would be using data from LP. The most popular bindings are, of course, for Python (versions both 2 and 3). You can do things like fetch packages (or their info) from a selected archive, get bug information, copy packages etc. Generally almost anything. Launchpad provides a few versions of its API - one stable (1.0) and one under constant development (devel). The stable one is rather outdated and it is generally recommended to use devel for most cases.

I won't be covering basics of Python or the Launchpad API in this blog-post - if you're interested in the API itself and how to use the Python bindings in general, make your way to these two pages: Launchpad API reference and Python Launchpadlib Introduction. The bindings are so easy to use that once you know how to login, all that's needed is the API reference - using the provided collections, methods and properties like in normal Python objects.

A new addition in the devel version of the LP API is the introduction of a binding between a binary_package_publishing_history's build object and its corresponding source_package_publishing_history. First some explanation though. Each Debian package that's available in the Ubuntu archives is a so-called binary package - it's a package that has been built from source for the selected architecture (e.g. i386, amd64, armhf). Each such binary package is generated from a source package. One source package can build multiple binary packages - for example, a source package foo can define and build the following binary packages: libfoo, libfoo-dev, foo-cli, foo-doc. All binary packages in Ubuntu are generally built in some archive - the main ones or PPA's.

Determining the source package from a binary package's contents is rather easy, but this generally means that to get the binary -> source conversion you have to download, extract and analyze the package directly before you can get this rather basic information. This is very hacky so at best we would like to get that information from the remote Ubuntu repositories almost instantly. Sadly, till recently this was not possible. One had to result to evil tricks to do that conversion: one way I found was to actually query the packages.ubuntu.com webpage through HTTP and parse the output - which was really slow and hacky.

Recently however, thanks to a merge by Colin Watson, the devel version of the Launchpad API finally exports a binding for this feature. Let's consider an example application to see how that's done:


 #!/usr/bin/python3
 #
 # Copyright (C) 2015 Lukasz 'sil2100' Zemczak <sil2100@vexillium.org>

 import argparse
 from launchpadlib.launchpad import Launchpad

 if __name__ == '__main__':
     parser = argparse.ArgumentParser(
         description='Convert binary package name to source package name')

     parser.add_argument(
         'distro_name', type=str, help='distribution, e.g. ubuntu')
     parser.add_argument(
         'series_name', type=str, help='series name, e.g. vivid')
     parser.add_argument(
         'arch_name', type=str, help='binary package arch, e.g. i386')
     parser.add_argument(
         'binary_package', type=str, help='binary package name, e.g. libfoo')

     args = parser.parse_args()

     # Login to LP - we only need read access so we can do it anonymously
     # Let's use the production server and the 'devel' level of the API
     lp = Launchpad.login_anonymously(
         'conversion', 'production', version='devel')

     # Fetch the distro, series and distro_arch_series objects from LP
     distro = lp.distributions[args.distro_name]
     series = distro.getSeries(name_or_version=args.series_name)
     arch_series = series.getDistroArchSeries(archtag=args.arch_name)
     # Now fetch all binary_package_publishing_history objects for the selected
     # binary package name, coming from the distro's main archive for the
     # requested arch and series
     bins = distro.main_archive.getPublishedBinaries(
         binary_name=args.binary_package,
         distro_arch_series=arch_series,
         exact_match=True)

     source_package = None
     if len(bins) > 0:
         build = bins[0].build
         if build is not None:
             # If we have some binary packages that fit our criteria, fetch the
             # corresponding source_package_publishing_history and get its name
             source = build.getLatestSourcePublication()
             if source is not None:
                 source_package = source.source_package_name

     if source_package is not None:
         print('Binary {} comes from source {}.'.format(
             args.binary_package, source_package))
     else:
         print('Cannot get source package name for binary')

As it's a fully working Python application, you can download and use it here: binary_to_source.py.

The inline comments should be self-sufficient, but let's take a quick look on what we do here. The key here is to simply get the binary package's binary_package_publishing_history object, fetch its corresponding build object and then call the newly added getLatestSourcePublication() method. Since a month every build in LP that corresponds to an archive build can return its source package publishing info - and that's all we need.

Launchpad API can be very useful if you work on tools related to LP or the Ubuntu archives. With the use of the API one can do almost everything that is achievable manually - especially when you're logged in with your credentials, not anonymously. I highly recommend checking it out. I use it very frequently as my current work in Foundations involves many Ubuntu-archive related chores. It's fun.
Recently, mostly because of my work as well, I seem to be using C and C++ less and less. It's a bit sad as I still consider C my main language of choice, although I do appreciate some of the conveniences that Python offers. But I do miss those times when I was writing all my tools like these in C... Might start doing that again!