sil2100//vx developer log

Originally this was supposed to be a compilation of different Launchpad API related tips and tricks but since I got preempted while writing it (and can't seem to find the time to get back to it), I decided to at least share this one story that I already managed to write down.
On the side note: if you want to see some Python 3 LP API in action, be sure to check-out my batch of useful scripts in the lp:landing-team bzr branch. Soon those tools (after some cleanup) will be available as an Snap in the store but before that happens, well, simply bzr branch or visit the project's webpage to get the code. Some of those scripts have been written in a hurry but they tend to 'do the job'. Anyway, back to the topic at hand...

A while ago I wrote about how to perform a binary-package-to-source-package resolve using the Launchpad API. The base idea was to get the binary_package_publishing_history object representing the selected binary package, fetching its build object and calling the getLatestSourcePublication() method on it to get the source_package_publishing_history. That all works excellent.

Now let's imagine you want to do something like this. You have this binary package name and want to know all the other binary packages that come from the same source. Easy, you might think. I just do the same thing as before, just on the final source_package_publishing_history call an additional getPublishedBinaries() and violla, you have all the binary packages you need. For most cases it will just work. But if suddenly you notice that the final call for getPublishedBinaries returns 0 entries, let me explain what happened.

In [1]: query = {
   ...:             'binary_name': "account-plugin-flickr",
   ...:             'version': "0.12+15.04.20160126-0ubuntu1",
   ...:             'order_by_date': True,
   ...:             'exact_match': True
   ...:         }

In [2]: ppa = lp.load('https://api.launchpad.net/devel/~ci-train-ppa-service/+archive/ubuntu/stable-phone-overlay')

In [3]: bins = ppa.getPublishedBinaries(**query)

In [4]: src = bins[0].build.getLatestSourcePublication()

In [5]: src
Out[5]: <source_package_publishing_history at https://api.launchpad.net/devel/~ci-train-ppa-service/+archive/ubuntu/landing-036/+sourcepub/5991548>

In [6]: len(src.getPublishedBinaries())
Out[6]: 0

The snippet above was taken from a short experimental session in lp-shell (a very useful tool btw.). As you can see in line 6, the overal number of binaries associated with the given source package is 0 - even though in reality it should be 96, since there's a bunch of them for each architecture. Why is that so? The key here is to understand one thing: source publications are specific to the selected archive. If, let's say, a package gets built in one archive (like a PPA) and then moved to another (copied and deleted from source), the build object for the binary will be still the one from the original archive (as that's where the binaries got built, right?). So, if you use getLatestSourcePublication() from that build object, it will give you the source_package_publication_history from the original archive. Now the second problem is, getPublishedBinaries() will only return active binary publications. So why 0 binaries? If we removed the packages of interest from the original package after copying, their binaries in that archive are in the Deleted state - so no longer active. Since the SPPH we got was specific to the archive the packages were built on originally, LP doesn't know the binaries got moved to a different place. It just sees that all of those are no longer present in the archive, thus returning 0.

There's no real easy way of working-around this sadly. One possible option is to use the resolved source package name to query the final archive (the one we know hosts the binaries right now) and calling getPublishedBinaries on the SPPH object there.
You might wonder how big of a problem this actually is, since it pretty much looks like an edge-case. It's actually pretty frequent for CI Train managed packages, since each of those is built in a silo PPA and moved (copied and deleted) to the destination archive during the publish step.

You don't know what CI Train (Bileto) is? It's a piece of infrastructure we use in Ubuntu Touch to land packaging changes to the archives. That's possibly something for a different story - in the meantime the LandingProcess wiki page can give you a general overview.