sil2100//vx developer log

Welcome to my personal web page

Custom library search paths

A beginners post today. The idea for this article came from my recent battles with packages and a question asked by a friend of mine - "how can I point where the system can find some private libraries of mine on GNU/Linux for my application?". There are a few methods for doing this: LD_LIBRARY_PATH, ld.so.conf and RPATH. A word of notice though - it is usually not wise to make your application depend on libraries in some private, normally unaccessible paths. But when you are forced to, here's what you can do.

2012-08-02 23:35

Empty

I'm pretty sure all of you know about the LD_LIBRARY_PATH environment variable. If you need to quickly point to an untracked directory with libraries when running an application (or want to override some libraries with other copies), all that is needed is setting LD_LIBRARY_PATH=/path/to/libraries before starting the target program.


 # Run foo, searching for libraries in the /usr/lib/baz folder besides the default 
 # library locations
 LD_LIBRARY_PATH=/usr/lib/baz ./foo

The other way probably known to almost everyone is using ld.so.conf. It is a special file located in /etc/ld.so.conf. In this file we can include custom search paths for libraries on the whole system. After adding entries to this file, we always need to re-build the cache by calling ldconfig though - its good to remember that.
In Ubuntu, we shouldn't edit ld.so.conf directly, since there is a special directory /etc/ld.so.conf.d/ where we can put our .conf files with paths one-per-line to our convenience. We can also add directories to the cache directly through the command line without editing or adding any files - using ldconfig -n.


 # We can either edit ld.so.conf (or put a new .conf file to ld.so.conf.d/) and rebuild
 # the cache by calling ldconfig, or simply run ldconfig adding the path from the command line
 ldconfig -n /usr/lib/baz

Now, if we do not want all the applications in our system to use the private path but just a selected one - and we do not want to bother by meddling with the environment variables every time - there is something called RPATH which we can use.

RPATH is an optional section in the .dynamic section of any ELF object - both executable and shared object. It is an embedded list of run-time search paths in a given object. By default, when looking for a library, ld.so first looks through all directories listed in the binary's RPATH - if none are specified or the given library is not found, it looks through the rest (ldconfig cache, LD_LIBRARY_PATH, default paths etc.). So it's an easy way of adding search paths to a selected object.

We can add entries to RPATH in various ways. If we are using ld directly, we can use the -rpath=path parameter. When using gcc/g++ for linking, we need to pass it using the -Wl property. The -Wl argument can be used to pass flags directly to the linker. A few examples:


 ld foo.o -o foo -rpath="/usr/lib/baz"  # When calling ld directly
 gcc foo.c -o foo -Wl,-rpath="/usr/lib/baz"  # When using gcc/g++
 gcc foo.c -o foo -R"/usr/lib/baz"  # Using the convenient -R notation

After adding the path, we can then confirm if RPATH has been set correctly by either calling objdump -x application | grep RPATH or using any other, similar tool (e.g. readelf).
For convenience, CMake helps out providing some easy-to-use properties for forcing RPATH for both the whole project and a single target. All information regarding that can be found on the CMake wiki. Really useful when using CMake.

WARNING! Just to be clear. As I already mentioned at the beginning, in most cases we should avoid using RPATH or any other non-standard library path meddling. A detailed study on why it is a problem can be found here. But anyway, it's always good to know how to deal with situations like these. Just remember - when building Debian packages from binaries that have custom RPATH's, get used to linian making a fuss about it. And it probably won't be accepted to any official Debian repository as well. No one likes RPATH it seems...