Some notes on the GNU toolchain

Order dependence of libraries passed to the linker

The order in which libraries to link to are passed to the linker is important. In particular, it is important when linking in static libraries, since the symbols from these are resolved at link-time. The way the linker works is that it keeps a current list of undefined symbols and then processes libraries in turn in the order specified on the command-line. If some of the libraries supplied on the command-line themselves have undefined symbols, these are added to the current list and subsequent libraries are checked for these.

The implication of this, however, is that if libraries are not supplied in the correct order, some symbols will not be defined. Consider the following simple scenario:

If you try to build libA as follows:

gcc -o libA -lB -lC

the resulting library will have its symbols fully resolved. If the order of libraries passed to the linker is reversed however:

gcc -o libA -lC -lB

then the undefined symbols in libB which are in libC may not be resolved, since they were not in the “current” list when libC was considered by the linker, which was before libB in this case.

Undefined VTables

A vtable is the data structure which implements polymorphic behaviour of C++ classes. They are sometimes involved in tricky (when using dynamic linking) run-time errors, in which the linker complains that the vtable is undefined.

An example of such an error is:

/home/bnikolic/d/n/casa-nrao/main/code/build/test/libncasa-display.so: undefined symbol: _ZTVN4casa8GLWidgetE

To interpret it correctly it is first necessary to de-mangle the symbol:

./c++filt _ZTVN4casa8GLWidgetE
vtable for casa::GLWidget

Technically, this means that the first non-inline non-abstract method (see FAQ) that is declared by the class is not defined. The reason for this message is that when the g++ compiler encounters this method definition, it also emits the vtable definition.

In this particular example, the reason for the missing vtable and first method is that the GLWidget class is a QT object, and the first method is normally defined in the “moc” file. Therefore, to resolve this error, the “moc” file must be generated, compiled with g++ and linked in.

Position independent code

Position-Independent Code (PIC) is code which is designed so that it may be moved to an arbitrary location in memory and executed without modification. (This is not as absolutely essential as it might seem at first as modern processors allocate an entire virtual address space to each program that is run.) There is plenty of documentation on the web regarding position independent code, e.g.:

• http://en.wikipedia.org/wiki/Position-independent_code
• http://www.gentoo.org/proj/en/hardened/pic-internals.xml?style=printable
• http://blog.flameeyes.eu/2009/11/02/the-pie-is-not-exactly-a-lie

There is one subtlety which should be noted, in particular in relation to CASA, which tends to mixes static and shared objects. Contrary to what may be expected, it is technically possible to build dynamically loadable libraries with object that are not position-independent. In this case, the linker needs the do a “text relocation” at the time of the loading of the library. This of course has a run-time impact in terms of both time and memory consumption since the objects are not shared.

The subtlety arises in that on x86-64 and/or some security-hardened systems the dynamic linker will refuse to do so. This can lead to errors in linking and loading of dynamic objects which are different on x86 and x86-64 systems. The solution is to, on x86-64, build all modules that may end up dynamically loaded or linked against shared library as PIC (-fPIC flag to gcc). Because of the revised instruction set of x86-64 and higher number of registers, the potential performance penalty of doing this is minuscule.

Autoconf for end-users

Many libraries use the GNU Autoconf/Automake build toolchain. Here are some generic notes on what end-users need to know about these.

Two-step process

Building a library from scratch is a two-step process:

1. Run the ./configure script, which looks for the various programs and libraries that the package needs
2. Run make to do the actual compilation

Passing options to configure

There is often two ways of letting the configure script know about locations of non-system wide packages and other options:

1. Setting environment variables. For example:

   PKG_CONFIG_PATH=/home/bnikolic/p/bnprog-main/lib/pkgconfig/ ./configure
   

   informs the pkg-config program, which is run by configure, to look for additional packages in the supplied directory /home/bnikolic/p/bnprog-main/lib/pkgconfig/

2. Passing command line options. For example:

   ./configure --prefix=/home/bnikolic/p/bnprog-main
   

   asks configure to ensure that the compiled library is eventually installed in /home/bnikolic/p/bnprog-main rather then the system-wide default location

Figuring out dependencies

The simplest way is to run ./configure and make note of any complaints about missing programs or libraries that it reports. Then install these missing programs/libraries, and repeat the process.

Non-root installation

When you run make the library/program is compiled, but it then simply remains in the directory in which it was built.

If you run make install, the program/library is also installed to a specific location and internal links in the program are updated to reflect this final installation location. By default the installation location are the system wide program directories rooted at /usr.

It is usually more appropriate to install in-development and custom software in a different location, often in users own file space. This can be done by passing the prefix argument already mentioned above:

./configure --prefix=/home/bnikolic/p/bnprog-main

Asks configure to set the install location to /home/bnikolic/p/bnprog-main. This will not by itself actually do the installation, that only happens when you do make install.

Note that the trailing slash in directory arguments to ./configure can be important. In the case of –prefix, there should be no trailing slash. Therefore do /home/bnikolic/p/bnprog-main and not /home/bnikolic/p/bnprog-main/

Specifying version of Python against which to build modules

External Python modules generally work with only the version of Python against which they were compiled. Therefore if you have more than one version of Python on your system, you need to ensure that you compile the modules against each of the Python versions that you will use in practice.

This is best done by specifying the PYTHON environment variable during configuration stage. For example:

PYTHON=/export/local/python2.6/bin/python2.6 LD_LIBRARY_PATH=$INSTALLDIR/lib PATH=$INSTALLDIR/bin:$PATH ./configure --prefix=$INSTALLDIR