developer toolbox

Rolling your own Debian packages (part 1)¶

Posted by bdf on Fri 20 Jan 2006 at 09:14
Tags: building packages, debian-specific commands

This two-part article explains how to make a Debian package of simple piece of software, presumably something you have written yourself. Although building a new package is more complex than rebuilding one or having one generated, the idea is that it is actually surprisingly simple to create basic Debian packages. In fact, if you can make software install into a temporary installation tree, you're already 90% done! This text provides a quick alternative to the more comprehensive Debian New Maintainers' Guide. Only knowledge of Makefiles and the basic Debian package tools is assumed.

The first part of this article will continue with some preliminary information about Debian packages. In the second part we walk through a concrete packaging example.
Why would you want to make a Debian package?
Perhaps you already know how to install software into /usr/local/bin using make install and wonder why you should bother to make a package? Contrary to what you might think, the main goal of packaging software does not have to be that you want to upload it to the official Debian distribution. A Debian package offers a uniform and integrated way to manage and install software on a Debian system. By packaging software, you can install, upgrade or remove the software using standard tools such as apt-get and dpkg. It also offers an easy way to distribute software, even to systems that don't have development tools installed. If you have to manage a number of machines, I'm sure you'll appreciate this.
What's inside a binary package?
To show how simple it is to create a package, let's first vulgarize the concept a bit: a .deb-file is nothing more than an archive with a bunch of files to be installed 'as such' in the filesystem tree, and a bit of control information. You can check this for yourself by looking at the internals of a binary package. The first part, the data files, usually consists of compiled programs, manpages and documentation (for the /usr/share/doc/packagename/ directory). The control part on the other hand, contains package management data (package name, version, dependencies,...) as well as MD5 checksums to verify the integrity of the first part. Additionally, it can hold some provisions for more advanced packages, such as helper scripts with additional remove and update actions. These two parts are sufficient for dpkg to provide its core features, such as installing packages, maintaining a list of installed packages and their files, verifying package dependencies, etc.

How is a Debian package created?¶

The command that initiates the creation of a .deb-file is dpkg-buildpackage. From the source tree of the software, it creates a clean installation tree in a temporary directory, and archives this tree to a .deb-file. It also packages the source tree as a source package. For these tasks, dpkg-buildpackage is heavily guided by the files in the special debian subdir of the source tree. In fact, almost all of the work is done by calling the debian/rules script, so the building process is largely controlled by the package source. This script should accept commands such as clean, build and binary and is normally implemented as a set of rules in a Makefile (hence its name). It will in turn invoke the original software build script (i.e. the original Makefile) as well as a number of tools from the debhelper suite. These tools, which all have a prefix "dh_" in their name, handle common tasks in the preparation of the installation tree and the consecutive .deb-archive. To name a few, these tasks include: installing files in the right place (dh_installxxxx), generating md5 checksums (dh_md5sums), fixing file permissions (dh_fixperms) and creating the resulting archive file (dh_builddeb).

Apart from the rules script, dpkg-buildpackage extracts basic information from the control and changelog files as well. We will cover these files in the second part of the text. Also, it is common to invoke dpkg-buildpackage from a wrapper called debuild. It will conveniently call a package checker such as linda or lintian after the build process. Since all the different commands in this layered design can be confusing, a schematic overview is presented below:

debuild
  -> dpkg-buildpackage
       -> debian/control
       -> debian/changelog
       -> debian/rules
            -> original build scripts (./configure, Makefile,...)
            -> debhelper suite (dh_*)
  -> lintian or linda
  -> debsign

Below is the command to set up a fairly decent package building environment. The package dpkg-dev houses dpkg-buildpackage; debuild and debsign are provided by devscripts. fakeroot is used to mimic root status when installing into the installation tree.

apt-get install dpkg-dev debhelper devscripts fakeroot linda

In the second part of this text, we will use these commands to build a concrete package for a simple command line program.

Rolling your own Debian packages (part 2)¶

Posted by bdf on Sat 21 Jan 2006 at 17:18
Tags: building packages, debian-specific commands

In the first part of this text, we introduced the principal concepts of Debian package building. We're now ready to build an example package of a simple command line program.

I picked svnyoungest, a small auxiliary program to report the latest revision of a Subversion repository. You can find the source tarball of svnyoungest here. To interface directly with the Subversion library, it is written in C, but you can package programs from any development environment.

The source code is provided with a Makefile (no configure script though) that accepts these standard targets: all (the default), clean, distclean and install.

Generating the debian subdir¶

The first step is to create a debian subdir with packaging information. Probably the easiest approach is to use dh_make to generate one from a standard template. The package providing this command is called dh-make (with hyphen, not underscore).

apt-get install dh-make

dh_make requires the source directory to be named according to the common pattern packagename-version. So once we have the source available in svnyoungest-0.1, we can issue the dh_make command with appropriate options: -n indicates a native package (i.e. that is packaged by the original author), which is slightly simpler for demonstration purposes, -s indicates a single binary and -e sets the maintainer e-mail address to be used:

_{/svnyoungest-0.1$ find .
.
./svnyoungest.1
./svnyoungest.c
./Makefile
./README}
/svnyoungest-0.1$ dh_make -n -s -e bruno@defraine.net

Maintainer name : Bruno De Fraine
Email-Address   : bruno@defraine.net 
Date            : Tue, 20 Dec 2005 10:28:12 +0100
Package Name    : svnyoungest
Version         : 0.1
License         : blank
Type of Package : Single
Hit  to confirm: 
Done. Please edit the files in the debian/ subdirectory now. You should also
check that the svnyoungest Makefiles install into $DESTDIR and not in / .

_{/svnyoungest-0.1$ rm debian/*.ex debian/*.EX}
/svnyoungest-0.1$ find debian/

debian/
debian/changelog
debian/compat
debian/dirs
debian/README.Debian
debian/copyright
debian/control
debian/rules
debian/docs
debian/README
~/svnyoungest-0.1$ rm debian/README debian/README.Debian

Notice that the standard template used by dh_make contains a number of example files (extensions .ex or .EX). These files illustrate features for advanced packages (such as helper scripts, cron jobs, installation dialogs,...) and can be handy if your package needs them. In this case however, they would make the presentation needlessly complex, so I chose to remove them for the time being (they can easily be recreated by issuing dh_make with the -a switch). The template also provides a README and README.Debian file. I chose to delete both of them, respectively because my original source already contains a README document, and because I don't have additional Debian-specific remarks. This leaves us with an uncluttered debian subdir.

Editing control files¶

The first file to edit is probably the main debian/control file. The format and contents of this file should look familiar if you know the output of apt-cache show, and you can always find additional info in the deb-control(5) manpage. Note that the first and second parts refer to the source and binary packages respectively. In this case, it is straightforward to fill in the fields set up by the template:

Source: svnyoungest
Section: devel
Priority: optional
Maintainer: Bruno De Fraine 
Build-Depends: debhelper (>= 4.0.0), libapr0-dev, libsvn0-dev
Standards-Version: 3.6.1

Package: svnyoungest
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: report the youngest revision of a svn repository path
 svnyoungest is a small tool to report the youngest revision of a path in
 a Subversion (svn) repository. It is similar to ''svnlook youngest'', but
 works on the client side.

Since the generated debian/rules file invokes a number of debhelper tools, dh_make already included a build dependency for them. As the svnyoungest program uses functionality from the APR and svn libraries, I added their development packages. Other than that, I only had to provide a description text for the package.

Notice that there is no version number in the control file. This is because it can be extracted from another mandatory file: debian/changelog. This file contains an entry for each released version and is later installed as /usr/share/doc/packagename/changelog.Debian.gz. dh_make did already set up debian/changelog to document the initial release, so we don't need to edit it now. If you need to update it for a subsequent releases, I recommend using the debchange helper (also from the devscripts package), since the file format is rather stringent.

A third mandatory file is debian/copyright. Its format is not stipulated, but it should contain the license restrictions under which the software is distributed. It is installed with the documentation of the package. In this example, I just copy the license section from the README file.
Editing the debian/rules script

Once we have the control data in place, it is the easiest to take a trial-and-error approach. dh_make already generated a debian/rules file for us, and before we edit anything, we try if it responds as expected to the standard targets:

~/svnyoungest-0.1$ fakeroot debian/rules clean
(... invokes make clean; no errors ...)

Although the target clean does not produce an error, there is a small mismatch to be aware of: in the underlying Makefile, a complete clean is referred to as distclean, not clean (the latter one only deletes intermediate files). So in this case, we need to invoke make distclean. This can be easily corrected by changing the actions of the clean target in debian/rules:

1. Add here commands to clean up after the build process.
   -$(MAKE) distclean

We can then rerun the clean target and try the other consecutive targets:

_{/svnyoungest-0.1$ fakeroot debian/rules clean
(... invokes make distclean; no errors ...)}
/svnyoungest-0.1$ debian/rules build
(... invokes make; no errors ...)
~/svnyoungest-0.1$ fakeroot debian/rules binary
(... invokes make install ...)
install: cannot create regular file `/usr/local/bin/svnyoungest': Permission denied
(...)

We notice that there is an error during the binary target: the underlying make install tries to install to /usr/local instead of the temporary directory where we build the installation tree. It is not difficult to see why: debian/rules passes in this path as the DESTDIR variable, whereas the Makefile expects a variable PREFIX. Here's how we correct the relevant part of the binary target in debian/rules:

1. Add here commands to install the package into debian/svnyoungest.
   $(MAKE) install PREFIX=$(CURDIR)/debian/svnyoungest/usr

We then retry building the binary package. After this succeeds, we can check the generated installation tree in the temporary directory (created by default under debian/packagename).

~/svnyoungest-0.1$ fakeroot debian/rules binary
(... invokes make install; no errors ...)

~/svnyoungest-0.1$ find debian/svnyoungest

debian/svnyoungest
debian/svnyoungest/usr
debian/svnyoungest/usr/bin
debian/svnyoungest/usr/bin/svnyoungest
debian/svnyoungest/usr/sbin
debian/svnyoungest/usr/man
debian/svnyoungest/usr/man/man1
debian/svnyoungest/usr/man/man1/svnyoungest.1.gz
debian/svnyoungest/usr/share
debian/svnyoungest/usr/share/doc
debian/svnyoungest/usr/share/doc/svnyoungest
debian/svnyoungest/usr/share/doc/svnyoungest/README
debian/svnyoungest/usr/share/doc/svnyoungest/copyright
debian/svnyoungest/usr/share/doc/svnyoungest/changelog.gz
debian/svnyoungest/DEBIAN
debian/svnyoungest/DEBIAN/md5sums
debian/svnyoungest/DEBIAN/control

~/svnyoungest-0.1$ cat debian/svnyoungest/DEBIAN/control

Package: svnyoungest
Version: 0.1
Section: devel
Priority: optional
Architecture: i386
Depends: libapr0 (>= 2.0.54), libc6 (>= 2.3.2.ds1-21), libsvn0 (>= 1.1.4-2)
Installed-Size: 64
Maintainer: Bruno De Fraine 
Description: report the youngest revision of a svn repository path
 svnyoungest is a small tool to report the youngest revision of a path in
 a Subversion (svn) repository. It is similar to ''svnlook youngest'', but
 works on the client side.

This seems OK. Notice that the documentation is installed under the right location by the tools from debhelper. If something was omitted, we could have checked the debian/docs file and the dh_installdocs(1) manpage to see how to fix it. Similarly, we notice that the dependencies of the binary package have been provided by dh_shlibdeps.
Validating the package

We proceed by trying a complete debuild run. This will also run a package validator to verify the newly created package. We pass in the options -us -uc to turn off the package signing step.

~/svnyoungest-0.1$ debuild -us -uc
(... regular build output ...)
Now running lintian...
E: svnyoungest: FSSTND-dir-in-usr usr/man/
Finished running lintian.

Apparently, something is blatantly wrong with our package, because the package checker lintian reports an error. This normally signifies a violation of the Debian Policy, and we can find more info at the lintian website. A quick investigation of the policy reveals that the manpages should go into /usr/share/man instead of /usr/man. Of course, this error was quite obvious to advanced packagers, but I wanted to demonstrate the importance of validating a package. The mistake can be corrected in debian/rules, by passing in a special MANDIR value:

DESTDIR=$(CURDIR)/debian/svnyoungest
...

    # Add here commands to install the package into $(DESTDIR).
    $(MAKE) install PREFIX=$(DESTDIR)/usr MANDIR=$(DESTDIR)/usr/share/man

If we then re-invoke debuild, we notice that the package now passes the lintian check. We finally obtain the desired Debian package, both in source and binary form:

_{/svnyoungest-0.1$ debuild -us -uc
(... regular build output ...)
Now running lintian...
Finished running lintian.}
/svnyoungest-0.1$ ls ../svnyoungest_0.1.*
../svnyoungest_0.1.dsc ../svnyoungest_0.1.tar.gz
~/svnyoungest-0.1$ ls ../svnyoungest_0.1_i386.*
../svnyoungest_0.1_i386.build ../svnyoungest_0.1_i386.deb
../svnyoungest_0.1_i386.changes

The resulting .deb file can be installed using dpkg -i.

Further Reading¶

More comprehensive information about creating Debian packages is available in the Debian New Maintainers' Guide and the Debian Developer's Reference.

If you would like to make your own .deb files downloadable by apt-get, check this article or this low-tech solution.

More advanced testing of packages can be done using pbuilder. Finally, this article describes how to integrate non-Debian packages with the standard reportbug tool.