2.1   How to build

In order to build an RPM, you should only need a base system, rpmbuild, python and GNU make.

make testrpm

In order to build the RPM from your version control system (probably subversion), you should have the svn command line client which has been configured to use the central repository (See Using alternate version control systems):

make rpm

In order to upload a package to the Open Build Service (OBS), you will need the osc command line client which has been configured in your environment with your credentials for connecting to the OBS. The credentials found in ~/.oscrc will be used:

make obs

That's the crash course!

2.2   Build process

For a bit more detail, here's what is happening when you build the rpm target. From the example directory, when you type make rpm, the following steps will occur.:

1. This will generate a correctly named specfile (i.e. atelerix.spec from specfile.in.
2. Connect to the VCS (svn, git or hg) and download a tarball of the source distribution, creating, e.g. atelerix-0.2.14.tar.gz.
3. Execute rpmbuild against the generated specfile.
4. Produce an RPM (FHS-style, with minor modifications) and SRPM.
5. Clean up any intermediate work and directories.
6. Leave the RPMs, SRPM, tarball and specfile in the ./dist/ directory.

If you want to restore a pristine directory, simply type make distclean to remove the ./dist/ directory. This is safe, since the package can always be rebuilt directly from the version control system.

3.1   Goals of the atelerix system

• Minimize the time between developer code change and potential deployment. (Works with Open Build Service here.)
• Allow easy support of predictable versioning of each piece of software.
• Provide guaranteed reproducibility of build for individual packages.
• Provide well-packaged, consistent RPMS to operations to ease the burden of software management.
• Provide tight integration with a version control system (subversion) so that packages can be built directly from the repository.
• Provide integration features with Open Build Service (OBS), a complete software platform (re)building tool. See also InternalOBS.

3.2   Big picture of usage

The Makefile itself should remain the same across all packages. Instructions for macro replacement and final installation can be found in the Makefile.build. Used together with specfile.in by the developer/packager to capture all code, data and configuration files that need to be shipped with the RPM.

4.1   Creating a new package (using svn)

1. Copy the required template files to a new directory (assuming new package called frobnitz and you are using svn).

   mkdir frobnitz/{trunk,branches,tags}
   svn add frobnitz
   svn commit frobnitz -m 'add new project frobnitz'
   cd example-packaging/trunk
   cp Makefile Makefile.build specfile.in ../../frobnitz/trunk
   # copy this if your project includes Python code
   cp -p setup.py.in ../../frobnitz/trunk
   # copy these two if your package includes services
   cp pkg/renserv.cfg.in pkg/start.in ../../frobnitz/trunk
   cd ../../frobnitz/trunk
   # don't forget to add some text to these! empty docs help no one
   touch README RELEASE-NOTES
   svn add *
   

2. Modify the specfile.in to change the Name and Version. You should also modify the Summary field and the %changelog (which must be in chronological order, newest stuff at the top).

3. Build the package with make testrpm. If that succeeds, then you have a package.

4. Commit the work with svn commit -m 'initial commit of frobnitz'.

5. Build the package out of subversion with make rpm.

6. Find the package in ./dist/.

7. Toss the package into OBS with make obs.

4.2   Features of the example package

• The Makefile provides Renesys standard (not far from FHS) locations for configuration directories, static data, a cache directory, available integration for daemon services, and documentation.

• The example package includes a start.in and a renserv.cfg.in file. The renserv.cfg.in file defines the environment in which (and user under which) you wish your process to run. If your program can (or should) be controlled by environment variables, put them (documented) into this file. Our operations group usually copies this file into their configuration management system for a given service.

• The start.in file contains the startup instructions for your daemon. It is very likely that the final command in this script will be something like:

  exec -a ${ARGV0:-@PACKAGE@} @PACKAGE_ROOT@/daemonic-utility

  All of the environment variables specified in renserv.cfg may be already set, but should be checked and have defaults set here (or in the eventual daemon).

  Of course, this start.in can do anything you want it to do, and it need not be shell.

  On the target host in question, start.in will turn into something like /usr/libexec/start-frobnitz-0.3.17 or similar.

• The packaging style supports the usage of POD for writing manpages. This could be extended to support other input formats.

• The Makefile supports connecting to an instance of the Open Build Service (OBS) to submit new versions of packages. The OBS system used depends entirely on your working environment (i.e. your ~/.oscrc).

5.1   Adding a configuration file

If any of your scripts needs a configuration file (separate from configuration data passed via environment variables), you should use the @PACKAGE_ETC@ macro and directory. Here's an example script called frobnitz which will need to read a configuration file.

1. Make your program name frobnitz.in.

2. Add frobnitz to BUILT_FILES in Makefile.build.

   BUILT_FILES := frobnitz
   

3. Create your configuration file main.conf and add to the repository.

4. Install the configuration file main.conf in the install target.

   install:
       mkdir -p $(DESTDIR)$(PACKAGE_ETC)
       install -m 0644 main.conf $(DESTDIR)$(PACKAGE_ETC)
   

5. After the package is installed, the configuration file will live here:

   /etc/renesys/frobnitz/
   /etc/renesys/frobnitz/main.conf
   

6. Therefore, add the configuration file and directory to specfile.in.

   %dir                   @PACKAGE_ETC@
   %config(noreplace)     @PACKAGE_ETC@/frobnitz.conf
   

5.2   Adding a manpage

1. If any of your scripts include inline POD (for manpage generation), you need only specify this in your Makefile.build. Find the line that says MANPAGES := and add your program name with a .1, e.g.

   MANPAGES := proggie.1
   

2. Now, add two lines to the install target that look like this:

   mkdir -p $(DESTDIR)$(MANDIR)/man1
   install -m 0644 proggie.1 $(DESTDIR)$(MANDIR)/man1
   

3. And, finally, add a line in the %files section of the specfile.in that refers to the newly added documentation, e.g.

   @MANDIR@/man1/proggie.1*
   

5.3   Adding a sample Apache configuration file

The below instructions assume that your proggie is going to be installed in your @PACKAGE_ROOT@ and be called frobnitz.py.

1. Identify the URL path on which your application will respond to requests. (Let's assume that this is /MyFrobnitzCGI.

2. Add a file called frobnitz.conf.in to your package with the following contents.

   <Directory "@PACKAGE_ROOT@">
     Options +ExecCGI -Includes -Indexes -MultiViews -FollowSymLinks
     Order allow,deny
     Allow from all
   </Directory>
   
   ScriptAliasMatch /MyFrobnitzCGI(.*$) "@PACKAGE_ROOT@/frobnitz.py$1"
   

3. Add the following snippet to the install target of Makefile.build:

   mkdir -p $(DESTDIR)$(APACHE_ROOT)/conf.d
   install -m 0644 frobnitz.conf $(DESTDIR)$(APACHE_ROOT)/conf.d
   

4. Add the following snippet to the %files section of specfile.in:

   %config(noreplace)     @APACHE_ROOT@/conf.d/frobnitz.conf
   

5.4   Adding a renserv (daemontools) service

1. Add a file called renserv.cfg.in to your package.

2. Add, at absolute least, the first two lines, which define the software (and version, if you must) and the user as which the software will run.

   SERVICE=@PACKAGE@
   USER=gradus
   

3. Add a start.in file which calls your daemon software.

   #! /bin/bash
   exec @PACKAGE_ROOT@/your-daemon
   

4. Modify your Makefile.build install target to look like this:

   mkdir -p $(DESTDIR)$(PACKAGE_SHARE)
   install -m 0644 renserv.cfg $(DESTDIR)$(PACKAGE_SHARE)
   #
   # -- install the start files for the daemontools services
   #
   mkdir -p $(DESTDIR)$(LIBEXEC)
   install -m 0755 start $(DESTDIR)$(LIBEXEC)/start-$(PACKAGE)
   install -m 0755 start $(DESTDIR)$(LIBEXEC)/start-$(PACKAGE)-$(VERSION)
   

Here's a brief summary of how the start.in and renserv.cfg.in files get used to run your service.

1. You write``start.in`` and renserv.cfg.in.
2. At build time, these are created as (for example), /usr/share/frobnitz/renserv.cfg and /usr/libexec/start-frobnitz.
3. You ask to have the software deployed and the service started.
4. Ops installs the package.
5. Ops copies /usr/share/frobnitz/renserv.cfg into /etc/renesys/services/frobnitz and adjusts the variables as necessary for production. (They probably use their configuration management system, DACS, to do this.)
6. Ops runs renserv start frobnitz.
7. The supervise finds the variables, as specified by ops in the renserv.cfg configuration file, puts them into the process environment and executes your start script. (Some details elided.)
8. Your start script, therefore will find configurables in the process environment.

5.5   Testing to see if a package builds successfully

You have finished editing your code, you have modified your specfile.in and your Makefile.build install target and you think the package should build. How do you test?

make testrpm

You will see a series of commands excute, and if the job exits successfully, you will find several files in ./dist/, which will include (assuming the package is named frobnitz-0.1.42::

dist/frobnitz-0.1.42.tar.gz
dist/frobnitz-0.1.42-1.src.rpm
dist/frobnitz-0.1.42-1.noarch.rpm
dist/frobnitz.spec

5.6   Listing the contents of a generated package

Supposing you have generated a package and you wish to see the list of files that have been included for distribution (and eventual installation on the end system). The rpm command operates not only on the installed software base on the system you are working on, but also operates on RPM files on disk, so you can point rpm at your generated package and have it show you the contents (these two commands produce the same result)::

rpm --query --list --package dist/frobnitz-0.1.42-1.noarch.rpm
rpm -qlp dist/frobnitz-0.1.42-1.noarch.rpm

The result of the above command would be something like the following::

/usr/lib/frobnitz
/usr/lib/frobnitz/script.py
/usr/libexec/start-frobnitz
/usr/libexec/start-frobnitz-0.1.42
/usr/share/doc/packages/frobnitz
/usr/share/doc/packages/frobnitz/README
/usr/share/doc/packages/frobnitz/RELEASE-NOTES
/usr/share/frobnitz
/usr/share/frobnitz/renserv.cfg

Supposing you need a bit more detail about the contents of the package::

rpm --query --list --verbose --package dist/frobnitz-0.1.42-1.noarch.rpm
rpm -qlvp dist/frobnitz-0.1.42-1.noarch.rpm

The results for increased verbosity::

drwxr-xr-x    2 root    root                0 Apr  9 02:50 /usr/lib/frobnitz
-rwxr-xr-x    1 root    root               58 Apr  9 02:50 /usr/lib/frobnitz/script.py
-rwxr-xr-x    1 root    root              433 Apr  9 02:50 /usr/libexec/start-frobnitz
-rwxr-xr-x    1 root    root              433 Apr  9 02:50 /usr/libexec/start-frobnitz-0.1.42
drwxr-xr-x    2 root    root                0 Apr  9 02:50 /usr/share/doc/packages/frobnitz
-rw-r--r--    1 root    root                0 Apr  9 01:57 /usr/share/doc/packages/frobnitz/README
-rw-r--r--    1 root    root                0 Apr  9 01:57 /usr/share/doc/packages/frobnitz/RELEASE-NOTES
drwxr-xr-x    2 root    root                0 Apr  9 02:50 /usr/share/frobnitz
-rw-r--r--    1 root    root              194 Apr  9 02:50 /usr/share/frobnitz/renserv.cfg

5.7   Increasing the version number of the software

After you are able to successfully build the software out of your working directory, using make testrpm, you will probably commit the software. The next step is to bump the version of the software. Minimally, modify the Version line in the specfile.in. This is most often the second line of the file.

head -n 2 specfile.in
Name:           frobnitz
Version:        0.1.42

It is good behaviour (though not required) to put in a reason for the package change in the %changelog.

%changelog
* Tue Mar 25 2014 Martin A. Brown <mabrown@renesys.com> [0.1.42-1]
  - adding support for mastication agents
  - improving efficiency of memory usage for >100k turnips
  - correcting typographical errors in punctuation

At this point, you should run make testrpm again to make sure that the package builds cleanly after your minor specfile.in changes. If that succeeds, then, it is time to commit your outstanding work to the version control system.

5.8   Adding a version control tag

The default version control system at Renesys is svn (subversion). A software tag is a way to take a snapshot of the state of all files in a given tree at a particular time. A common convention is to name the tag after the software version.

1. Set the version number as described in Increasing the version number of the software.
2. Move into the ./trunk/ directory.
3. Commit any pending changes. (e.g. svn commit)
4. Check that you have no pending work. (e.g. svn status)
5. Run make tag.

This will create a new tag in the version control system that will immortalize (well, depending on your VCS) the current state of the tree. You will be able in the future to rebuild that version of the software (see below).

5.9   Building the package from the version control system

This is trivial, and is part of the major point of atelerix.

1. cd tags/frobnitz-0.1.42
2. make rpm (See below.)

5.10   Submitting a package to OBS

After creating a package, you will want to put it into the Open Build Service (InternalOBS) instance that Renesys runs internally. From there, the package will get rebuilt for all platforms and architectures that we are supporting.

Assuming you have created a tag for your software, and you wish to distribute that internally (to the dev:internal repository, let's say):

1. cd tags/frobnitz-0.1.42
2. make obsdev

The RPM will be rebuilt directly from the version control system. The resulting package will be sent to the Open Build Service instance.

You may determine whether the package goes to the dev:internal project home:$USER (where $USER comes from your ~/.oscrc file).

• make obs to send it to home:$USER
• make obsdev to send it to dev:internal

Supposing you haven't created a tag yet, and you still want to send the package to the OBS, but maybe just for your own testing:

1. cd trunk
2. make testobs

The RPM will be rebuilt from the current working directory, and uploaded to the OBS project home:$USER

5.11   Using alternate version control systems

This system supports the use of alternate version control systems. Since git supports bidirectional communication with the git-svn tool, there is some (incomplete) support for interacting with the Renesys subversion repository and packaging system with git. If you have questions about this, just ask--several people are using these tools.

8.1   Using the macros:

Suppose you want to refer to the PACKAGE_ROOT macro in your code so that you can have /usr/lib/frobnitz inline in your code.

• In Makefile.build, you should use the Makefile variable $(PACKAGE_ROOT).
• In specfile.in and any *.in files (a.k.a. $(BUILT_FILES)), use the replacement string/macro with at signs: @PACKAGE_ROOT@.

8.2   Available macros:

• PACKAGE: frobnitz (e.g.)

  This variable will contain the name of the package as obtained from the specfile.in. This is the only place the name of the package needs to be entered. In our example case, this would be frobnitz.

• VERSION: 0.1.42 (e.g.)

  This variable will contain the version of the package as obtained from the specfile.in. This is the only place the version of the package needs to be updated. In our example case, this would be 0.1.42.

• MAJOR_VERSION: 0

• MAJOR_PACKAGE: frobnitz-0

• MINOR_VERSION: 0.1

• MINOR_PACKAGE: frobnitz-0.1

• PACKAGE_ROOT: /usr/lib/@PACKAGE@

  This is the root location for any data or code that is specific to the package and not intended to be used on the command line. This is the ideal location for programs that are intended to be used as daemons and never touched by a user.

  The developer should feel free to place any executable code and data under this directory. This directory is solely for the purpose of this package.

  In our example this would be /usr/lib/frobnitz.

• PACKAGE_SHARE: /usr/share/@PACKAGE@

  This is the best choice of path for any static package data. By convention, we put our sample renserv.cfg files here, too. The usual path for this will be /usr/share/frobnitz. The package has complete control of everything under this directory.

• PACKAGE_ETC: /etc/renesys/@PACKAGE@

  Any configuration file data should be stored in this directory. It will map to /etc/renesys/frobnitz.

• PACKAGE_CACHE: /var/cache/@PACKAGE@

  This directory is on the /var partition and is for any intermediate work required by the application. If your application uses files to store data between runs and needs to have access to data files between runs, then this is the appropriate directory to use.

• PACKAGE_SPOOL: /var/spool/@PACKAGE@

  This directory is on the /var partition and is for any intermediate work required by the application. If your application uses files to store data between runs and needs to have access to data files between runs, then this is the appropriate directory to use.

• APACHE_ROOT: /etc/httpd (or /etc/apache2)

  This variable is available for placing files that are Apache configuration files. If you are using this macro, the correct location will be selected for the host distribution. In our environment, this will always map to /etc/httpd/. See Adding a sample Apache configuration file.

• LIBEXEC: /usr/libexec

  This directory is primarily for the purpose of the daemontools start scripts. See Adding a renserv (daemontools) service.

• MANDIR: /usr/man

  This is the base directory for manual pages. Under this directory should be the man1, man3 and such directories. You will need to make the manX subdirectory in your Makefile.build. See also Adding a manpage.

• BINDIR: /usr/bin

  This is for any binaries (or scripts) you wish users to use. The path will be /usr/bin for conventional packages.

• SBINDIR: /usr/sbin

  This is for any binaries (or scripts) you wish only superusers to use. The path will be /usr/sbin for conventional packages.

• ETC: /etc

  This is the base configuration data directory. It is usually /etc/.

• VAR: /var

  The /var directory.

• SHARE: /usr/share

  The base directory in which package data is usually stored. This is, customarily, /usr/share. Please do not use this directory. See, PACKAGE_SHARE instead.

• PACKAGE_TMP: /var/tmp/@PACKAGE@

  This directory is on the /var partition and is for any large temporary files that the package may need to create. You should not expect that this directory will contain any files written in a previous run (this directory may even be a RAM disk). Consider also respecting the common $TMPDIR environment variable.

  (Deprecated. We have run afoul of the pruning strategies of tmpwatch with using this directory as a package-owned directory. Better would be to have a short-running program use the $TMPDIR environment variable, use /tmp/ or /var/tmp with safe-file and directory handling or, perhaps the best option, use @PACKAGE_CACHE@.)

• ROOT: /

  This will be set to the root of the filesystem from which all of the other software will be found. It will be set to the empty string, so that everything will be installed relative to the Unix root directory /.