Steeldump_BETA

Building Steeldump

(You can skip this section if you just want to use steeldump packages normally. See above for installation instructions.)

To help debugging or developing Steeldump, the following steps should be enough to build your own steeldump packages:

• Set up a chroot environment or virtual machine to compile steeldump in.
  
  If you really do not want to do this, keep this in mind: Steeldump must be compiled in the exact same location where it is going to be installed into, /opt/steeldump. So you cannot run the steeldump scripts in a filesystem where steeldump packages are already installed.
  
  Although it does not address safety issues, you can obviously build a mock-up chroot very easily using mount -o bind, so this should not be too much of an issue.
• Create a new directory /opt/steeldump. In the build environment, this directory should be owned by a non-root user. (The build scripts will only run sudo briefly for each package to fix file ownership before packaging the files up. Building should be done normally under your non-root account.)
• Check out the steeldump scripts from Subversion:

  $ cd /opt/steeldump 
  $ svn co svn://common-lisp.net/project/steeldump/svn/trunk/scripts

• Run the first script:

  /opt/steeldump$ ./scripts/init

  -- or just look at what it does. It merely creates a few directories and checks whether required software is installed. CMUCL is used for building (you can use the debian-provided cmucl package), and dpkg-dev tools as well as sudo are needed.
• The first package you need to download and build is SBCL:

  $ ./scripts/fetch-sbcl
  $ ./scripts/build-sbcl
  $ ./scripts/makedeb-sbcl

Congratulations: If that worked, you have your first .deb package in /opt/steeldump/pool.

• Now you can build the actual steeldumped applications using the same steps:

  $ ./scripts/fetch-all
  $ ./scripts/build-all
  $ ./scripts/makedeb-all

And that's it. After makedeb-all, you can find all packages in the pool directory.

While debugging, however, you will probably want to build individual packages instead of all in one go:

• For every system, there is a separate fetch-foo script. These fetch-scripts do not track dependencies for you. When not using fetch-all, make sure to download everything you need before trying to build.
• The build-foo scripts can be invoked in an arbitrary order. For example, building climacs will automatically compile mcclim if it had not been compiled yet. However, it does not actually create the dumpfile for mcclim, you will still have to call build-mcclim eventually before you can make its .deb file.
• Finally, for makedeb-all note that it has one advantage over the individual makedeb-foo scripts: Before calling out to the others, makedeb-all relocates the heap files to non-overlapping locations, speeding up loading of the heap files a little.

If you got this far and have working packages in /opt/steeldump/pool, send me a postcard.

Extending Steeldump

(You can skip this section if you just want to use steeldump packages normally. See above for installation instructions.)

To add a new package called "blubba", create these files:

• /opt/steeldump/scripts/fetch-blubba
  This script must put the source code into /opt/steeldump/src/blubba. Remove version numbers from the directory name, if any.
  Look at the other fetch scripts for ideas. Usually it is enough to call one of the helper scripts, aux/fetch-url, aux/fetch-cvs, or aux/fetch-svn. If you have to patch the source code, this script is the right place to do that.
• /opt/steeldump/scripts/build-blubba
  Usually this script is trivial and just calls build-system. Copy over /opt/steeldump/scripts/build-SAMPLE and change the system name from "SAMPLESSYTEMNAME" to "blubba". This script is the right place to call "make" if the system includes C code.
• /opt/steeldump/scripts/lisp/build-blubba.lisp
  This Lisp script is less trivial. Starting from the sample file /opt/steeldump/scripts/lisp/build-SAMPLE.lisp, change the system name from "SAMPLE" to "blubba", but review the heap dumping logic carefully. The heap dumper needs to be told about:
  • The packages that are to be dumped (look out for multiple defpackage forms). This is the first argument to dump-system.
  • The ASDF systems involved. If the .asd file contains multiple system definitions, all of them must be listed manually in the build script. (The :system argument.)
  • The name of the package the .asd files defines and uses. (The :system-package argument.)
  You now have a first draft of the dumping script. Beware that dumping is not the part where things tend to fail, it is the loading and running of heap files where mistakes show up. The heuristic used by the heapdumper is that it thinks in terms of packages. For many systems, you will have to supply more information than just the package names, because the software often installs objects into variables contained in other packages, slots of objects the heap dumper cannot know about, etc.
  • Refer to sb-heapdump documentation on the precise logic used by the dumper.
  • For CLIM systems in particular, application-defined command tables and presentation types need to be extracted from McCLIM-internal tables. CLIM also has methods that are eql-specializing on objects like +flipping-ink+, so we must guarantee uniqueness of these objects. For details, see the build scripts for gsharp and climacs.
  • Any CLOS usage can be tricky. The heap dumper will (a) include generic functions and all their methods in the package the generic function's name is in (for example, the MCCLIM package), and (b) additional methods in a different package if those methods specialize on a class named by a symbol in that other package (for example, methods defined for classes in the CLIMACS package). One corollary of these rules is that we must not dump the McCLIM package after having loaded Climacs into the same core, because then loading of McCLIM would fail trying to find the CLIMACS package.
• /opt/steeldump/scripts/descriptions/blubba
  This file ends up as the Description: header in the Debian package's control file. Again there is a skeleton file: /opt/steeldump/scripts/descriptions/SAMPLE.
• /opt/steeldump/scripts/makedeb-blubba
  This scripts collect all files to be installed and creates the .deb archive. Starting with /opt/steeldump/scripts/makedeb-SAMPLE, replace SAMPLESYSTEMNAME with "blubba", and DEPENDENCIES with other steeldump packages that "blubba" depends on. Omit the "steeldump-" prefix and version number, the helper script inserts those for you.