The Linux Boot Sequence

Introduction

The term “boot” is a contraction of “bootstrapping”, originating in the idea of "lifting yourself up by your bootstraps". The boot process is the execution of a series of programs performing incremental steps, each step doing a specific job, and passing control to the next program.

The very first step is a "power on system test" (POST). A specialized chip on the motherboard has as its only purpose holding a small program that starts the boot sequence as soon as power is applied to it. This program is the Basic Input Output System (BIOS); it checks a few basic hardware components, such as processor, memory, disk drives, and I/O ports. Configuration of the hardware can be somewhat involved, and you request presentation of its user interface by pressing one or another keys early in the boot sequence (usually the F1 or F2 function keys, but also F10 or DEL on some systems). Control passes from the BIOS to something called a boot loader.

The boot loader is another small program; rather than being held in nonvolatile memory in a piece of silicon, it is stored in the first 512 bytes of the hard disk. It cannot be stored as a file, since no file system is present until the operating system is able to load. This region of the disk is therefore known as the master boot record (MBR). The BIOS uses POST to obtain the physical layout of the disk, thereby locating the MBR data.

Before flushing itself from memory, BIOS loads the program stored in the MBR into memory, and the processor then begins executing it. The boot loader program itself is independent of any particular operating system, and allows you to boot several systems, through a rudimentary user interface similar to that of BIOS, but rather simpler. Most GNU/Linux distributions ship with a choice of two boot loaders, Lilo (LInux LOader) and GRUB (GRand Unified Bootloader). Since 512 bytes requires very economical code, the boot loader's job is only to load enough of the OS kernel to see things farther on their way. Data for GRUB is located in the filesystem at /boot/grub, but it is in a specific physical location on the disk so that GRUB can find some of it before the kernel has even booted the file system. GRUB proceeds as follows:

• BIOS code reads the MBR from a disk (inspecting the last 2 bytes to verify that it is indeed a MBR).

• Starts executing bootloader code (GRUB stage 1).

• Bootloader jumps to location (sector num) of next stage. This sector num is stored at a particular location in the bootloader "code" at GRUB install time and usually points to a stage 1.5, which is located in the first 30 kilobytes of disk space immediately following the MBR.

• Stage 1.5 knows about the boot filesystem so it opens the filesystem on a partition that is specified at install-time. Stage 1.5 is filesystem-specific code used to find the operating system image on the "boot" filesystem. It looks for the stage 2 executable here and executes it. Note since stage 1.5 knows about the boot filesystem it gives much greater flexibility in upgrading stage 2, the kernel, and so forth, as their new locations don't need to be written to the earlier GRUB stages.

• Stage 2 contains most of the GRUB logic. When GRUB Stage 2 receives control, it presents an interface to the user in order to select which operating system to boot. This normally takes the form of a graphical menu, although if this is not available or the user wishes further control, GRUB has its own command prompt, where the user can manually specify the boot parameters. GRUB can also be set to automatically load a particular kernel after a timeout period. It loads the menu.lst file and executes the statements, usually providing a menu to the user, and so on.

• When GRUB starts booting one of the entries, it reads the initial ramdisk and starts the kernel running, telling it about the ramdisk.

• The initial ramdisk finds the location of the filesystem it itself is on and passes that to the kernel as its root filesystem. This allows for greater flexibility of the devices the kernel resides on.

• The kernel reads its root filesystem and executes /bin/init by default. This in turn parses /etc/inittab which usually sets up the login consoles and starts executing the scripts in /etc/init.d

• These scripts start various subsystems, culminating in starting X. X in turn starts the display manager which gives a login prompt.

The next critical priority is to prepare a file system. Most file systems will be built into the kernel directly, but sometimes are loaded into a RAM disk. The RAM disk (or initdisk or initrd) is a memory-based file system that is loaded by the boot loader before Linux is run. The Linux kernel, as its final act during boot-up, runs init. This is a program normally stored as /sbin/init. It is init, and not the kernel, which then configures the system, starts services (a more correct term for applications such as Apache) and opens virtual terminals. And this is where the real fun begins!

By default, init is located in the filesystem at /sbin/init, and is the first place the kernel will look, followed by /etc/init and /bin/init, in that order. Should all of these fail, a shell (/bin/sh) will be executed instead, enabling you to recover the system. You can also specify a different location for init by passing it as an argument to the kernel.

Once the kernel has started, init prepares the machine to work in a particular way: what software should be running, what terminals should be open, and which network services are to be allowed. It does this using runlevels. Each specific runlevel describes a unique state of the system as it runs. As with most programs under Linux, init has its own configuration file stored in /etc. It is called inittab and consists of configuration data that indicates what, where and when particular services are to be run. The inittab file itself is well commented (lines beginning with # character), and you should familiarize yourself with its contents.

Run Levels

There are a number of runlevels that are, by convention, used to describe a working Linux system, with 2, 3 or 5 being typical for a multi-user configuration. The complete list is given below (according the Linux Standards Base - LSB).

• Runlevel 0 (halt) - Shuts down everything and brings the system to halt.

• Runlevel 1 (single user) - Useful for maintenance work.

• Runlevel 2 (multi-user) - multi-user, but with no network services exported.

• Runlevel 3 (multi-user) - normal/full multi-user mode.

• Runlevel 4 (multi-user) - reserved for local use. Usually the same as 3.

• Runlevel 5 (multi-user) - multiuser, but boots up into X Window, using xdm, or similar.

• Runlevel 6 (Reboot) - As 0, but reboots after closing everything down.

Notice runlevel 0 is called “halt”. A runlevel describes what services should be running; a runlevel that terminates all services and runs a program to halt the processor leads to a valid state of the system. Given the laws of thermodynamics and of computing, this is by far the most stable state of the system!

inittab: Configuration of the init Program

The first lines of the inittab file reveal the following:

1. The default runlevel.

id:2:initdefault:

Each line in inittab has four fields, each separated by a colon. The first portion is an identifier for the action, and can be anything provided it is unique within the file (with the exception of the lines which spawn virtual terminals, which will be covered later). The second indicates the runlevel(s) to which this rule applies. In the above case it means runlevel 2 only, while the case below would work in all “multi-user” levels.

message:2345:wait:echo "In multi-user mode"

Parameter three is called the action. It indicates how the command (given in parameter four) is to be run. There are numerous actions available, and most are used in the inittab provided by most default distributions of GNU/Linux. The common ones are shown in the list below.

• wait – Execute the command, and wait until it completes before moving onto the next one. Used mostly for running software in sequence. If a program can not be run, init will work through the rest of the file. init does not stop on errors, but will report them to the console. • respawn – Execute the command, but respawn it when the process completes. Used for virtual terminals that need to re-run the login prompt (through getty) whenever the user logs out. • ctrlaltdel – Whenever the “three fingered salute” is given, run this program. It is usually configured to reboot the machine. • off – Disables the entry, without deleting it from the file. • initdefault – The default runlevel used if init is called without an argument. • sysinit – The command is run first before anything else, when the system boots (runlevels are ignored).

Finally, the executable in parameter four may be a command, script or daemon, and can include arguments if required. But both the command and its arguments may be omitted, as we saw with initdefault.

Although the kernel starts init at the default runlevel, runlevel can be changed at any time, provided you are the superuser, without rebooting the machine by issuing the following command:

1. init 2

This puts the system into runlevel 2, starting all services that should be running at this runlevel, and killing those that shouldn’t. Using runlevels as a profile in this manner lets you remove services during system maintenance, such as the network, very simply.

Finally, consider the following:

1. init 0

to place the machine in the halting runlevel. This is equivalent to the more descriptive:

1. shutdown -h now

Although longer, the latter is preferable because it is more flexible; a time argument allows you to shutdown in half an hour, for example.

Resource Configuration (The rc Directory Structure)

init is a simple program. It proceeds, line by line in inittab, executing every command (relevant to the current runlevel) in the order listed in the file. The first command invariably performs system initialization, by specifying the sysinit action.

si::sysinit:/etc/init.d/rcS

Its purpose is to configure any system-wide parameters (such as the system clock, or the serial port), regardless of runlevel. Once init has handled the system initialization it switches to the default runlevel and continues reading the rest of the file.

The sysinit start-up code is handled by the /etc/init.d/rcS script, which starts each process that is catalogued in the /etc/rcS.d directory. Since the boot-up sequence doesn’t have a runlevel, a pseudo-runlevel called ‘N’ is used.

When switching between runlevels, the set of running services must also change. While this is possible to do from inside inittab and the /etc directory, it is cumbersome. To ease the pain, Linux uses a set of directories, one per runlevel (called /etc/rc0.d, /etc/rc1.d, /etc/rc2.d etc), that lists some services that must (and some that must not) be active in that specific runlevel. A script called /etc/init.d/rc is then responsible for starting and stopping them. The directory structure and absolute location of rc0.d and init.d can be inferred from the inittab file.

As an example, runlevel 2 will be directed from /etc/rc2.d which contains a number of files. Those beginning with the letter ‘S’ are services that will start when we change into this runlevel, and those beginning with ‘K’ are services that will be killed.

init is clever enough to do things in the correct order established by the system administrator, as well as working efficiently, i.e, not stopping, and then restarting, any service that appears in the both the new runlevel, and the previous one.

S10sysklogd

S12kerneld

S14ppp

S19bind

S19nfs-common

S20anacron

This fragment of the /etc/rc2.d directory from a GNU/Debian box shows that sysklogd will be started first, followed by kerneld, then ppp, and so on. The /etc/init.d/rc script will execute each of these files in order, passing it either the argument “start” or “stop”, depending on whether the filename begins with an ‘S’ or ‘K’. This gives the /etc/rc2.d directory similar functionality to the Windows Startup folder. But while Windows hides the order and method by which its startup program runs, Linux makes them available with ease, executing them directly from the rc script.

The sysinit directory /etc/rcS.d works in exactly the same way, but as it contains only system configuration information, no ‘K’ files are required. In contrast, booting up into single-user mode (runlevel 1) has almost nothing except kill files, to stop all the old services.

The missing link

The files in the /etc/rc2.d directory (or the other rcN.d directories), however, are not actually files. They are links to scripts that do the real work! The script for S10sysklogd, for instance, would reside as /etc/init.d/sysklogd (the S10 having been stripped off) and would start or stop the service based on its argument.

All the startup and shutdown scripts are in this directory (/etc/init.d), so controlling services on-the-fly is very easy. You can stop, start, or restart them with one simple command. Namely,

1. /etc/init.d/apache start

Some scripts will also support the restart directive. This removes the obligation to kill each process, and restart them manually, whenever a change to the configuration file is made. It also removes the need to reboot after installing new software, since the start command can be called directly.

You can add these links yourself with the ln command:

1. ln -s /etc/init.d/apache /etc/rc2.d/S20apache

However, adding the same link to several runlevels (and its equivalent kill version to the halt, shutdown and single-user runlevels) can be a little monotonous, and therefore prone to user error. So there is a script that helps. It’s called update-rc.d, and has the usage:

usage: update-rc.d [-n] [-f] <basename> remove

update-rc.d [-n] [-f] <basename> defaults [NN | sNN kNN]

update-rc.d [-n] [-f] <basename> start|stop NN runlvl runlvl . 

 -n: not really

 -f: force

The basename is the name of the script, and would be apache in our example.

1. update-rc.d apache remove

This would remove all the Apache symlinks in the /etc/rc?.d directories (where “?” can be any character). You must make sure the /etc/init.d/apache script is also removed, either by manually deleting it, or by use of the -f flag.

The defaults option will start the service in all multi-user runlevels, and stop it in the single user, halt and reboot runlevels. If you want to add services to specific runlevels, the “start|stop” option must be used. This requires the order parameter. The runlevels are given as separate parameters, and terminated with an all-important full stop.

1. update-rc.d apache start 20 3 4 5 .

Adding system startup for /etc/init.d/apache ...
 /etc/rc3.d/S20apache -> ../init.d/apache
 /etc/rc4.d/S20apache -> ../init.d/apache
 /etc/rc5.d/S20apache -> ../init.d/apache

The NN symbol has been replaced with a number indicating the placement of the service in the start-up sequence. This represents the positional number we saw earlier on each of the symlinks, and can be omitted when using defaults. The default value is 20—a sensible choice since the network components are ready by this time.

The end of innocence

Once the various services have started, the inittab script arrives at the actions to configure the three-finger salute (ctrl-alt-delete) and prepare the virtual terminals. These work simply by running the /sbin/getty program (or equivalent) on each specified terminal. The respawn action is required to repopulate the virtual terminals after a user has logged out. If the terminal is connected via modem, inittab can handle that too by using mgetty instead of getty. At this point we have a login prompt, and our journey is over.

Conclusion

The apparently simple act of getting a prompt on-screen is very involved. However, it is not as complex as it first appears since it is comprised of several small steps, each building on the previous one.

Each step increases our understanding and helps us streamline our system.

FOOTNOTES

To give a hexdump of this first sector of the disk you can do dd bs=512 count=1 if=/dev/hda | od -Ax -tx1z -v (changing /dev/hda as appropriate for your setup)

Lilo, for example, uses /etc/lilo.conf to determine which operating system kernel, or kernels, can be loaded at boot time. This file can also be used to pass options from Lilo into the kernel. In order to create or change the Lilo boot loader you need to edit /etc/lilo.conf and run the /sbin/lilo program. This takes the loader and places it directly into the MBR. The most common change of this kind occurs after compiling a new kernel, as the boot loader needs to know where the new kernel lives. The next evident activity will come from the Linux kernel itself. The kernel does little more than simply initialize itself.

Linux, regardless of version, will always issue a flood of messages to the screen as it runs. These describe the computer system, as the kernel sees it, to aid troubleshooting. The messages themselves are also written into a log file at /var/log/kern.log, or can be retrieved using the dmesg command.

A more exotic system configuration can be created using the /linuxrc file to load other drivers, or mount additional components into the file system.

Linuxrc may be either a program, or a script. However, the latter would require a shell interpreter in order to run, and is thus rarer. Lilo users, for example, can include an append line in lilo.conf to use a completely different program. This line can also specify a runlevel that will take precedence over the one in inittab. append="init=/sbin/myinit 5"