Saturday, January 15, 2011

Kernel Compilation

Linux Kernel Compilation

Steps in compiling a kernel:

* Installing the sources.
* Configuring the kernel (choosing which features and Drivers to compile).
* Compiling the kernel (i.e. typing a single command, and watching...).
* Installing the compiled kernel.
* Updating the boot loader to recognize the new kernel.
* Booting...
* Making the new kernel become the default.

Kernel "Types"

* The Linux kernel comes in two variants - the "vanilla" kernel, and the distribution's kernel.
* The "vanilla" kernel is the kernel officially released by Linus, or by a member of the community appointed by Linus (e.g. kernels version 2.4.X are officially maintained and released by Marcelo Tosatti).
* The distribution's kernel is normally a "vanilla" kernel, with many patches on top of it that either did not get accepted to the "vanilla" kernel, or that the distribution's maker back-ported from newer kernels.
* Thus, normally the latest "vanilla" kernel has more features...
* ... while the distribution's kernel has gone through more orderly testing and could be more stable.

Getting The Source

* The sources of linux kernels are available via the Internet, or on the distribution's CDs.
* Of-course, there are many mirrors, and since we're talking about not-so-small files (around 30MB), we better know our mirrors.
* In Israel, one may try Iglu's mirror, at http://www.iglu.org.il/, or check Hamakor's Israeli mirrors list, at http://mirror.hamakor.org.il/.

The Source Of The Distribution's Kernel

* The source code of the distribution's kernel comes as another package on the installation CDs.
* On RedHat 9, for example, it is stored in a file named 'kernel-source-2.4.20-8.i386.rpm'.
* Installing it is done like installing any other RPM package:

rpm -Uvh /path/to/kernel-source-2.4.20-8.i386.rpm


* If The distribution's maker updated the kernel, the new kernel sources would be found among the updates.
* In RedHat 9, the current file is kernel-2.4.20-28.9.src.rpm, found in the SRPMs (Source-RPMs) directory.

The Source Of The Vanilla Kernel

* The "vanilla" kernel is normally kept at ftp.kernel.org, as a tar.gz (or tar.bz2 - better compressed) file.
* For example, the latest 2.4 kernel for now is found at ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.24.tar.bz2
* Unpack these kernel sources as root:

cd /usr/src/
tar xjf /path/to/linux-2.4.24.tar.bz2


* You will get a new directory named 'linux-2.4.24', containing the sources.

"Readying" The Sources

* Before we do anything with the sources, we should make sure the source tree does not contain any old object files or configuration information...
* ... That got there accidentally.
* This is especially true for distribution kernels - they tend to contain various stale files.
* To do this, go into the source directory and run the command:

make mrproper

Configuring The Kernel

* Before we compile the kernel, we need to configure it.
* This includes telling it which drivers and features to compile ...
* ... and how to compile them (as modules or inside the kernel's main file).
* Several configuration programs are supported by the kernel, to be launched using one of the following commands:

make config
A simple text-mode program, that asks a zillion questions one after the other. Not recommended.
make menuconfig
A full-screen text-mode program. Use it if you don't have X windows running, or if you're connected from a remote location.
make xconfig
A Graphical program. Use this one when you can.

* There is yet another, which will be shown later...

Tips Regarding Configuring The Kernel

* Each item has a help section - read it.
* Each help text contains a suggestion of what to do if you're not sure. Use these suggestions, indeed.
* The first time around, take a tour around the different options, without changing them.
* The configuration process generates a file called ".config" in the top directory of the kernel sources. Keep a backup before making changes.
* Don't configure a kernel when you don't have enough free time - it's a long task (1-2 hours) the first time around.
* Do not despair - eventually, you'll configure kernels in 5 minutes ;)

Compiling The Kernel

* Once configured, Compiling the kernel is easy.
* First, make sure we start afresh (takes a few seconds):

make clean

* Then, prepare the dependencies list (might take a minute):

make depend

Compiling The Kernel (Cont.)

* Then, compile the kernel's main part (might take 5-30 minutes):

make bzImage

After this step, we should have the following new file:

[root@simey linux]# ls -l arch/i386/boot/bzImage
-rw-r--r-- 1 root root 1064017 Jan 16 01:53 arch/i386/boot/bzImage


* Then, compile the kernel modules (might take 5-30 minutes):

make modules

Compilation Errors

* A normal compilation process might result several warnings.
* But errors should not happen, unless you're compiling a non-stable kernel.
* Make sure you started from a fresh compilation (i.e. ran 'make clean').
* Make sure that you have read/write access to the source tree (i.e. if the sources belong to 'root', compile them as user 'root').

Installing The New Kernel

* Installing the kernel is split into two parts:
1. First, installing the kernel itself.
2. Then, installing the kernel modules.
* Before we install the kernel, we want to make sure we do not overrun our current kernel, or a previously existing kernel.
* So we will install the kernel itself manually.
* Avoiding overrunning the kernel modules is more difficult, since the location of the modules is hard-coded into the kernel (its version number) and the module loading tools.

Installing The Kernel's Main File

* Runnable kernels are expected to be in the /boot directory.
* Simple way to install the kernel (assuming it is version 2.4.20-8):

cp arch/i386/boot/bzImage /boot/vmlinuz-2.4.20-8

* Make sure the name is unique, and especially different then the current kernel...
* ... cause if we got a broken kernel, we will want to be able to switch back to the current kernel.

Installing The Kernel Modules

* The kernel modules will normally be placed in /lib/modules/
* If the compiled kernel is of the same version as an existing kernel, we better first back-up the modules library:

cp -rp /lib/modules/2.4.20-8 /lib/modules/2.4.20-8.old

* This assumes that the older kernel can boot into a working system without working loadable modules...
* ... or that we can boot from a rescue CD/floppy to restore the modules.
* And now install the modules:

make modules_install

The "initrd" (Init Ram-Disk) File

* Sometimes, we need to pack modules that are needed before the kernel can access the disk partitions, into the 'initrd' file.
* To do this, we need to use the 'mkinitrd' command:

mkinitrd /boot/initrd-2.4.20-8.img 2.4.20-8

This creates an initrd file named '/boot/initrd-2.4.20-8.img', containing hard-disk and file-system related modules, for kernel version 2.4.20-8.
* The initrd command uses the modules installed under /lib/modules/, so it must be executed after make modules_install.

Updating The Boot Loader

* Once we installed the new kernel and its new modules, we need to tell the boot loader about it.
* Generally, we need to supply the following information:
o Kernel image file path (e.g. /boot/vmlinuz-2.4.20-8).
o Partition of the root directory - copy it from the spec of the current kernel.
o Kernel parameters - may be copied from the current kernel, if it has not changed drastically.
o Optional initrd file path.
o Label.

Instructions For "lilo"

* If your system uses "lilo" as the boot loader, the config file is normally at /etc/lilo.conf
* A normal entry for our example kernel would look like this:

image = /boot/vmlinuz-2.4.20-8
initrd=/boot/initrd-2.4.20-8.img
label = rh9-mykernel
append="hdc=ide-scsi"
root=/dev/hda1

* the 'root' entry may be omitted, if we have 'root=current' in the global section.
* If we do not need initrd, the initrd entry may be omitted.

Instructions For "lilo" (Cont.)

* Finally, run "lilo" to make the actual update of the boot loader:

[root@simey ~]# lilo
Added linux-2.4.18
Added rh9-mykernel
Added dos

MANY people forget this step!

Instructions For "grub"

* For systems with "grub" as their boot loader, the config file is normally at /boot/grub/grub.conf
* An entry for our example kernel would look like this:

title Red Hat Linux (2.4.20-8)
root (hd0,0)
kernel /boot/vmlinuz-2.4.20-8 ro root=/dev/hda1 hdc=ide-scsi
initrd /boot/initrd-2.4.20-8.img

* The disks and partitions are numbered by bus numbers, rather then by letters: hd0,0 means /dev/hda1.
* If we have /boot on its own partitions, the path names should NOT contain /boot, and the "root" entry should point to the /boot partition.
* No need to run anything after updating the config file - grub will read it during system boot.

Booting The New Kernel

* Once everything is set, reboot your machine.
* At the boot loader's prompt/menu, chose the new kernel.
* Watch the boot messages - are they similar to what you are familiar with? Are there any errors or failures?
* If boot fails - reboot again with the previous (working) kernel.
* If boot succeeds, check that everything works (including networking, sound card, modem...).

Troubleshooting The Boot Process

* The boot process logs all its messages into /var/log/messages
* Possibly also in /var/log/dmesg
* Look in those files for error messages, and try to analyze them or look them up on the Internet.
* Sometimes you'll notice a problem resulting from kernel configuration - reconfigure, recompile from scratch (make clean...), re-install and try again.
* If you see that all modules fail to load with unresolved symbols, make sure you compiled and installed the modules.

Boot Problems - No Root Device

* Symptom: you get the message that no root device was found.
* Reboot to the previous kernel.
* Check the exact syntax of your lilo/grub entry for the new kernel. Are you sure you specified the root device entry properly?
* Make sure you compiled the drivers for your hard-disk and the partitions into the kernel image, or you have them in the initrd image.

Boot Problems - "lilo" says "LI"

* Symptom: you get the message "LI" (or a similar message) and then nothing, or an endless loop of garbage.
* Reboot to the previous kernel.
* Check the syntax of the lilo entry for the kernel - something there is terribly wrong.
* If you can't see what is wrong, delete the entry, and create it again. Often mistakes done in the first copy+paste do not occur in the second attempt.

When Installing A Newer Kernel...

* Check the file 'Documentation/Changes' in the source directory, for packages that might need upgrading before booting the new kernel.
* Don't configure the new kernel from scratch - copy your previous kernel's '.config' file to the new kernel's source directory, and run make oldconfig - you will only need to answer questions about new features.
* If you used a distribution's kernel previously, it normally has a config file matching the installed kernel.
* On RedHat, this file is usually under /boot/config-

External Device Drivers

* Various types of hardware might have drivers available not as part of the kernel sources.
* Perhaps this is a new driver, that didn't yet get accepted into the normal kernel.
* Or the driver is maintained by the manufacturer of the hardware...
* ... and possibly delivered in binary-only mode?
* We need to be able to install such drivers on our own, then.

External Device Drivers Formats

* An external device driver might come with full sources, that we need to compile. This is the best scenario.
* An external device driver might come as a binary-only module. In this case we need to get a driver that was compiled specifically for the kernel version we are using.
* If we use a less-commonly-used distribution, we might not find a driver for our distribution's kernel...
* At which time we either dumb the hardware, or take the best "vanilla" kernel for which there is a binary driver available.
* The driver could come as a hybrid - half source, and half binary. the Source-part need to be compiled against our kernel, and it'll make sure the binary part works.

Notes On External Device Drivers Compilation

* You need to have your kernel source tree configured properly, matching your running kernel.
* You don't have to actually compile the kernel source tree in this case.
* Sometimes the driver has a script that will compile the driver for you, hiding the actual compilation process.
* If it thinks your kernel sources are under /lib/modules/2.4.20-8/build, this is ok - this should be a symbolic link pointing to the right location of the sources....
* ... But it does not hurt to make sure.
* Otherwise, read the driver's installation instructions, and follow them carefully.

No comments:

Post a Comment