Comment goes here
You should log in and post some comments! The link is up there in the toolbar. Go nuts!
 

Intermediate Level Course: The Linux Kernel

The Linux Kernel

What is the Linux kernel

The Linux kernel is essentially what Linus Torvalds started work on in 1991. A kernel is the part of the operating system that handles the communication between software and your machine's hardware. In the decade since Linus' first kernel was released, its complexity has grown enormously and so has the number of people working on it. It is now the largest collaborative project in the world. The reason for this is because Linus released the Linux kernel under the terms of the GNU General Public License, which makes it freely distributable and allows anyone to make changes to it. In this lesson we are not going to be talking about making changes to the source code itself, but we will talk about choosing certain files of this source code in order to create a custom kernel.

Where to get the kernel source

As I mentioned, the kernel source code is available for anyone to download it and you can find it at kernel.org.

At the time of this writing, the latest stable version of the Linux kernel is 2.4.22. Though the arrival of version 2.6 is imminent, we will be dealing with the 2.4.X version of the kernel in this lesson.

The latest stable version of the kernel comes in at just under 30 megabytes in a bzip'd tarball. A gzip'd version is also available and is slightly larger. You should download the format of your choice and untar it in the directory /usr/src/.

Kernel configuration
Compiling a new kernel is a serious thing. It's a lot like changing the motor in your car. If you haven't done it correctly, the machine, which is like your whole car, will not run. For this reason, you should not attempt this on a mission critical system the first time around. On the other hand, you shouldn't be afraid of it either. Like any upgrade, if you have taken the appropriate steps and have made back-ups you shouldn't have any problems.

First, you need to know what hardware is in your machine in order to get a good kernel at the end of the process. There are also other concerns. For example, you need to decide what kinds of file systems your machine needs to work with (ext2, Reiser, VFAT, NTFS, etc), what character support to include in the kernel, what networking protocols you need to use or whether the machine will be used as a router or firewall, to name a few. This is important because what sets a Linux kernel apart from, say, a Microsoft Windows one is that you can pick and choose what goes into it and then your kernel will be much leaner. This customization is what makes Linux operating systems more stable. So it's important to know what you need. It makes no sense to compile support for USB scanners into your kernel if you're not going to be using a USB scanner. On the other hand, if you have this type scanner and you don't compile USB support into the kernel, you won't be able to use it.

There are three main interfaces that you can choose from when you are preparing the configuration of a new kernel. They are 'config', 'menuconfig' and 'xconfig'. The first one is an old-style command-line interface that is not very attractive nor do I recommend it if this is something that you've never done before. The last one, xconfig, is a GUI based configuration tool that requires you have X-window running. This may not be the best option if you have a server because there's no need to be running x-window in that environment. As a general rule, I stick with the middle option, 'menuconfig' because it has an ncurses based GUI (text mode with colors) and therefore it is visual and straightforward. It can also be used to configure a kernel on a remote machine, something that xconfig is just not suited to.

To use this, just type:
make menuconfig

inside your /usr/src/linux/ directory. You will see something very similar to the screenshot above. What you have here are the major categories of kernel configuration options. Since I don't know what each machine out there is going to need, it would be impossible to go over every option. However, we'll look at the more common options and use those as examples.

The kernel configuration process creates a .config ([dot]config) file with your options in the /usr/src/linux/ directory. If you have systems with similar hardware, you can use it between machines. This also comes in handy if you do a complete switch to another distribution, for example. If you use the same .config, you'll need to fire up 'menuconfig' once so your .config file gets "registered" with the new system.

Using kernel modules
One of the most popular features of the Linux kernel is its support for modules. A module is code that provides for support for certain things. These modules can be inserted or removed from a kernel while its running. Normally, support for, say, a network card would be built-in to the kernel. That is, the kernel would always include that support. It would form part of the kernel itself. If your network card were to fail at some point and you weren't able to find that kind of card, you would have to recompile your kernel to get it to provide support. However, if you indicated that you wanted to provide support for various types of network cards as modules when you configure your kernel, you could swap out a new card supported by one of the modules. You would then have a program called 'insmod' activate the modules to provide support. Besides this, a kernel with all of its support options compiled into it will be bigger. It may become so big that you may not be able to make a floppy rescue disk from it. It's perhaps a good idea to configure non-permanent items as modules. (SCSI support for pen drives, if your hardware is IDE, for example). Configuring items as modules requires you to choose this option at the beginning.

Loadable module support --->

If you activate this, you should indicate (M) if you want something configured as a module. An asterisk (*) will compile support directly into the kernel.
Other kernel options

Directly following the modules support options is the processor type option.

Processor type and features --->

Here, you should use the processor type for your machine (Pentium III, AMD, etc.). You should obviously not try to get support for an AMD Athlon if what you have is an Intel 486. Again, you need to know what's in your machine before you even start.

You will find the rest of the options to fairly self-explanatory. If you're going to be using the machine as a mail server and/or web server, you need support for your network card, tcp/ip support and perhaps some other protocols plus iptables support is recommmended if you're planning on implementing a firewall. Choosing support for things like this then leads to opening sub-categories to see more options for this support. Again, it's impossible to go through all of these options, but fortunately the kernel maintainers have done a lot of work to provide explanations for most of them. They are to be commended for their labors. By pressing < Help >, you'll see explanations and even some recommendations. If you think you might need this support, it's a good idea to read through the explanations thoroughly. And as the kernel maintainers point out in all of these help bits, if you don't know what this means, you can probably safely choose 'no'.

If you're planning on "rolling your own" kernel, as they say in the Linux community, please take the necessary time to do this carefully. Napoleon once said: 'Dress me slowly, I'm in a hurry'. The time you take at the beginning to make sure that you've included all the features you want will save you time on the other end, just as Napoleon knew that the time spent in making sure the buttons were all lined up prevented his dresser from having to unfasten everything and start again. I have been in situations where I rushed through a kernel configuration only to boot the machine and have sound cards not work or firewalls give error messages just because I left things out in my haste.

When you exit 'menuconfig' you will be prompted to save your configuration. Once you've done this, you can proceed to compile your new kernel.
Compiling a new kernel

I remember the days when compiling the kernel took hours (or even days on slow machines). With today's powerful CPUs and cheap memory, you should probably be able to get a new kernel up and running in under a half an hour (not including configuration time).

Your first step in compiling the kernel will be to create the interdependency configuration for the new kernel. For example, if you've got a BTTV web cam you also need video support for Linux. If you've chosen BTTV support for your kernel and not 'Video for Linux', then your dependencies will break. To make sure you've got everything you need, you need to type the following:

make dep


You'll start seeing a lot of activity. If you see error messages at the end, then you've got some unresolved dependency. You'll need to go back to 'menuconfig' and add support according to what the error messages show. If this isn't your first try at building the kernel, you can invoke the following command to clean up files left behind from previous attempts. It is optional.

make clean


Now we're ready to create our kernel. You can now invoke the following command to compile your new kernel:

make bzImage


Again, if you're working with recent hardware, this will not take too long. If you're curious, you can watch what it's doing. If you've got a slow machine and you're not well-versed in C programming, it might seem like watching paint dry. Sooner or later you'll have a nice new kernel, deposited in /usr/src/linux/arch/i386/boot/. This doesn't mean that you're finished, however. If we've chosen to use kernel modules, we need to deal with those now. We need to do this:

make modules


This command will create the modules. Now you must move them to /lib/modules. To to this, you need to use this command:

make modules_install


Please note that there's an underscore (_) between those words. Even though that's the last command we're going to invoke related directly to the kernel, we're still not finished. Now we need to move our new kernel to the /boot directory. This is accomplished by simply doing:

cp /usr/src/linux/arch/i386/boot/bzImage /boot/


If you've already got a bzImage, this will overwrite it. If you want to keep the previous kernel image, you may rename either one of the files (kernelnew, bzImage.old, etc.).

We still have one more step to go. We need to tell the bootloader about your new kernel. This will differ depending on the bootloader you're using.

LILO and GRUB
In case you're wondering, we're still talking about the kernel, not a Walt Disney movie. LILO and GRUB aren't cartoon characters. They are two bootloaders that are used to boot the kernel on Linux systems.

LILO
LILO has traditionally been the most popular bootloader on Linux systems. We talked briefly about LILO in our section on configuration files. Here we're going to talk more in-depth about it. LILO is a program that allows you to choose how you're going to boot your computer. You can configure it to boot one or more Linux kernels or even boot into another operating system, like Microsoft Windows. LILO is commonly installed into the master boot record or your hard drive, though you may also choose to install it into a sector of a primary partition. It has a configuration file, lilo.conf, which controls what it does. This can be found in /etc/lilo/. Here's the sample lilo.conf we saw in a previous lesson:

# device to boot
boot=/dev/hda
# our root partition
root=/dev/hda1
# map file
map=/boot/map
# delay in 10ths of a second before booting
delay=20
# kernels to boot
default=linux
image=/boot/bzImage
      label=linux
      read-only
      append="hdc=ide-scsi"
image=/boot/vmlinuz
      label=linux_old
      read-only


This is well-commented and not tremendously difficult to understand. First, we specify which device contains the operating systems and/or kernels to boot. In this case, it's in the first IDE hard disk of the machine. The location of the partition containing these is the first partition, (dev/hda1). Then we have the reference to the 'map' file. This file contains information about your kernel that may be needed by programs that access it. The next lines are the actual kernels to be booted, starting with a notation with name of the label of the kernel/operating system to be booted first (default=linux). It's standard procedure to include a label so that the lilo menu you see at boot time can identify the different boot options. Kernels are generally labeled read-only. The last item in the first kernel is an 'append' parameter.

In this example, we see instructions to treat an IDE CD-RW as a SCSI device. This is an example of post-kernel configuration. Even though we may have compiled SCSI support into our kernel, we still need to tell the kernel to use it in case of our drive actually being IDE. This little trick is something we can't compile into our kernel, so we need to "append" this behavior here. You will also see that we are choosing between two kernels. We can add more here if we want. That's what we're going to do now:

image=/boot/bzImage.new

      label=linuxnew

      read-only


We must add the above line to lilo.conf to make our system aware of the new kernel. After we save the changes, we need to invoke LILO so that the changes take effect. Simply type:

lilo


This last step is extremely important. If you don't do this and you shutdown or restart, your system will not be able to boot from the hard drive. You'll need to insert a floppy or CD rescue disk and mount the hard disk partition where lilo.conf resides and run lilo again.

Now you're free to re-boot and try out the changes.

GRUB
GRUB is a GNU bootloading utility. It is now widely used in mainstream Linux distributions like RedHat and Mandrake. If you're using one of these distributions, GRUB has been configured for you during the installation process and like LILO, is normally installed into the master boot record. Here's my grub.conf for my RedHat workstation:

default=0
timeout=10
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
title Red Hat Linux (2.4.20-8)
        root (hd0,0)
        kernel /boot/vmlinuz-2.4.20-8 ro root=LABEL=/
        initrd /boot/initrd-2.4.20-8.img


As we did with LILO, we'll add some lines so that we can use our new kernel:

title Custom Linux Kernel 2.4.22
    root (hd0,0)
    kernel /boot/bzImage ro root=/dev/hda1 hdc=ide-scsi


'title' is what will appear in the splash screen so that you can choose at boot time. what follows 'root' is how GRUB understands your hard drive. Linux will boot from the first partition of the first hard drive. This is the hd0,0 part. GRUB starts counting at zero, though it may sound confusing at first. Next we tell GRUB where the kernel image is, to boot it read-only, where our root (/) directory is located. Finally, like the LILO 'append' parameter, we tell GRUB we've got a IDE CD-RW drive that should be treated like a SCSI device.

Unlike LILO, we don't need to type something like grub so that our changes get registered. After you save your changes in grub.conf, you just need to reboot and choose the new kernel.
Enjoy your new kernel

Now you can enjoy your new kernel. It probably runs better than the one that's offered out of the box by most distribution vendors. Also, keep in mind that users of proprietary operating systems can't change their kernel for better performance. That's another reason for enjoying it - the freedom that Linux provides.