The Linux Kernel: Compiling and Installing

[DevynCJohnson]

Series Index - http://www.linux.org/threads/linux-kernel-reading-guide.5384/

Aloha! After you have spent a lot of time configuring your kernel to make the kernel you need, you can now compile it. The source code is C code in the form of plain text. This is readable to humans but not computers. Compiling the code converts the code to a form computers understand called binary (ones [on] and zeros [off]). Compilation will also make all of those kernel files one file called the kernel.

To compile the kernel, type "make" in a terminal that is in the same directory as the kernel's source code folders. This will take some time. Once done, the modules must be compiled by typing "make modules". To make the compiling process easier from the beginning, type "make; make modules". This will make the kernel and then the modules instead of the user coming back later to type "make modules".

WARNING: Before you install a kernel, backup all of the important data. Be sure to make a copy of /boot/ onto a FAT32 memory card. This helps to repair the system if the installation fails. FAT32 does not store permissions, so it will be easier to use a live disk to copy the files back. Remember to set the original file permissions and executable bits.

Once the compilation has finished successfully, we can then install the kernel to the local system (I will explain how to install the kernel on other systems in a moment [cross-compiling]). In the same terminal, after compilation, type "make install". This will place some files in the /boot/ directory. The "vmlinuz" (or some similar name) is the kernel itself. "initrd" is the temporary RAM-based filesystem that is placed in memory and used during boot-up. The "System-map" contains a list of kernel symbols. These are global variables and functions used in the kernel's code. "config" is the configuration file for the kernel. grub.cfg will automatically be updated. However, some other bootloaders may need to be manually configured. The kernel installer can configure the Grub, LILO, and SysLinux bootloader automatically. Bootloaders like BURG may need to be manually configured. The modules must also be installed by typing "make modules_install".

NOTE: Both the kernel and modules can be installed using one line - “make install && make modules_install”

Once that process is complete, the user can ensure the kernel was installed by restarting the system and typing "uname -r" in a terminal when the system is back on. If the system fails to boot or uname reports a different version number than expected, the issue may be due to one of many issues. Either the bootloader was improperly setup, feature/configuration conflict, compilation error, improperly installed, or some other reason. The best way to start finding the source of the issue is to look at the systems logs (if the system boots up enough to produce logs). "dmesg" is a command that prints the kernels logs to the screen. Look for any errors, warnings, or unexpected results. If the system does not boot or does not boot-up enough to produce logs, use a live Linux disc to perform diagnostics and repairs. If all else fails, compile the kernel again and make sure you installed the kernel as Root or used "sudo".

NOTE: The best way to repair such a system is to use a live Linux distro to remove the new/broken kernel and then manually fix (or paste a backup) Grub's files.

Some Linux users like to have the documentation installed as well, but this is not required. For those that like to have the documentation installed, type this line where version is the kernel version - "install -d /usr/share/doc/linux-VERSION && cp -r Documentation/* /usr/share/doc/linux-VERSION" (VERSION is the kernel's version number). Obviously, Root privileges are required.

To compile a newer kernel with the same features as your current kernel, then type this command "zcat /proc/config.gz > .config". This file may not exist, if so, you may be able to ask the developers of your distro/kernel for the file. The "zcat" command uncompresses the data and places it in the file ".config". Remember to type where you want ".config". This file is to be placed in the Linux kernel directory and allow it to replace the current file. Then, compile and install the kernel as you normally would.

Cross-compiling is slightly different. Configure the kernel for the intended system. Make sure that when the kernel was configured, that it was configured with cross-piling in mind. When cross-compiling, their are two terms to be familiar with. "Host" is the system performing the compilation. The "Target" is the system that will receive the new kernel. Make sure that the host system has the proper compilers. For example, to cross-compile for ARM systems, users will need gcc-arm-linux-gnueabi on the host system. Generally, the developer can do a search in their package manager or Google for the proper/best cross-compiler for their needs. The specific command used to cross-compile for ARM systems is "make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-". The "ARCH=arm" refers to the target's processor type and "CROSS_COMPILE" declares the cross-compiler. Notice that the cross-compiler is missing the "gcc-" at the beginning and ends in a dash. That is the format users must use when using the cross-compiler as a parameter. The modules can be cross-compiled by typing "make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules". To install the kernel on the target system, copy the Linux kernel folder to the target system. Once the files are on the target system and a terminal is open in that directory, type "make install && make modules_install". Of course, you must be Root or use "sudo".

INFO: Kernel.org hosts a list of supported cross-compilers (https://www.kernel.org/pub/tools/crosstool/).

FUN FACT: Some Linux distros store the kernel's config file in the /boot/ directory. Developers wanting to compile a kernel with the same settings as their current kernel can copy the file to their build directory. cp /boot/config-$(uname -r) /PATH/TO/.config



Compilation and Installation Summary:

Standard:

make && make modules && make install && make modules_install

Make a newer version or remix of your kernel:

zcat /proc/config.gz > .config &&  make && make modules && make install && make modules_install

Cross-compile:

make ARCH={TARGET-ARCHITERCTURE} CROSS_COMPILE={COMPILER}; make ARCH={TARGET-ARCHITERCTURE} CROSS_COMPILE={COMPILER} modules && make install && make modules_install

In the next article, we will discuss adding and activating modules. Mahalo!

 

[ryanvade]

make install && make modules_install

can be

make modules_install install

 

[lobo]

# make bzImage modules && make modules_install

Is all you need...

Though you should try to build kernel packages for your distro.

 

[Esdras Said]

Really interesting! I'm going to read from the first article!

 

[Waqar]

I compiled kernel and modules and they work perfectly fine on my computer. Now I want to move this kernel to an identical system but without 'make'ing. Moving kernel and ramdisk is fine and easy, just some grub entries. What do I do to move modules to other system? If I try to copy /lib/modules/(myKernelVersion) to another location it copies 'source' and make folders (available in folder as a link) with it as well that is 2GB of load. Can I get away without these 2 folders? Will system work fine if I skip those 2 folders while copying?

 

[DevynCJohnson]

I compiled kernel and modules and they work perfectly fine on my computer. Now I want to move this kernel to an identical system but without 'make'ing. Moving kernel and ramdisk is fine and easy, just some grub entries. What do I do to move modules to other system? If I try to copy /lib/modules/(myKernelVersion) to another location it copies 'source' and make folders (available in folder as a link) with it as well that is 2GB of load. Can I get away without these 2 folders? Will system work fine if I skip those 2 folders while copying?

Yes. I do not have those two folders in my /lib/modules/kernel-version/. The "source" and "make" folder is only needed for compiling/making modules.

 

[Waqar]

Yes. I do not have those two folders in my /lib/modules/kernel-version/. The "source" and "make" folder is only needed for compiling/making modules.

Sorry the folders available as a link were "source" and make"build", I typed wrong earlier. For "source", yes it is not there in distro kernel's folder so seems it can be skipped as you said. But "build" folder is also there. each of them is 1.0GB. skipping one means skipping 1.0GB but other 1.0G is still huge. Is it required in any case than module development?
build folder points to linux headers in distro kernel, so may be it was required to compile modules. But once compiled, do I need those in any case?

 

[DevynCJohnson]

Sorry the folders available as a link were "source" and make"build", I typed wrong earlier. For "source", yes it is not there in distro kernel's folder so seems it can be skipped as you said. But "build" folder is also there. each of them is 1.0GB. skipping one means skipping 1.0GB but other 1.0G is still huge. Is it required in any case than module development?
build folder points to linux headers in distro kernel, so may be it was required to compile modules. But once compiled, do I need those in any case?

You do not need it. If for some reason you do have problems, make a new "build" link that points to the headers on the new/second system.

 

[Waqar]

You do not need it. If for some reason you do have problems, make a new "build" link that points to the headers on the new/second system.

Although that completes the answer, but just to be on safer side, these other files in this folder 'modules.order', 'modules.xxxmap', etc are not something system specific and will they work fine with other systems? (off course same architecture)

 

[DevynCJohnson]

Although that completes the answer, but just to be on safer side, these other files in this folder 'modules.order', 'modules.xxxmap', etc are not something system specific and will they work fine with other systems? (off course same architecture)

Copy over the modules files. You can disregard links.

 

[Waqar]

Copy over the modules files. You can disregard links.

View attachment 1142

Nice! but that induced another question I don't have an 'initrd' folder here, should be ok I guess as this kernel was loading fine.

 

[DevynCJohnson]

Nice! but that induced another question I don't have an 'initrd' folder here, should be ok I guess as this kernel was loading fine.

Yes, it should all be fine.

 

[Ornik]

I'm not sure whether it fits in here, but perhaps it might be worth a mention, for those using Ubuntu flavours, of the alternative of using something like

make-kpkg --initrd kernel_image

followed by a "dpkg -i" command, as it combines several steps?

Also, if you are compiling on a machine with a multicore processor, you may be able to speed up the compiling process by setting the CONCURRENCY_VALUE, j. For example:

make-kpkg --initrd -j5 kernel_image

for a cpu with 4 cores (the j value being the number of cores + 1; so an 8-core cpu would have -j9). With the addition of this single option, I cut compiling time from 30 minutes to less than 10 minutes.

 

[DevynCJohnson]

I'm not sure whether it fits in here, but perhaps it might be worth a mention, for those using Ubuntu flavours, of the alternative of using something like

make-kpkg --initrd kernel_image

followed by a "dpkg -i" command, as it combines several steps?

Also, if you are compiling on a machine with a multicore processor, you may be able to speed up the compiling process by setting the CONCURRENCY_VALUE, j. For example:

make-kpkg --initrd -j5 kernel_image

for a cpu with 4 cores (the j value being the number of cores + 1; so an 8-core cpu would have -j9). With the addition of this single option, I cut compiling time from 30 minutes to less than 10 minutes.

Thanks for the helpful tip.

 

[Priya Agarwal]

I did not change any configuration option while I did configuration and tried to boot the new kernel. But it did not boot. How is it possible?
Also after booting if I want to make changes in configuration options then how do I do it. As in building again the whole kernel will take a long time, so using make and make modules command doesn't make sense. How do I make only that part which I changed so that it gets over in less time?

 

[DevynCJohnson]

I did not change any configuration option while I did configuration and tried to boot the new kernel. But it did not boot. How is it possible?
Also after booting if I want to make changes in configuration options then how do I do it. As in building again the whole kernel will take a long time, so using make and make modules command doesn't make sense. How do I make only that part which I changed so that it gets over in less time?

If you compile the kernel's source code without changing anything, it will very rarely work because the defaults are not set to work for a majority of computers. I do not know how the kernel developers determine what a default should be, but do not always assume all defaults are good. True, in most cases, the defaults are good, but not always.

Okay, so you are asking about making changes to the kernel after it is booted or compiled. Well, once you compile the kernel, you must configure and compile a new one. As for an active system, all that you can do is add, remove, disable, and enabled modules, but nothing to the kernel itself. You can tweak some limits and settings with the sysctl command or the /etc/sysctl.conf file. However, you must still configure and compile a new kernel.

When you configure a kernel, you can save the settings. This article (http://www.linux.org/threads/the-linux-kernel-configuring-the-kernel-part-1.4274/) briefly mentions the different interfaces/methods for configuring the kernel. Most permit saving your choices to a file that can be loaded when needed to configure a kernel. My favorite interface is initiated with the "make menuconfig" command (be sure to have ncurses install [libncurses5-dev]).

If you have compiled a kernel, but lost the configuration settings, you may still be able to retrieve them. Look for this file "/proc/config.gz" after booting into the kernel that has the needed configuration. You can go here (http://www.linux.org/threads/the-linux-kernel-configuring-the-kernel-part-2.4318/) and go to the paragraph discussing "IKCONFIG".

This is helpful, but I am sure you may have already seen it - http://www.linux.org/threads/linux-kernel-reading-guide.5384/

I hope this helps.

 

[Priya Agarwal]

priya@priya-Rev-1-0:~/linux-3.15.1$ make
scripts/kconfig/conf --silentoldconfig Kconfig
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
make[2]: *** [silentoldconfig] Error 1
make[1]: *** [silentoldconfig] Error 2
make: *** No rule to make target `include/config/auto.conf', needed by `include/config/kernel.release'.  Stop.

Thanks. I loaded the old config file and then made changes and next time I built using make it took lesser time. And also when I tried to boot the default configuration it booted up, I wasn't waiting long enough for screen to show Ubuntu desktop.
However today I installed the latest version of Ubuntu and tried to build the kernel again. In make menuconfig I did not change any configuration option and saved the default. Then I typed make command. It is showing the error as shown. It is unable to find .config file. However when I open the folder 'linux-3.15.1', hello.config (my file) is present.
How to solve this?

P.S. Earlier it wasn't showing any such error. I had Ubuntu 12.04 till yesterday. I uninstalled it for some reason and then Installed Ubuntu 14.04 today. Is the error because of this? :/

 

[DevynCJohnson]

priya@priya-Rev-1-0:~/linux-3.15.1$ make
scripts/kconfig/conf --silentoldconfig Kconfig
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
make[2]: *** [silentoldconfig] Error 1
make[1]: *** [silentoldconfig] Error 2
make: *** No rule to make target `include/config/auto.conf', needed by `include/config/kernel.release'.  Stop.

Thanks. I loaded the old config file and then made changes and next time I built using make it took lesser time. And also when I tried to boot the default configuration it booted up, I wasn't waiting long enough for screen to show Ubuntu desktop.
However today I installed the latest version of Ubuntu and tried to build the kernel again. In make menuconfig I did not change any configuration option and saved the default. Then I typed make command. It is showing the error as shown. It is unable to find .config file. However when I open the folder 'linux-3.15.1', hello.config (my file) is present.
How to solve this?

P.S. Earlier it wasn't showing any such error. I had Ubuntu 12.04 till yesterday. I uninstalled it for some reason and then Installed Ubuntu 14.04 today. Is the error because of this? :/

By default, the "make" command and configuration tool will look for the hidden file "/.config". To have the tools use your "hello.config", you must load that configuration file in the configuration tool and/or pass a parameter to the make command to tell it to use "hello.config".

I doubt changing distros would cause your config files to disappear or not be found. Changing distros can cause "make" errors if you are missing a header file or developer's tool in your library.

 

[Priya Agarwal]

How can I pass parameter to tell make command to use hello.config?

Is there any command that will tell me which file 'make' was using all this file. I make changes to hello.config after loading that in configuration window, then save and exit and then I run make; make modules. If make wasnt using hello.config then which file was it using?

 

[DevynCJohnson]

How can I pass parameter to tell make command to use hello.config?

Is there any command that will tell me which file 'make' was using all this file. I make changes to hello.config after loading that in configuration window, then save and exit and then I run make; make modules. If make wasnt using hello.config then which file was it using?

Well, most developers rename their config file to ".config". By default, the ".config" file is used.