Linux Kernel - http://www.linux.org/threads/linux-kernel-reading-guide.5384/ Many of us have heard of virtualization, guest operating systems, hypervisors, and many other related terms. Virtualization allows users to run an operating system (OS) inside the "real" OS as if it were an application, or virtualize some device or network. But, what exactly is virtualization and how does it all work? NOTE: This article discusses virtualization as in operating system virtualization. This article does not discuss virtualization as seen in Java's virtual machine, runtime environments, and similar systems. Virtualization is the creation of a system or component that is not truly real. In other words, virtualization makes something seem genuine. The virtual object is not exactly real, but it is not entirely fake either. In computer technology, virtualization can be used to make "semi-real" networks, devices, computer systems, etc. Users may make a virtual device that the operating system thinks is a genuine physical device. The user knows the device is entirely software, but the computer cannot see the difference between a virtual device and a physical device. There is also OS virtualization which is where an operating system (called the guest) runs inside (or on top) of the OS on the hardware (called the host). The host is running on the physical computer while the guest system thinks it is running on bare-metal hardware. The guest runs as it would on real hardware, but it cannot see or access the physical hardware. Instead, it thinks it is running on the hardware that the virtualization software is virtualizing. Virtualization is not emulation. Emulation is the process of pretending (or imitating) some other device or system. Think about some game emulators like ZSNES (which mimics Super Nintendo). They behave like the real console just as ZSNES acts like the Super Nintendo consoles. The emulator performs the functions of the real device, but it does not truly act like the device. Emulators copy the behavior of real devices enough to complete the same tasks they perform. For instance, Super Nintendo games work on the console and SNES ROMs (files containing an SNES game) work on ZSNES. On both systems, the game functions perfectly, or at least nearly perfect on the emulator. However, emulators do not act or appear exactly like the real system. For instance, most game emulators support the ability to save the game's current state/progress while consoles usually use a different way to save the game (like checkpoints). Also, emulators may provide the user with additional functionality. In contrast, virtualization is nearly real. Virtualization is faster than emulation in nearly all cases. A virtual machine is a piece of software that represents a computer system. Guest operating systems run inside of virtual machines (like Oracle's VirtualBox). The guest OS thinks the virtual machine is real hardware. The hypervisor creates and manages the virtual machine. Multiple hypervisors and guest systems can be run on a host system as long as there is enough “real” hardware to satisfy the needs of the guests. A guest can also be run in a guest. NOTE: Usually, when many guests are run, a single hypervisor manages all of the virtual machines containing guests. Virtual machines can represent hardware of an architecture that is very different from the host. This is possible because of the way the guest's system calls are mapped (or "translated"). Users can run 32-bit guests on 64-bit hosts. Operating systems designed for one processor type can be run on a host that uses an entirely different CPU. However, not all processors, BIOS chips, and hypervisors support certain guest architectures. For instance, some 64-bit hosts cannot support a 64-bit guest (usually due to limitations in the CPU and/or BIOS). Because of this fact, be sure to understand the virtualization capabilities of your BIOS, CPU, hypervisor, and host kernel when choosing the right hypervisor+guest+host combination for your needs. To run a particular guest, you may need a different hypervisor, host OS, or entirely different physical hardware. FUN FACT: Under very special conditions, a 64-bit guest can run on a 32-bit host. This is possible if the physical hardware is 64-bit and has a CPU that offers certain virtualization extensions. Hypervisors may work at different levels of the host system depending on the particular hypervisor's abilities. KVM (Kernel-based Virtual Machine) is a hypervisor in the Linux kernel (or as a module) that runs virtual machines in separate userspaces. QEMU and Oracle's VirtualBox (VBox) are hypervisors that run in the host's userspace as an application. Some hypervisors may work at the hardware level. Such hypervisors may exist as firmware or act as a compatibility layer under the OS kernels (such as Xen which is a hypervisor in the form of a microkernel). Many hypervisors exist, each with their advantages and disadvantages as well as supported features. Not all hardware supports the same hypervisors. Some hypervisors require special CPU commands (instruction sets and extensions). Computers with a CPU that lacks virtualization extensions will only be able to use hypervisors that run as applications or use paravirtualization (discussed below). NOTE: Processors with virtualization instructions/extensions can help speed-up virtualization. This is known as hardware-assisted virtualization. Virtual machines often use virtual devices, and they may use virtual networks. Many guest systems use a file as a hard-drive (device virtualization). ISO files are examples of virtual optical discs (like CDs and DVDs), and “.img” files are often used as virtual floppies. Many virtual machines allow users to use ISO files on their host system as discs on the guest. Guests can form a virtual network with other systems like other guests, the host, and external systems. Some hypervisors allow the host's directories to be mounted in the guest as a drive. This helps users transfer files between systems. Another type of virtualization exists called paravirtualization (in contrast to full virtualization). This is a form of virtualization where the guest knows it is running in a virtual machine. Thus, the guest will modify its behavior slightly to enhance its performance. The guest does so by using a special set of system calls referred to as "hypercalls". Not all operating systems support hypercalls. Operating systems that support hypercalls either have the hypercalls built into the kernel or as a driver. Hypercalls use the paravirtualization application-programming-interface (para-API). Hypercalls come from the kernel and are sent to the hypervisor while the guest's userspace still sends system calls to the guest's kernel. Using paravirtualization (also called para-virt or PV) does not require the processor to have virtualization extensions. The Xen hypervisor supports paravirtualization. NOTE: Hypercalls are to hypervisors as system calls are to kernels. (Analog from http://wiki.xen.org/wiki/Hypercall) Guest's userspace >>(syscalls)>> Guest's kernel >>(hypercalls)>> Hypervisor Instead of hypervisors, OS–level virtualization can be used. In this virtualization process, the kernel runs multiple sets of userspaces. Hypercalls and virtual machines are not used since the userspaces interact with the kernel directly. The userspaces communicate with the kernel using the typical system calls like “normal” systems. There is only one kernel running on the physical hardware. Since there is only one kernel that interacts with the userspaces directly, each userspace must be compatible with the kernel. This means OS-level virtualization is only possible among systems that use the same kernel. For instance, users running a Linux kernel in this fashion can only run userspaces of various Linux distros. Each userspace instance is referred to as a "container" in contrast to the term “virtual machine”. The machine can act as a server that uses one IP address with each userspace using its own hostname or domain-name. Alternately, the userspaces can each be set to use a different port. Either way, only one IP address is used, thus saving IP addresses for other uses. There is also another form of OS-level virtualization. LXC is a hypervisor that allows each userspace to have its own Linux kernel. This is possible because LXC divides the hardware resources for each kernel (via cgroups) as if the hardware were multiple computers. Such systems do not use the concept of "host and guest". Technically speaking, each OS is its own host. NOTE: OS-level virtualization is not the same as multibooting. In summary, operating systems can be run in another operating system by using virtualization or paravirtualization. Virtualization also allows multiple operating systems to run on the same piece of hardware simultaneously. Hypervisors manage and create virtual machines that contain the guest OS. All this works together to allow users to take advantage of other operating systems to perform tasks that may not be easily performed using their usual OS. Understanding the basics of virtualization can help users with creating virtual systems.