Become a kickstart Guru

[dos2unix]

What is kickstart? It's simply an automated installation application.
If you only build out one computer, once a year, kickstart won't be of much benefit to you.
However, if you build out several computers fairly frequently, have I got a deal for you.
It can also be used for consistency, even if you only build that one computer a year, but it needs
to be exactly like the one you built last year, kickstart will take care of that also.

The Linux installers have gotten better over the years, but I still have to create users, set a root password,
set user passwords, create filesystems, install specific packages, create repo, set up the network, set up the
default language and keyboard configuration. If I only have to do this once or twice a year, not a big deal.
But if I work in a data center with hundreds of computers, and I have to rebuild them every few weeks.
Well, now it gets to be a little time consuming.

Kickstart is really only made for redhat/fedora Linux types. I've used it for CentOS, AlmaLinux, RockyLinux,
OracleLinux, Redhat, Fedora, and ScientificOS. I've had some success with OpenSuSE as well.
There is some limited support for Debian/Ubuntu, but they aren't officially supported, so we will skip those for now.

You can run a kickstart server on pretty much any Computer, with pretty much any flavor of Linux.
It's best to have a dedicated computer, that's really not being used for anything else.
A switch is highly recommended, a small LinkSys, Netgear, or TP-Link 5 port will work just fine.
A large hard drive of at least 200GB is recommended. You have to have at least one Ethernet interface.
It has to be Ethernet, sorry Wi-Fi, won't work for Kickstart. Really your kickstart server should have two network
interfaces. One can be Wi-Fi, but at least one has to be Ethernet. Some USB to Ethernet adapters do not support
PXE boot, so a built-in Ethernet port usually works better.

The client computers have to have an Ethernet port as well, they have to support something call PXE boot.
Kickstart works with both legacy BIOS and EFI. As long as the client computer supports one or the other.

Next, we will get into the set-up, but this can be time consuming the first time.
This also works best if you've installed the client computer the manual USB drive method at least once.

 

[dos2unix]

On your server computer, you will need to install three applications.
A dhcp server, a web server, and a tftp-server. For fedora/redhat it looks like this.

dnf install nginx tftp-server dhcp-server

You can also use apache/httpd, but I find nginx a little easier to configure.
If there is enough interest, I will go over the apache setup as well.

You will need to open the firewall for these services. For fedora/redhat it looks like this.

firewall-cmd --add-service httpd --perm
firewall-cmd --add-service dhcp --perm
firewall-cmd --add-service tftp --perm
firewall-cmd --reload

and finally, we will need to enable these services.

systemctl enable dhcpd
systemctl enable nginx
systemctl enable tfp.socket

That last one may be a little different than you are used to, it's a socket, not a service.

If you try to start these, you may notice that dhcpd will not start, that's because it needs a configuration file.

Next, setting up dhcpd and indexing for our web server.

 

[dos2unix]

On fedora/redhat type systems, the dhcpd configuration is in /etc/dhcp.
The name of the file is dhcpd.conf. You can edit the existing one, or you can simply copy and paste this one.

# see dhcpd.conf(5) man page
#
# Note /etc/xinetd.d/tftp is no longer needed, this path is set in the tftp.service file.

allow booting;
allow bootp;

set vendorclass = option vendor-class-identifier;
option pxe-system-type code 93 = unsigned integer 16;
set pxetype = option pxe-system-type;

option domain-name "kickstart.com";

# Set the rsyslog facility to local7 and redirect logs to a file
# log-facility local7;

subnet 192.168.7.128 netmask 255.255.255.128 {
  option domain-name-servers 10.0.0.1;
  option broadcast-address 192.168.7.255;
  option routers 192.168.7.129;
  default-lease-time 14400;
  max-lease-time 28800;

  if substring(vendorclass, 0, 9)="PXEClient" {
    if pxetype = 00:06 or pxetype = 00:07 {
      filename "efi/shimx64.efi";
    } else {
      filename "pxelinux/pxelinux.0";
    }
  }

  # Add the class for HTTP clients
  class "httpclients" {
    match if substring(option vendor-class-identifier, 0, 10) = "HTTPClient";
    option vendor-class-identifier "HTTPClient";
    filename "http://192.168.7.129/pub/fedora39srv/dvd/EFI/BOOT/BOOTX64.EFI";
  }

  # Define the pool inside the subnet
  pool {
    range 192.168.7.135 192.168.7.250;
  }

  # Define next-server outside of the pool
  next-server 192.168.7.129;

}

This configuration assumes that I have an Ethernet interface setup with a static IP address of
192.168.7.129 . It must be a static IP. It can pretty much be any IP you like, but you will want this
to be off of your main LAN network. Otherwise, you might be kickstarting all your other computers
every time you reboot them. In this example I'm using a subnet of 255.255.255.128, that means
I could kickstart over 120 computers at the same time if I have enough ports on my switch.

Next, setting up file indexing on our web server.

 

[APTI]

dumb question here, why are we setting up a DHCP server when that is generally handled by the router. Wouldn't this create a conflict? The PXE should only boot computers set in BIOS to boot PXE as one of the options so you will not boot all computers on PXE only the ones that have it set in BIOS...
That means no need for secondary network?

Or am I wrong here?

 

[dos2unix]

dumb question here, why are we setting up a DHCP server when that is generally handled by the router. Wouldn't this create a conflict? The PXE should only boot computers set in BIOS to boot PXE as one of the options so you will not boot all computers on PXE only the ones that have it set in BIOS...
That means no need for secondary network?

Normally consumer/home routers don't have PXE boot on them.
This is why you need two interfaces. One for your LAN/admin network, and another for the kickstart network. It has to isolated
( a whole other subnet ) than your management subnet.

Notice near the bottom of the dhcpd.conf file, there is a line that says "next-server".
This is where the tftp-server ( the same IP address as my dhcp-server ) send the boot kernel to the PXE client.

The PXE should only boot computers set in BIOS to boot PXE as one of the options so you will not boot all computers on PXE only the ones that have it set in BIOS...

That can be true, but frequently PXE boot is the first boot option. ( I notice this was true on the last two MSI motherboards I bought ). I suppose they figure you don't already have an OS installed, which makes sense.

 

[APTI]

I have not done network booting since the days of Novel. But that was a boot rom not PXE. I have never done anything with PXE but sounds like something I may use in shop when building. are you saying that the routers will not pass PXE along?
I deal with home routers along with cisco and juniper

 

[dos2unix]

Theoretically you could pass the PXE boot protocol through a managed router running dhcp.
But I know of no way to pass on the "next service". In this case tftp.

 

[dos2unix]

Now we need to set up file indexing on the web server. As mentioned earlier, I use nginx, but apache will work also.
My /etc/nginx/nginx.conf file looks like this.

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80;
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
            autoindex on;
        }

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }

# Settings for a TLS enabled server.
#
#    server {
#        listen       443 ssl http2;
#        listen       [::]:443 ssl http2;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate "/etc/pki/nginx/server.crt";
#        ssl_certificate_key "/etc/pki/nginx/private/server.key";
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers PROFILE=SYSTEM;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        error_page 404 /404.html;
#        location = /404.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#        location = /50x.html {
#        }
#    }

}

This is pretty the stadard file with the exception of the ..

        location / {
            autoindex on;
        }

section.
If you're running SElinux, it doesn't like you to be able to see the files on your webserver with indexing.
So you can either disable it, or add permissions to the /usr/share/nginx/html/pub directory.

 chcon -R -t httpd_sys_content_t /pub
 ls -Z /pub
 setsebool -P httpd_can_network_connect on

... next setting up the PXE boot kernels.

 

[dos2unix]

This will create our bootloader directories.
While we're at it, let's extract our OS from the iso images.

## This example uses a Fedora 39 iso image, but the steps are the same for any Distro.
## It is assumed you have already downloaded an iso image, and placed it in the /pub directory.
mount /pub/Fedora-Server-dvd-x86_64-39-1.2.iso -o loop /mnt
mkdir -p /pub/fed39srv/dvd
mkdir -p /pub/fed39srv/OSupdates
cp -rvf /mnt/* /pub/fed39srv/dvd
umount /mnt

# alma 85
mount /pub/AlmaLinux-8.9-x86_64-dvd.iso -o loop /mnt
mkdir -p /pub/alma/89/dvd
mkdir -p /pub/alma/89/OSupdates
cp -rvf /mnt/* /pub/alma/89/dvd
umount /mnt

## This part of the script will have to be modified for your particular secondary interface name.
## If you change this IP, you will have modify several scripts.
nmcli con mod enp0s3 ip4 192.168.7.227/24 ipv4.method manual autoconnect yes

## These instructions assume the target device is using a UEFI.
## There are different instructions for Legacy BIOS.
mkdir -p /var/lib/tftpboot/efi/fed39
cp /pub/fed39srv/dvd/isolinux/vmlinuz /var/lib/tftpboot/efi/fed39
cp /pub/fed39srv/dvd/isolinux/initrd.img /var/lib/tftpboot/efi/fed39

mkdir -p /var/lib/tftpboot/efi/alma/89
cp /pub/alma/89/dvd/isolinux/vmlinuz /var/lib/tftpboot/efi/alma/89
cp /pub/alma/89/dvd/isolinux/initrd.img /var/lib/tftpboot/efi/alma/89

chown -R 744 /var/lib/tftpboot/efi/*

cd /tmp
dnf reinstall -y grub2-efi-x64 --downloadonly --downloaddir=/tmp
dnf reinstall -y shim-x64 --downloadonly --downloaddir=/tmp

rpm2cpio shim-x64*rpm | cpio -dimv
rpm2cpio grub2-efi-x64*rpm | cpio -dimv
cp -rvf /tmp/boot/efi/EFI/BOOT/* /var/lib/tftpboot/efi
cp -rvf /tmp/boot/efi/EFI/fedora/* /var/lib/tftpboot/efi
cp -rvf /tmp/boot/grub2/fonts /var/lib/tftpboot/efi

 

[dos2unix]

Now we need to build the kickstart menu's.
Create this file at ...
/var/lib/tftpboot/efi
It must be named "grub.cfg". You cannot change the name of this file.

set default="1"

function load_video {
  insmod all_video
}

load_video
set gfxpayload=keep
insmod gzio
insmod part_gpt
insmod ext2
insmod chain

set timeout=5

menuentry 'Exit this grub' {
        exit
}

menuentry 'Install Fedora 39 VM'  --class fedora --class gnu-linux --class gnu --class os {
    linuxefi efi/fed39/vmlinuz  inst.repo=http://192.168.7.129/pub/fedora39srv/dvd/ ip=dhcp inst.ks=http://192.168.7.129/pub/fedora39srv/fed39-mate-vm.cfg
    initrdefi efi/fed39/initrd.img
}

menuentry 'Install Alma 8.9 VM'  --class redhat --class gnu-linux --class gnu --class os {
    linuxefi efi/alma89/vmlinuz  inst.repo=http://192.168.7.129/pub/alma89srv/dvd ip=dhcp inst.ks=http://192.168.7.129/pub/alma89srv/alma89-smg3-vm.cfg
        initrdefi efi/alma89/initrd.img   
}

For this example, I only have 2 OS choices in my menu, but I have seen 12 or more OS's on some kickstart server.

Notice that there is a path that points the "dvd" directory where we extracted the ISO images.
If you extracted them to some other place, then you'll have to change the path here accordingly.

Now we can make a soft link to "/pub" on our webserver.

ln -s /pub /usr/share/nginx/html/pub

Now if your web server is started. You should be able to see these files in browser pointed at your web server.
If you used the configuration examples I provided, it would be at ...

http://192.168.7.129/pub

Obviously if you setup a different static IP, it will be there.

We're almost done. The only thing really left is the OS build anaconda files.
That's where the real meat and potatoes of kickstart are.

However if you've gotten this far. You should at least be able to get to the kickstart menu
on your kickstart client now.

 

[dos2unix]

This script will put most of what we've already done together for you.

#!/usr/bin/bash

# Install required packages
dnf install -y nginx nginx-all-modules nginx-filesystem
dnf install -y dhcp-server
dnf install -y tftp-server
dnf install -y ansible
dnf install -y git

# Configure firewall rules
firewall-cmd --add-service=http --permanent
firewall-cmd --add-service=dhcp --permanent
firewall-cmd --add-service=tftp --permanent

# Enable services
systemctl enable nginx
systemctl enable dhcpd
systemctl enable tftp.socket

# Disable SELinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0

# Create symbolic link for non-standard locations
ln -s /pub /usr/share/nginx/html/pub

# Mount Fedora 39 ISO image
mount /pub/Fedora-Server-dvd-x86_64-39-1.2.iso -o loop /mnt
mkdir -p /pub/fed39srv/dvd
mkdir -p /pub/fed39srv/OSupdates
cp -rvf /mnt/* /pub/fed39srv/dvd
umount /mnt

# Mount AlmaLinux 8.9 ISO image
mount /pub/AlmaLinux-8.9-x86_64-dvd.iso -o loop /mnt
mkdir -p /pub/alma/89/dvd
mkdir -p /pub/alma/89/OSupdates
cp -rvf /mnt/* /pub/alma/89/dvd
umount /mnt

# Configure secondary network interface
nmcli con mod enp0s3 ip4 192.168.7.227/24 ipv4.method manual autoconnect yes

# Copy UEFI boot files for Fedora 39
mkdir -p /var/lib/tftpboot/efi/fed39
cp /pub/fed39srv/dvd/isolinux/vmlinuz /var/lib/tftpboot/efi/fed39
cp /pub/fed39srv/dvd/isolinux/initrd.img /var/lib/tftpboot/efi/fed39

# Copy UEFI boot files for AlmaLinux 8.9
mkdir -p /var/lib/tftpboot/efi/alma/89
cp /pub/alma/89/dvd/images/pxeboot/vmlinuz /var/lib/tftpboot/efi/alma/89
cp /pub/alma/89/dvd/images/pxeboot/initrd.img /var/lib/tftpboot/efi/alma/89

echo "Kickstart server setup completed successfully."

You will likely need to change the enp0s3 interface to whatever your computer has.

 

[dos2unix]

Here is a standard fedora anaconda file. Almost all redhat ( rpm ) based Linux systems will have a similar
file in /root called "anaconda-ks.cfg".

It gets put there when you build the system. It doesn't matter if it was kickstarted or installed from a USB.

# Generated by Anaconda 39.32.6
# Generated by pykickstart v3.48
#version=DEVEL
# Use graphical install
graphical

# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_US.UTF-8

# Network information
network  --bootproto=dhcp --device=eno1 --ipv6=auto --activate
network  --hostname=devNUC1

%packages
@^server-product-environment
@development-tools
@editors
@headless-management
@rpm-development-tools
@text-internet

%end

# Run the Setup Agent on first boot
firstboot --enable

# Generated using Blivet version 3.8.1
ignoredisk --only-use=nvme0n1
# Partition clearing information
clearpart --all --initlabel
# Disk partitioning information
part /boot --fstype="xfs" --ondisk=nvme0n1 --size=3072
part pv.370 --fstype="lvmpv" --ondisk=nvme0n1 --size=238596
part /boot/efi --fstype="efi" --ondisk=nvme0n1 --size=370 --fsoptions="umask=0077,shortname=winnt"
volgroup fedora_devnuc1 --pesize=4096 pv.370
logvol /data --fstype="xfs" --size=87040 --name=data --vgname=fedora_devnuc1
logvol / --fstype="xfs" --size=40960 --name=root --vgname=fedora_devnuc1
logvol /var --fstype="xfs" --size=30720 --name=var --vgname=fedora_devnuc1
logvol /opt/esco --fstype="xfs" --size=20480 --name=opt_esco --vgname=fedora_devnuc1
logvol swap --fstype="swap" --size=8192 --name=swap --vgname=fedora_devnuc1
logvol /home --fstype="xfs" --size=51200 --name=home --vgname=fedora_devnuc1

# System timezone
timezone America/Los_Angeles --utc

# Root password
rootpw --iscrypted --allow-ssh $abcdefghijklmnopqrstuvwxyz1234
user --groups=wheel --name=coolUser -password=$abcdefghijklmnopqrstuvwxyz1234 --iscrypted --gecos="Elvis Presley"

This is basically the one from my local Linux systems. The username and password string have been changed.
But other than the interface device, disk device and the hostname, it's probably very similar to what you have on your redhat/fedora/alma/rocky/aws/centOS/suse/oracle system.

This is a good place to start. It won't work with a PXE boot kickstart server quite yet, we'll need to change a few lines,
but it's pretty close.

 

[dos2unix]

Between the

lang en_US.UTF-8

# Network information

section of the file above, add this.

# repo --name="Extras" --baseurl='http://192.168.7.129/pub/fedora39srv/extras'
url --url='http://192.168.7.129/pub/fedora39srv/dvd/'

# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_US.UTF-8

eula --agreed
firstboot --disable
# selinux --disabled

The # at the beginning of the line, just makes that line like a comment, that line doesn't get executed.

and also change this section

# Use graphical install
graphical

to look like this.

# Use text based install
text
skipx

and finally change this line...

network  --bootproto=dhcp --device=eno1 --ipv6=auto --activate

to device= (whatever your interface name is)

Save this file to "/pub/fedora39srv/mycustom.cfg"

You can name it whatever you want. But it has to be in this directory.

Back in post #10, in this thread there was a line like this in my grub menu file.

linuxefi efi/fed39/vmlinuz  inst.repo=http://192.168.7.129/pub/fedora39srv/dvd/ ip=dhcp inst.ks=http://192.168.7.129/pub/fedora39srv/fed39-mate-vm.cfg

Change the "fed39-mate-vm.cfg" to whatever you named your cfg file.

That's it, you should have a functioning kickstart server now.

I will add instructions for Legacy BIOS bootloaders soon.
I will also custom repo's so you can add packages that don't come on the install USB image.
Finally, we will add multiple users, ( which you can't do with a USB install ).

 

[dos2unix]

Back in post #12, at the bottom of the anaconda.cfg file you added a user named Elvis.

But what if I want to add more users while I'm kickstarting?
It's pretty easy, but the user has to already exist on another Linux system.

cat /etc/shadow

You will add the users on the system in this file. A user line looks like this. The part you care about is the password hash.
It's the long string of characters after the first ":" It includes the $ character. Copy this all the way to the next ":"

mikeT:$j9T$nQAKKxn2ixOVCYMr655RJ/$xEYdnygFt4cCJpJN9wDmE0srTnnj804oELXmT21g0L.:19755::::::
johnK:$82fh23gf88gh5g35g5ghh5g4h0-6jhk556mk68k-kcmwvg9h4h94hhj68hj6jh8j8.:1892:::::
bobM:$y$j9T$iHKfoeWVyFDEolOffR63urAD$zBY7jAdfg1PklOWk/kXdRCjjjDjaqfLNqwmU.ArjN6::0:99999:7:::

Let's say we wanted to add these three users, directory beneath the Elvis user at the bottom of our anaconda file we could simply paste.

user --name=joeM --groups=wheel --iscrypted --password=$6$NQxcaeY.Pvm1FWBl$LriLt5PFtqUUs0sJgUhpAwOc4n9dwJ0sx1qPDVXHZzXq0GnA8ZpuLkJG9QoGb5JwUv2/3JZLJBjDTUJXIP3bS. --gecos="Joe Montana"

Just add as many users as you like. Be sure to paste the correct password hash string for each user.

Some other useful user flags here are...

--name= - Provides the name of the user. This option is required.


--gecos= - Provides the GECOS information for the user. This is a string of various system-specific fields separated by a comma. It is frequently used to specify the user's full name, office number, and so on. See the passwd(5) man page for more details.

--groups= - In addition to the default group, a comma separated list of group names the user should belong to. The groups must exist before the user account is created. See the group command.

--homedir= - The home directory for the user. If not provided, this defaults to /home/username.

--lock - If this option is present, this account is locked by default. This means that the user will not be able to log in from the console. This option will also disable the Create User screens in both the graphical and text-based manual installation.

--password= - The new user's password. If not provided, the account will be locked by default.

--iscrypted - If this option is present, the password argument is assumed to already be encrypted. This option is mutually exclusive with --plaintext.

--shell= - The user's login shell. If not provided, the system default is used.

--uid= - The user's UID (User ID). If not provided, this defaults to the next available non-system UID.

--gid= - The GID (Group ID) to be used for the user's group. If not provided, this defaults to the next available non-system group ID.