Install CentOS Stream 9 booting VNC installer with kexec

Lately, dedicated servers come with Remote management consoles like IPMI KVM or iLO, or DRAC, but they are still slow to initiate the process of installing a system.

main menu
kexec execute

Consider a server (dedicated or not) should be installed in a remote colocation with the help of only the server’s network. The system administrator just receives an administrative shell access and nothing more and the server should be installed with the proper and secured software, in this case, the CentOS Stream 9. Using kexec the user can boot a new kernel from a different Linux Distribution and initiate automated network installation of the system and it is not needed any Remote management consoles. The only thing needed is the ability of the current system/kernel to be able to use kexec, which is pretty standard for 8 to 10 years old Linux systems. There is a good chance the colocations’ rescue CD/DVD/USB flash drives or the PXE rescue images support kexec, because they tend to upgrade their rescue systems, which the user may boot if he has problems.
Still, using kexec to initiate another kernel or Linux Distribution like CentOS Stream 9 with VNC installer, for example, it a powerful tool to safely replace a currently running system with only shell access.
This article has chosen to start the CentOS Stream 9 VNC installer just for demonstration purposes. Booting a downloaded kernel may be used for just anything from booting a system over the network, booting an installer, booting an unattended automation installation, and so on. There are a couple of simple things to check before booting the new kernel.
This article will show just one use case – reinstalling a system with CentOS Stream 9 over the network using the CentOS VNC Install. The purpose is to show how simple, fast, and easy is to install a modern Linux system only by having console access. No scripts are required if manual installation is performed.
To boot a CentOS Stream 9 VNC Installer the kexec command needs the following options.

The kexec commands need the following options:

  • Networkingdevice interface name, IP, netmask, gateway and DNS servers
  • Kernel options – these options will initiate scripts from the initramfs.
  • inst.vnc – a kernel option, which will start a VNC server with no password on the default port and network device. Using it with another inst.vncpassword=[PASSWORD] the VNC server will require the password – [PASSWORD]. The password should be a maximum of 8 characters because the VNC server will not start if it is with more!
  • inst.repo=[HTTP/HTTPS://repository] – a kernel option, which sets the CentOS HTTP/HTTPS repository.

The kexec command to boot the CentOS Stream 9 VNC Installer is:

kexec --initrd=./initrd.img -l ./vmlinuz --command-line="bootdev=eno1 ip=10.10.10.20::10.10.10.1:24:srv.example.com:eno1:none nameserver=8.8.8.8 inst.vnc inst.vncpassword=cha3hae4ahZaqueev1ee inst.repo=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/"

The kernel (i.e. vmlinuz) and the initramfs (i.e. initrd.img) should be downloaded in the current directory before executing the above command. The above line will order the kernel to load the new kernel, but to boot it another command must be executed:

kexec -e

Keep on reading!

Ubuntu 16/18 LTS – load a new kernel without rebooting the server

Here are the commands needed to load a new kernel without rebooting your server or desktop computer. Why you need this? As said in our first article for CentOS 7 – sometime rebooting a server could take 5 to 10 minutes and loading a new kernel is just up to a minute. In fact in most cases loading the new kernel and starting the system then is just under 20-30 seconds, so upgrading your server even with new kernel is super easy lately. We tested it on Ubuntu 16 and Ubuntu 18 servers and it was successful. The system uses systemd and the process is really easy and safe for the systems.

When the processes is initiated the system shutdowns normally (shutting down all running service with systemd) and then load the system immediately with the new kernel and starts the services as usual!

So no need to worry about unflushed data or not proper shutdown of a service! It’s like a normal reboot but without a hardware reboot and is a lot faster!
Here is what is required to load a kernel without hardware rebooting your computer box:

  1. kexec-tools
  2. Load the new kernel, initram file and the command line arguments with “kexec”
  3. Start a systemd target – kexec.target

Ubuntu 16/18 LTS using kexec to load a new kernel

The real commands only for Ubuntu 16/18 LTS:

sudo apt -y install kexec-tools
sudo kexec -l /boot/vmlinuz-4.15.0-33-generic --initrd=/boot/initrd.img-4.15.0-33-generic --command-line="root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro  quiet splash"
sudo systemctl start kexec.target

Here is a real world example with all the output:
And again update your system to see if there is a new kernel and install “kexec-tools”. In our case indeed there is a new kernel – vmlinuz-4.15.0-33-generic

myuser@srv:~$ sudo apt -y upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  libllvm5.0
Use 'sudo apt autoremove' to remove it.
The following NEW packages will be installed:
  amd64-microcode autotools-dev dblatex debhelper dh-strip-nondeterminism docbook-dsssl docbook-utils docbook-xml docbook-xsl fonts-lato fonts-lmodern fonts-texgyre
  intel-microcode iucode-tool jadetex javascript-common kernel-common kernel-package kernel-patch-scripts kernel-wedge kerneloops kerneloops-applet kernelshark kerneltop
  libfile-homedir-perl libfile-stripnondeterminism-perl libfile-which-perl libjs-jquery libllvm6.0 libmail-sendmail-perl libosp5 libostyle1c2 libpotrace0 libptexenc1
  libqpdf21 libruby2.3 libsgmls-perl libsp1c2 libsynctex1 libsys-hostname-long-perl libtexlua52 libtexluajit2 libwebpdemux1 libxml2-utils libzzip-0-13
  linux-headers-4.15.0-33 linux-headers-4.15.0-33-generic linux-image-4.15.0-33-generic linux-modules-4.15.0-33-generic linux-modules-extra-4.15.0-33-generic lmodern lynx
  lynx-common openjade po-debconf preview-latex-style prosper ps2eps python-apt rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit
  ruby2.3 rubygems-integration sgml-data sgmlspl sp tex-common tex-gyre texlive texlive-base texlive-bibtex-extra texlive-binaries texlive-extra-utils texlive-font-utils
  texlive-fonts-recommended texlive-fonts-recommended-doc texlive-generic-recommended texlive-latex-base texlive-latex-base-doc texlive-latex-extra
  texlive-latex-extra-doc texlive-latex-recommended texlive-latex-recommended-doc texlive-luatex texlive-math-extra texlive-pictures texlive-pictures-doc texlive-pstricks
  texlive-pstricks-doc tipa trace-cmd xmlto xsltproc
The following packages will be upgraded:
.....
.....
Running mktexlsr /var/lib/texmf ... done.
Building format(s) --all.
        This may take some time... done.
Processing triggers for linux-image-4.15.0-33-generic (4.15.0-33.36~16.04.1) ...
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-4.15.0-33-generic
/etc/kernel/postinst.d/vboxadd:
VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel modules.
/etc/kernel/postinst.d/zz-update-grub:
Generating grub configuration file ...
Warning: Setting GRUB_TIMEOUT to a non-zero value when GRUB_HIDDEN_TIMEOUT is set is no longer supported.
Found linux image: /boot/vmlinuz-4.15.0-33-generic
Found initrd image: /boot/initrd.img-4.15.0-33-generic
Found linux image: /boot/vmlinuz-4.13.0-36-generic
Found initrd image: /boot/initrd.img-4.13.0-36-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
done

myuser@srv:~$ sudo apt -y install kexec-tools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libllvm5.0
Use 'sudo apt autoremove' to remove it.
The following NEW packages will be installed:
  kexec-tools
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 77,4 kB of archives.
After this operation, 276 kB of additional disk space will be used.
Get:1 http://bg.archive.ubuntu.com/ubuntu xenial-updates/main amd64 kexec-tools amd64 1:2.0.16-1ubuntu1~16.04.1 [77,4 kB]
Fetched 77,4 kB in 0s (707 kB/s)      
Preconfiguring packages ...
Selecting previously unselected package kexec-tools.
(Reading database ... 253895 files and directories currently installed.)
Preparing to unpack .../kexec-tools_1%3a2.0.16-1ubuntu1~16.04.1_amd64.deb ...
Unpacking kexec-tools (1:2.0.16-1ubuntu1~16.04.1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu21.4) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up kexec-tools (1:2.0.16-1ubuntu1~16.04.1) ...
Generating /etc/default/kexec...
Processing triggers for systemd (229-4ubuntu21.4) ...
Processing triggers for ureadahead (0.100.0-19) ...

     ┌──────────────────────────────────────────────────────────────────┤ Configuring kexec-tools ├───────────────────────────────────────────────────────────────────┐
     │                                                                                                                                                                │ 
     │ If you choose this option, a system reboot will trigger a restart into a kernel loaded by kexec instead of going through the full system boot loader process.  │ 
     │                                                                                                                                                                │ 
     │ Should kexec-tools handle reboots (sysvinit only)?                                                                                                             │ 
     │                                                                                                                                                                │ 
     │                                                 <Yes>                                                    <No>                                                  │ 
     │                                                                                                                                                                │ 
     └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ 

On the above configuration question mark “” and press Enter.

So we performed an update and there was a new kernel vmlinuz-4.15.0-33-generic, which we would like to load without hardware reboot.
Here is our new kernel in “/boot”

myuser@srv:~$ ls -altr /boot/
total 130420
-rw-r--r--  1 root root   184840 Jan 28  2016 memtest86+_multiboot.bin
-rw-r--r--  1 root root   184380 Jan 28  2016 memtest86+.elf
-rw-r--r--  1 root root   182704 Jan 28  2016 memtest86+.bin
-rw-------  1 root root  3879946 Feb 17  2018 System.map-4.13.0-36-generic
-rw-r--r--  1 root root     2850 Feb 17  2018 retpoline-4.13.0-36-generic
-rw-r--r--  1 root root   213220 Feb 17  2018 config-4.13.0-36-generic
-rw-r--r--  1 root root  1501359 Feb 17  2018 abi-4.13.0-36-generic
-rw-r--r--  1 root root  7710912 May 17 16:50 vmlinuz-4.13.0-36-generic
-rw-------  1 root root  4041375 Aug 16 00:00 System.map-4.15.0-33-generic
-rw-r--r--  1 root root        0 Aug 16 00:00 retpoline-4.15.0-33-generic
-rw-r--r--  1 root root   216913 Aug 16 00:00 config-4.15.0-33-generic
-rw-r--r--  1 root root  1537455 Aug 16 00:00 abi-4.15.0-33-generic
-rw-------  1 root root  8108600 Aug 16 21:58 vmlinuz-4.15.0-33-generic
drwxr-xr-x 24 root root     4096 Sep  7 14:15 ..
-rw-r--r--  1 root root 51290506 Sep  7 14:15 initrd.img-4.13.0-36-generic
-rw-r--r--  1 root root 54451680 Sep  7 14:15 initrd.img-4.15.0-33-generic
drwxr-xr-x  3 root root     4096 Sep  7 14:15 .
drwxr-xr-x  5 root root     4096 Sep  7 14:16 grub

Now we know the kernel and initram file names we just check the kernel arguments in the kernel, load them with kexec and start an systemd target to load the new kernel:

myuser@srv:~$ grep vmlinuz-4.15.0-33-generic /boot/grub/grub.cfg 
        linux   /boot/vmlinuz-4.15.0-33-generic root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro  quiet splash $vt_handoff
                linux   /boot/vmlinuz-4.15.0-33-generic root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro  quiet splash $vt_handoff
                linux   /boot/vmlinuz-4.15.0-33-generic root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro  quiet splash $vt_handoff init=/sbin/upstart
                linux   /boot/vmlinuz-4.15.0-33-generic root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro recovery nomodeset 
myuser@srv:~$ sudo kexec -l /boot/vmlinuz-4.15.0-33-generic --initrd=/boot/initrd.img-4.15.0-33-generic --command-line="root=UUID=061b2936-34bf-4da3-b7d2-b8bde0899f03 ro  quiet splash"
myuser@srv:~$ sudo systemctl start kexec.target
Connection to srv closed by remote host.
Connection to srv closed.

Use the first line of the grep output above (or you can cat the file and see what is in it if you have any doubts) to take the proper kernel boot arguments and do not include anything starting with “$”.

As you can see systemd performs a normal shutdown of all services and targets.

main menu
Normal shutdown

The ssh connection is immediately closed because the reboot is initiated.
After 10-15 seconds our host is alive and the new kernel is loaded successfully:

root@test ~ $ ssh root@srv
root@srv's password: 
Last login: Wed Sep  5 17:15:08 2018 from test
[root@srv ~]# uname -a
Linux srv.local 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:21:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@srv ~]# 

Because we do not wanted to mess up the two output in one article we decided to split it in two separate ones, so here is the previous one for CentOS 7 – “CentOS 7 – load a new kernel without rebooting the CentOS 7 server”

CentOS 7 – load a new kernel without rebooting the server

Here are the commands needed to load a new kernel without rebooting your server or desktop computer. Why you need this? Sometime rebooting a server could take 5 to 10 minutes and loading a new kernel is just up to a minute. In fact in most cases loading the new kernel and starting the system then is just under 20-30 seconds, so upgrading your server even with new kernel is super easy lately. We tested it on CentOS 7 server and it was successful. The system uses systemd and the process is really easy and safe for the systems.

When the processes is initiated the system shutdowns normally (shutting down all running service with systemd) and then load the system immediately with the new kernel and starts the services as usual!

So no need to worry about unflushed data or not proper shutdown of a service! It’s like a normal reboot but without a hardware reboot and is a lot faster!
Here is what is required to load a kernel without hardware rebooting your computer box:

  1. kexec-tools
  2. Load the new kernel, initram file and the command line arguments with “kexec”
  3. Start a systemd target – kexec.target

CentOS 7 using kexec to load a new kernel

The real commands only for CentOS 7:

yum install -y kexec-tools
kexec -l /boot/vmlinuz-3.10.0-862.11.6.el7.x86_64 --initrd=/boot/initramfs-3.10.0-862.11.6.el7.x86_64.img --command-line="root=/dev/mapper/centos_srv-root ro crashkernel=auto rd.lvm.lv=centos_srv/root rd.lvm.lv=centos_srv/swap rhgb quiet LANG=en_US.UTF-8"
systemctl start kexec.target

Here is a real world example with all the output:
As you can see it is important to load the initram file and the exact arguments to the kernel. You should take them from the grub 2 configuration – /boot/grub2/grub.cfg
Here is the real output:

[root@srv ~]# yum -y update
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirrors.neterra.net
 * extras: mirrors.neterra.net
 * updates: mirrors.neterra.net
base                                                                                                                                                 | 3.6 kB  00:00:00     
centos-sclo-rh                                                                                                                                       | 3.0 kB  00:00:00     
centos-sclo-sclo                                                                                                                                     | 2.9 kB  00:00:00     
extras                                                                                                                                               | 3.4 kB  00:00:00     
updates                                                                                                                                              | 3.4 kB  00:00:00     
(1/4): extras/7/x86_64/primary_db                                                                                                                    | 187 kB  00:00:00     
(2/4): updates/7/x86_64/primary_db                                                                                                                   | 5.2 MB  00:00:01     
(3/4): centos-sclo-sclo/x86_64/primary_db                                                                                                            | 292 kB  00:00:01     
(4/4): centos-sclo-rh/x86_64/primary_db                                                                                                              | 3.7 MB  00:00:02     
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:3.10.0-862.11.6.el7 will be installed
---> Package kernel-headers.x86_64 0:3.10.0-862.3.2.el7 will be updated
---> Package kernel-headers.x86_64 0:3.10.0-862.11.6.el7 will be an update
---> Package kernel-tools.x86_64 0:3.10.0-862.3.2.el7 will be updated
---> Package kernel-tools.x86_64 0:3.10.0-862.11.6.el7 will be an update
---> Package kernel-tools-libs.x86_64 0:3.10.0-862.3.2.el7 will be updated
---> Package kernel-tools-libs.x86_64 0:3.10.0-862.11.6.el7 will be an update
....
....
[root@srv ~]# yum install -y kexec-tools
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.neterra.net
 * extras: mirrors.neterra.net
 * updates: mirrors.neterra.net
Package kexec-tools-2.0.15-13.el7.x86_64 already installed and latest version
Nothing to do

So we performed an update and there was a new kernel kernel.x86_64 0:3.10.0-862.11.6.el7, which we would like to load without hardware reboot.
Here is our new kernel in “/boot”

[root@srv ~]# ls -altr /boot/
total 171812
-rw-------.  1 root root  3228420 22 Aug  2017 System.map-3.10.0-693.el7.x86_64
-rw-r--r--.  1 root root   140894 22 Aug  2017 config-3.10.0-693.el7.x86_64
-rw-r--r--.  1 root root      166 22 Aug  2017 .vmlinuz-3.10.0-693.el7.x86_64.hmac
-rwxr-xr-x.  1 root root  5877760 22 Aug  2017 vmlinuz-3.10.0-693.el7.x86_64
-rw-r--r--.  1 root root   293027 22 Aug  2017 symvers-3.10.0-693.el7.x86_64.gz
drwxr-xr-x.  3 root root       17 20 Feb  2018 efi
drwxr-xr-x.  2 root root       27 20 Feb  2018 grub
-rw-r--r--.  1 root root   611343 20 Feb  2018 initrd-plymouth.img
-rw-------.  1 root root 51315067 20 Feb  2018 initramfs-0-rescue-cc7889764e86441b8d1eb54e29e81a91.img
-rwxr-xr-x.  1 root root  5877760 20 Feb  2018 vmlinuz-0-rescue-cc7889764e86441b8d1eb54e29e81a91
-rw-------.  1 root root  3409912 21 May 23,50 System.map-3.10.0-862.3.2.el7.x86_64
-rw-r--r--.  1 root root   147823 21 May 23,50 config-3.10.0-862.3.2.el7.x86_64
-rw-r--r--.  1 root root      170 21 May 23,50 .vmlinuz-3.10.0-862.3.2.el7.x86_64.hmac
-rwxr-xr-x.  1 root root  6228832 21 May 23,50 vmlinuz-3.10.0-862.3.2.el7.x86_64
-rw-r--r--.  1 root root   304943 21 May 23,52 symvers-3.10.0-862.3.2.el7.x86_64.gz
dr-xr-xr-x. 17 root root      224 14 Jun  7,18 ..
-rw-------.  1 root root 12997841 14 Jun  7,19 initramfs-3.10.0-693.el7.x86_64kdump.img
-rw-------.  1 root root 20771492 14 Jun  7,20 initramfs-3.10.0-693.el7.x86_64.img
-rw-------.  1 root root 13007444 14 Jun  7,23 initramfs-3.10.0-862.3.2.el7.x86_64kdump.img
-rw-r--r--.  1 root root   147859 14 Aug 22,02 config-3.10.0-862.11.6.el7.x86_64
-rw-------.  1 root root  3414344 14 Aug 22,02 System.map-3.10.0-862.11.6.el7.x86_64
-rw-r--r--.  1 root root      171 14 Aug 22,02 .vmlinuz-3.10.0-862.11.6.el7.x86_64.hmac
-rwxr-xr-x.  1 root root  6242208 14 Aug 22,02 vmlinuz-3.10.0-862.11.6.el7.x86_64
-rw-r--r--.  1 root root   305158 14 Aug 22,05 symvers-3.10.0-862.11.6.el7.x86_64.gz
-rw-------.  1 root root 20774393  5 Sep 15,20 initramfs-3.10.0-862.11.6.el7.x86_64.img
drwx------.  5 root root       97  5 Sep 15,20 grub2
dr-xr-xr-x.  5 root root     4096  5 Sep 15,21 .
-rw-------.  1 root root 20782404  5 Sep 15,21 initramfs-3.10.0-862.3.2.el7.x86_64.img

Now we know the kernel and initram file names we just check the kernel arguments in the kernel, load them with kexec and start an systemd target to load the new kernel:

[root@srv ~]# grep vmlinuz-3.10.0-862.11.6.el7.x86_64 /boot/grub2/grub.cfg 
        linux16 /vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos_srv-root ro crashkernel=auto rd.lvm.lv=centos_srv/root rd.lvm.lv=centos_srv/swap rhgb quiet LANG=en_US.UTF-8
[root@srv ~]# kexec -l /boot/vmlinuz-3.10.0-862.11.6.el7.x86_64 --initrd=/boot/initramfs-3.10.0-862.11.6.el7.x86_64.img --command-line="root=/dev/mapper/centos_srv-root ro crashkernel=auto rd.lvm.lv=centos_srv/root rd.lvm.lv=centos_srv/swap rhgb quiet LANG=en_US.UTF-8"
[root@srv ~]# systemctl start kexec.target
Connection to srv closed by remote host.
Connection to srv closed.

As you can see systemd performs a normal shutdown of all services and targets.

main menu
Normal Shutdown

The ssh connection is immediately closed because the reboot is initiated.
After 10-15 seconds our host is alive and the new kernel is loaded successfully:

root@test ~ $ ssh root@srv
root@srv's password: 
Last login: Wed Sep  5 15:16:08 2018 from test
[root@srv ~]# uname -a
Linux srv.local 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@srv ~]# 

Because we do not wanted to mess up the two output from different linux distros in one article we decided to split it in two separate ones, so here is the one for Ubuntu 16/18 LTS – “Ubuntu 16/18 LTS – load a new kernel without rebooting the server