lxc and interface lo does not exist in virtualized server

Virtualizing a real server with an LXC container is pretty easy – do a rsync and run it. Sometimes there are some glitches when starting the LXC container for the first time. Such errors like the following – no networking available at the start, but when attached to the started container it seems to have the network interfaces with no IPs. Even, though it is possible to set the IPs manually the init scripts do not work.

[root@srv ~]# lxc-start -F -n n7763.node-int.info
lxc-start: live300.mytv.bg: start.c: proc_pidfd_open: 1607 Function not implemented - Failed to send signal through pidfd
INIT: version 2.88 booting

   OpenRC 0.12.4 is starting up Gentoo Linux (x86_64) [LXC]

 * /proc is already mounted
 * Mounting /run ... * /run/openrc: creating directory
 * /run/lock: creating directory
 * /run/lock: correcting owner
 * Caching service dependencies ... [ ok ]
 * setting up tmpfiles.d entries for /dev ... [ ok ]
 * Creating user login records ... [ ok ]
 * Wiping /tmp directory ... [ ok ]
 * Bringing up network interface lo ...RTNETLINK answers: File exists
 [ ok ]
 * Updating /etc/mtab ... [ ok ]
 * Bringing up interface lo
 *   ERROR: interface lo does not exist
 *   Ensure that you have loaded the correct kernel module for your hardware
 * ERROR: net.lo failed to start
 * setting up tmpfiles.d entries ... [ ok ]
INIT: Entering runlevel: 3
 * Loading iptables state and starting firewall ... [ ok ]
 * Bringing up interface lo
 *   ERROR: interface lo does not exist
 *   Ensure that you have loaded the correct kernel module for your hardware
 * ERROR: net.lo failed to start
 * Bringing up interface eth0
 *   ERROR: interface eth0 does not exist
 *   Ensure that you have loaded the correct kernel module for your hardware
 * ERROR: net.eth0 failed to start

And it appeared that the old /dev was still in place, which messed up with virtualization and the init scripts.
The solution is simple just

  1. remove the existing /dev
  2. create a new empty one

And the LXC container of the real server will start with a network as usual.

So when virtualizing a real server into LXC container after doing RSYNC of the storage, it is mandatory to create an empty /dev, /proc, and /sys directories!

More on the LXC containers – Run LXC CentOS 8 container with bridged network under CentOS 8.

Software comparison Ubuntu server 22.04 LTS vs Fedora 36 server edition – head-to-head

The following article compares two different Linux distros with Ubuntu Server 22.04 LTS (Jammy) and Fedora 36 Server Edition. These are the latest versions released in the past month. Bear in mind, that they have a really different release cycle Ubuntu Server 22.04 LTS (Jammy) support will end after 10 years and Fedora 36 support is just 13 months! Still, the two systems offer server editions with mostly the latest stable Linux software and they offer easy distribution upgrades.

In the Desktop world upgrading to the new and latest version of a Linux distribution is almost mandatory, but in the server world, upgrading is more complicated. The first step in updating or choosing the right Linux distribution and version for a server is to check what software versions come with the (new) distribution and then check whether the running custom (application) software supports the software versions. For example, updating to a new distribution version, which comes with PHP 8.1, but the current application supports only 7.4 is not very wise and in addition, the current version may have years of support in the future.

Lately, with the virtualization software and the distribution upgrades made easy is more common to choose a more short-living or a rolling distribution for a server. Of course, when a Linux version distro has a 13-month life cycle, it is expected to include bleeding-edge software and minor differences between consecutive releases.

SoftwareUbuntu 22.04Fedora 36
Support10 years (April 2032)~ 13 months (July 2023)
Linux kernel5.15.05.17.9
libc2.352.35
OpenSSL.
3.0.2
1.1.1n
3.0.2
GNU GCC9.4.0
10.3.0
11.2.0
12-20220302
.
.
.
12.1.1
PHP8.18.1.6
Python2.7.18
3.10.4
2.7.18
3.10.4
Perl5.34.05.34.1
Ruby3.03.1.2
OpenJDK8u312-b07
11.0.15
17.0.3
18~36ea-1
8.0.332.b09
11.0.15.0.10
17.0.3.0.7
18.0.1.0.10
Go lang1.13.8
1.17.3
1.18.1
.
.
1.18.2
Rust1.58.11.60.0
llvm.
.
.
.
11.1.0
12.0.1
13.0.1
14.0.0
7.0.1
8.0.1
9.0.1
10.0.0
11.1.0
12.0.1
13.0.1
14.0.0
nodejs12.22.916.14.0
Subversion1.14.11.14.1
Git2.34.12.36.1
Apache2.4.522.4.53
Nginx1.18.01.20.2
MySQL server8.0.298.0.28
MariaDB10.6.710.5.15
PostgreSQL14.214.1
SQLite2.8.17
3.37.2
2.8.17
3.36.0
Xorg X server1.22.1.11.20.14
Gnome Shell42.042.1
qemu6.26.2
docker|podman20.10.124.1.0
lxc5.0.04.0.10

Similar series for comparing Ubuntu 20.04 LTS and Ubuntu 22.04 LTS could be found here – Upgrading Ubuntu 20 to Ubuntu 22 – software versions upgrade table – head-to-head and for the old, but still supported Ubuntu 18.04 LTS – Upgrading Ubuntu 18 to Ubuntu 20 – software versions upgrade table – head to head.

Software and technical details of Fedora Server 36 including cockpit screenshots

This article is for those of you who do not want to install a whole new operating system only to discover some technical details about the default installation like disk layout, packages included, software versions, and so on. Here we are going to review in several sections what is like to have a default installation of Fedora 36 Server using a real not virtual machine!
The kernel is 5.17.9 it detects successfully the Threadripper 1950X AMD and the system is stable (we booted in UEFI mode).
The installation procedure uses default options for all installation setups – Minimal network installation of Fedora 36 Server

Software

With Fedora Server 36 you can have

  • linux kernel – 5.17.9 (5.17.9-300.fc36.x86_64)
  • System
    • linux-firmware – version: 20220509, release: 20220509-132.fc36.
    • libc – 2.35 (2.35-9.fc36)
    • GNU GCC – 12.1.1 (12.1.1-1.fc36)
    • OpenSSL – 3.0.2 (1:3.0.2-5.fc36) and 1.1.1n (1:1.1.1n-1.fc36)
    • coreutils – 9.0 (9.0-5.fc36)
    • yum – Depricated and replaced with dnf
    • dnf – 4.12.0 (4.12.0-1.fc36)
    • rsyslog – 8.2204.0 (8.2204.0-1.fc36)
    • NetworkManager – 1.38.0 (1:1.38.0-1.fc36)
  • Servers
    • Apache – 2.4.53 (2.4.53-1.fc36)
    • Nginx – 1.20.2 (1:1.20.2-2.fc36)
    • MySQL server – 8.0.28 (8.0.28-1.fc36)
    • MariaDB server – 10.5.15 (3:10.5.15-1.fc36)
    • PostgreSQL – 14.1 (14.1-3.fc36)
  • Programming
    • PHP – 8.1.6 (8.1.6-1.fc36)
    • python – 3.10.4 (3.10.4-1.fc36) and also includes the older 2.7.18 (2.7.18-21.fc36)
    • perl – 5.34.1 (4:5.34.1-486.fc36)
    • ruby – 3.1.2 (3.1.2-164.fc36)
    • OpenJDK – the latest 18 – 18.0.1.0.10 (1:18.0.1.0.10-1.rolling.fc36) and also includes 17.0.3.0.7 (1:17.0.3.0.7-1.fc36), 11.0.15.0.10 (1:11.0.15.0.10-1.fc36) and 8.0.332.b09 (1:1.8.0.332.b09-1.fc36)
    • Go – 1.18.2 (1.18.2-1.fc36)
    • Rust – 1.60.0 (1.60.0-1.fc36)
    • llvm – the latest 14.0.0 (14.0.0-1.fc36) and the old 7.0.1 (7.0.1-7.fc36.4), 8.0.1 (8.0.1-2.fc36), 9.0.1 (9.0.1-15.fc35), 10.0.0 (10.0.0-13.fc35), 11.1.0 (11.1.0-6.fc35), 12.0.1 (12.0.1-2.fc35) and 13.0.1 (13.0.1-1.fc36)
    • Subversion – 1.14.1 (1.14.1-11.fc36)
    • Git – 2.36.1 (2.36.1-1.fc36)

Note: Not all of the above software comes installed by default. The versions above are valid as of May 2022, these are the minimal versions you get with Fedora Server 36 now and updating it after the initial date may update some of the above packages with newer versions.

Installed packages are 672 occupying 1.8G space:. Note, this is Fedora Server Install, not minimal install. The server install includes the web console – cockpit version 254.

[root@srv ~]# dnf list installed|wc -l
672
[root@srv ~]# df -h /
Filesystem                      Size  Used Avail Use% Mounted on
/dev/mapper/fedora_fedora-root   15G  1.8G   14G  12% /

Keep on reading!

Minimal network installation of Fedora 36 Server

This tutorial will show you the simple steps of installing a modern Linux Distribution Fedora 36 Server edition. Fedora line offers many bleeding-edge Linux technologies than the more enterprise CentOS of the same RPM Linux family.

In fact, if the user needs a server with the latest Linux stable software Fedora server is the right and easy choice for a server!

Here are some basic data from the default installation setup settings:

  1. Installed packages – ~671 occupying 1.8G of space.
  2. 3 partitions when using automatic patition layout – boot efi, boot and root.
  3. xfs used for the root parition.

The Fedora 36 Server comes and updates to the latest stable Linux:

  • Linux kernel : 5.17.9.
  • Python : 3.10.4
  • GLibc : 2.35
  • OpenSSL : 3.0.2
  • systemd : 250.3

Of course, one can expect latest version of GCC (12.1.1), PHP (8.1.6), GO (1.18.2), MySQL Server (8.0.28), PostgreSQL (14.1), Nginx (1.20.2), Apache (2.4.53) and so on. Almost all of them are the latest stable version on their Internet sites.
Just be careful, the Fedora life cycle is 13 months from the release to the EOL (End of Life)! Of course, a dist-upgrade is supported and indeed, it has been flawless for years!

We used the following ISO for the installation process from https://getfedora.org/en/server/download/:

https://download.fedoraproject.org/pub/fedora/linux/releases/36/Server/x86_64/iso/Fedora-Server-netinst-x86_64-36-1.5.iso

It is a LIVE image so you can try it before installing it. The easiest way is to download the image and burn it to a DVD or USB stick disk and then follow the installation below (a USB flash drive could be also created from this ISO). The netinstall installation is as simple as having a good Internet connection to download the packages, the installation wizard automatically detects the closest mirror, from which it will download the packages. Essentially, the network does not differ from the ordinary installation except it expects to download the packages from the Internet.

SCREENSHOT 1) If you booted from the DVD you would get this first screen – select “Install Fedora 36” and hit Enter

main menu
Start Fedora 36 Server

Keep on reading!

git status and bus error on SSD – fix READ errors by recovering part of the file

SSD and Linux encryption may not be the best idea, especially without the TRIM (allow-discards) option (or never executed fstrim?). Nevertheless, this error may occur not only on an SSD device, but just where there is a corrupted file system or device.
In our case, the SSD has some read errors. Apparently, some files or some parts of files could not be read by the git command:

[myuser@dekstop kernel]# git status -v
Bus error 84/115708)

In the case of SSD bad reads, the only working solution is to find and overwrite the problem file(s) or remove the file(s) and recreate them. A more sophisticated solution is to dump the file with dd and skip errors option enabled to another location and then overwrite the old file with the new one. So only the corrupted area of the file will be lost, which in most cases is just one or two sectors, i.e. one or two 512 bytes of data.

STEP 1) Find the bad files with the find command.

Use find Linux command and read all the files with the cat Linux command, so a bad sector will output an input/output error on READ. On write errors won’t be generated, but the sector will be automatically moved to a healthy one (the bad sector is marked and never used more).

[myuser@dekstop kernel]#  find -type f -exec cat {} > /dev/null \;
cat: ./servers/logo_description.txt: Input/output error

If multiple files are found repeat the procedure with each file.

STEP 2) Copy the healthy portion of the file.

The easiest way to remove the error is just to delete the file (or overwrite it), but if the healthy portion of the file is desirable the dd utility may be used to recover it:
Keep on reading!

Software and technical overview of Ubuntu server 22.04 LTS

Ubuntu server 22.04 LTS server edition offers the following software and versions:

Software

  • linux kernel – 5.15.0 – 5.15.0-33-generic
  • System
    • linux-firmware – 20220329.git681281e4-0ubuntu3. Apparently, this is a git clone on 29 March 2022.
    • libc – 2.35 – 2.35-0ubuntu3
    • GNU GCC – multiple versions available – 9.4.0, 10.3.0, 11.2.0 and 12-20220302. The exact versions – 9.4.0-5ubuntu1, 10.3.0-15ubuntu1, 11.2.0-19ubuntu1 and 12-20220319-1ubuntu1
    • OpenSSL – 3.0.2 – 3.0.2-0ubuntu1.2
    • coreutils – 8.328.32-4.1ubuntu1
    • apt – 2.4.5
    • rsyslog – 8.2112.0 – 8.2112.0-2ubuntu2.2
  • Servers
    • Apache – 2.4.52 – 2.4.52-1ubuntu4
    • Nginx – 1.18.0 – 1.18.0-6ubuntu14.1
    • MySQL server – 8.0.298.0.29-0ubuntu0.22.04.2
    • MariaDB server – 10.6.7 – 1:10.6.7-2ubuntu1
    • PostgreSQL – 14+238
  • Programming
      LTS the user may install

    • PHP – 8.1 – 2:8.1+92ubuntu1
    • python – 3.10.4 (3.10.4-0ubuntu2) and also includes 2.7.18-3
    • perl – 5.34.0 (5.34.0-3ubuntu1)
    • ruby – 3.0 – 1:3.0~exp1
    • OpenJDK – includes multiple versions – 8u312-b07, 11.0.15, 17.0.3 and 18~36ea-1. The exact versions are 8u312-b07-0ubuntu1, 11.0.15+10-0ubuntu0.22.04.1, 17.0.3+7-0ubuntu0.22.04.1 and 8u312-b07-0ubuntu1
    • Go lang – multiple versions – 1.13.8, 1.17.3 and 1.18.1. The exact versions – 1.13.8-1ubuntu2, 1.17.3-1ubuntu2 and 1.18.1-1ubuntu1.
    • Rust – 1.58.1 – 1.58.1+dfsg1~ubuntu1-0ubuntu2
    • Subversion – 1.14.1 – 1.14.1-3build4
    • Git – 2.34.1 – 1:2.34.1-1ubuntu1.2
    • llvm – multiple versions – 11.1.0, 12.0.1, 13.0.1 and 14. The exact versions – 1:11.1.0-6, 1:12.0.1-19ubuntu3, 1:13.0.1-2ubuntu2 and 1:14.0.0-1ubuntu1
  • Graphical User Interface
    • Xorg X server – ??? – 1.22.1.1
    • GNOME (the GUI) – 42.0 – Gnome Shell – 42.0 (42.0-2ubuntu1)

Note: Not all of the above software comes installed by default. The versions above are valid as of May 2022, these are the minimum versions you get with Ubuntu server 22.04 LTS now, and updating it after the initial date may update some of the above packages with newer versions.
The installation procedure you can find here – link.
Installed packages are 617 occupying 2.3G space:. Note, this is Ubuntu server 22.04 LTS, not the minimalistic install (used for embedded systems). This server install includes OpenSSH server.

root@srv:~# dpkg-query -l|wc -l
617
[root@srv ~]# df -h /
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv   57G   11G   44G  20% /
root@srv:~# du -h -d 1 -x /
36K     /root
2.2G    /usr
4.0K    /opt
5.4M    /etc
432M    /var
64K     /tmp
4.0K    /mnt
16K     /lost+found
28K     /home
4.0K    /media
4.0K    /srv
24K     /snap
11G     /

The swap file is 8G, so the real occupying space by the Ubuntu server 22.04 LTS is around 2.3G.
Keep on reading!

Review of freshly installed Ubuntu Desktop 22.04 LTS (Gnome GUI)

After the article of Ubuntu Desktop 22.04 LTS installation steps Install Ubuntu Desktop 22.04 LTS on the entire disk and Install Ubuntu Desktop 22.04 LTS on a PC with existing windows 10 – dual boot this article is mainly to see what to expect from a freshly installed Ubuntu Desktop 22.04 LTS – the look and feel of the GUI (Gnome – version 42.0).

  • Xorg X server – 1.22.1.1
  • GNOME (the GUI) – 42.0
  • linux kernel – 5.15.0

The idea of this tutorial is to see what to expect from Ubuntu Desktop 22.04 LTSthe look and feel of the GUI, the default installed programs, and their look and how to do some basic steps with them. Here you’ll find more than 114 screenshots and not so many text we do not want to turn this review of many texts and version information and 3 meaningless screenshots , which you cannot see anything for the user interface, which these days is the primary goal of a Desktop system. You can expect more of this kind of review in the future…
You can find a similar article for Fedora Workstation 36 – (coming soon).
Real workstations, not virtual environments are used for all installation and review tutorials!

SCREENSHOT 1) Ubuntu is selected by default

Wait for 10 seconds or hit Enter to boot the Ubuntu. The GNU GRUB version is 2.06.

main menu
grub entry boot

SCREENSHOT 2) Select the user.

main menu
login screen

SCREENSHOT 3) Enter the password.

main menu
login screen password

SCREENSHOT 4) Gnome Shell Overview.

main menu
desktop – gnome shell

Keep on reading!

Install and use collectd-ping under CentOS 8 to monitor latency

Tracking the network latency of the servers’ network is not an easy job. Most monitoring software is capable to monitor the state of the server, but how to monitor the state of the connectivity and the network latency and even the Internet connectivity with some respectful addresses like 1.1.1.1 or 8.8.8.8? It should be easy to do it with ICMP and ping command but using the collectd daemon and one of its plugins offers collectd-ping from https://collectd.org/wiki/index.php/Plugin:Ping to save all the history in a time series back-end and using grafanahttps://grafana.com/ (or other graphs/histograms and etc software) to make graphs.
Using the collectd-ping plugin in conjunction with grafana may reach the similar effect as using the old and gold smokeping.
CentOS 7 included the collectd-ping plugin in its official repository, but in CentOS 8 the plugin is missing! Under Cent OS 8 the CentOS SIG OpsTools https://wiki.centos.org/SpecialInterestGroup/OpsTools includes the collectd-ping plugin in their repository. More on SIG and OpsTools may be obtained in the later page. In general, it is safe to use this repository it would not break user’s system.
Here is how to install and configure it. Real grafana examples are also included at the end.

The example here assumes there is a grafana server installed with influxdb backend.

STEP 1) Add OpsTools repository and install the collectd and collectd-ping.

The OpsTools repository is installed with centos-release-opstools package.
Here is what is going to install:

dnf install -y centos-release-opstools
dnf install -y collectd collectd-ping

Keep on reading!

Upgrading Ubuntu 20 to Ubuntu 22 – software versions upgrade table – head-to-head

In the following article a comparison between two LTS versions of Ubuntu is presented – Ubuntu 20.04 LTS (Focal) versus Ubuntu 22.04 LTS (Jammy). The latest version of Ubuntu 20.04 and Ubuntu 22.04 (13.05.2022) is used to generate the software versions below.

In the Desktop world upgrading to the new and latest version of a Linux distribution is almost mandatory, but in the server world, upgrading is more complicated. The first step in updating a server is to check what software versions come with the new distribution version and then check whether the running custom (application) software supports the software versions. For example, updating to a new distribution version, which comes with PHP 8.1, but the current application supports only 7.4 is not very wise and in addition, the current version may have years of support in the future.

But as it may be seen below, the new Ubuntu server 22.04 LTS brings much newer versions compared to even the latest updates from Ubuntu server 20.04 LTS most of the latest server software.

There is a previous version of this article comparing the Ubuntu 18.04 LTS and Ubuntu 20.04 LTSUpgrading Ubuntu 18 to Ubuntu 20 – software versions upgrade table – head to head

SoftwareUbuntu 22.04Ubuntu 20.04
Linux kernel.
.
.
.
.
.
.
5.15.0
5.4.0
5.6.0
5.8.0
5.10.0
5.11.0
5.13.0
5.14.0
libc2.352.31
OpenSSL3.0.21.1.1f
GNU GCC.
.
9.4.0
10.3.0
11.2.0
12-20220302
8.4.0
7.5.0
9.4.0
10.3.0
PHP8.17.4
Python2.7.18
3.10.4
2.7.17
3.8.10
Perl5.34.05.30.0

6.d
Ruby3.02.7
OpenJDK8u312-b07
11.0.15
.
.
17.0.3
18~36ea-1
8u312-b07
11.0.15
13.0.7
16.0.1
17.0.3
Go lang1.13.8


1.17.3
1.18.1
1.13
1.14
1.16
Rust1.58.11.57.0
llvm.
.
.
.
.
11.1.0
12.0.1
13.0.1
14.0.0
6.0.1
7.0.1
8.0.1
9.0.1
10.0.0
11.0.0
12.0.0
nodejs12.22.910.19.0
Subversion1.14.11.13.0
Git2.34.12.25.1
Apache2.4.522.4.41
Nginx1.18.01.18.0
MySQL server8.0.298.0.29
MariaDB10.6.710.3.34
PostgreSQL14.212.10
SQLite2.8.17
3.37.2
2.8.17
3.31.1
Xorg X server1.22.1.11.20.13
Gnome Shell42.03.36.9
qemu6.24.2
docker20.10.1220.10.12
lxc5.0.04.0.6

Install Ubuntu Desktop 22.04 LTS on the entire disk

This tutorial will show the simple steps of installing the latest version of UbuntuUbuntu Desktop 22.04 LTS. This is the simplest set up. One hard disk device in the system is installed, which is detected as sda and the entire disk will be used for the installation of Ubuntu Desktop 22.04 LTS. All disk information in sda disk device will be permanently deleted by the installation wizard!
Ubuntu Desktop 22.04 LTS comes with the following software:

  • Xorg X server – 1.22.1.1
  • GNOME (the GUI) – 42.0
  • linux kernel – 5.15.0
  • linux-firmware – 20220329.git681281e4
  • QT – 5.13.3 and 6.2.4
  • libc – 2.35
  • gnu gcc – 9.4.0, 10.3.0, 11.2.0 and 12-20220302
  • coreutils – 8.32
  • python2.7 (possible to install) – 2.7.18
  • python3 (default) – 3.10.4
  • perl – 5.34.0
  • apt – 2.4.5
  • cups – 2.4.1

A more complex installation with a dual boot set up could found here – Install Ubuntu Desktop 22.04 LTS on a PC with existing windows 10 – dual boot.

We used the following ISO for the installation process – Ubuntu 22.04 LTS (Jammy Jellyfish):

https://releases.ubuntu.com/22.04/ubuntu-22.04-desktop-amd64.iso

It is a LIVE image so you can try it before installing it. The easiest way is just to download the image and burn it to a DVD disk and then follow the installation below:

SCREENSHOT 1) Select to “Try or Install Ubuntu” and hit enter.

To install Ubuntu from your DVD or USB you must boot from it, so change your BIOS accordingly – first boot devide should be the DVD or USB drive with Ubuntu installation. If you do it successfully you would see the screenshot below.

main menu
Grub2 try install linux

Keep on reading!