Easy install the latest docker-compose with pip3 under Ubuntu

At present, the latest docker-compose version, which could be installed under Ubuntu 18, 20, and 21 is the 1.25 and 1.27 versions. There may be significant changes included in the latest versions and if one wants to install it there are two options:

For example, depends_on.service.condition: service_healthy is added with version 1.28. Using this new feature it is fairly easy to implement waiting for a docker container (service) before starting another docker.

Here is how easy it is to install and to have the latest stable docker-compose version, which is 1.29.2 at the writing of this article:

STEP 1) Update and upgrade.

Do this step always before installing new software.

sudo apt update
sudo apt upgrade -y

STEP 2) Install pip3 and docker.

pip 3 is the package installer for Python 3. When using docker-compose it is supposed to have the very Docker software, too.

apt install python3-pip docker
systemctl start docker

STEP 3) Install docker-compose using pip3.

pip3 install docker-compose

And here is what a version command prints:

root@srv:~# docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.2
CPython version: 3.8.10
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

Just to note, installing packages using other programs other than apt may lead to future conflicts!

The whole console output of the pip3 installing docker-compose

root@srv:~# apt update
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]              
Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]                  
Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1514 kB]        
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [33.3 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1069 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [575 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6324 B]
Get:14 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2668 B]
Get:15 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1070 kB]
Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [790 kB]
Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [30.1 kB]
Get:18 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [525 kB]
Fetched 19.0 MB in 1s (16.7 MB/s)                          
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
root@srv:~# apt upgrade -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root@srv:~# apt install -y python3-pip docker
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential ca-certificates cpp cpp-9 dirmngr dpkg-dev fakeroot file g++ g++-9 gcc gcc-9 gcc-9-base gnupg gnupg-l10n gnupg-utils
  gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libasn1-8-heimdal libassuan0 libatomic1
  libbinutils libbsd0 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-9-dev
  libgdbm-compat4 libgdbm6 libglib2.0-0 libglib2.0-data libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libicu66 libisl22
  libitm1 libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common liblocale-gettext-perl liblsan0 libmagic-mgc libmagic1 libmpc3 libmpdec2 libmpfr6 libnpth0 libperl5.30 libpython3-dev
  libpython3-stdlib libpython3.8 libpython3.8-dev libpython3.8-minimal libpython3.8-stdlib libquadmath0 libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db
  libsqlite3-0 libssl1.1 libstdc++-9-dev libtsan0 libubsan1 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxml2 linux-libc-dev make manpages manpages-dev mime-support
  netbase openssl patch perl perl-modules-5.30 pinentry-curses python-pip-whl python3 python3-dev python3-distutils python3-lib2to3 python3-minimal python3-pkg-resources
  python3-setuptools python3-wheel python3.8 python3.8-dev python3.8-minimal readline-common shared-mime-info tzdata wmdocker xdg-user-dirs xz-utils zlib1g-dev
Suggested packages:
  binutils-doc cpp-doc gcc-9-locales dbus-user-session libpam-systemd pinentry-gnome3 tor debian-keyring g++-multilib g++-9-multilib gcc-9-doc gcc-multilib autoconf automake libtool flex
  bison gdb gcc-doc gcc-9-multilib parcimonie xloadimage scdaemon glibc-doc git bzr gdbm-l10n libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap
  libsasl2-modules-otp libsasl2-modules-sql libstdc++-9-doc make-doc man-browser ed diffutils-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl libb-debug-perl
  liblocale-codes-perl pinentry-doc python3-doc python3-tk python3-venv python-setuptools-doc python3.8-venv python3.8-doc binfmt-support readline-doc
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential ca-certificates cpp cpp-9 dirmngr docker dpkg-dev fakeroot file g++ g++-9 gcc gcc-9 gcc-9-base gnupg gnupg-l10n
  gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libasn1-8-heimdal libassuan0
  libatomic1 libbinutils libbsd0 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-9-dev
  libgdbm-compat4 libgdbm6 libglib2.0-0 libglib2.0-data libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libicu66 libisl22
  libitm1 libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common liblocale-gettext-perl liblsan0 libmagic-mgc libmagic1 libmpc3 libmpdec2 libmpfr6 libnpth0 libperl5.30 libpython3-dev
  libpython3-stdlib libpython3.8 libpython3.8-dev libpython3.8-minimal libpython3.8-stdlib libquadmath0 libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db
  libsqlite3-0 libssl1.1 libstdc++-9-dev libtsan0 libubsan1 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxml2 linux-libc-dev make manpages manpages-dev mime-support
  netbase openssl patch perl perl-modules-5.30 pinentry-curses python-pip-whl python3 python3-dev python3-distutils python3-lib2to3 python3-minimal python3-pip python3-pkg-resources
  python3-setuptools python3-wheel python3.8 python3.8-dev python3.8-minimal readline-common shared-mime-info tzdata wmdocker xdg-user-dirs xz-utils zlib1g-dev
.....
.....
0 upgraded, 128 newly installed, 0 to remove and 0 not upgraded.
Need to get 84.6 MB of archives.
After this operation, 370 MB of additional disk space will be used.
Processing triggers for ca-certificates (20210119~20.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root@srv:~# pip3 install docker-compose
Collecting docker-compose
  Downloading docker_compose-1.29.2-py2.py3-none-any.whl (114 kB)
     |████████████████████████████████| 114 kB 12.4 MB/s 
Collecting requests<3,>=2.20.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 355 kB/s 
Collecting jsonschema<4,>=2.5.1
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
     |████████████████████████████████| 56 kB 3.4 MB/s 
Collecting websocket-client<1,>=0.32.0
  Downloading websocket_client-0.59.0-py2.py3-none-any.whl (67 kB)
     |████████████████████████████████| 67 kB 3.3 MB/s 
Collecting texttable<2,>=0.9.0
  Downloading texttable-1.6.4-py2.py3-none-any.whl (10 kB)
Collecting PyYAML<6,>=3.10
  Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
     |████████████████████████████████| 662 kB 76.9 MB/s 
Collecting dockerpty<1,>=0.4.1
  Downloading dockerpty-0.4.1.tar.gz (13 kB)
Collecting docker[ssh]>=5
  Downloading docker-5.0.2-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 119.5 MB/s 
Collecting distro<2,>=1.5.0
  Downloading distro-1.6.0-py2.py3-none-any.whl (19 kB)
Collecting docopt<1,>=0.6.1
  Downloading docopt-0.6.2.tar.gz (25 kB)
Collecting python-dotenv<1,>=0.13.0
  Downloading python_dotenv-0.19.0-py2.py3-none-any.whl (17 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 141.1 MB/s 
Collecting charset-normalizer~=2.0.0; python_version >= "3"
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 133.3 MB/s 
Collecting idna<4,>=2.5; python_version >= "3"
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 1.6 MB/s 
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0-cp38-cp38-manylinux1_x86_64.whl (118 kB)
     |████████████████████████████████| 118 kB 131.3 MB/s 
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from jsonschema<4,>=2.5.1->docker-compose) (45.2.0)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting attrs>=17.4.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |████████████████████████████████| 53 kB 899 kB/s 
Collecting paramiko>=2.4.2; extra == "ssh"
  Downloading paramiko-2.7.2-py2.py3-none-any.whl (206 kB)
     |████████████████████████████████| 206 kB 147.7 MB/s 
Collecting cryptography>=2.5
  Downloading cryptography-3.4.8-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
     |████████████████████████████████| 3.2 MB 147.4 MB/s 
Collecting bcrypt>=3.1.3
  Downloading bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl (63 kB)
     |████████████████████████████████| 63 kB 1.4 MB/s 
Collecting pynacl>=1.0.1
  Downloading PyNaCl-1.4.0-cp35-abi3-manylinux1_x86_64.whl (961 kB)
     |████████████████████████████████| 961 kB 139.4 MB/s 
Collecting cffi>=1.12
  Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB)
     |████████████████████████████████| 411 kB 84.0 MB/s 
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
     |████████████████████████████████| 112 kB 140.9 MB/s 
Building wheels for collected packages: dockerpty, docopt
  Building wheel for dockerpty (setup.py) ... done
  Created wheel for dockerpty: filename=dockerpty-0.4.1-py3-none-any.whl size=16604 sha256=d6f2d3d74bad523b1a308a952176a1db84cb604611235c1a5ae1c936cefe7889
  Stored in directory: /root/.cache/pip/wheels/1a/58/0d/9916bf3c72e224e038beb88f669f68b61d2f274df498ff87c6
  Building wheel for docopt (setup.py) ... done
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=f8c389703e63ff7ec3734b240ba8d62c8f8bd99f3b05ccdcb0de1397aa523655
  Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c
Successfully built dockerpty docopt
Installing collected packages: urllib3, charset-normalizer, certifi, idna, requests, pyrsistent, six, attrs, jsonschema, websocket-client, texttable, PyYAML, dockerpty, pycparser, cffi, cryptography, bcrypt, pynacl, paramiko, docker, distro, docopt, python-dotenv, docker-compose
Successfully installed PyYAML-5.4.1 attrs-21.2.0 bcrypt-3.2.0 certifi-2021.5.30 cffi-1.14.6 charset-normalizer-2.0.4 cryptography-3.4.8 distro-1.6.0 docker-5.0.2 docker-compose-1.29.2 dockerpty-0.4.1 docopt-0.6.2 idna-3.2 jsonschema-3.2.0 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0 pyrsistent-0.18.0 python-dotenv-0.19.0 requests-2.26.0 six-1.16.0 texttable-1.6.4 urllib3-1.26.6 websocket-client-0.59.0

Minimal installation of Ubuntu server 20.04 LTS

This tutorial will show you the simple steps of installing a modern Linux Distribution – Ubuntu server 20.04 LTS edition. Following most of the default options during the setup configuration for simplicity.

Here are some basic data from the default installation setup settings:

  1. Installed packages – ~582 occupying 11G of space.
  2. 3 partitions when using automatic patition layout – boot efi, swap and root.
  3. ext4 used for the root parition.

We used the following ISO for the installation process – Ubuntu 20.04 LTS (Focal Fossa):

http://releases.ubuntu.com/focal/ubuntu-20.04-live-server-amd64.iso

It is a LIVE image so you can try it before installing it. The easiest way is just to download the image and burn it to a DVD disk and then follow the installation below:

SCREENSHOT 1) Boot from the disk or USB – whatever you made after downloading the ISO file from Ubuntu official source.

On the image here the DVD is used to boot in UEFI mode installation.

main menu
boot uefi dvd

Keep on reading!

Install Fedora 31 KDE Plasma Desktop (KDE GUI)

This tutorial will show you the simple steps of installing a modern Linux Distribution Fedora 31 KDE Plasma Desktop with KDE for the user graphical interface. First, we present the basic steps for installing the Operating system in addition to your present operating systems (here we have two: Windows 10) and then you can see some screenshots of the installed system and the look and feel of it. We have another tutorial showing more screenshots of the installed and working Fedora 31 (Gnome and KDE plasma) – so you can decide which of them to try first – coming soon.

The Fedora 31 KDE Plasma Desktop comes with

  • Xorg X server – 1.20.5 XWayland is used by default
  • linux kernel – 5.3.7
  • KDE Plasma version: 5.16.5
  • KDE Frameworks version: 5.61.0
  • QT version: 5.12.5

The installation process is very similar to the old Install Fedora 27 KDE Plasma Desktop and Install Fedora 29 KDE Plasma Desktop (KDE GUI). Our system is relatively new – Asus X399 with AMD Ryzen Threadripper 1950X and NVIDIA 1080 TI and the setup loaded successfully and there were no problems till the end.

We used the following ISO for the installation process:

https://download.fedoraproject.org/pub/fedora/linux/releases/31/Spins/x86_64/iso/Fedora-KDE-Live-x86_64-31-1.9.iso

It is a LIVE image so you can try it before installing it. The easiest way is just to download the image and burn it to a DVD disk and then follow the installation below:

SCREENSHOT 1) Here is our “UEFI BIOS->Boot->Boot Override” and in most modern motherboard you can choose to override the default boot devices.

Choose the “UEFI: HL-DT-STDVDRAM…” to boot and install Fedora KDE Plasma Desktop 31 with UEFI support. You should do this, because most of the new hardware like video cards would not work properly without beeing in UEFI mode.

main menu
Boot from DVD/USB Installation

Keep on reading!

Gentoo – UnicodeEncodeError: ‘ascii’ codec can’t encode character ‘\xc4’ in position 83: ordinal not in range(128)

Recently emerging package =dev-lang/go-1.13.4 under Gentoo failed on one of our virtual servers with:

--- /usr/lib/go/test/fixedbugs/issue27836.dir/
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/_MergeProcess.py", line 234, in _spawn
    prev_mtimes=self.prev_mtimes, counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 1788, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5385, in merge
    counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4548, in treewalk
    rval = self._merge_contents(srcroot, destroot, cfgfiledict)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4828, in _merge_contents
    self.settings["EPREFIX"].lstrip(os.sep), cfgfiledict, mymtime):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5225, in mergeme
    encoding=_encodings['merge'])
  File "/usr/lib64/python3.6/site-packages/portage/util/movefile.py", line 256, in movefile
    selinux.rename(src, dest)
  File "/usr/lib64/python3.6/site-packages/portage/__init__.py", line 246, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/_selinux.py", line 71, in rename
    os.rename(src, dest)
UnicodeEncodeError: 'ascii' codec can't encode character '\xc4' in position 83: ordinal not in range(128)

>>> Failed to install dev-lang/go-1.13.4, Log file:

>>>  '/var/tmp/portage/dev-lang/go-1.13.4/temp/build.log'

The above log shows that the emerge fails in the installation phase when moving the files to the proper path in the system. The problem there is a non-ASCII character in the file name or path, but the environment is set to use ASCII as language.

The solution is to check if the environment LANG is set and what it contains. In this case, we should set the LANG environment to utf8.

export LANG=en_US.UTF-8

Most of the cases this kind of error could occur with virtual servers, docker (or the other kind of containers like lxc, podman and so on) containers and chroot jails or screens with changed user with su or sudo! In our case, the LANG just got missed because of a switch user procedure in a container and the emerge failed with the above error. When the LANG is missing probably the default value is “C”. In fact, check not only LANG but also the “LC_ALL” environment variable (it may have different value, which is wrong!), which also should be “en_US.UTF-8”:

export LC_ALL=en_US.UTF-8

Or try removing it at all with

unset LC_ALL

Keep on reading!

Install Fedora Workstation 31 (Gnome GUI)

This tutorial will show you the simple steps of installing a modern Linux Distribution like Fedora 31 Workstation with Gnome for the user graphical interface. First, we present the basic steps for installing the Operating system in addition to your present operating systems (here we also have Windows 10) and then you can see some screenshots of the installed system and the look and feel of it. We have other tutorials showing more screenshots of the installed and working Fedora 31 (Gnome and KDE plasma) – so you can decide which of them to try first – coming soon.

The Fedora 31 Workstation comes with

  • Xorg X server – 1.20.5 XWayland is used by default
  • GNOME (the GUI) – 3.34.1
  • linux kernel – 5.3.7

Check out our article about what software is included in comming soon.

The installation process is very similar to the old Fedora Workstation 27, Fedora Workstation 28, Fedora Workstation 29, Install Fedora Workstation 30 (Gnome GUI) , in fact the main difference is the creation of an user, which the setup is not responsible anymore, the creation of an user is done by the first boot after installation. Our system was pretty good – Asus X399 with AMD Ryzen Threadripper 1950X and NVIDIA 1080 Ti and the setup loaded successfully and there were no problems till the end.

We used the following ISO for the installation process:

https://download.fedoraproject.org/pub/fedora/linux/releases/31/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-31-1.9.iso

It is a LIVE image so you can try it before installing. The easiest way is just to download the image and burn it to a DVD disk and then follow the installation below:

SCREENSHOT 1) Here is our “UEFI BIOS->Boot->Boot Override” and in most modern motherboard you can choose to override the default boot devices.

Choose the “UEFI: HL-DT-STDVDRAM…” to boot and install Fedora Workstation 31 with UEFI support. You should do this, because most of the new hardware like video cards would not work properly without beeing in UEFI mode.

main menu
Boot from DVD/USB Installation

Keep on reading!

Install CentOS 8 over the old OS and preserve the storage partitions

Always put your root partition separate from the storage (aka data) partitions. root partition should be only for system files and nothing more! Keeping this simple rule you may easily change your operating system (or clean install or clean upgrade) without deleting the user’s data thus preserving the old storage partitions.
Our storage has 2 storage partitions, which means they hosts only data and no system files and there are separate partitions for Linux booting (grub2) and system files (root partition). Here is the partitions layout:

[root@srv0 ~]# parted /dev/sda --script print
Model: AVAGO SMC3108 (scsi)
Disk /dev/sda: 48.0TB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  53.7GB  53.7GB                     raid
 2      53.7GB  54.8GB  1075MB                     raid
 3      54.8GB  55.0GB  211MB   fat16              raid
 4      55.0GB  69.4GB  14.4GB                     raid
 5      71.8GB  48.0TB  47.9TB  ext4

[root@srv0 ~]# parted /dev/sdb --script print
Model: AVAGO SMC3108 (scsi)
Disk /dev/sdb: 48.0TB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  53.7GB  53.7GB                     raid
 2      53.7GB  54.8GB  1075MB                     raid
 3      54.8GB  55.0GB  211MB   fat16              raid
 4      55.0GB  69.4GB  14.4GB                     raid
 5      71.8GB  48.0TB  47.9TB  ext4
[root@srv0 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         16G     0   16G   0% /dev
tmpfs            16G     0   16G   0% /dev/shm
tmpfs            16G  250M   16G   2% /run
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/md127       50G  1.7G   45G   4% /
/dev/md125      991M  133M  792M  15% /boot
/dev/md124      201M   12M  190M   6% /boot/efi
/dev/sda5        44T   23T   22T  52% /mnt/storage1
/dev/sdb5        44T   14T   30T  32% /mnt/storage2
tmpfs           3.2G     0  3.2G   0% /run/user/0

Of course, when there are partitions above 2T the GPT is mandatory.
You can skip the software RAID1 setup if you use only one controller or you have system partitions only in one disk (virtual drive and so on). Here we have two hardware controllers, which we want to use both for the system partitions.
4 RAID1 devices:

  1. EFI partition (/boot/efi)
  2. swap partition
  3. boot parition (/boot)
  4. root partition (/)

The best practice is have total between 30G and 50G for the 4 partitions (in fact, boot partition could be skipped). Have in mind most modern Linux distributions cannot be installed on less than 10G~20G and for optimal results just separate between 30G and 50G for 4 partitions above (or 3 if you choose to skip the boot one).

Upgrade to CentOS 8 with clean install over our old CentOS 7 system partitions preserving the big data partitions.

Couple of things before start:

  • UEFI installation will be selected. So boot in UEFI mode.
  • IPMI KVM is used to install the new Linux distribution – CentOS 8
  • The installation disk is mounted in the Virtual CD/DVD IPMI KVM device – with Mount and boot ISO file from windows share in Supermicro IPMI Virtual media (CD-ROM)
  • All system parititions will be removed (grub, boot, root) and a clean minimal installation will be performed.
  • Network installation – using CentOS-8-x86_64-1905-boot.iso

SCREENSHOT 1) The Server is starting. This is the IPMI KVM window. Press F11 to Boot in Boot Menu.

main menu
SUPERMICRO Server starting – KVM

Keep on reading!

How to do a network installation of CentOS 8 (8.0.1950) – minimal server installation

Minimal net install is useful when a dedicated server is installed from a IPMI KVM or Dell iDRAC, HP iLO, IBM IMM or where the initial client side download of files need to be minimal.
For amd64 CentOS 8 BaseOS the net install bootable media is located here (now the current latest release is 8.0.1950, but you can check the last directory with 8. for the time you follow this howto):

http://mirror.leaseweb.com/centos/8/isos/x86_64/CentOS-8-x86_64-1905-boot.iso

Note there is no minimal CD for offline installation anymore. Boot CD is to just boot and make “netinstall” installation and a big fat DVD of 6.6Gbytes to install offline.
Software details of CentOS 8 minimal install could be found here – Technical details of a default CentOS 8.0.1950 minimal installation

Download it and put it on a cd or usb, the boot from it and follow the steps bellow:

STEP 1) Enter the BIOS of your machine (desktop or server) to change the boot order.

You can try F11 on most servers to have the boot menu. Here we use DEL to enter the UEFI BIOS.

main menu
Enter BIOS – change boot order

Keep on reading!

Minimal quagga bgpd configuration to run and remote configure it

There are the three steps to configure your Quagga bgpd daemon to be able to run and configure remotely. The idea of this article is to show you how you can run the quagga bgpd with the minimal configuration and probably you might give the credential to a network administrator.
Summary – 3 files to change:

  1. /etc/quagga/daemons – enable BGPD daemon
  2. /etc/quagga/debian.conf – which IP to listen to
  3. /etc/quagga/bgpd.conf – BGP daemon configuration

Here are the steps:
Keep on reading!

Install Fedora Workstation 30 (Gnome GUI)

This tutorial will show you the simple steps of installing a modern Linux Distribution like Fedora 30 Workstation with Gnome for the user graphical interface. First, we present the basic steps for installing the Operating system in addition to your present operating systems (here we also have Windows 10) and then you can see some screenshots of the installed system and the look and feel of it. We have other tutorials showing more screenshots of the installed and working Fedora 29 (Gnome and KDE plasma) – so you can decide which of them to try first – coming soon.

The Fedora 30 Workstation comes with

  • Xorg X server – 1.20.4 XWayland is used by default
  • GNOME (the GUI) – 3.32.1
  • linux kernel – 5.0.9

Check out our article about what software is included in comming soon.

The installation process is very similar to the old Fedora Workstation 27, Fedora Workstation 28 and Fedora Workstation 29, in fact the main difference is the creation of an user, which the setup is not responsible anymore, the creation of an user is done by the first boot after installation. Our system was pretty good – Asus X399 with AMD Ryzen Threadripper 1950X and NVIDIA 1080 Ti and the setup loaded successfully and there were no problems till the end.

We used the following ISO for the installation process:

https://download.fedoraproject.org/pub/fedora/linux/releases/30/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-30-1.2.iso

It is a LIVE image so you can try it before installing. The easiest way is just to download the image and burn it to a DVD disk and then follow the installation below:

SCREENSHOT 1) Here is our “UEFI BIOS->Boot->Boot Override” and in most modern motherboard you can choose to override the default boot devices.

Choose the “UEFI: HL-DT-STDVDRAM…” to boot and install Fedora Workstation 30 with UEFI support. You should do this, because most of the new hardware like video cards would not work properly without being in UEFI mode.

main menu
Boot from DVD/USB Installation

Keep on reading!

Access Violation error when compiling packages in Gentoo – symlink

Here is another example of an Access violation error when building packages in Gentoo. This time the build process could not make a symbolic link in “/usr/bin” and the build process of the package failed with:

 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied

A detail explanation is available in our first article on the subject here – Access Violation error, when compiling packages in Gentoo.
All packages are built in a sandbox and there is a sandbox configuration in

/etc/sandbox.d/00default

, which instruct the build process where could write. If you get such an error in 99.99% there is a bug in the package and if you do not want to wait for fixing it (report it!) you can manually edit the SANDBOX_WRITE variable and add the path, which causes the build failure. Build the package and remove the added path!!! Or you risk making your system less secure!

We have problem with building the package “net-libs/srt-1.3.1”

srv1 src # emerge -v net-libs/srt

These are the packages that would be merged, in order:

[ebuild  N     ] net-libs/srt-1.3.1::gentoo  USE="-doc -gnutls -libressl" ABI_X86="32 (64) (-x32)" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) net-libs/srt-1.3.1::gentoo
 * srt-1.3.1.tar.gz BLAKE2B SHA512 size ;-) ...                                                                                                        [ ok ]
>>> Unpacking source...
.....
.....
>>> Install srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/ category net-libs
 * abi_x86_32.x86: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_32.x86"
make -j6 -l10 install 
[ 28%] Built target haicrypt_virtual
[ 40%] Built target srtsupport_virtual
[ 80%] Built target srt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-file-transmit
[ 95%] Built target srt-live-transmit
[100%] Built target srt-multiplex
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
 * abi_x86_64.amd64: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_64.amd64"
make -j6 -l10 install 
[ 11%] Built target srtsupport_virtual
[ 52%] Built target srt_virtual
[ 80%] Built target haicrypt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-multiplex
[ 95%] Built target srt-file-transmit
[100%] Built target srt-live-transmit
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
>>> Completed installing srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/

 * Final size of build directory: 14632 KiB (14.2 MiB)
 * Final size of installed tree:   5324 KiB ( 5.1 MiB)

 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-25570.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 
 * --------------------------------------------------------------------------------

>>> Failed to emerge net-libs/srt-1.3.1, Log file:

>>>  '/var/tmp/portage/net-libs/srt-1.3.1/temp/build.log'

In the installation phase occurred the package failure leaving half installed package. So we edited the “/etc/sandbox.d/00default” and added “:/usr/bin” at the end of SANDBOX_WRITE:

SANDBOX_WRITE="/usr/tmp/conftest:/usr/lib/conftest:/usr/lib32/conftest:/usr/lib64/conftest:/usr/tmp/cf:/usr/lib/cf:/usr/lib32/cf:/usr/lib64/cf:/usr/bin"

Then rebuild the package with emerge and remove the added path “:/usr/bin”. This is a dirty workaround, but it will allow you to use srt (and as a dependency to another packages’ installation).