Admin? Anyone? Let's say Anyhelp is appreciated!
Linux rules the world, right?
A quick tip for GNU Nano – the text editor. Ever receiving the terminal error when using nano?
[root@srv ~]# nano Error opening terminal: unknown. [root@srv ~]#
Under chroot or container (like docker) environment try exporting the TERM environment:
export TERM=linux
And then execute nano the chances are you will enter nano normally and do your work (faster than vim 🙂 )?
You may use it without exporting to your shell, but just the for the nano:
TERM=linux nano
This article is for those of you who do not want to install a whole new operating system only to discover some technical details about the default installation like disk layout, packages included, software versions, and so on. Here we are going to review in several sections what is like to have a default installation of CentOS 8.0.
Despite the kernel is 4.18 it detects successfully the new RYZEN/Threadripper AMD and the system is stable (we booted in UEFI mode).
The CentOS 8.0 (8.0.1950) you can have
Note: Not all of the above software comes installed by default.
The installation procedure you can find here – How to do a network installation of CentOS 8 (8.0.1950) – minimal server installation Minimal install with “Standard” Addons selected.
Installed packages are 547 occupying 1.8G space:.
[root@srv ~]# dnf list installed|wc -l 547 [root@srv ~]# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/cl_srv-root 50G 1.8G 49G 4% /
Note: if you do not select “Standard Addon” the installation occupies less space – 1.6Gbytes and installs only 397 packages.
Keep on reading!
When upgrading to 17.1 profile you need to perform additional steps to get to a healthy Gentoo portage system using profile 17.1.
Here what we stumbled when we tried to upgrade from 13 to 17.1:
>>> Emerging (1 of 3) dev-libs/elfutils-0.177::gentoo * Fetching files in the background. * To view fetch progress, run in another terminal: * tail -f /var/log/emerge-fetch.log * elfutils-0.177.tar.bz2 BLAKE2B SHA512 size ;-) ... [ ok ] * Please follow the instructions in the news item: * 2019-06-05-amd64-17-1-profiles-are-now-stable * or choose the 17.0 profile. * ERROR: dev-libs/elfutils-0.177::gentoo failed (setup phase): * ERROR: 17.1 migration has not been performed!! * * Call stack: * ebuild.sh, line 591: Called __source_all_bashrcs * ebuild.sh, line 410: Called __try_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc' * ebuild.sh, line 467: Called __qa_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc' * ebuild.sh, line 112: Called source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc' * profile.bashrc, line 6: Called die * The specific snippet of code: * die "ERROR: 17.1 migration has not been performed!!" * * If you need support, post the output of `emerge --info '=dev-libs/elfutils-0.177::gentoo'`, * the complete build log and the output of `emerge -pqv '=dev-libs/elfutils-0.177::gentoo'`. * The complete build log is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/die.env'. * Working directory: '/var/tmp/portage/dev-libs/elfutils-0.177/homedir' * S: '/var/tmp/portage/dev-libs/elfutils-0.177/work/elfutils-0.177'
As you can see there is news, which consists of 12 steps to perform a successful upgrade to profile 17.1.
You can read the news with (the news output is included at the end of this article, just for completeness):
eselect news read 28
Specific note for our upgrade – in fact, our Gentoo installation was a chroot and it had not been updated for three years. The profile was 13.0 and portage supported only EAPI 5, which is a real problem. The new portage tree (we installed the latest) has no profile 13.0 and our emerge was half broken. We have preliminary steps described in Failed to install sys-apps/portage – Called pkg_preinst – portage._compat_upgrade.default_locations
Because we upgrade from 13.0 to 17.1, first, we need to perform the 17.0 upgrade (check the original steps here – https://gentoo.org/support/news-items/2017-11-30-new-17-profiles.html)
srv ~ # eselect profile set default/linux/amd64/17.0
You won’t be able to build any GNU GCC and you cannot continue with the next step.
Our current GNU GCC is really old – 4.9.3. The latest version is gcc-9.2.0-r1 and we are going to use it.
Keep on reading!
Minimal net install is useful when a dedicated server is installed from a IPMI KVM or Dell iDRAC, HP iLO, IBM IMM or where the initial client side download of files need to be minimal.
For amd64 CentOS 8 BaseOS the net install bootable media is located here (now the current latest release is 8.0.1950, but you can check the last directory with 8.
http://mirror.leaseweb.com/centos/8/isos/x86_64/CentOS-8-x86_64-1905-boot.iso
Note there is no minimal CD for offline installation anymore. Boot CD is to just boot and make “netinstall” installation and a big fat DVD of 6.6Gbytes to install offline.
Software details of CentOS 8 minimal install could be found here – Technical details of a default CentOS 8.0.1950 minimal installation
Download it and put it on a cd or usb, the boot from it and follow the steps bellow:
You can try F11 on most servers to have the boot menu. Here we use DEL to enter the UEFI BIOS.
Keep on reading!
Recently there was a problem with one compute node servers, which Gentoo (in fact, the chroot environment had not been updated for 3 years) could not emerge the new portage because it kept failing in the pkg_preinst with:
srv ~ # USE="-rsync-verify" emerge -va '=sys-apps/portage-2.3.76::gentoo'
.....
.....
>>> Installing (1 of 1) sys-apps/portage-2.3.76::gentoo
* checking 1999 files for package collisions
1000 files checked ...
^[[B>>> Merging sys-apps/portage-2.3.76 to /
/usr/bin/python2.7: No module named _compat_upgrade
* ERROR: sys-apps/portage-2.3.76::gentoo failed (preinst phase):
* (no error message)
*
* Call stack:
* ebuild.sh, line 133: Called pkg_preinst
* environment, line 3238: Called die
* The specific snippet of code:
* env -u DISTDIR -u PORTAGE_OVERRIDE_EPREFIX -u PORTAGE_REPOSITORIES -u PORTDIR -u PORTDIR_OVERLAY PYTHONPATH="${D%/}${PYTHON_SITEDIR}${PYTHONPATH:+:${PYTHONPATH}}" "${PYTHON}" -m portage._compat_upgrade.default_locations || die;
*
* If you need support, post the output of `emerge --info '=sys-apps/portage-2.3.76::gentoo'`,
* the complete build log and the output of `emerge -pqv '=sys-apps/portage-2.3.76::gentoo'`.
* The complete build log is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/environment'.
* Working directory: '/var/tmp/portage/._portage_reinstall_.RKkxLW/pym'
* S: '/var/tmp/portage/sys-apps/portage-2.3.76/work/portage-2.3.76'
!!! FAILED preinst: 1
>>> Failed to install sys-apps/portage-2.3.76, Log file:
>>> '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'
Note this is the installation phase, where everything from the build process passes smoothly, but it fails to install it with some vague error of a dying line with a couple of python and portage environments. The problem is the current portage does not support EAPI 6 and 7 and some packages were unable to be installed!
And we fixed the issue by just using the method described in our previous article – Fix your broken or old portage in Gentoo, fix the emerge command
Cloning a new portage software and using it to emerge (install) a newer (or reinstall the current) portage in the system to fix the system’s portage and the emerge command:
cd /root git clone https://anongit.gentoo.org/git/proj/portage.git /root/portage/bin/emerge --sync eselect profile set default/linux/amd64/17.1 /root/portage/bin/emerge -v portage
Now the latest and stable Gentoo profile is 17.1, so we set it to be sure we use it (in future this may change).
Keep on reading!
Using Gentoo for servers and chroots it might happen to deal with old and not regularly updated Gentoo environment, which in some cases is completely non-updatable using the regular process with emerge command. And no emerge means no installs from the Gentoo packaging system. Or you maybe have a broken portage after failed update with missing emerge command or missing or broken portage modules
In general, this article is for those, who cannot update their “sys-apps/portage” or even the emerge command exits with errors and is of no use.
In cases such as:
A real world errors are included at the bottom for each case above.
You need just a working python 2.7.x and git to clone “https://anongit.gentoo.org/git/proj/portage.git” or at least some tool to download the portage package.
Unpack to a globally accessible directory, DO NOT USE “/root“, because emerge changes user to non-privileges and needs execute permission for other for the all the path to the directory of emerge. Look below for more details.
Our example is to repair a completely broken portage with error:
srv ~ # emerge -va portage
Traceback (most recent call last):
File "/usr/lib/python-exec/python2.7/emerge", line 42, in <module>
import portage
ImportError: No module named portage
If you do not have a git client you can always check the URL in the sys-app/portage ebuild and download the URL. Still, if you cannot find the tarball’s URL, you can just simply get a mirror such as https://mirror.leaseweb.com/gentoo/distfiles, wait for the loading of the page (12Mbytes, it’s huge, all Gentoo tarball files’ names are here) and grep in it by the name “portage”. Or you can check here https://dev.gentoo.org/~zmedico/portage/archives/. Download the latest tarball and unpack it in the /opt directory.
Using git is the easiest method, but it is the same with downloading and unpacking the tarball in the “/opt”.
srv opt # git clone https://anongit.gentoo.org/git/proj/portage.git Cloning into 'portage'... remote: Enumerating objects: 151654, done. remote: Counting objects: 100% (151654/151654), done. remote: Compressing objects: 100% (39915/39915), done. remote: Total 151654 (delta 112542), reused 149864 (delta 111349) Receiving objects: 100% (151654/151654), 22.38 MiB | 550.00 KiB/s, done. Resolving deltas: 100% (112542/112542), done. srv opt # cd portage/ srv portage # ls -al total 232 drwxr-xr-x. 11 root root 4096 Oct 6 04:52 . drwx------. 4 root root 4096 Oct 6 04:51 .. drwxr-xr-x. 6 root root 4096 Oct 6 04:52 bin drwxr-xr-x. 5 root root 4096 Oct 6 04:52 cnf -rw-r--r--. 1 root root 1840 Oct 6 04:52 COPYING -rw-r--r--. 1 root root 6646 Oct 6 04:52 DEVELOPING drwxr-xr-x. 5 root root 4096 Oct 6 04:52 doc -rw-r--r--. 1 root root 200 Oct 6 04:52 .editorconfig drwxr-xr-x. 7 root root 4096 Oct 6 04:52 .git -rw-r--r--. 1 root root 80 Oct 6 04:52 .gitignore drwxr-xr-x. 4 root root 4096 Oct 6 04:52 lib -rw-r--r--. 1 root root 18092 Oct 6 04:52 LICENSE -rwxr-xr-x. 1 root root 1272 Oct 6 04:52 make.conf.example-repatch.sh drwxr-xr-x. 3 root root 4096 Oct 6 04:52 man -rw-r--r--. 1 root root 290 Oct 6 04:52 MANIFEST.in drwxr-xr-x. 2 root root 4096 Oct 6 04:52 misc -rw-r--r--. 1 root root 17845 Oct 6 04:52 NEWS -rw-r--r--. 1 root root 0 Oct 6 04:52 .portage_not_installed -rw-r--r--. 1 root root 2246 Oct 6 04:52 README -rw-r--r--. 1 root root 68122 Oct 6 04:52 RELEASE-NOTES drwxr-xr-x. 6 root root 4096 Oct 6 04:52 repoman -rwxr-xr-x. 1 root root 4889 Oct 6 04:52 runtests -rwxr-xr-x. 1 root root 19743 Oct 6 04:52 setup.py drwxr-xr-x. 2 root root 4096 Oct 6 04:52 src -rwxr-xr-x. 1 root root 81 Oct 6 04:52 tabcheck.py -rw-r--r--. 1 root root 1864 Oct 6 04:52 TEST-NOTES -rw-r--r--. 1 root root 571 Oct 6 04:52 testpath -rw-r--r--. 1 root root 348 Oct 6 04:52 tox.ini -rw-r--r--. 1 root root 584 Oct 6 04:52 .travis.yml
This is the latest portage with working emerge command in the bin sub-directory.
Keep on reading!
Inspired by our article – SSD cache device to a hard disk drive using LVM, which uses SSD driver as a cache device to a single hard drive, we decided to make a new article, but this time using two hard drives in raid setup (in our case RAID1 for redundancy) and a single NVME SSD drive.
The goal:
Caching RAID1 consisting of two 8T hard drive with a single 1T NVME SSD drive. Caching reads and writes, i.e. the write-back is enabled.
Our setup:
Only this step is different on different Linux distributions. We included three of them:
Ubuntu 16+:
sudo apt update && apt upgrade -y sudo apt install lvm2 -y systemctl enable lvm2-lvmetad systemctl start lvm2-lvmetad
CentOS 7:
yum update yum install -y lvm2 systemctl enable lvm2-lvmetad systemctl start lvm2-lvmetad
Gentoo:
emerge --sync emerge -v sys-fs/lvm2 /etc/init.d/lvm start rc-update add default lvm
If you tried recently to build your kernel with genkernel (4.0.0_beta17, but you will have the same problem with 3.x) you may end up with the following error:
* ERROR: Unable to generate splash, 'splash_geninitramfs' was not found!
The splash_geninitramfs was part of the removed Gentoo ebuild package – “media-gfx/splashutils“. The software in the package splashutils had not been maintained for a really long time and eventually, it was removed. At present, there is no package offering the splash_geninitramfs. Still, the same functionality could be archived with “sys-boot/plymouth“, but the genkernel does not have the option to use it.
The genkernel command option, which triggers the error is “–splash” you must remove it. And check whether the following option in “/etc/genkernel.conf” is set to “no” (at least, in genkernel version 3.x is the default set to “yes”):
SPLASH="no"
When you instruct the genkernel not to add boot splash using splashutils the error will be gone and the kernel build will be successful.
We include the output error when using “–splash” with genkernel and the last part of genkernel.log.
livecd ~ # genkernel --splash --install --clean --no-mrproper --menuconfig all * Gentoo Linux Genkernel; Version 4.0.0_beta17 * Using genkernel configuration from '/etc/genkernel.conf' ... * Running with options: --splash --install --clean --no-mrproper --menuconfig all * Working with Linux kernel 5.3.1-gentoo for x86_64 * Using kernel config file '/usr/share/genkernel/arch/x86_64/generated-config' ... * * Note: The version above is subject to change (depends on config and status of kernel sources). * kernel: >> Initializing ... * >> Running 'make clean' ... * >> --no-mrproper is set; Skipping 'make mrproper' ... * >> Running 'make oldconfig' ... * >> Invoking menuconfig ... * >> Re-running 'make oldconfig' due to changed kernel options ... * >> Kernel version has changed (probably due to config change) since genkernel start: * We are now building Linux kernel 5.3.1-gentoo-x86_64 for x86_64 ... * >> Compiling 5.3.1-gentoo-x86_64 bzImage ... * >> Compiling 5.3.1-gentoo-x86_64 modules ... * >> Installing 5.3.1-gentoo-x86_64 modules (and stripping) ... * >> Generating module dependency data ... * >> Saving config of successful build to '/etc/kernels/kernel-config-5.3.1-gentoo-x86_64' ... * initramfs: >> Initializing ... * >> Appending devices cpio data ... * >> Appending base_layout cpio data ... * >> Appending auxilary cpio data ... * >> Appending blkid cpio data ... * >> Appending busybox cpio data ... * >> Appending mdadm cpio data ... * >> Appending modprobed cpio data ... * >> Appending splash cpio data ... * ERROR: Unable to generate splash, 'splash_geninitramfs' was not found! * Please consult '/var/log/genkernel.log' for more information and any * errors that were reported above. * * Report any genkernel bugs to bugs.gentoo.org and * assign your bug to genkernel@gentoo.org. Please include * as much information as you can in your bug report; attaching * '/var/log/genkernel.log' so that your issue can be dealt with effectively. * * Please do *not* report kernel compilation failures as genkernel bugs! livecd ~ # cat /var/log/genkernel.log ..... ..... * >> Appending splash cpio data ... * ERROR: Unable to generate splash, 'splash_geninitramfs' was not found! * Please consult '/var/log/genkernel.log' for more information and any * errors that were reported above. * * Report any genkernel bugs to bugs.gentoo.org and * assign your bug to genkernel@gentoo.org. Please include * as much information as you can in your bug report; attaching * '/var/log/genkernel.log' so that your issue can be dealt with effectively. * * Please do *not* report kernel compilation failures as genkernel bugs! * * mount: >> '/boot' is not a mountpoint; Nothing to restore ... >>> Ended on: 2019-09-30 18:11:20 (after 0 days 0 hours 08 minutes 06 seconds)
Moving towards the firewalld software and especially the systemd some good old init scripts got missing! For example, one of those good scripts is the init script for iptables firewall, which allows saving iptables rules and during boot, it loads them again. With the init iptables script we have persistence of the iptables rules. Meanwhile, we can always call the init script with “save” argument to update the currently saved rules. Many different Linux distributions have this init script – “/etc/init.d/iptables”, but in systemd world, it has been removed and replaced with nothing (probably, because you are encouraged to use firewalld, which is not a bad thing!).
There are two packages “iptables-persistent” and “netfilter-persistent”, which work together to have iptables persistence over reboots. The rules are saved and restored automatically during system startup.
sudo apt install netfilter-persistent iptables-persistent
During the iptables–persistent installation the setup asks the user to save the current iptables rules. Hit “Yes” if you want to save the current iptables rules, which will be automatically loaded the next time the system starts up.
So it is safe to install it on a live system – the current iptables rules won’t be deleted.
Second, ensure the boot script to restore the iptables rules is enabled
sudo systemctl enable netfilter-persistent
Saving the current state of the iptables rules:
myuser@myubuntupc:~$ sudo /usr/sbin/netfilter-persistent save run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save
Restore the original state of the iptables rules:
sudo systemctl restart netfilter-persistent
And all commands you can do – start, stop, restart, reload, flush, save. You can use the script directly (it is not mandatory to use systemctl to restart, i.e. restore rules and etc.)
myuser@myubuntupc:~$ sudo /usr/sbin/netfilter-persistent Usage: /usr/sbin/netfilter-persistent (start|stop|restart|reload|flush|save)
The script netfilter-persistent executes 2 other scripts as plugins:
/usr/share/netfilter-persistent/plugins.d/15-ip4tables /usr/share/netfilter-persistent/plugins.d/25-ip6tables
The iptables rules are saved respectively in files
/etc/iptables/rules.v4 /etc/iptables/rules.v6
And you can always edit them manually or save/restore with iptables-save and iptables-restore redirecting the output to the above files.
It’s normal the state of the “active (exited)”. The service is “enabled” as you can see (by default the setup automatically enables the service on Ubuntu, but always check it to be sure, it’s the firewall!).
myuser@myubuntupc:~$ sudo systemctl status netfilter-persistent ● netfilter-persistent.service - netfilter persistent configuration Loaded: loaded (/lib/systemd/system/netfilter-persistent.service; enabled; vendor preset: enabled) Active: active (exited) since Thu 2019-01-17 20:44:08 EST; 14min ago Main PID: 666 (code=exited, status=0/SUCCESS) CGroup: /system.slice/netfilter-persistent.service Jan 17 20:44:08 myubuntupc systemd[1]: Starting netfilter persistent configuration... Jan 17 20:44:08 myubuntupc netfilter-persistent[666]: run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables start Jan 17 20:44:08 myubuntupc netfilter-persistent[666]: run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables start Jan 17 20:44:08 myubuntupc systemd[1]: Started netfilter persistent configuration.
Getting the following error may be deceiving:
Error! echo Your kernel headers for kernel 3.10.0-1062.1.1.el7.x86_64 cannot be found at /lib/modules/3.10.0-1062.1.1.el7.x86_64/build or /lib/modules/3.10.0-1062.1.1.el7.x86_64/source.
Because you may have already installed the kernel-headers package for the current kernel and still to get the same error. So what is missing?
In fact, the kernel headers for compiling a kernel module is in kernel-devel package.
[root@localhost ~]# yum install kernel-devel Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.wwfx.net * extras: mirror.wwfx.net * updates: mirror.wwfx.net Resolving Dependencies --> Running transaction check ---> Package kernel-devel.x86_64 0:3.10.0-1062.1.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================================ Installing: kernel-devel x86_64 3.10.0-1062.1.1.el7 updates 18 M Transaction Summary ============================================================================================================================================================================ Install 1 Package Total download size: 18 M Installed size: 38 M Is this ok [y/d/N]: y Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. kernel-devel-3.10.0-1062.1.1.el7.x86_64.rpm | 18 MB 00:00:02 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : kernel-devel-3.10.0-1062.1.1.el7.x86_64 1/1 Verifying : kernel-devel-3.10.0-1062.1.1.el7.x86_64 1/1 Installed: kernel-devel.x86_64 0:3.10.0-1062.1.1.el7 Complete!
If you have used other Linux distribution the “kernel headers”/”linux headers” package just means what it is named. In the CentOS 7 world there are two packages:
[root@localhost ~]# yum info kernel-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.wwfx.net
* extras: mirror.wwfx.net
* updates: mirror.wwfx.net
Installed Packages
Name : kernel-devel
Arch : x86_64
Version : 3.10.0
Release : 1062.1.1.el7
Size : 38 M
Repo : installed
From repo : updates
Summary : Development package for building kernel modules to match the kernel
URL : http://www.kernel.org/
License : GPLv2
Description : This package provides kernel headers and makefiles sufficient to build modules
: against the kernel package.
[root@localhost ~]# yum info kernel-headers
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.wwfx.net
* epel: mirrors.neterra.net
* extras: mirror.wwfx.net
* updates: mirror.wwfx.net
Installed Packages
Name : kernel-headers
Arch : x86_64
Version : 3.10.0
Release : 1062.1.1.el7
Size : 3.7 M
Repo : installed
From repo : updates
Summary : Header files for the Linux kernel for use by glibc
URL : http://www.kernel.org/
License : GPLv2
Description : Kernel-headers includes the C header files that specify the interface
: between the Linux kernel and userspace libraries and programs. The
: header files define structures and constants that are needed for
: building most standard programs and are also needed for rebuilding the
: glibc package.