removing the default kernel in CentOS 8 – remove elrepo kernel

Removing the default kernel aka the loaded kernel in CentOS 8 maybe challenging because the package is protected and cannot be removed by the yum or dnf.
Here is the case: an elrepo kernel-ml loaded and the dnf prints it cannot remove the package, because it is protected:

[root@srv ~]# dnf remove kernel-ml kernel-ml-core kernel-ml-modules
Error: 
 Problem: The operation would result in removing the following protected packages: kernel-ml-core
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
[root@srv ~]# uname -a
Linux srv.localhost 5.10.4-1.el8.elrepo.x86_64 #1 SMP Tue Dec 29 11:04:23 EST 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@srv ~]# grubby --default-kernel
/boot/vmlinuz-5.10.4-1.el8.elrepo.x86_64

The system is booted up with the kernel we are trying to remove, which is impossible.

The solution is to set a new default kernel and load it. Then dnf will be able to remove the first kernel.

For CentOS 7, just use the yum instead of dnf command.
Using grubby is really easy and straightforward:

STEP 1) List all installed and available to boot kernels

[root@srv ~]# grubby --info=ALL |grep ^kernel
kernel="/boot/vmlinuz-5.10.4-1.el8.elrepo.x86_64"
kernel="/boot/vmlinuz-4.18.0-259.el8.x86_64"
kernel="/boot/vmlinuz-4.18.0-257.el8.x86_64"
kernel="/boot/vmlinuz-0-rescue-45e12f0814fd4947b99cbdcb88950361"

STEP 2) Select the kernel to load the next time

[root@srv ~]# grubby --set-default "/boot/vmlinuz-4.18.0-259.el8.x86_64"
The default is /boot/loader/entries/45e12f0814fd4947b99cbdcb88950361-4.18.0-259.el8.x86_64.conf with index 1 and kernel /boot/vmlinuz-4.18.0-259.el8.x86_64

Keep on reading!

Gentoo emerge GO lang failed – atomic_amd64x.go: too many errors

Upgrading GO lang under Gentoo maybe a little bit tricky. The upgrading go lang from 1.13.7 to 1.15.5 failed with strange error:

# runtime/internal/atomic
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:18:6: Load redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:16:24
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:24:6: Loadp redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:22:32
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:30:6: Load64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:28:26
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:36:6: LoadAcq redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:34:27
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:41:6: Xadd redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:39:37
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:44:6: Xadd64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:42:39
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:47:6: Xadduintptr redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:45:47
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:50:6: Xchg redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:48:36
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:53:6: Xchg64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:51:38
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: Xchguintptr redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:54:45
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: too many errors

Googling a little bit and it appeared there might be a conflict with the old version in the same directory. Deleting the temporary build directory didn’t help…

Removing the GO lang package by unmerge command and then emerging the newest GO lang package is successful.

So the solution is to unmerge it and then immediately emerge the newest version with:

emerge -vC dev-lang/go
emerge -v dev-lang/go

The whole Gentoo output of the failed emerge command

Keep on reading!

Booting network installation from ipxe disk using IPMI KVM

There is a project for extended PXE Boot features https://ipxe.org/. This article is not for describing what this project may offer, but to show how to boot any Linux distribution (in fact, Windows 10, too) network installation wizard using the virtual CD/DVD of an IPMI KVM, DELL’s DRAC, HP iLO, IBM RSA/IMM and in general, KVM over IP.
Using the iPXE CD bootable disk mounted in the virtual CD/DVD of the server’s remote console (IPMI KVM and so on) will allow:

  • Booting from a CD/DVD with only a 1M size.
  • Extends the PXE features of the server’s network card.
  • Manual set IP address, i.e. not relying on DHCP server. In addition of the DHCP feature, but DHCP feature requires DHCP server, which is not always the case.
  • Load a Linux kernel and initramfs from a URL.
  • Boot a Linux live or installation CD/DVD from an URL. The server could load the instllation wizard from an official mirror in the Internet.
  • Manual install – boot from 1M CD and continue with multi gigabyte installation from an URL. For comparision the CentOS 8 network installation disk is more than 600M versus 1M iPXE CD disk. Booting directly from a 600M CentOS 8 network installation disk is unstable and really slow when the disk is mounted in user’s KVM. And not alwyas is possible to mount a disk next to the server location (or in the same co-location).
  • Automated install – simple unattended installation with kickstart files without the need of speacial features of the dedicated service provider.
  • No software installation or code writting needed.

This article uses the iPXE CD to boot and manually set an IP and then load the Linux kernel and initramfs of the CentOS 8 installation disk using an official URL mirror on the Internet. All types of server’s KVM, which supports CD/DVD virtual device, can be used.

Just 1 Mbytes of CD/DVD is required to boot an installation of a (connected to the Internet) server/machine.

Here are the steps and correct (all lines are tested) command lines to boot an installation wizard. The server is a SUPERMICRO server with IPMI KVM for remote management.
The iPXE ISO file is located here http://boot.ipxe.org/ipxe.iso

SCREENSHOT 1) Open the IPMI KVM and click on “Virtual Storage” menu to open the image mount dialog.

main menu
Virtual Storage menu

Keep on reading!

gentoo network interface with hyphen in the name

Using the OpenRC (i.e. init system) and network names with special symbols like hyphen in the name may lead to errors of “command not found” and “No such file or directory

The hyphen in the network interface name must be replaced in the configuration file with an underline and the init name file should be with the hyphen.

Proper configuration for network interface name with hyphen mv-eth0

  • In the configuration file /etc/conf.d/net:
    config_mv_eth0="
    192.168.0.202/24
    "
    routes_mv_eth0="
    default via 192.168.0.1
    "
    
  • The network interface init file is with hyphen:
    root@srv /etc/init.d # ln -s net.lo net.mv-eth0
    

And starting the network is successful:

root@srv ~ # /etc/init.d/net.mv-eth0 start
 * Caching service dependencies ...                                                                                                                                                     [ ok ]
 * Bringing up interface mv-eth0
 *   Caching network module dependencies
 *   192.168.0.202/24 ...                                                                                                                                                               [ ok ]
 *   Adding routes
 *     default via 192.168.0.1 ...                                                                                                                                                      [ ok ]
 *   Waiting for tentative IPv6 addresses to complete DAD (5 seconds) ..

Virtualization software may include to the network interface name not so typical alphabets. For example, systemd-nspawn will give name to the guest’s macvlan network with mv-{host_network_name} and iv-{host_network_name} for ipvlan.

Wrong configuration with a hyphen in the network interface name.

The configuration file /etc/conf.d/net:

config_mv-eth0="
192.168.0.202/24
"
routes_mv-eth0="
default via 192.168.0.1
"

Starting the network with such configuration will result in multiple errors:

root@srv ~ # /etc/init.d # /etc/init.d/net.mv-eth0 start
 * Caching service dependencies ...
/etc/init.d/../conf.d/net: line 3: config_mv-eth0=
192.168.0.202/24
: No such file or directory
/etc/init.d/../conf.d/net: line 6: $'routes_mv-eth0=\ndefault via 192.168.0.1\n': command not found
/etc/init.d/../conf.d/net: line 3: config_mv-eth0=
192.168.0.202/24
: No such file or directory
/etc/init.d/../conf.d/net: line 6: $'routes_mv-eth0=\ndefault via 192.168.0.1\n': command not found                                                                                  [ ok ]
/etc/init.d/../conf.d/net: line 3: config_mv-eth0=
192.168.0.202/24
: No such file or directory
/etc/init.d/../conf.d/net: line 6: $'routes_mv-eth0=\ndefault via 192.168.0.1\n': command not found
 * net.mv-eth0: error loading /etc/init.d/../conf.d/net
 * ERROR: net.mv-eth0 failed to start

xdg and autostart in Linux X server regardless the desktop environment

There is a tool xdg, which manages application integration with the different GUI Desktops in the Linux world. One of the features it offers is to autostart an application when the X window system starts and it is perfectly normal to have a bunch of running programs that cannot be found in the Windowing manager settings like KDE System Settings -> Autostart, GNOME Tweak tool and Autostart and so on.

xdg offers autostart of Linux appilcations mainly Desktop when the GUI windowing system starts

There two main paths to look for entries to autostart:

  1. /etc/xdg/autostart – called system-wide and most of the application will place files when they are installed.
  2. [user’s home]/.config/autostart – user’s applications to start when the user logs in .

With xdg autostart feature the user can explain himself why the Windowing systems like KDE or GNOME start tens of applications (not exactly related to the base GUI windowing system).

There is a security problem here, which is sometimes installing a package will place an autostart file there because the maintainer decided it is important but the package might be just a dependency and the next time the user logs in unwanted program might execute and open ports!

For example, Rygel is an open-source UPnP/DLNA MediaServer and it might be installed as a dependency but it places an autostart file, which starts a UPnP/DLNA server and exports the /home/[user’s directory]/Videos, /home/[user’s directory]/Pictures and more to the local network. Another example is with the GNOME index system tracker and the tracker-store, which may easily eat the RAM, disk, CPU, battery on a system without GNOME but with a different GUI!

Here is what a typical Ubuntu 18.04 system might autostart

Keep on reading!

multiple random crashes of firefox in a docker container under Linux

Multiple random crashes in Firefox under Linux, when started in a docker container with errors of the kind:

[myuser@92ee57f7f63a ~]$ Exiting due to channel error.
Exiting due to channel error.
[myuser@92ee57f7f63a ~]$ firefox 
ExceptionHandler::GenerateDump cloned child 4864
ExceptionHandler::SendContinueSignalToChild sent continue signal to child
ExceptionHandler::WaitForContinueSignal waiting for continue signal...
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
Bus error (core dumped)
[myuser@92ee57f7f63a ~]$ 
###!!! [Parent][MessageChannel] Error: (msgtype=0x5A001C,name=PHttpChannel::Msg_DeleteSelf) Channel error: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x5A001C,name=PHttpChannel::Msg_DeleteSelf) Channel error: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x5A001C,name=PHttpChannel::Msg_DeleteSelf) Channel error: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x5A001C,name=PHttpChannel::Msg_DeleteSelf) Channel error: cannot send/recv
###!!! [Parent][MessageChannel] Error: (msgtype=0x5A001C,name=PHttpChannel::Msg_DeleteSelf) Channel error: cannot send/recv

The chances to stop these multiple random crashes if you just increase the memory for the /dev/shm device in the docker container is really good! The default values in the docker /dev/shm are really low (in the tested machine only 64Mbytes) and the recommended values are at least 2Gbytes. More shared tabs more shared memory is needed.

Such strange and random crashes might cause a testing case to fail in an automation testing suite using the Selenium driver to start a Firefox headless instance.

To increase the shared memory size in the docker container, the container should be started with “–shm-size” option. For example:

docker run -it --shm-size=2048m my-gui-fedora:31

If the option –shm-size is missing (probably an old docker software) mapping an already mounted tmpfs directory in the host machine to the container is also a solution:

mkdir /dev/dockershm
mount -t tmpfs -o size=2048m tmpfs /dev/shmdocker
chmod 1777 /dev/shmdocker
docker run -d -v /mnt/shmdocker:/dev/shm my-gui-fedora:31

Remounting in the docker container is also possible:

srv myuser # docker exec -it my-gui bash
[myuser@92ee57f7f63a ~]$ sudo mount -o remount,size=2048m /dev/shm

Note the container should be run with –privileged option to be able to remount the /dev/shm.

The build docker command also has the –shm-size for the purpose of the building process not changing the default shared memory size in the containers based on the image afterward.

Using xtrabackup to make fast MySQL backups – backup and restore

Percona provides a really interesting tool for MySQL backups! It works on a live running MySQL server by copying the MySQL binary files from the data directory and because the tool knows how the engines work (InnoDB, MyISAM, and so on) it can make point-in-time consistent MySQL data files. Of course, using this tool on a database with InnoDB tables only is the best case, because no write lock will be used.

There are two main tasks when making a backup and one, which is not mandatory:

  1. Execute xtrabackup with –backup option to copy the MySQL data files and additional information for the tool
  2. Execute xtrabackup with –prepare option to prepare the MySQL (InnoDB) files ready for use of a MySQL server. The files are consistented, i.e. the whole backup is consistent despite the copy of the different files that happened at different times.
  3. Execute xtrabackup with –prepare option again to further prepare the MySQL (InnoDB) files ready for use of a MySQL server. Additional preparation such as InnoDB log files and more. This step is not mandatory and it may be skipped because the MySQL server, which uses the data files, will create the InnoDB log files.

The directory with MySQL copied files contains not only the MySQL files but additional information plus my.cnf (MySQL current configuration) backup. The backup files may be prepared (and restore) in a different server than the original, on which the backup was made.

xtrabackup may enable compressing on-the-fly when copying the binary files with the option –compress but the steps to use (restore) the backup in a new server are different – additional step to decompress before prepare. So the above steps become:

  1. xtrabackup –backup –compress – copy and compress on-the-fly.
  2. xtrabackup –decompress – decompress the backup files
  3. xtrabackup –prepare – make point-in-time consistent MySQL data files, which are ready for a MySQL server.
  4. xtrabackup –prepare – additional preparation to make the start up of the MySQL server with those files faster.

And here is a real-world example:

STEP 1) Install the xtrabackup utility

yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm
yum install -y percona-xtrabackup-24 qpress

The percona-xtrabackup-24 provides the percona xtrabackup tool for backup of MySQL 5.1, 5.5, 5.6 and 5.7 servers, as well as Percona Server for MySQL with XtraDB. Install percona-xtrabackup-80 for use with MySQL 8 and later.
The whole output is included in the Bonus 3 section below.

STEP 2) Make backup

Make the backup. The datadir of the MySQL server is needed and a new direcotry for the backup files. The MySQL server is running and serving requests.

[root@srv ~]# xtrabackup --backup --slave-info --datadir=/var/lib/mysql/ --target-dir=/mnt/backups/sql-xtrabackup/
xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --log_bin --server-id=1 --open_files_limit=5000 --innodb_buffer_pool_size=256M --innodb_log_buffer_size=32M --innodb_log_files_in_group=2 --innodb_flush_log_at_trx_commit=0 --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --datadir=/var/lib/mysql/ 
xtrabackup: recognized client arguments: --password=* --backup=1 --slave-info=1 --target-dir=/mnt/backups/sql-xtrabackup/ 
200919 14:35:11  version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup' (using password: YES).
200919 14:35:11  version_check Connected to MySQL server
200919 14:35:11  version_check Executing a version check against the server...
200919 14:35:11  version_check Done.
200919 14:35:11 Connecting to MySQL server host: localhost, user: not set, password: set, port: not set, socket: not set
Using server version 5.7.31-log
xtrabackup version 2.4.20 based on MySQL server 5.7.26 Linux (x86_64) (revision id: c8b4056)
xtrabackup: uses posix_fadvise().
xtrabackup: cd to /var/lib/mysql/
xtrabackup: open files limit requested 5000, set to 5000
xtrabackup: using the following InnoDB configuration:
xtrabackup:   innodb_data_home_dir = .
xtrabackup:   innodb_data_file_path = ibdata1:12M:autoextend
xtrabackup:   innodb_log_group_home_dir = ./
xtrabackup:   innodb_log_files_in_group = 2
xtrabackup:   innodb_log_file_size = 50331648
xtrabackup: using O_DIRECT
InnoDB: Number of pools: 1
200919 14:35:11 >> log scanned up to (1135932912)
xtrabackup: Generating a list of tablespaces
InnoDB: Allocated tablespace ID 34 for mywordpress/wp_users, old maximum was 0
200919 14:35:12 [01] Copying ./ibdata1 to /mnt/backups/sql-xtrabackup/ibdata1
200919 14:35:12 >> log scanned up to (1135932912)
200919 14:35:13 >> log scanned up to (1135932912)
200919 14:35:14 [01]        ...done
200919 14:35:14 >> log scanned up to (1135932912)
200919 14:35:15 >> log scanned up to (1135932912)
200919 14:35:16 [01] Copying ./mywordpress/wp_users.ibd to /mnt/backups/sql-xtrabackup/mywordpress/wp_users.ibd
200919 14:35:16 [01]        ...done
200919 14:35:16 [01] Copying ./mywordpress/wp_gglcptch_whitelist.ibd to /mnt/backups/sql-xtrabackup/mywordpress/wp_gglcptch_whitelist.ibd
200919 14:35:16 [01]        ...done
200919 14:35:16 [01] Copying ./mywordpress/wp_usermeta.ibd to /mnt/backups/sql-xtrabackup/mywordpress/wp_usermeta.ibd
200919 14:35:16 [01]        ...done
200919 14:35:16 >> log scanned up to (1135932912)
200919 14:35:17 [01] Copying ./mywordpress/wp_postmeta.ibd to /mnt/backups/sql-xtrabackup/mywordpress/wp_postmeta.ibd
200919 14:35:17 [01]        ...done
200919 14:35:17 [01] Copying ./mywordpress/wp_posts.ibd to /mnt/backups/sql-xtrabackup/mywordpress/wp_posts.ibd
200919 14:35:17 >> log scanned up to (1135932912)
200919 14:35:18 >> log scanned up to (1135932912)
200919 14:35:19 [01]        ...done
200919 14:35:19 >> log scanned up to (1135932912)
.....
.....
200919 14:36:30 [01] Copying ./mysql/columns_priv.MYD to /mnt/backups/sql-xtrabackup/mysql/columns_priv.MYD
200919 14:36:30 [01]        ...done
200919 14:36:30 >> log scanned up to (1135933567)
200919 14:36:31 [01] Copying ./mysql/proxies_priv.frm to /mnt/backups/sql-xtrabackup/mysql/proxies_priv.frm
200919 14:36:31 [01]        ...done
200919 14:36:31 [01] Copying ./mysql/help_category.frm to /mnt/backups/sql-xtrabackup/mysql/help_category.frm
200919 14:36:31 [01]        ...done
200919 14:36:31 [01] Copying ./mysql/proc.frm to /mnt/backups/sql-xtrabackup/mysql/proc.frm
200919 14:36:31 [01]        ...done
200919 14:36:31 Finished backing up non-InnoDB tables and files
200919 14:36:31 [00] Writing /mnt/backups/sql-xtrabackup/xtrabackup_slave_info
200919 14:36:31 [00]        ...done
200919 14:36:31 [00] Writing /mnt/backups/sql-xtrabackup/xtrabackup_binlog_info
200919 14:36:31 [00]        ...done
200919 14:36:31 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...
xtrabackup: The latest check point (for incremental): '1135933558'
xtrabackup: Stopping log copying thread.
.200919 14:36:31 >> log scanned up to (1135933567)

200919 14:36:32 Executing UNLOCK TABLES
200919 14:36:32 All tables unlocked
200919 14:36:32 [00] Copying ib_buffer_pool to /mnt/backups/sql-xtrabackup/ib_buffer_pool
200919 14:36:32 [00]        ...done
200919 14:36:32 Backup created in directory '/mnt/backups/sql-xtrabackup/'
MySQL binlog position: filename 'srv-bin.000001', position '1004'
200919 14:36:32 [00] Writing /mnt/backups/sql-xtrabackup/backup-my.cnf
200919 14:36:32 [00]        ...done
200919 14:36:32 [00] Writing /mnt/backups/sql-xtrabackup/xtrabackup_info
200919 14:36:32 [00]        ...done
xtrabackup: Transaction log of lsn (1135932903) to (1135933567) was copied.
200919 14:36:33 completed OK!

In the end, there must be written: “completed OK!”.
The whole output of the command is included in the Bonus 1 section below. An example with compress option enabled is in Bonus 2 section below.
Here is what contains the backup diectory:
Keep on reading!

gitlab in podman cannot create unix sockets in glusterfs because of SELinux

Installing gitlab-ee (and gitlab-ce) under CentOS 7 with enabled SELinux (i.e. enforcing mode) looped endlessly the container in restarting the installation process! There were multiple errors for missing sockets in the podman logs of the gitlab container. Here are some of the errors:
Missing postgresql unix socket in “/var/opt/gitlab/postgresql”:

Recipe: gitlab::database_migrations
  * bash[migrate gitlab-rails database] action run
    [execute] rake aborted!
              PG::ConnectionBad: could not connect to server: No such file or directory
                Is the server running locally and accepting
                connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:53:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'
              Tasks: TOP => gitlab:db:configure
              (See full trace by running task with --trace)
    
    
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
.....
.....
Running handlers:
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database] (gitlab::database_migrations line 55) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20200915-35-lemic5" ----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:53:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR: 
---- End output of "bash"  "/tmp/chef-script20200915-35-lemic5" ----
Ran "bash"  "/tmp/chef-script20200915-35-lemic5" returned 1

Missing redis socket in

Running handlers:
There was an error running gitlab-ctl reconfigure:

redis_service[redis] (redis::enable line 19) had an error: RuntimeError: ruby_block[warn pending redis restart] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/resources/service.rb line 65) had an error: RuntimeError: Execution of the command `/opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket INFO` failed with a non-zero exit code (1)
stdout: 
stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No such file or directory

It should be noted that the /var/opt/gitlab directory has been mapped in /mnt/storage/podman/gitlab/data. GlusterFS is used for /mnt/storage, so the gitlab files resides on a GlusterFS volume.

ERROR 1) Cannot create unix socket.

Checking the /var/log/audit/audit.log reveiled the problem immediately:
Keep on reading!

gpg list key and display key details from a file (without importing the key)

Files with GPG keyspublic or private. Here is how to get more information without importing the keys.
GPG cli could give enough information for an explored key in a file:

  • public or private key
  • encrypted or unencrypted key
  • user id description (including email)
  • key id and issuer fpr v4
  • when the key was generated and when it will expire
  • the algo for the encrypted key
  • more

The key may be in binary or ascii format. No difference.
Here is the GNU GPG cli command:

gpg --list-packets < ./filewith.key

All examples below are made with gpg (GnuPG) 2.2.19.
Keep on reading!

syslog – UDP local to syslog-ng and send remote. Forward syslog to remote server.

After writing an article for the rsyslog daemon about forwarding local UDP logging to a remote server using TCP – UDP local to rsyslog and send remote with TCP and compression this time going to use syslog-ng daemon for those who use it as default in their Linux distribution.
As mentioned in the previous article always use a non-blocking way of writing logs using UDP locally and then transfer (forward) the logs to the centralized log server(s). The example here transfers the web server’s access logs to a remote server. The web server is an Nginx web server.
The goal is to use

  • UDP for the client program (Nginx in the case) for non-blocking log writes.
  • TCP between our local machine and the remote syslog server – to be sure not to lose messages on bad connectivity.
  • local caching for our client machine – not to lose messages if the remote syslog is temporary unreachable.

The configuration and the commands are tested on CentOS 7, CentOS 8, Gentoo and Ubuntu 18 LTS. Check out UDP remote logging here – nginx remote logging to UDP rsyslog server (CentOS 7) to see how to build the server-side part – the syslog server accepting the syslog messages and writing them into files.

STEP 1) Listen for local UDP connections

Configuration file /etc/syslog-ng/syslog-ng.conf

source udp_local {
    network(ip(127.0.0.1) port(514) transport("udp") so_rcvbuf(67108864) log_fetch_limit(1000) max-connections(1000) log-iw-size(1000000));
};

Keep on reading!