Category: tips

Overwrite Return-Path with postfix because of “550-Sender verification is required but failed”

Sending emails from web applications like PHP may result in rejecting the emails from some servers. Fighting spam emails results in too strict filters and rules, which reject the mails even before the anti-spam service of the accepting server. Here is an error:

Apr  1 04:10:18 srv-mail postfix/pickup[26902]: AB13578FAB3: uid=1015 from=<www-data>
Apr  1 04:10:18 srv-mail postfix/cleanup[21182]: AB13578FAB3: message-id=<20200401041018.AB13578FAB3@www.mydomain.com>
Apr  1 04:10:18 srv-mail postfix/qmgr[6485]: AB13578FAB3: from=<www-data@www.mydomain.com>, size=7923, nrcpt=1 (queue active)
Apr  1 04:10:19 srv-mail postfix/smtp[45689]: AB13578FAB3: to=<mailbox@example.com>, relay=mx.example.com[1.1.1.1]:25, delay=11, delays=0.02/0.01/0.65/10, dsn=5.0.0, status=bounced (host mx.example.com[1.1.1.1] said: 550-Sender verification is required but failed. (ID:550:0:5 550 (smtp1.mx.example.com)): www-data@mydomain.com (in reply to MAIL FROM command))

The receiving server has too strict rules!

It just expects the “From” and the “Return-Path” headers to contain the same string – the sender’s email box.

As you can see, from the example above, the application sends all emails (from let’s say web forms) from the www-data@mydomain.com and probably the www-data is the username of the OS user, under which the application executes.
Or you want to overwrite the Return-Path because it uses the username of the application, which sent the email like “web”, “apache”, “www-data” and so on.
Here is how to overwrite the Return-Path with postfix mail system.

STEP 1) Edit postfix configuration

Add a line in /etc/postfix/main.cf (it is perfectly fine to be on the last line):

smtp_generic_maps = hash:/etc/postfix/generic

And create the file /etc/postfix/generic with mapping “old@mailbox.com new@mail.com”:

www-data@mydomain.com no-reply@domain.com

The domains of the emails may be different or the same. It doesn’t matter. If you do not know what is your “www-data@mydomain.com” the mail logs in /var/log/messages or /varlog/mail maight help to find the emailbox or just send yourself an email and look for the Return-Path.
And a real-world example for /etc/postfix/generic

www-data@www.mydomain.com no-reply@ahelpme.com

STEP 2) Generate the hash file, which postfix will use. Reload the postfix.

The postfix will use the hash file add in the configuration. Just execute:

postmap /etc/postfix/generic

The above command will create a binary file /etc/postfix/generic.db, which will be used by the postfix mail system. Do not edit the file directly. To add entry, just use a text editor and edit /etc/postfix/generic (without the “.db” suffix) and then reload/restart the postfix to enable the new configuration.
And reload (or restart) postfix with

systemctl reload postfix

or for init systems:

/etc/init.d/postfix restart

nginx proxy cache and expires directive – pass-through the origin cache control

Proxying static content sometimes requires to modify the expire directive on the proxy server, but sometimes it may just need to pass the origin expire directive. What if the “expires” is defined in the server section and ones need to pass through the value from the origin by the proxy server?

Simply switch off the expires by “expires off”. It will disable an earlier definition in the block it is used and the proxy answer to the client will include the origin header for the cache control.

It won’t disable the cache control meaning to add no-cache in proxy answer to the client. So if it is used “expires” in a block and a pass-through from the origin is required, just make a location block with “expires off”:

        server {
                listen          10.10.10.10:443 ssl http2;
                server_name     srv1.example.com;

                ssl_certificate  /etc/ssl/nginx/srv1.example.com.chain.crt;
                ssl_certificate_key /etc/ssl/nginx/srv1.example.com.key;

                resolver 8.8.8.8;

                client_max_body_size 12m;

                expires         -1;
                root            /mnt/storage/web/root;
                access_log      /mnt/storage/web/logs/srv1.example.com;
                error_log       /mnt/storage/web/logs/srv1.example.com warn;

                location / {
                        #proxy
                        proxy_buffer_size   128k;
                        proxy_buffers   4 256k;
                        proxy_busy_buffers_size   256k;
                        proxy_buffering off;
                        proxy_read_timeout 600;
                        proxy_send_timeout 600;
                        proxy_store off;
                        proxy_cache off;
                        proxy_redirect off;
                        proxy_no_cache $cookie_nocache $arg_nocache $arg_comment;
                        proxy_no_cache $http_pragma $http_authorization;
                        proxy_cache_bypass $cookie_nocache $arg_nocache $arg_comment;
                        proxy_cache_bypass $http_pragma $http_authorization;

                        proxy_pass https://https_backend;
                        proxy_http_version 1.1;
                        proxy_set_header Connection "";
                        proxy_set_header Host $host;
                        proxy_set_header        X-Real-IP-EXAMPLE       $remote_addr;
                        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                }

                location ~* \.jpeg {
                        expires off;
                        #proxy
                        proxy_buffer_size   128k;
                        proxy_buffers   4 256k;
                        proxy_busy_buffers_size   256k;
                        proxy_buffering off;
                        proxy_read_timeout 600;
                        proxy_send_timeout 600;
                        proxy_store off;
                        proxy_cache off;
                        proxy_redirect off;
                        proxy_no_cache $cookie_nocache $arg_nocache $arg_comment;
                        proxy_no_cache $http_pragma $http_authorization;
                        proxy_cache_bypass $cookie_nocache $arg_nocache $arg_comment;
                        proxy_cache_bypass $http_pragma $http_authorization;
                        proxy_ignore_headers "Expires" "Cache-Control";


                        proxy_pass https://https_backend;
                        proxy_http_version 1.1;
                        proxy_set_header Connection "";
                        proxy_set_header Host $host;
                        proxy_set_header        X-Real-IP-EXAMPLE       $remote_addr;
                        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                }

                 location ~* \.(jpg|gif|png|css|mcss|js|mjs|woff|woff2)$ {
                        expires 30d;
                        #proxy
                        proxy_buffer_size   128k;
                        proxy_buffers   4 256k;
                        proxy_busy_buffers_size   256k;
                        proxy_buffering off;
                        proxy_read_timeout 600;
                        proxy_send_timeout 600;
                        proxy_store off;
                        proxy_cache off;
                        proxy_redirect off;
                        proxy_no_cache $cookie_nocache $arg_nocache $arg_comment;
                        proxy_no_cache $http_pragma $http_authorization;
                        proxy_cache_bypass $cookie_nocache $arg_nocache $arg_comment;
                        proxy_cache_bypass $http_pragma $http_authorization;

                        proxy_pass http://http_backend;
                        proxy_http_version 1.1;
                        proxy_set_header Connection "";
                        proxy_set_header Host $host;
                        proxy_set_header        X-Real-IP-EXAMPLE       $remote_addr;
                        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                }
        }

The above sample configuration defines 3 location blocks for 3 different “expires” cases:

  1. Uses the server block “expires -1” – proxy for dynamic code such as proxy site application code. No caching.
  2. Uses the location block .jpeg “expires off” – the proxy will pass-through the value from the origin server. Any value from the origin.
  3. Uses the location block .(gif|png|css|mcss|js|mjs|woff|woff2) “expires 30d” – it will set +30 days expire header no matter of the origin value. Origin server value is replaced by the today +30 days in the future cache cache header.

Dracut boot failed with missing device – exit and continue normal booting!

This issue deserves a much more article, in fact, a straightforward tip:

You may be able to continue a normal boot only by typing “exit” and hitting enter in the “Dracut” console.

Most of the time this Dracut console entering is caused because the system administrator of the server/machine added, replaced or deleted a RAID or similar device and forgot to update the configuration (grub2 probably). And in most of these cases, the raid is not critical for machine normal boot from the root partition, but it may be critical for the services lately. Booting in normal mode, even without some devices, is the main goal because under the normal mode it easier to repair the system.
Check out the two articles on the topic (especially the first one):

SCREENSHOT 1) Just type “exit” and hit enter.

It’s worth noting that if you executed some commands in the console and/or mounted devices to test they are with healthy file system or for whatever reason you did it, the boot process may not continue after typeing exit and probablly a reboot is required. The server will go once more in this mode and then just typing will work.

main menu
type exit

Keep on reading!

Dual 10Gbit network using PCI 2.0 (5GT/s) x4 – what is the maximum bandwidth?

Ever wondered what is the maximum bandwidth of a Dual 10Gbit LAN card, which can be reached using a dual 10Gbit ports card in a PCI Express 2.0 (Speed 5GT/s) and Width x4?
Here is the graph:
h3>SCREENSHOT 1) The bandwidth never exceeds 13.90Gbps (performed with only synthetic tests and mixed synthetic plus real http traffic).

main menu
Max graph bandwidth – below 14Gbps

As you can see the total of the two network ports is a little bit under 14Gbps. We are using intel dual-port controller:

Intel Corporation Ethernet Server Adapter X520-2

Even the dmesg reports the card is not in the right place:

[ 2.541813] ixgbe 0000:82:00.0: (Speed:5.0GT/s, Width: x4, Encoding Loss:20%)
[ 2.541832] ixgbe 0000:82:00.0: This is not sufficient for optimal performance of this card.
[ 2.541854] ixgbe 0000:82:00.0: For optimal performance, at least 20GT/s of bandwidth is required.
[ 2.541876] ixgbe 0000:82:00.0: A slot with more lanes and/or higher speed is suggested.
[ 2.541978] ixgbe 0000:82:00.0: MAC: 2, PHY: 19, SFP+: 5, PBA No: FFFFFF-0FF
[ 2.541996] ixgbe 0000:82:00.0: 00:16:31:fd:03:b8
[ 2.543027] ixgbe 0000:82:00.0: Intel(R) 10 Gigabit Network Connection
[ 2.694839] ixgbe 0000:82:00.1: Multiqueue Enabled: Rx Queue count = 48, Tx Queue count = 48 XDP Queue count = 0
[ 2.695531] ixgbe 0000:82:00.1: PCI Express bandwidth of 16GT/s available
[ 2.696087] ixgbe 0000:82:00.1: (Speed:5.0GT/s, Width: x4, Encoding Loss:20%)
[ 2.696631] ixgbe 0000:82:00.1: This is not sufficient for optimal performance of this card.
[ 2.697181] ixgbe 0000:82:00.1: For optimal performance, at least 20GT/s of bandwidth is required.
[ 2.697723] ixgbe 0000:82:00.1: A slot with more lanes and/or higher speed is suggested.
[ 2.698352] ixgbe 0000:82:00.1: MAC: 2, PHY: 19, SFP+: 6, PBA No: FFFFFF-0FF
[ 2.698890] ixgbe 0000:82:00.1: 00:16:31:fd:03:b9
[ 2.700436] ixgbe 0000:82:00.1: Intel(R) 10 Gigabit Network Connection

The controller is in the PCI Express slot – PCI 2.0 (Speed 5.0GT/s) Width x4, but the capability of the card is Speed 5GT/s, Width x8. This can be seen with “lspci -vvv” and the meanings with simple words:

  • LnkCap – it is the device capability. In fact, this is the hightest possible speed of the device put in the slot.
  • LnkSta – the actual speed of the PCI Express link.

If the device capacity (LnkCap) is higher than the actual speed (LnkSta) you could put the device in another slot with a higher capacity to take full advantage of the device.

In our case, the maximum bandwidth of the two ports of the Dual 10G port Intel card was just below 14Gbps (13.85Gbps ~ 13.95Gbps). After we move the very same card in another slot with the capability of Speed 5GT/s Width x8, the card’s maximum bandwidth increased to 19.20Gbps ~ 19.40Gbps.

SCREENSHOT 2) After changing the slot of the network card, which supports PCI 2.0 (5GT/s) Width x8, the bandwidth tops arround 19.40Gbps in synthetic tests (performed with iperf3).

main menu
Max graph bandwidth – almost 20Gbps

Keep on reading!

send access logs in json to Elasticsearch using rsyslog

Here is a simple example of how to send well-formatted JSON access logs directly to the Elasticsearch server.

It is as simple as Nginx (it could be any webserver) sends the access logs using UDP to the rsyslog server, which then sends well-formatted JSON data to the Elasticsearch server.

No other server program like logstash is used. The data is transformed in rsyslog and it is passed through a couple of modules to ensure the JSON is valid and Elasticsearch would not complain (and missing logs entry!).
Objectives:

  1. Nginx to send access logs using UDP to the rsyslog server.
  2. rsyslog server to accept UDP messages.
  3. rsyslog server transforms the web-server access logs from the Nginx server to JSON.
  4. rsyslog server sends the validated JSON to the Elasticsearch server.

The configuration and the commands are tested on CentOS 7, CentOS 8 and Ubuntu 18 LTS (just replace yum with apt).

STEP 1) Nginx to send access logs using UDP to the rsyslog server.

It is simple enough to send Nginx’ access logs to a UDP server (local or remote) there are two articles here: nginx remote logging to UDP rsyslog server (CentOS 7) and syslog – UDP local to rsyslog and send remote with TCP and compression. For simplicity, Nginx will send to the remote rsyslog server using UDP.
Instruct the Nginx to send access logs using UDP to the remote rsyslog server.
Define a new access log format in http serction:

        log_format mainJSON escape=json '@cee: {'
                '"vhost":"$server_name",'
                '"remote_addr":"$remote_addr",'
                '"time_iso8601":"$time_iso8601",'
                '"request_uri":"$request_uri",'
                '"request_length":"$request_length",'
                '"request_method":"$request_method",'
                '"request_time":"$request_time",'
                '"server_port":"$server_port",'
                '"server_protocol":"$server_protocol",'
                '"ssl_protocol":"$ssl_protocol",'
                '"status":"$status",'
                '"bytes_sent":"$bytes_sent",'
                '"http_referer":"$http_referer",'
                '"http_user_agent":"$http_user_agent",'
                '"upstream_response_time":"$upstream_response_time",'
                '"upstream_addr":"$upstream_addr",'
                '"upstream_connect_time":"$upstream_connect_time",'
                '"upstream_cache_status":"$upstream_cache_status",'
                '"tcpinfo_rtt":"$tcpinfo_rtt",'
                '"tcpinfo_rttvar":"$tcpinfo_rttvar"'
                '}';

It is a valid JSON object, but sometimes in user agent or referer contain non-standard and not valid characters, so it breaks the JSON format, which may lead to problems in Elasticsearch (read ahead).

In a server section of Nginx configuration file /etc/nginx/nginx.conf:

server {
     .....
     access_log      /var/log/nginx/example.com_access.log main;
     access_log      syslog:server=10.10.10.2:514,facility=local7,tag=nginx,severity=info mainJSON;
     .....
}

Keep on reading!

Patch and resume compilation of a failed package in Gentoo – ebuild, local repository or ctrl+Z

A dependency package failed to compile throwing error and existing the emerge of a queue with a hundred and more packages. Or worse you installed a new version of a package and multiple rebuilds are pulled, but one of the dependencies fails and you may end up with a broken system? What can you do? There is no new version of the failed package and yes, there is a bug in the Gentoo’s Bugzilla – https://bugs.gentoo.org/. And there is a solution with a patch, which has not made its way to the production and in Gentoo portage yet.
The package in the portage is broken, no new fixed package is released, but there is a patch to fix your issue. Here is what you can do:

  • Make your own package with the fixed version of the original package and put it in your local repository (not the official one, because on every emerge –sync it will be deleted). You should make a local repository and put the ebuild and all necessary files.
  • Or just download the patch and patch the source in the directory, which still holds the source of the failed package and resume the compilation manually. Then install it. Using this tutorial – Resume installation after a package build error, when emerging firefox under Gentoo
  • Just after the uncompress operation of the emerge press CTRL+Z to put the operation in the background and download and patch. Then bring back the emerge from the background with “fg” command.

The second and third options are not permanent solutions, but they are fast enough to be used in some situations.
Here are steps for the first and second option you may have:

OPTION 1) Make your own package.

Create a local repository (for details Simple steps to create Gentoo custom repository and add a package): 

root@srv ~ # mkdir -p /var/db/repos/my-local-portage/{metadata,profiles}
root@srv ~ # cat  << 'EOF' > /var/db/repos/my-local-portage/metadata/layout.conf
masters = gentoo
auto-sync = false
EOF
root@srv ~ # cat  << 'EOF' > /etc/portage/repos.conf/my-local-portage.conf
[my-local-portage]
location = /var/db/repos/my-local-portage 
EOF
root@srv ~ # cat  << 'EOF' > /var/db/repos/my-local-portage/profiles/repo_name
my-local-portage
EOF

Copy the ebuild file of the package you want to modify in the custom repository directory created above (it’s a good idea to copy all the sub-directories, too):
Keep on reading!

Simple steps to create Gentoo custom repository and add a package

Creating a custom repository would give you a chance to fast edit (ebuild) files of existing packages and drop better versions to the custom repository, which then will be used to install in the system. Here is the simplest way to create a Gentoo custom repository without installing any mandatory software. You may check the two Gentoo articles on the subject – https://wiki.gentoo.org/wiki/Custom_repository, which uses repoman (and additional software to install) and https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/CustomTree#Defining_a_custom_repository, which is part of a bigger article and without a clear example with a package as we are going to show.
Our custom repository name is “my-local-portage”.

STEP 1) Create the directories and basic configuration files for the new custom repository

Just two mandatory directories. 

mkdir -p /var/db/repos/my-local-portage/{metadata,profiles}

The minimal configuration in two files:

cat  << 'EOF' > /var/db/repos/my-local-portage/metadata/layout.conf
masters = gentoo
auto-sync = false
EOF
cat  << 'EOF' > /var/db/repos/my-local-portage/profiles/repo_name
my-local-portage
EOF

Fix the permissions

chown -R portage:portage /var/db/repos/my-local-portage

The custom repository is set up. Now only the emerge should get the configuration to check for it in the next step.

STEP 2) Portage global configuration.

Add a file pointing for your custom repository in the Portage global configuration directory “/etc/portage”:

cat  << 'EOF' > /etc/portage/repos.conf/my-local-portage.conf
[my-local-portage]
location = /var/db/repos/my-local-portage
EOF

STEP 3) Add a package in the new custom repository.

The package version may be the same version as in the official Gentoo repository, but the package form the custom repository will be used if no repository is included in the “emerge” command.
For simplicity, we are going not to modify the ebuild file of a copied official package, but the idea is to copy an existing ebuild file and then change it for the user’s needs and the steps are the same as follow.
Just copy the file (and edit it). The package “app-text/calibre” was randomly selected for the example.

mkdir /var/db/repos/my-local-portage/app-text/calibre
cp /usr/portage/app-text/calibre/calibre-4.9.1-r1.ebuild /var/db/repos/my-local-portage/app-text/calibre/

Create the manifest files and you are ready:

cd /var/db/repos/my-local-portage/app-text/calibre/
ebuild calibre-4.9.1-r1.ebuild manifest
>>> Downloading 'ftp://ftp.free.fr/mirrors/ftp.gentoo.org/distfiles/2e/calibre-4.9.1.tar.xz'
--2020-02-06 18:04:59--  ftp://ftp.free.fr/mirrors/ftp.gentoo.org/distfiles/2e/calibre-4.9.1.tar.xz
           => '/usr/portage/distfiles/calibre-4.9.1.tar.xz.__download__'
Resolving ftp.free.fr... 212.27.60.27, 2a01:e0c:1:1598::1
Connecting to ftp.free.fr|212.27.60.27|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /mirrors/ftp.gentoo.org/distfiles/2e ... done.
==> SIZE calibre-4.9.1.tar.xz ... 37529656
==> PASV ... done.    ==> RETR calibre-4.9.1.tar.xz ... done.
Length: 37529656 (36M) (unauthoritative)

calibre-4.9.1.tar.xz                       100%[========================================================================================>]  35.79M  5.53MB/s    in 8.0s    

2020-02-06 18:05:08 (4.48 MB/s) - '/usr/portage/distfiles/calibre-4.9.1.tar.xz.__download__' saved [37529656]

>>> Creating Manifest for /var/db/repos/my-local-portage/app-text/calibre

Fix the permissions with

chown -R portage:portage /var/db/repos/my-local-portage

The manifest file contains the hash of the ebuild file and all the additional files if any (for this package there are no additional files). All file needed for the operation will be downloaded so they must be network accessed in the time of executing the command (except in the cases when they have already existed in the distfiles directory and/or the subdirectories of the /var/db/repos/my-local-portage/app-text/calibre)
Keep on reading!

more than the default 4 parallel processes using distributed compiling with distcc

Distributed compilation could greatly speed the build process of Gentoo packages (and not only Gentoo, of course). If you tend to use Gentoo on a laptop or a relatively old CPU you may want to build packages distributively across multiple hosts.
Different (Linux) distributions use different configurations and environment scheme and sometimes it is difficult to sift the configuration, which could be applied to your setup. This is not a tutorial on how to enable parallel processing in Gentoo but it is just our client-site setup.

By default, there is a limit of 4 parallel processes, which is utterly insufficient, because nowadays most servers have more than 8 cores/logical compute units (not to mention that probably most would have 16 and above cores compute units).

The environment variable DISTCC_HOSTS controls, which hosts will receive files for the compilation of what they support and what is the limit of parallel processes.

In Gentoo we set this variable in the /etc/portage/make.conf. Here what you may include in make.conf to have 16 parallel remote processes and up to maximum 4 local (if the remote fails):

MAKEOPTS="-j16 -l4"
FEATURES="distcc"
DISTCC_HOSTS="192.168.0.101/16"

We use the environment DISTCC_HOSTS (here in Gentoo put in the make.conf, but in another Linux distribution an environment variable with this name should be set) because it is easy to set up and control globally for the Gentoo emerge system.
According to the documents:

In order, distcc looks in the $DISTCC_HOSTS environment variable, the user’s $DISTCC_DIR/hosts file, and the system-wide host file.

So when using emerge to build the packages, the emerge will rely on $DISTCC_HOSTS in make.conf (/etc/portage/make.conf or /etc/make.conf if you still use the old path), “/var/tmp/portage/.distcc/” (the build process uses “portage” user and group, not root!) and “/etc/distcc/hosts”. The first option used in the order above will be set the hosts and the limitation for the distributed processing. So if you use $DISTCC_HOSTS in make.conf (or environment) you wouldn’t need to set the “hosts” file.
Separate the different hosts with white space if you have more than one and always use the notation “/LIMIT” for each host. The default value is only 4 parallel processes (i.e it is implicitly added /4 to each hosts in the configuration!)
Keep on reading!

display packages from the Gentoo emerge resume list

A quick tip and an easy hack to see the emerge resume list, which will be used by the “emerge –resume” command. There are two resume lists – resume and resume_backup in the binary file /var/cache/edb/mtimedb.
The first two commands are python 3+.
The resume and resume_backup lists

python -c 'import portage; print(portage.mtimedb.get("resume", {}).get("mergelist"))'
python -c 'import portage; print(portage.mtimedb.get("resume_backup", {}).get("mergelist"))'

If you need the command for the python 2 you should use the following:

python -c 'import portage; print portage.mtimedb.get("resume", {}).get("mergelist")'
python -c 'import portage; print portage.mtimedb.get("resume_backup", {}).get("mergelist")'

Real world example

Resume list for

desktop ~ # emerge -v --nodeps $(qlist -IC|grep dev-qt)

These are the packages that would be merged, in order:

[ebuild     U  ] dev-qt/designer-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="declarative webkit -debug -test" 8,493 KiB
[ebuild     U  ] dev-qt/linguist-tools-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -test" 0 KiB
[ebuild     U  ] dev-qt/qdbus-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtbluetooth-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -qml -test" 2,727 KiB
[ebuild   R    ] dev-qt/qtchooser-66::gentoo  USE="-test" 32 KiB
[ebuild     U  ] dev-qt/qtconcurrent-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 48,549 KiB
[ebuild     U  ] dev-qt/qtcore-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="icu -debug (-systemd) -test" 0 KiB
[ebuild     U  ] dev-qt/qtdbus-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtdeclarative-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="jit widgets -debug -gles2 -localstorage -test" 20,753 KiB
[ebuild     U  ] dev-qt/qtgraphicaleffects-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 13,713 KiB
[ebuild     U  ] dev-qt/qtgui-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="accessibility dbus egl evdev gif ibus jpeg libinput png udev vnc wayland%* xcb -debug -eglfs -gles2 -test -tslib -tuio" 0 KiB
[ebuild     U  ] dev-qt/qthelp-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtimageformats-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="mng -debug -test (-jpeg2k%)" 1,768 KiB
[ebuild     U  ] dev-qt/qtlocation-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 5,974 KiB
[ebuild   R    ] dev-qt/qtlockedfile-2.4.1_p20171024::gentoo  USE="-doc" 694 KiB
[ebuild     U  ] dev-qt/qtmultimedia-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="alsa gstreamer openal pulseaudio qml widgets -debug -gles2 -test" 3,671 KiB
[ebuild     U  ] dev-qt/qtnetwork-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="libproxy networkmanager ssl -bindist -connman -debug -sctp -test" 0 KiB
[ebuild     U  ] dev-qt/qtopengl-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtpaths-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtpositioning-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -geoclue -test" 0 KiB
[ebuild     U  ] dev-qt/qtprintsupport-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="cups -debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtquickcontrols2-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test -widgets" 7,947 KiB
[ebuild     U  ] dev-qt/qtquickcontrols-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="widgets -debug -test" 5,843 KiB
[ebuild     U  ] dev-qt/qtscript-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="jit scripttools -debug -test" 2,584 KiB
[ebuild     U  ] dev-qt/qtsensors-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -qml -test" 1,996 KiB
[ebuild   R    ] dev-qt/qtsingleapplication-2.6.1_p20171024::gentoo  USE="X -doc" 0 KiB
[ebuild     U  ] dev-qt/qtspeech-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 99 KiB
[ebuild     U  ] dev-qt/qtsql-5.14.0:5/5.14.0::gentoo [5.12.2:5/5.12.2::gentoo] USE="mysql sqlite -debug -freetds -oci8 -odbc -postgres -test" 0 KiB
[ebuild     U  ] dev-qt/qtsvg-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 1,822 KiB
[ebuild     U  ] dev-qt/qttest-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qttranslations-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 1,302 KiB
[ebuild     U  ] dev-qt/qtvirtualkeyboard-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="spell xcb -debug -handwriting -test" 10,705 KiB
[ebuild     U  ] dev-qt/qtwayland-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="libinput xcomposite -debug -test" 532 KiB
[ebuild     U  ] dev-qt/qtwebchannel-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -test" 192 KiB
[ebuild     U  ] dev-qt/qtwebengine-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="alsa pulseaudio system-icu widgets -bindist -debug -designer -jumbo-build -pax_kernel -system-ffmpeg -test (-geolocation%*)" 235,904 KiB
[ebuild     U  ] dev-qt/qtwebkit-5.212.0_pre20190629:5/5.212::gentoo [5.212.0_pre20180120:5/5.212::gentoo] USE="X geolocation hyphen jit multimedia opengl printsupport qml -gles2 -gstreamer -nsplugin -orientation -webp" 12,166 KiB
[ebuild     U  ] dev-qt/qtwidgets-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="gtk png xcb -debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtx11extras-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 125 KiB
[ebuild     U  ] dev-qt/qtxml-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtxmlpatterns-5.14.0:5/5.14::gentoo [5.11.1:5/5.11::gentoo] USE="-debug -qml% -test" 1,362 KiB

Total: 40 packages (37 upgrades, 3 reinstalls), Size of downloads: 388,941 KiB


>>> Verifying ebuild manifests
>>> Running pre-merge checks for dev-qt/qtwebkit-5.212.0_pre20190629

>>> Emerging (1 of 40) dev-qt/designer-5.14.0::gentoo
 * Fetching files in the background.
 * To view fetch progress, run in another terminal:
 * tail -f /var/log/emerge-fetch.log
 * qttools-everywhere-src-5.14.0.tar.xz BLAKE2B SHA512 size ;-) ...                                                                                                  [ ok ]
!!! Failed to set new SELinux execution context. Is your current SELinux context allowed to run Portage?
>>> Unpacking source...
>>> Unpacking qttools-everywhere-src-5.14.0.tar.xz to /var/tmp/portage/dev-qt/designer-5.14.0/work
^C

Exiting on signal 2
sandbox:stop  caught signal 2 in pid 22886
sandbox:stop  Send signal 4 more times to force SIGKILL
Sandboxed process killed by signal: Interrupt
^C * The ebuild phase 'die_hooks' has been killed by signal 2.

 * Messages for package dev-qt/designer-5.14.0:

We interrupted it on purpose with Ctrl+C.

The output of the above python command:

desktop ~ # python -c 'import portage; print(portage.mtimedb.get("resume", {}).get("mergelist"))'
[['ebuild', '/', 'dev-qt/designer-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/linguist-tools-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qdbus-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtbluetooth-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtchooser-66', 'merge'], ['ebuild', '/', 'dev-qt/qtconcurrent-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtcore-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtdbus-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtdeclarative-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtgraphicaleffects-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtgui-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qthelp-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtimageformats-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtlocation-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtlockedfile-2.4.1_p20171024', 'merge'], ['ebuild', '/', 'dev-qt/qtmultimedia-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtnetwork-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtopengl-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtpaths-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtpositioning-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtprintsupport-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qtquickcontrols2-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtquickcontrols-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtscript-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsensors-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsingleapplication-2.6.1_p20171024', 'merge'], ['ebuild', '/', 'dev-qt/qtspeech-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsql-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsvg-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qttest-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qttranslations-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtvirtualkeyboard-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwayland-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebchannel-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebengine-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebkit-5.212.0_pre20190629', 'merge'], ['ebuild', '/', 'dev-qt/qtwidgets-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtx11extras-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtxml-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtxmlpatterns-5.14.0', 'merge']]
desktop ~ # python -c 'import portage; print(portage.mtimedb.get("resume_backup", {}).get("mergelist"))'
None

rsync daemon in CentOS 8

Apparently, the team behind the CentOS 8 decided to split the rsync functionality to two packages – one for the client-side and for the server-side, despite the binary rsync is only one and offers the client-size and server-side.
So there two packages in CentOS 8:

  1. rsync – provides the client-side and server-side as usual
  2. rsync-daemon – provides configuration example file and the systemd to start it as a service.

So if you wonder where is your rsync service after installing the rsync package under CentOS 8 you must install additional package “rsync-daemon”.

Of course, you may just create anywhere “rsyncd.conf” (the best place for the configuration file is in /etc, but could be placed anywhere with “–config=PATH/FILE” option) file and start the daemon as usual with “–daemon” option included to have the rsync server-side up and running.

rsync --daemon --config=/etc/rsyncd.conf

Just create yourself the configuration “/etc/rsyncd.conf” file.

Install the rsync program – the client and the daemon

dnf install rsync

Install the configuration and systemd files

dnf install rsync-daemon

rsync and rsync-daemon files

The files included in the two packages:

[root@srv ~]# dnf repoquery -l rsync-daemon
Last metadata expiration check: 0:33:02 ago on Wed 22 Jan 2020 02:57:06 PM UTC.
/etc/rsyncd.conf
/etc/sysconfig/rsyncd
/usr/lib/systemd/system/rsyncd.service
/usr/lib/systemd/system/rsyncd.socket
/usr/lib/systemd/system/rsyncd@.service
/usr/share/man/man5/rsyncd.conf.5.gz
[root@srv ~]# dnf repoquery -l rsync
Last metadata expiration check: 0:33:06 ago on Wed 22 Jan 2020 02:57:06 PM UTC.
/usr/bin/rsync
/usr/lib/.build-id
/usr/lib/.build-id/6e
/usr/lib/.build-id/6e/bdbf5032a966a6293cea003621ba766ab36ff1
/usr/share/doc/rsync
/usr/share/doc/rsync/NEWS
/usr/share/doc/rsync/OLDNEWS
/usr/share/doc/rsync/README
/usr/share/doc/rsync/support
/usr/share/doc/rsync/support/Makefile
/usr/share/doc/rsync/support/atomic-rsync
/usr/share/doc/rsync/support/cvs2includes
/usr/share/doc/rsync/support/deny-rsync
/usr/share/doc/rsync/support/file-attr-restore
/usr/share/doc/rsync/support/files-to-excludes
/usr/share/doc/rsync/support/git-set-file-times
/usr/share/doc/rsync/support/instant-rsyncd
/usr/share/doc/rsync/support/logfilter
/usr/share/doc/rsync/support/lsh
/usr/share/doc/rsync/support/lsh.sh
/usr/share/doc/rsync/support/mapfrom
/usr/share/doc/rsync/support/mapto
/usr/share/doc/rsync/support/mnt-excl
/usr/share/doc/rsync/support/munge-symlinks
/usr/share/doc/rsync/support/rrsync
/usr/share/doc/rsync/support/rsync-no-vanished
/usr/share/doc/rsync/support/rsync-slash-strip
/usr/share/doc/rsync/support/rsyncstats
/usr/share/doc/rsync/support/savetransfer.c
/usr/share/doc/rsync/tech_report.tex
/usr/share/licenses/rsync
/usr/share/licenses/rsync/COPYING
/usr/share/man/man1/rsync.1.gz