Tag: emerge

Gentoo ERROR: Python module pytevent of version 0.10.2 not found, and bundling disabled

Emerging the sys-libs/ldb-2.3.0-r1 package may fail with an error for a missing Python mode, despite the sys-libs/tevent with a python USE flag is presented in the system:

Checking for system tevent (>=0.10.2)                                                           : yes 
ERROR: Python module pytevent of version 0.10.2 not found, and bundling disabled
 * ERROR: sys-libs/ldb-2.3.0-r1::gentoo failed (configure phase):
 *   configure failed
 * 
 * Call stack:

Indeed, the tevent (>=0.10.2) is found, but not the Python module! And the checking pase of the setup fails.
First, check whether the USE of sys-libs/tevent has python and the right version PYTHON_SINGLE_TARGET=”python3_8″ is used (the Python version may vary here):

root@srv # emerge -pv =tevent-0.10.2::gentoo

[ebuild   R    ] sys-libs/tevent-0.10.2::gentoo  USE="python" ABI_X86="32 (64) (-x32)" PYTHON_SINGLE_TARGET="python3_8 -python3_9" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

This system uses Python 3.8 and the library sys-libs/tevent was built with this USE flag.

The problem here is the tevent is installed under a its own directory: /usr/lib64/tevent. Using the ldconig utility the problem quickly has been resolved. Just add a file /etc/ld.so.conf.d/tevent_ldb.conf with the path to the library and then regenerate the ldconfig:

root@srv # cat /etc/ld.so.conf.d/tevent_ldb.conf
/usr/lib64/tevent
root@srv # ldconfig
root@srv #

Do not forget to run the “ldconfig“, because tevent library won’t be added to the LD cache. Emerging the sys-libs/ldb and then samba was successful after this quick workaround! There is a Gentoo bug reported, but the problem still exists – https://bugs.gentoo.org/590026

Gentoo emerge virtualbox- Mesa / GLU: Mesa not found at, Mesa headers not found

Emerging the package app-emulation/virtualbox the following error occurs:

Checking for Mesa / GLU: 
  Mesa not found at -L/usr/X11R6/lib -L/usr/X11R6/lib64 -L/usr/local/lib -lXext -lX11 -lGL -I/usr/local/include or Mesa headers not found
  Check the file /var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/configure.log for detailed error information.
Check /var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/configure.log for details
 * ERROR: app-emulation/virtualbox-6.1.18::gentoo failed (configure phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_configure
 *   environment, line 5504:  Called doecho './configure' '--with-gcc=x86_64-pc-linux-gnu-gcc' '--with-g++=x86_64-pc-linux-gnu-g++' '--disable-dbus' '--disable-kmods' '--disable-alsa' '--disable-docs' '--disable-devmapper' '--disable-pulse' '--disable-python' '--enable-webservice' '--enable-vnc'
 *   environment, line 1538:  Called die
 * The specific snippet of code:
 *       "$@" || die

The configure script reports the mesa is missing, but the package media-libs/mesa is installed. Reinstalling does not fix the problem.
Farther investigation in the logs by checking the configure.log reveals the real problem:

srv ~ # tail -n 16 /var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/configure.log
***** Checking Mesa / GLU *****
compiling the following source file:
#include <cstdio>
#include <X11/Xlib.h>
#include <GL/glx.h>
#include <GL/glu.h>
extern "C" int main(void)
{
  return 0;
}
using the following command line:
x86_64-pc-linux-gnu-g++  -fPIC -g -O -Wall -o /var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/.tmp_out /var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/.tmp_src.cc "-L/usr/X11R6/lib -L/usr/X11R6/lib64 -L/usr/local/lib -lXext -lX11 -lGL -I/usr/local/include"
/var/tmp/portage/app-emulation/virtualbox-6.1.18/work/VirtualBox-6.1.18/.tmp_src.cc:4:10: fatal error: GL/glu.h: No such file or directory
    4 | #include <GL/glu.h>
      |          ^~~~~~~~~~
compilation terminated.

The glu part of mesa is missing. In Gentoo, the glu (https://gitlab.freedesktop.org/mesa/glu) is not included in the media-libs/mesa and it is a separate package media-libs/glu.

The solution is to emerge media-libs/glu and then the app-emulation/virtualbox.

emerge -v media-libs/glu

Another Linux distribution may include glu in the main mesa package.

Here, the conclusion is to always check the configure.log, because it reports the exact error and not to trust the generic output of the configure script.

Gentoo emerge GO lang failed – atomic_amd64x.go: too many errors

Upgrading GO lang under Gentoo maybe a little bit tricky. The upgrading go lang from 1.13.7 to 1.15.5 failed with strange error:

# runtime/internal/atomic
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:18:6: Load redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:16:24
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:24:6: Loadp redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:22:32
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:30:6: Load64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:28:26
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:36:6: LoadAcq redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:34:27
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:41:6: Xadd redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:39:37
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:44:6: Xadd64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:42:39
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:47:6: Xadduintptr redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:45:47
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:50:6: Xchg redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:48:36
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:53:6: Xchg64 redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:51:38
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: Xchguintptr redeclared in this block
        previous declaration at /usr/lib/go/src/runtime/internal/atomic/atomic_amd64.go:54:45
/usr/lib/go/src/runtime/internal/atomic/atomic_amd64x.go:56:6: too many errors

Googling a little bit and it appeared there might be a conflict with the old version in the same directory. Deleting the temporary build directory didn’t help…

Removing the GO lang package by unmerge command and then emerging the newest GO lang package is successful.

So the solution is to unmerge it and then immediately emerge the newest version with:

emerge -vC dev-lang/go
emerge -v dev-lang/go

The whole Gentoo output of the failed emerge command

Keep on reading!

Patch and resume compilation of a failed package in Gentoo – ebuild, local repository or ctrl+Z

A dependency package failed to compile throwing error and existing the emerge of a queue with a hundred and more packages. Or worse you installed a new version of a package and multiple rebuilds are pulled, but one of the dependencies fails and you may end up with a broken system? What can you do? There is no new version of the failed package and yes, there is a bug in the Gentoo’s Bugzilla – https://bugs.gentoo.org/. And there is a solution with a patch, which has not made its way to the production and in Gentoo portage yet.
The package in the portage is broken, no new fixed package is released, but there is a patch to fix your issue. Here is what you can do:

  • Make your own package with the fixed version of the original package and put it in your local repository (not the official one, because on every emerge –sync it will be deleted). You should make a local repository and put the ebuild and all necessary files.
  • Or just download the patch and patch the source in the directory, which still holds the source of the failed package and resume the compilation manually. Then install it. Using this tutorial – Resume installation after a package build error, when emerging firefox under Gentoo
  • Just after the uncompress operation of the emerge press CTRL+Z to put the operation in the background and download and patch. Then bring back the emerge from the background with “fg” command.

The second and third options are not permanent solutions, but they are fast enough to be used in some situations.
Here are steps for the first and second option you may have:

OPTION 1) Make your own package.

Create a local repository (for details Simple steps to create Gentoo custom repository and add a package): 

root@srv ~ # mkdir -p /var/db/repos/my-local-portage/{metadata,profiles}
root@srv ~ # cat  << 'EOF' > /var/db/repos/my-local-portage/metadata/layout.conf
masters = gentoo
auto-sync = false
EOF
root@srv ~ # cat  << 'EOF' > /etc/portage/repos.conf/my-local-portage.conf
[my-local-portage]
location = /var/db/repos/my-local-portage 
EOF
root@srv ~ # cat  << 'EOF' > /var/db/repos/my-local-portage/profiles/repo_name
my-local-portage
EOF

Copy the ebuild file of the package you want to modify in the custom repository directory created above (it’s a good idea to copy all the sub-directories, too):
Keep on reading!

more than the default 4 parallel processes using distributed compiling with distcc

Distributed compilation could greatly speed the build process of Gentoo packages (and not only Gentoo, of course). If you tend to use Gentoo on a laptop or a relatively old CPU you may want to build packages distributively across multiple hosts.
Different (Linux) distributions use different configurations and environment scheme and sometimes it is difficult to sift the configuration, which could be applied to your setup. This is not a tutorial on how to enable parallel processing in Gentoo but it is just our client-site setup.

By default, there is a limit of 4 parallel processes, which is utterly insufficient, because nowadays most servers have more than 8 cores/logical compute units (not to mention that probably most would have 16 and above cores compute units).

The environment variable DISTCC_HOSTS controls, which hosts will receive files for the compilation of what they support and what is the limit of parallel processes.

In Gentoo we set this variable in the /etc/portage/make.conf. Here what you may include in make.conf to have 16 parallel remote processes and up to maximum 4 local (if the remote fails):

MAKEOPTS="-j16 -l4"
FEATURES="distcc"
DISTCC_HOSTS="192.168.0.101/16"

We use the environment DISTCC_HOSTS (here in Gentoo put in the make.conf, but in another Linux distribution an environment variable with this name should be set) because it is easy to set up and control globally for the Gentoo emerge system.
According to the documents:

In order, distcc looks in the $DISTCC_HOSTS environment variable, the user’s $DISTCC_DIR/hosts file, and the system-wide host file.

So when using emerge to build the packages, the emerge will rely on $DISTCC_HOSTS in make.conf (/etc/portage/make.conf or /etc/make.conf if you still use the old path), “/var/tmp/portage/.distcc/” (the build process uses “portage” user and group, not root!) and “/etc/distcc/hosts”. The first option used in the order above will be set the hosts and the limitation for the distributed processing. So if you use $DISTCC_HOSTS in make.conf (or environment) you wouldn’t need to set the “hosts” file.
Separate the different hosts with white space if you have more than one and always use the notation “/LIMIT” for each host. The default value is only 4 parallel processes (i.e it is implicitly added /4 to each hosts in the configuration!)
Keep on reading!

env python3.5: no such file or directory

The dependency conflicts may require a lot of time to resolve. On the contrary, uninstalling old software is easy, but it might lead to multiple broken programs and strange errors. Here is one of them, after removing the old python 3.4 and 3.5 (or whatever version it prints in your case) under Gentoo despite the rebuilding all dependencies as required by the emerge command for the new python 3.6 an error occurred trying to emerge the “app-misc/geoclue-2.5.3-r2”

[20/75] /usr/lib/python-exec/python3.6/meson --internal exe --unpickle /var/tmp/portage/app-misc/geoclue-2.5.3-r2/work/geoclue-2.5.3-build/meson-private/meson_exe_glib-mkenums_af4feac41c42ec7289410f8b20070c0528a3a7c0.dat
FAILED: public-api/gclue-enum-types.c 
/usr/lib/python-exec/python3.6/meson --internal exe --unpickle /var/tmp/portage/app-misc/geoclue-2.5.3-r2/work/geoclue-2.5.3-build/meson-private/meson_exe_glib-mkenums_af4feac41c42ec7289410f8b20070c0528a3a7c0.dat
/usr/bin/env: âpython3.5â: No such file or directory
ninja: build stopped: subcommand failed.
 * ERROR: app-misc/geoclue-2.5.3-r2::gentoo failed (compile phase):
 *   ninja -v -j15 -l14 -C /var/tmp/portage/app-misc/geoclue-2.5.3-r2/work/geoclue-2.5.3-build failed
 * 
 * Call stack:
 *     ebuild.sh, line  125:  Called src_compile
 *   environment, line 2847:  Called meson_src_compile
 *   environment, line 1904:  Called eninja '-C' '/var/tmp/portage/app-misc/geoclue-2.5.3-r2/work/geoclue-2.5.3-build'
 *   environment, line 1274:  Called die
 * The specific snippet of code:
 *       "$@" || die "${nonfatal_args[@]}" "${*} failed"
 *

Despite the lines starting with “/usr/lib/python-exec/python3.6/meson”, there is an error for missing “python 3.5“.

Here is another example, just trying to execute the distcc-config and it immediately throws the error for missing “python 3.5“.

root@srv ~ # distcc-config 
/usr/bin/env: ‘python3.5’: No such file or directory

The python script distcc-config starts with:

root@srv ~ $ head /usr/bin/distcc-config
#!/usr/bin/env python3.5
# Copyright 1999-2018 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

import os, re, signal, subprocess, sys

options=[
        '--get-hosts',
        '--set-hosts',
        '--get-verbose',

Apparently, multiple programs use this technic to explicitly put a specific version of python in the header of the python script to be used. Thus grantees the version in the header will be used even you change the system or user-preferred Python interpreter.

But uninstalling this python version all these programs will stop working till you rebuild them with emerge.
Keep on reading!

display packages from the Gentoo emerge resume list

A quick tip and an easy hack to see the emerge resume list, which will be used by the “emerge –resume” command. There are two resume lists – resume and resume_backup in the binary file /var/cache/edb/mtimedb.
The first two commands are python 3+.
The resume and resume_backup lists

python -c 'import portage; print(portage.mtimedb.get("resume", {}).get("mergelist"))'
python -c 'import portage; print(portage.mtimedb.get("resume_backup", {}).get("mergelist"))'

If you need the command for the python 2 you should use the following:

python -c 'import portage; print portage.mtimedb.get("resume", {}).get("mergelist")'
python -c 'import portage; print portage.mtimedb.get("resume_backup", {}).get("mergelist")'

Real world example

Resume list for

desktop ~ # emerge -v --nodeps $(qlist -IC|grep dev-qt)

These are the packages that would be merged, in order:

[ebuild     U  ] dev-qt/designer-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="declarative webkit -debug -test" 8,493 KiB
[ebuild     U  ] dev-qt/linguist-tools-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -test" 0 KiB
[ebuild     U  ] dev-qt/qdbus-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtbluetooth-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -qml -test" 2,727 KiB
[ebuild   R    ] dev-qt/qtchooser-66::gentoo  USE="-test" 32 KiB
[ebuild     U  ] dev-qt/qtconcurrent-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 48,549 KiB
[ebuild     U  ] dev-qt/qtcore-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="icu -debug (-systemd) -test" 0 KiB
[ebuild     U  ] dev-qt/qtdbus-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtdeclarative-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="jit widgets -debug -gles2 -localstorage -test" 20,753 KiB
[ebuild     U  ] dev-qt/qtgraphicaleffects-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 13,713 KiB
[ebuild     U  ] dev-qt/qtgui-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="accessibility dbus egl evdev gif ibus jpeg libinput png udev vnc wayland%* xcb -debug -eglfs -gles2 -test -tslib -tuio" 0 KiB
[ebuild     U  ] dev-qt/qthelp-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtimageformats-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="mng -debug -test (-jpeg2k%)" 1,768 KiB
[ebuild     U  ] dev-qt/qtlocation-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 5,974 KiB
[ebuild   R    ] dev-qt/qtlockedfile-2.4.1_p20171024::gentoo  USE="-doc" 694 KiB
[ebuild     U  ] dev-qt/qtmultimedia-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="alsa gstreamer openal pulseaudio qml widgets -debug -gles2 -test" 3,671 KiB
[ebuild     U  ] dev-qt/qtnetwork-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="libproxy networkmanager ssl -bindist -connman -debug -sctp -test" 0 KiB
[ebuild     U  ] dev-qt/qtopengl-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtpaths-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtpositioning-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -geoclue -test" 0 KiB
[ebuild     U  ] dev-qt/qtprintsupport-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="cups -debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtquickcontrols2-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test -widgets" 7,947 KiB
[ebuild     U  ] dev-qt/qtquickcontrols-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="widgets -debug -test" 5,843 KiB
[ebuild     U  ] dev-qt/qtscript-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="jit scripttools -debug -test" 2,584 KiB
[ebuild     U  ] dev-qt/qtsensors-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -qml -test" 1,996 KiB
[ebuild   R    ] dev-qt/qtsingleapplication-2.6.1_p20171024::gentoo  USE="X -doc" 0 KiB
[ebuild     U  ] dev-qt/qtspeech-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 99 KiB
[ebuild     U  ] dev-qt/qtsql-5.14.0:5/5.14.0::gentoo [5.12.2:5/5.12.2::gentoo] USE="mysql sqlite -debug -freetds -oci8 -odbc -postgres -test" 0 KiB
[ebuild     U  ] dev-qt/qtsvg-5.14.0-r1:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 1,822 KiB
[ebuild     U  ] dev-qt/qttest-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qttranslations-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 1,302 KiB
[ebuild     U  ] dev-qt/qtvirtualkeyboard-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="spell xcb -debug -handwriting -test" 10,705 KiB
[ebuild     U  ] dev-qt/qtwayland-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="libinput xcomposite -debug -test" 532 KiB
[ebuild     U  ] dev-qt/qtwebchannel-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="qml -debug -test" 192 KiB
[ebuild     U  ] dev-qt/qtwebengine-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="alsa pulseaudio system-icu widgets -bindist -debug -designer -jumbo-build -pax_kernel -system-ffmpeg -test (-geolocation%*)" 235,904 KiB
[ebuild     U  ] dev-qt/qtwebkit-5.212.0_pre20190629:5/5.212::gentoo [5.212.0_pre20180120:5/5.212::gentoo] USE="X geolocation hyphen jit multimedia opengl printsupport qml -gles2 -gstreamer -nsplugin -orientation -webp" 12,166 KiB
[ebuild     U  ] dev-qt/qtwidgets-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="gtk png xcb -debug -gles2 -test" 0 KiB
[ebuild     U  ] dev-qt/qtx11extras-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 125 KiB
[ebuild     U  ] dev-qt/qtxml-5.14.0:5/5.14::gentoo [5.12.2:5/5.12::gentoo] USE="-debug -test" 0 KiB
[ebuild     U  ] dev-qt/qtxmlpatterns-5.14.0:5/5.14::gentoo [5.11.1:5/5.11::gentoo] USE="-debug -qml% -test" 1,362 KiB

Total: 40 packages (37 upgrades, 3 reinstalls), Size of downloads: 388,941 KiB


>>> Verifying ebuild manifests
>>> Running pre-merge checks for dev-qt/qtwebkit-5.212.0_pre20190629

>>> Emerging (1 of 40) dev-qt/designer-5.14.0::gentoo
 * Fetching files in the background.
 * To view fetch progress, run in another terminal:
 * tail -f /var/log/emerge-fetch.log
 * qttools-everywhere-src-5.14.0.tar.xz BLAKE2B SHA512 size ;-) ...                                                                                                  [ ok ]
!!! Failed to set new SELinux execution context. Is your current SELinux context allowed to run Portage?
>>> Unpacking source...
>>> Unpacking qttools-everywhere-src-5.14.0.tar.xz to /var/tmp/portage/dev-qt/designer-5.14.0/work
^C

Exiting on signal 2
sandbox:stop  caught signal 2 in pid 22886
sandbox:stop  Send signal 4 more times to force SIGKILL
Sandboxed process killed by signal: Interrupt
^C * The ebuild phase 'die_hooks' has been killed by signal 2.

 * Messages for package dev-qt/designer-5.14.0:

We interrupted it on purpose with Ctrl+C.

The output of the above python command:

desktop ~ # python -c 'import portage; print(portage.mtimedb.get("resume", {}).get("mergelist"))'
[['ebuild', '/', 'dev-qt/designer-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/linguist-tools-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qdbus-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtbluetooth-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtchooser-66', 'merge'], ['ebuild', '/', 'dev-qt/qtconcurrent-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtcore-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtdbus-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtdeclarative-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtgraphicaleffects-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtgui-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qthelp-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtimageformats-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtlocation-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtlockedfile-2.4.1_p20171024', 'merge'], ['ebuild', '/', 'dev-qt/qtmultimedia-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtnetwork-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtopengl-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtpaths-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtpositioning-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtprintsupport-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qtquickcontrols2-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtquickcontrols-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtscript-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsensors-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsingleapplication-2.6.1_p20171024', 'merge'], ['ebuild', '/', 'dev-qt/qtspeech-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsql-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtsvg-5.14.0-r1', 'merge'], ['ebuild', '/', 'dev-qt/qttest-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qttranslations-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtvirtualkeyboard-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwayland-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebchannel-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebengine-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtwebkit-5.212.0_pre20190629', 'merge'], ['ebuild', '/', 'dev-qt/qtwidgets-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtx11extras-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtxml-5.14.0', 'merge'], ['ebuild', '/', 'dev-qt/qtxmlpatterns-5.14.0', 'merge']]
desktop ~ # python -c 'import portage; print(portage.mtimedb.get("resume_backup", {}).get("mergelist"))'
None

gentoo emerge resume – Invalid resume list and clean the list

Under Gentoo and emerge it will eventually end up using “–resume” paired with “–skipfirst” emerge command, which should resume last list of packages to build without the first one. Each execution of emerge command should reset the list but apparently it might not be the case (in fact two lists are kept – resume and resume_backup, read below).
The command just reads a list of packages and emerges them – it’s fast and it does not go through the whole process of dependancy check! But sometimes the list could be old or get corrupted or it just throws errors for “Invalid resume list“:

dekstop ~ # emerge --resume --skipfirst

These are the packages that would be merged, in order:

Calculating dependencies... done!
 * Invalid resume list:
 * 
 *   ('ebuild', '/', 'kde-frameworks/kpty-5.66.0', 'merge')
 *   ('ebuild', '/', 'kde-plasma/libkscreen-5.17.5', 'merge')
 *   ('ebuild', '/', 'kde-apps/kmbox-19.12.1', 'merge')
 *   ('ebuild', '/', 'kde-frameworks/kdesignerplugin-5.66.0', 'merge')
 *   ('ebuild', '/', 'kde-plasma/kwayland-integration-5.17.5', 'merge')
 *   ('ebuild', '/', 'kde-plasma/xembed-sni-proxy-5.17.5', 'merge')
 *   ('ebuild', '/', 'net-libs/accounts-qml-0.7-r1', 'merge')
 *   ('ebuild', '/', 'kde-apps/akonadi-notes-19.12.1', 'merge')
....
....
 * One or more packages are either masked or have missing dependencies:
 * 
 *   dev-lang/python:3.4[xml(+)] pulled in by:
 *     (dev-python/setuptools-40.6.2:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.5[xml(+)] pulled in by:
 *     (dev-python/setuptools-40.6.2:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.4 pulled in by:
 *     (dev-python/certifi-2018.8.24:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.5 pulled in by:
 *     (dev-python/certifi-2018.8.24:0/0::gentoo, installed)
 * 
 *   dev-libs/openssl:0/0= pulled in by:
 *     (dev-lang/python-2.7.17-r1:2.7/2.7::gentoo, installed)
 * 
 *   dev-lang/python:3.5 pulled in by:
 *     (dev-util/glib-utils-2.56.2:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.4 pulled in by:
 *     (sys-apps/file-5.35:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.5 pulled in by:
 *     (sys-apps/file-5.35:0/0::gentoo, installed)
 * 
 *   >=dev-libs/icu-58.1:0/60.2= pulled in by:
 *     (dev-lang/spidermonkey-52.9.1_pre1:52/52::gentoo, installed)
....
....
 *   dev-lang/python:3.4 pulled in by:
 *     (virtual/python-enum34-1:0/0::gentoo, installed)
 * 
 *   dev-lang/python:3.5 pulled in by:
 *     (virtual/python-enum34-1:0/0::gentoo, installed)
 * 
 *   dev-libs/openssl:0/0= pulled in by:
 *     (dev-lang/python-3.7.6:3.7/3.7m::gentoo, installed)
 * 
 * The resume list contains packages that are either masked or have
 * unsatisfied dependencies. Please restart/continue the operation
 * manually, or use --skipfirst to skip the first package in the list and
 * any other packages that may be masked or have missing dependencies.

Not all packages are included above, but it is clear that the list includes packages not only from the last emerge command. The list includes multiple old packages and dependencies about python:3.4, python:3.5, openssl:0/0.
The list is located in the binary file: /var/cache/edb/mtimedb and to clean the list you should execute:

desktop ~ # emaint --fix cleanresume
Emaint: fix cleanresume    100% [============================================>]

It is safe to the directory, too:

rm -Rf /var/cache/edb/

emaint is part of the sys-apps/portage package, so the program should always be available in the system.

*Bonus 1 – old packages

There is one more problem in the above example! In fact, you may clear the resume list with emaint or remove the whole directory “/var/cache/edb”, but still to receive such errors let’s say resuming log lists like KDE, gnome, qt packages (packages with many dependencies).

This problem persists not only with “–resume”, but also with “–keep-going” option!

When using “–keep-going” option you’ll get the same errors after the line “One or more packages are either masked or have missing dependencies”. So it is not an option, which may replace the “–resume” option.

Even all dependencies were resolved when the emerge list was generated when resuming a list of packages may throw errors for packages, which pull old packages because they had not been rebuilt against the updated of some of their dependencies. For example, simple enough from the above emerge resume command:
Some packages of our KDE resume list depends on the “dev-python/setuptools”, which has not been rebuilt against the new python (or after a removed old version python 3.4 and 3.5 or even another [old] package depends on the old python 3.5 and dev-python/setuptools, which has not been rebuilt after the python 3.4/3.5 removal!) and now this is a problem to resume the emerge list.

The solution is to take all those packages (after the line “One or more packages are either masked or have missing dependencies”) and rebuild or remove them.

Keep on reading!

libselinux – undefined reference to pcre_version

Emerging the “sys-libs/libselinux-2.9-r1” failed with this link errors of undefined references

/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_version':
regex.c:(.text+0x30): undefined reference to `pcre_version'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_writef':
regex.c:(.text+0x95): undefined reference to `pcre_fullinfo'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.c:(.text+0xf6): undefined reference to `pcre_fullinfo'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_data_free':
regex.c:(.text+0x1eb): undefined reference to `pcre_free'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.c:(.text+0x200): undefined reference to `pcre_free_study'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_prepare_data':
regex.c:(.text+0x26d): undefined reference to `pcre_compile'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.c:(.text+0x28f): undefined reference to `pcre_study'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_load_mmap':
regex.c:(.text+0x385): undefined reference to `pcre_fullinfo'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.c:(.text+0x3ed): undefined reference to `pcre_fullinfo'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_match':
regex.c:(.text+0x494): undefined reference to `pcre_exec'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.lo: in function `regex_cmp':
regex.c:(.text+0x502): undefined reference to `pcre_fullinfo'
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: regex.c:(.text+0x51a): undefined reference to `pcre_fullinfo'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:148: libselinux.so.1] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/var/tmp/portage/sys-libs/libselinux-2.9-r1/work/libselinux-2.9-abi_x86_32.x86/src'
make: *** [Makefile:44: all] Error 1
 * ERROR: sys-libs/libselinux-2.9-r1::gentoo failed (compile phase):
 *   emake failed
 *
 * If you need support, post the output of `emerge --info '=sys-libs/libselinux-2.9-r1::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-libs/libselinux-2.9-r1::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sys-libs/libselinux-2.9-r1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-libs/libselinux-2.9-r1/temp/environment'.
 * Working directory: '/var/tmp/portage/sys-libs/libselinux-2.9-r1/work/libselinux-2.9-abi_x86_32.x86'
 * S: '/var/tmp/portage/sys-libs/libselinux-2.9-r1/work/libselinux-2.9'

>>> Failed to emerge sys-libs/libselinux-2.9-r1, Log file:

>>>  '/var/tmp/portage/sys-libs/libselinux-2.9-r1/temp/build.log'

 * Messages for package sys-libs/libselinux-2.9-r1:

 * ERROR: sys-libs/libselinux-2.9-r1::gentoo failed (compile phase):
 *   emake failed
 *
 * If you need support, post the output of `emerge --info '=sys-libs/libselinux-2.9-r1::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-libs/libselinux-2.9-r1::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sys-libs/libselinux-2.9-r1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-libs/libselinux-2.9-r1/temp/environment'.
 * Working directory: '/var/tmp/portage/sys-libs/libselinux-2.9-r1/work/libselinux-2.9-abi_x86_32.x86'
 * S: '/var/tmp/portage/sys-libs/libselinux-2.9-r1/work/libselinux-2.9'

The solution was to rebuild the dev-libs/libpcre and dev-libs/libpcre2 libraries.

emerge -va dev-libs/libpcre dev-libs/libpcre2

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] dev-libs/libpcre-8.43:3::gentoo [8.42:3::gentoo] USE="bzip2 cxx jit pcre16 readline recursion-limit (split-usr%*) (static-libs) (unicode) zlib -libedit -pcre32" ABI_X86="32 (64) (-x32)" 1540 KiB
[ebuild     U  ] dev-libs/libpcre2-10.34::gentoo [10.32::gentoo] USE="bzip2 jit pcre16 pcre32* readline recursion-limit (split-usr%*) unicode zlib -libedit -static-libs" ABI_X86="32 (64) (-x32)" 1676 KiB

Total: 2 packages (2 upgrades), Size of downloads: 3216 KiB

if you encounter the error above even you are not using Gentoo probably the problem is your libpcre/libpcre2 library and try to rebuild it or include the proper path to the library and its headers.

Gentoo – UnicodeEncodeError: ‘ascii’ codec can’t encode character ‘\xc4’ in position 83: ordinal not in range(128)

Recently emerging package =dev-lang/go-1.13.4 under Gentoo failed on one of our virtual servers with:

--- /usr/lib/go/test/fixedbugs/issue27836.dir/
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/_MergeProcess.py", line 234, in _spawn
    prev_mtimes=self.prev_mtimes, counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 1788, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5385, in merge
    counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4548, in treewalk
    rval = self._merge_contents(srcroot, destroot, cfgfiledict)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4828, in _merge_contents
    self.settings["EPREFIX"].lstrip(os.sep), cfgfiledict, mymtime):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5225, in mergeme
    encoding=_encodings['merge'])
  File "/usr/lib64/python3.6/site-packages/portage/util/movefile.py", line 256, in movefile
    selinux.rename(src, dest)
  File "/usr/lib64/python3.6/site-packages/portage/__init__.py", line 246, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/_selinux.py", line 71, in rename
    os.rename(src, dest)
UnicodeEncodeError: 'ascii' codec can't encode character '\xc4' in position 83: ordinal not in range(128)

>>> Failed to install dev-lang/go-1.13.4, Log file:

>>>  '/var/tmp/portage/dev-lang/go-1.13.4/temp/build.log'

The above log shows that the emerge fails in the installation phase when moving the files to the proper path in the system. The problem there is a non-ASCII character in the file name or path, but the environment is set to use ASCII as language.

The solution is to check if the environment LANG is set and what it contains. In this case, we should set the LANG environment to utf8.

export LANG=en_US.UTF-8

Most of the cases this kind of error could occur with virtual servers, docker (or the other kind of containers like lxc, podman and so on) containers and chroot jails or screens with changed user with su or sudo! In our case, the LANG just got missed because of a switch user procedure in a container and the emerge failed with the above error. When the LANG is missing probably the default value is “C”. In fact, check not only LANG but also the “LC_ALL” environment variable (it may have different value, which is wrong!), which also should be “en_US.UTF-8”:

export LC_ALL=en_US.UTF-8

Or try removing it at all with 

unset LC_ALL

Keep on reading!