Tag: portage

Gentoo – bash: su: command not found – missing su flag

Upgrading multiple packages may lead to interesting results especially if the queue has not finished yet or the fails with an error! Apparently, there are two main ways to have the basic command SU in the system installed:

  1. sys-apps/shadow
  2. sys-apps/util-linux

At some point, the default inclusion of su flags in the above packages had changed from sys-apps/shadow to sys-apps/util-linux, which may lead to the following interesting error:

user@srv ~ $ su
bash: /bin/su: command not found

Just check, which of the above packages includes the su flag and re-emerge it. At present, sys-apps/util-linux includes it by default and it should work without any explicit activation in portage package use (i.e. /etc/portage/package.use/mybase or /etc/portage/make.conf, for example) file.

At the moment, here is the default:

user@srv ~ # emerge -vp shadow util-linux

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-apps/shadow-4.11.1:0/4::gentoo  USE="acl (audit) nls pam (selinux) (split-usr) xattr -bcrypt -cracklib -skey -su" 0 KiB
[ebuild   R    ] sys-apps/util-linux-2.37.2-r3::gentoo  USE="(audit) (caps) cramfs hardlink logger ncurses nls pam python readline (selinux) (split-usr) su suid udev (unicode) -build -cryptsetup -fdformat -kill -magic (-rtas) -slang -static-libs -systemd -test -tty-helpers" ABI_X86="32 (64) (-x32)" PYTHON_TARGETS="python3_8 -python3_9 -python3_10" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB

su flag is missing in sys-apps/shadow and is included in sys-apps/util-linux

Gentoo – UnicodeEncodeError: ‘ascii’ codec can’t encode character ‘\xc4’ in position 83: ordinal not in range(128)

Recently emerging package =dev-lang/go-1.13.4 under Gentoo failed on one of our virtual servers with:

--- /usr/lib/go/test/fixedbugs/issue27836.dir/
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/_MergeProcess.py", line 234, in _spawn
    prev_mtimes=self.prev_mtimes, counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 1788, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5385, in merge
    counter=counter)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4548, in treewalk
    rval = self._merge_contents(srcroot, destroot, cfgfiledict)
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 4828, in _merge_contents
    self.settings["EPREFIX"].lstrip(os.sep), cfgfiledict, mymtime):
  File "/usr/lib64/python3.6/site-packages/portage/dbapi/vartree.py", line 5225, in mergeme
    encoding=_encodings['merge'])
  File "/usr/lib64/python3.6/site-packages/portage/util/movefile.py", line 256, in movefile
    selinux.rename(src, dest)
  File "/usr/lib64/python3.6/site-packages/portage/__init__.py", line 246, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib64/python3.6/site-packages/portage/_selinux.py", line 71, in rename
    os.rename(src, dest)
UnicodeEncodeError: 'ascii' codec can't encode character '\xc4' in position 83: ordinal not in range(128)

>>> Failed to install dev-lang/go-1.13.4, Log file:

>>>  '/var/tmp/portage/dev-lang/go-1.13.4/temp/build.log'

The above log shows that the emerge fails in the installation phase when moving the files to the proper path in the system. The problem there is a non-ASCII character in the file name or path, but the environment is set to use ASCII as language.

The solution is to check if the environment LANG is set and what it contains. In this case, we should set the LANG environment to utf8.

export LANG=en_US.UTF-8

Most of the cases this kind of error could occur with virtual servers, docker (or the other kind of containers like lxc, podman and so on) containers and chroot jails or screens with changed user with su or sudo! In our case, the LANG just got missed because of a switch user procedure in a container and the emerge failed with the above error. When the LANG is missing probably the default value is “C”. In fact, check not only LANG but also the “LC_ALL” environment variable (it may have different value, which is wrong!), which also should be “en_US.UTF-8”:

export LC_ALL=en_US.UTF-8

Or try removing it at all with 

unset LC_ALL

Keep on reading!

emerge – ERROR: 17.1 migration has not been performed – upgrade from 13 to 17.1

When upgrading to 17.1 profile you need to perform additional steps to get to a healthy Gentoo portage system using profile 17.1.
Here what we stumbled when we tried to upgrade from 13 to 17.1:

>>> Emerging (1 of 3) dev-libs/elfutils-0.177::gentoo
 * Fetching files in the background.
 * To view fetch progress, run in another terminal:
 * tail -f /var/log/emerge-fetch.log
 * elfutils-0.177.tar.bz2 BLAKE2B SHA512 size ;-) ...                                                                                                                [ ok ]
 * Please follow the instructions in the news item:
 * 2019-06-05-amd64-17-1-profiles-are-now-stable
 * or choose the 17.0 profile.
 * ERROR: dev-libs/elfutils-0.177::gentoo failed (setup phase):
 *   ERROR: 17.1 migration has not been performed!!
 * 
 * Call stack:
 *        ebuild.sh, line 591:  Called __source_all_bashrcs
 *        ebuild.sh, line 410:  Called __try_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *        ebuild.sh, line 467:  Called __qa_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *        ebuild.sh, line 112:  Called source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *   profile.bashrc, line   6:  Called die
 * The specific snippet of code:
 *              die "ERROR: 17.1 migration has not been performed!!"
 * 
 * If you need support, post the output of `emerge --info '=dev-libs/elfutils-0.177::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-libs/elfutils-0.177::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/die.env'.
 * Working directory: '/var/tmp/portage/dev-libs/elfutils-0.177/homedir'
 * S: '/var/tmp/portage/dev-libs/elfutils-0.177/work/elfutils-0.177'

As you can see there is news, which consists of 12 steps to perform a successful upgrade to profile 17.1.
You can read the news with (the news output is included at the end of this article, just for completeness):

eselect news read 28

Specific note for our upgrade – in fact, our Gentoo installation was a chroot and it had not been updated for three years. The profile was 13.0 and portage supported only EAPI 5, which is a real problem. The new portage tree (we installed the latest) has no profile 13.0 and our emerge was half broken. We have preliminary steps described in Failed to install sys-apps/portage – Called pkg_preinst – portage._compat_upgrade.default_locations

Because we upgrade from 13.0 to 17.1, first, we need to perform the 17.0 upgrade (check the original steps here – https://gentoo.org/support/news-items/2017-11-30-new-17-profiles.html)

STEP 1) Select the previous profile 17.0 temporarily.

srv ~ # eselect profile set default/linux/amd64/17.0

You won’t be able to build any GNU GCC and you cannot continue with the next step.

STEP 2) Upgrade to GNU GCC 6.4.0 or later.

Our current GNU GCC is really old – 4.9.3. The latest version is gcc-9.2.0-r1 and we are going to use it.
Keep on reading!

Failed to install sys-apps/portage – Called pkg_preinst – portage._compat_upgrade.default_locations

Recently there was a problem with one compute node servers, which Gentoo (in fact, the chroot environment had not been updated for 3 years) could not emerge the new portage because it kept failing in the pkg_preinst with:

srv ~ # USE="-rsync-verify" emerge -va '=sys-apps/portage-2.3.76::gentoo'
.....
.....
>>> Installing (1 of 1) sys-apps/portage-2.3.76::gentoo
 * checking 1999 files for package collisions
1000 files checked ...
^[[B>>> Merging sys-apps/portage-2.3.76 to /
/usr/bin/python2.7: No module named _compat_upgrade
 * ERROR: sys-apps/portage-2.3.76::gentoo failed (preinst phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line 133:  Called pkg_preinst
 *   environment, line 3238:  Called die
 * The specific snippet of code:
 *       env -u DISTDIR -u PORTAGE_OVERRIDE_EPREFIX -u PORTAGE_REPOSITORIES -u PORTDIR -u PORTDIR_OVERLAY PYTHONPATH="${D%/}${PYTHON_SITEDIR}${PYTHONPATH:+:${PYTHONPATH}}" "${PYTHON}" -m portage._compat_upgrade.default_locations || die;
 * 
 * If you need support, post the output of `emerge --info '=sys-apps/portage-2.3.76::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-apps/portage-2.3.76::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/environment'.
 * Working directory: '/var/tmp/portage/._portage_reinstall_.RKkxLW/pym'
 * S: '/var/tmp/portage/sys-apps/portage-2.3.76/work/portage-2.3.76'
!!! FAILED preinst: 1

>>> Failed to install sys-apps/portage-2.3.76, Log file:

>>>  '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'

Note this is the installation phase, where everything from the build process passes smoothly, but it fails to install it with some vague error of a dying line with a couple of python and portage environments. The problem is the current portage does not support EAPI 6 and 7 and some packages were unable to be installed!

And we fixed the issue by just using the method described in our previous article – Fix your broken or old portage in Gentoo, fix the emerge command
Cloning a new portage software and using it to emerge (install) a newer (or reinstall the current) portage in the system to fix the system’s portage and the emerge command:

cd /root
git clone https://anongit.gentoo.org/git/proj/portage.git
/root/portage/bin/emerge --sync
eselect profile set default/linux/amd64/17.1
/root/portage/bin/emerge -v portage

Now the latest and stable Gentoo profile is 17.1, so we set it to be sure we use it (in future this may change).
Keep on reading!

Fix your broken or old portage in Gentoo, fix the emerge command

Using Gentoo for servers and chroots it might happen to deal with old and not regularly updated Gentoo environment, which in some cases is completely non-updatable using the regular process with emerge command. And no emerge means no installs from the Gentoo packaging system. Or you maybe have a broken portage after failed update with missing emerge command or missing or broken portage modules

In general, this article is for those, who cannot update their “sys-apps/portage” or even the emerge command exits with errors and is of no use.

In cases such as:

  • Missing portage files (broken portage)
  • Broken emerge command (broken portage)
  • no update of new portage is possible – very very old portage, which does not support EAPI of all current “sys-apps/portage” (now is EAPI=5) and you do not have old portage ebuild files
  • Easy update the portage to the newest in an old Gentoo system.

A real world errors are included at the bottom for each case above.

You need just a working python 2.7.x and git to clone “https://anongit.gentoo.org/git/proj/portage.git” or at least some tool to download the portage package.

Unpack to a globally accessible directory, DO NOT USE “/root“, because emerge changes user to non-privileges and needs execute permission for other for the all the path to the directory of emerge. Look below for more details.

Our example is to repair a completely broken portage with error:

srv ~ # emerge -va portage
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/emerge", line 42, in <module>
    import portage
ImportError: No module named portage

STEP 1) Clone the portage package from the git or download the portage tarball.

If you do not have a git client you can always check the URL in the sys-app/portage ebuild and download the URL. Still, if you cannot find the tarball’s URL, you can just simply get a mirror such as https://mirror.leaseweb.com/gentoo/distfiles, wait for the loading of the page (12Mbytes, it’s huge, all Gentoo tarball files’ names are here) and grep in it by the name “portage”. Or you can check here https://dev.gentoo.org/~zmedico/portage/archives/. Download the latest tarball and unpack it in the /opt directory.
Using git is the easiest method, but it is the same with downloading and unpacking the tarball in the “/opt”.

srv opt # git clone https://anongit.gentoo.org/git/proj/portage.git
Cloning into 'portage'...
remote: Enumerating objects: 151654, done.
remote: Counting objects: 100% (151654/151654), done.
remote: Compressing objects: 100% (39915/39915), done.
remote: Total 151654 (delta 112542), reused 149864 (delta 111349)
Receiving objects: 100% (151654/151654), 22.38 MiB | 550.00 KiB/s, done.
Resolving deltas: 100% (112542/112542), done.
srv opt # cd portage/
srv portage # ls -al
total 232
drwxr-xr-x. 11 root root  4096 Oct  6 04:52 .
drwx------.  4 root root  4096 Oct  6 04:51 ..
drwxr-xr-x.  6 root root  4096 Oct  6 04:52 bin
drwxr-xr-x.  5 root root  4096 Oct  6 04:52 cnf
-rw-r--r--.  1 root root  1840 Oct  6 04:52 COPYING
-rw-r--r--.  1 root root  6646 Oct  6 04:52 DEVELOPING
drwxr-xr-x.  5 root root  4096 Oct  6 04:52 doc
-rw-r--r--.  1 root root   200 Oct  6 04:52 .editorconfig
drwxr-xr-x.  7 root root  4096 Oct  6 04:52 .git
-rw-r--r--.  1 root root    80 Oct  6 04:52 .gitignore
drwxr-xr-x.  4 root root  4096 Oct  6 04:52 lib
-rw-r--r--.  1 root root 18092 Oct  6 04:52 LICENSE
-rwxr-xr-x.  1 root root  1272 Oct  6 04:52 make.conf.example-repatch.sh
drwxr-xr-x.  3 root root  4096 Oct  6 04:52 man
-rw-r--r--.  1 root root   290 Oct  6 04:52 MANIFEST.in
drwxr-xr-x.  2 root root  4096 Oct  6 04:52 misc
-rw-r--r--.  1 root root 17845 Oct  6 04:52 NEWS
-rw-r--r--.  1 root root     0 Oct  6 04:52 .portage_not_installed
-rw-r--r--.  1 root root  2246 Oct  6 04:52 README
-rw-r--r--.  1 root root 68122 Oct  6 04:52 RELEASE-NOTES
drwxr-xr-x.  6 root root  4096 Oct  6 04:52 repoman
-rwxr-xr-x.  1 root root  4889 Oct  6 04:52 runtests
-rwxr-xr-x.  1 root root 19743 Oct  6 04:52 setup.py
drwxr-xr-x.  2 root root  4096 Oct  6 04:52 src
-rwxr-xr-x.  1 root root    81 Oct  6 04:52 tabcheck.py
-rw-r--r--.  1 root root  1864 Oct  6 04:52 TEST-NOTES
-rw-r--r--.  1 root root   571 Oct  6 04:52 testpath
-rw-r--r--.  1 root root   348 Oct  6 04:52 tox.ini
-rw-r--r--.  1 root root   584 Oct  6 04:52 .travis.yml

This is the latest portage with working emerge command in the bin sub-directory.
Keep on reading!

portage is blocked by “the current version of portage supports EAPI ‘6’. You must upgrade”

We’ve synced the portage tree before upgrading our old portage package (big mistake! always upgrade the portage package before sync) and then do the sync. After the sync the portage upgrade was impossible, because a dependency package supported only a new portage API (probably a new package) in our case EAPI 7 and the offensive package was “app-eselect/eselect-pinentry“.
So there are two options:

  1. Find the last version of the portage,which does not depend on the package – app-eselect/eselect-pinentry
  2. Find if some of the USE flags disable the inclusion of this dependency – app-eselect/eselect-pinentry

We chose the second option and found that if we compiled the portage package with

-rsync-verify

the portage did not pull the dependency “app-eselect/eselect-pinentry” and then after a successful upgrade we had the portage supported EAPI 7 and reinstalled it with activated “-rsync-verify”.
Keep on reading!