emerge – ERROR: 17.1 migration has not been performed – upgrade from 13 to 17.1

When upgrading to 17.1 profile you need to perform additional steps to get to a healthy Gentoo portage system using profile 17.1.
Here what we stumbled when we tried to upgrade from 13 to 17.1:

>>> Emerging (1 of 3) dev-libs/elfutils-0.177::gentoo
 * Fetching files in the background.
 * To view fetch progress, run in another terminal:
 * tail -f /var/log/emerge-fetch.log
 * elfutils-0.177.tar.bz2 BLAKE2B SHA512 size ;-) ...                                                                                                                [ ok ]
 * Please follow the instructions in the news item:
 * 2019-06-05-amd64-17-1-profiles-are-now-stable
 * or choose the 17.0 profile.
 * ERROR: dev-libs/elfutils-0.177::gentoo failed (setup phase):
 *   ERROR: 17.1 migration has not been performed!!
 * 
 * Call stack:
 *        ebuild.sh, line 591:  Called __source_all_bashrcs
 *        ebuild.sh, line 410:  Called __try_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *        ebuild.sh, line 467:  Called __qa_source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *        ebuild.sh, line 112:  Called source '/usr/portage/profiles/default/linux/amd64/17.1/profile.bashrc'
 *   profile.bashrc, line   6:  Called die
 * The specific snippet of code:
 *              die "ERROR: 17.1 migration has not been performed!!"
 * 
 * If you need support, post the output of `emerge --info '=dev-libs/elfutils-0.177::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-libs/elfutils-0.177::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-libs/elfutils-0.177/temp/die.env'.
 * Working directory: '/var/tmp/portage/dev-libs/elfutils-0.177/homedir'
 * S: '/var/tmp/portage/dev-libs/elfutils-0.177/work/elfutils-0.177'

As you can see there is news, which consists of 12 steps to perform a successful upgrade to profile 17.1.
You can read the news with (the news output is included at the end of this article, just for completeness):

eselect news read 28

Specific note for our upgrade – in fact, our Gentoo installation was a chroot and it had not been updated for three years. The profile was 13.0 and portage supported only EAPI 5, which is a real problem. The new portage tree (we installed the latest) has no profile 13.0 and our emerge was half broken. We have preliminary steps described in Failed to install sys-apps/portage – Called pkg_preinst – portage._compat_upgrade.default_locations

Because we upgrade from 13.0 to 17.1, first, we need to perform the 17.0 upgrade (check the original steps here – https://gentoo.org/support/news-items/2017-11-30-new-17-profiles.html)

STEP 1) Select the previous profile 17.0 temporarily.

srv ~ # eselect profile set default/linux/amd64/17.0

You won’t be able to build any GNU GCC and you cannot continue with the next step.

STEP 2) Upgrade to GNU GCC 6.4.0 or later.

Our current GNU GCC is really old – 4.9.3. The latest version is gcc-9.2.0-r1 and we are going to use it.
Keep on reading!

Failed to install sys-apps/portage – Called pkg_preinst – portage._compat_upgrade.default_locations

Recently there was a problem with one compute node servers, which Gentoo (in fact, the chroot environment had not been updated for 3 years) could not emerge the new portage because it kept failing in the pkg_preinst with:

srv ~ # USE="-rsync-verify" emerge -va '=sys-apps/portage-2.3.76::gentoo'
.....
.....
>>> Installing (1 of 1) sys-apps/portage-2.3.76::gentoo
 * checking 1999 files for package collisions
1000 files checked ...
^[[B>>> Merging sys-apps/portage-2.3.76 to /
/usr/bin/python2.7: No module named _compat_upgrade
 * ERROR: sys-apps/portage-2.3.76::gentoo failed (preinst phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line 133:  Called pkg_preinst
 *   environment, line 3238:  Called die
 * The specific snippet of code:
 *       env -u DISTDIR -u PORTAGE_OVERRIDE_EPREFIX -u PORTAGE_REPOSITORIES -u PORTDIR -u PORTDIR_OVERLAY PYTHONPATH="${D%/}${PYTHON_SITEDIR}${PYTHONPATH:+:${PYTHONPATH}}" "${PYTHON}" -m portage._compat_upgrade.default_locations || die;
 * 
 * If you need support, post the output of `emerge --info '=sys-apps/portage-2.3.76::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-apps/portage-2.3.76::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-apps/portage-2.3.76/temp/environment'.
 * Working directory: '/var/tmp/portage/._portage_reinstall_.RKkxLW/pym'
 * S: '/var/tmp/portage/sys-apps/portage-2.3.76/work/portage-2.3.76'
!!! FAILED preinst: 1

>>> Failed to install sys-apps/portage-2.3.76, Log file:

>>>  '/var/tmp/portage/sys-apps/portage-2.3.76/temp/build.log'

Note this is the installation phase, where everything from the build process passes smoothly, but it fails to install it with some vague error of a dying line with a couple of python and portage environments. The problem is the current portage does not support EAPI 6 and 7 and some packages were unable to be installed!

And we fixed the issue by just using the method described in our previous article – Fix your broken or old portage in Gentoo, fix the emerge command
Cloning a new portage software and using it to emerge (install) a newer (or reinstall the current) portage in the system to fix the system’s portage and the emerge command:

cd /root
git clone https://anongit.gentoo.org/git/proj/portage.git
/root/portage/bin/emerge --sync
eselect profile set default/linux/amd64/17.1
/root/portage/bin/emerge -v portage

Now the latest and stable Gentoo profile is 17.1, so we set it to be sure we use it (in future this may change).
Keep on reading!

Fix your broken or old portage in Gentoo, fix the emerge command

Using Gentoo for servers and chroots it might happen to deal with old and not regularly updated Gentoo environment, which in some cases is completely non-updatable using the regular process with emerge command. And no emerge means no installs from the Gentoo packaging system. Or you maybe have a broken portage after failed update with missing emerge command or missing or broken portage modules

In general, this article is for those, who cannot update their “sys-apps/portage” or even the emerge command exits with errors and is of no use.

In cases such as:

  • Missing portage files (broken portage)
  • Broken emerge command (broken portage)
  • no update of new portage is possible – very very old portage, which does not support EAPI of all current “sys-apps/portage” (now is EAPI=5) and you do not have old portage ebuild files
  • Easy update the portage to the newest in an old Gentoo system.

A real world errors are included at the bottom for each case above.

You need just a working python 2.7.x and git to clone “https://anongit.gentoo.org/git/proj/portage.git” or at least some tool to download the portage package.

Our example is to repair a completely broken portage with error:

srv ~ # emerge -va portage
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/emerge", line 42, in <module>
    import portage
ImportError: No module named portage

STEP 1) Clone the portage package from the git or download the portage tarball.

If you do not have a git client you can always check the URL in the sys-app/portage ebuild and download the URL. Still, if you cannot find the tarball’s URL, you can just simply get a mirror such as https://mirror.leaseweb.com/gentoo/distfiles, wait for the loading of the page (12Mbytes, it’s huge, all Gentoo tarball files’ names are here) and grep in it by the name “portage”. Or you can check here https://dev.gentoo.org/~zmedico/portage/archives/. Download the latest tarball and unpack it in the /root directory.
Using git is the easiest method, but is the same with downloading and unpacking the tarball in the “/root”.

srv root # git clone https://anongit.gentoo.org/git/proj/portage.git
Cloning into 'portage'...
remote: Enumerating objects: 151654, done.
remote: Counting objects: 100% (151654/151654), done.
remote: Compressing objects: 100% (39915/39915), done.
remote: Total 151654 (delta 112542), reused 149864 (delta 111349)
Receiving objects: 100% (151654/151654), 22.38 MiB | 550.00 KiB/s, done.
Resolving deltas: 100% (112542/112542), done.
srv root # cd portage/
srv portage # ls -al
total 232
drwxr-xr-x. 11 root root  4096 Oct  6 04:52 .
drwx------.  4 root root  4096 Oct  6 04:51 ..
drwxr-xr-x.  6 root root  4096 Oct  6 04:52 bin
drwxr-xr-x.  5 root root  4096 Oct  6 04:52 cnf
-rw-r--r--.  1 root root  1840 Oct  6 04:52 COPYING
-rw-r--r--.  1 root root  6646 Oct  6 04:52 DEVELOPING
drwxr-xr-x.  5 root root  4096 Oct  6 04:52 doc
-rw-r--r--.  1 root root   200 Oct  6 04:52 .editorconfig
drwxr-xr-x.  7 root root  4096 Oct  6 04:52 .git
-rw-r--r--.  1 root root    80 Oct  6 04:52 .gitignore
drwxr-xr-x.  4 root root  4096 Oct  6 04:52 lib
-rw-r--r--.  1 root root 18092 Oct  6 04:52 LICENSE
-rwxr-xr-x.  1 root root  1272 Oct  6 04:52 make.conf.example-repatch.sh
drwxr-xr-x.  3 root root  4096 Oct  6 04:52 man
-rw-r--r--.  1 root root   290 Oct  6 04:52 MANIFEST.in
drwxr-xr-x.  2 root root  4096 Oct  6 04:52 misc
-rw-r--r--.  1 root root 17845 Oct  6 04:52 NEWS
-rw-r--r--.  1 root root     0 Oct  6 04:52 .portage_not_installed
-rw-r--r--.  1 root root  2246 Oct  6 04:52 README
-rw-r--r--.  1 root root 68122 Oct  6 04:52 RELEASE-NOTES
drwxr-xr-x.  6 root root  4096 Oct  6 04:52 repoman
-rwxr-xr-x.  1 root root  4889 Oct  6 04:52 runtests
-rwxr-xr-x.  1 root root 19743 Oct  6 04:52 setup.py
drwxr-xr-x.  2 root root  4096 Oct  6 04:52 src
-rwxr-xr-x.  1 root root    81 Oct  6 04:52 tabcheck.py
-rw-r--r--.  1 root root  1864 Oct  6 04:52 TEST-NOTES
-rw-r--r--.  1 root root   571 Oct  6 04:52 testpath
-rw-r--r--.  1 root root   348 Oct  6 04:52 tox.ini
-rw-r--r--.  1 root root   584 Oct  6 04:52 .travis.yml

This is the latest portage with working emerge command in the bin sub-directory.
Keep on reading!

portage is blocked by “the current version of portage supports EAPI ‘6’. You must upgrade”

We’ve synced the portage tree before upgrading our old portage package (big mistake! always upgrade the portage package before sync) and then do the sync. After the sync the portage upgrade was impossible, because a dependency package supported only a new portage API (probably a new package) in our case EAPI 7 and the offensive package was “app-eselect/eselect-pinentry“.
So there are two options:

  1. Find the last version of the portage,which does not depend on the package – app-eselect/eselect-pinentry
  2. Find if some of the USE flags disable the inclusion of this dependency – app-eselect/eselect-pinentry

We chose the second option and found that if we compiled the portage package with

-rsync-verify

the portage did not pull the dependency “app-eselect/eselect-pinentry” and then after a successful upgrade we had the portage supported EAPI 7 and reinstalled it with activated “-rsync-verify”.
Keep on reading!