Mirror the official Ubuntu repositories using aptly

This article is to show mainly how to work with aptly by mirroring an official Ubuntu mirror. If you want to know how to install and a brief description of what is aptly you may want to read our previous article – Install aptly under Ubuntu 18 LTS with Nginx serving the packages and the first steps

What we are going to do – this is what you need to have a mirror of an external application repository:

  1. Install aptly in Ubuntu 18 LTS
  2. Create a mirror in aptly
  3. Create a snapshot of the mirror created before
  4. Publish the snapshot to be used in other servers.

and at the last step there is an example how to use the mirror in your local machines.

STEP 1) Install aptly in Ubuntu 18.04 LTS.

As mentioned already you may follow our article on the subject – Install aptly under Ubuntu 18 LTS with Nginx serving the packages and the first steps. The following steps are based on this installation!
The aptly home directory is in “/srv/aptly”. We use the “aptly” user and change to it to manipulate the aptly installation.

STEP 2) Create a mirror in aptly.

Prepare the keys (aptly needs to have the Ubuntu keys in its trustedkeys keyring):

aptly@srv:~$ gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg --export | gpg --no-default-keyring --keyring trustedkeys.gpg --import
gpg: key 3B4FE6ACC0B21F32: 3 signatures not checked due to missing keys
gpg: key 3B4FE6ACC0B21F32: public key "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>" imported
gpg: key D94AA3F0EFE21092: 3 signatures not checked due to missing keys
gpg: key D94AA3F0EFE21092: public key "Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>" imported
gpg: key 871920D1991BC93C: 1 signature not checked due to a missing key
gpg: key 871920D1991BC93C: public key "Ubuntu Archive Automatic Signing Key (2018) <ftpmaster@ubuntu.com>" imported
gpg: Total number processed: 3
gpg:               imported: 3
gpg: public key of ultimately trusted key 212A3D20E4D3351D not found
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u

Probably you would like to have “main” and “universe” for the three bionic, bionic updates and bionic security.
First, main and universe for bionic repository. main is ~16G and universe is ~136, these numbers will vary in future when more packages are added.
Two commands are need for the aptly mirror:

  1. create – create the mirror.
  2. update – download the repository contents locally.

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-main http://nl.archive.ubuntu.com/ubuntu/ bionic main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-main' to download repository contents.
aptly@srv:~$ aptly mirror update bionic-main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/main/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/main/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/main/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/main/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/main/source/Sources.gz...
Building download queue...
Download queue: 17044 items (16.13 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/b/binutils/binutils-s390x-linux-gnu_2.30-15ubuntu1_amd64.deb...
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/p/pymongo/python3-pymongo_3.6.1+dfsg1-1_amd64.deb...
.....
.....
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/l/language-pack-gnome-ml/language-pack-gnome-ml_18.04+20180423.dsc...
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/l/language-pack-gnome-ml/language-pack-gnome-ml_18.04+20180423.tar.xz...

Mirror `bionic-main` has been successfully updated.
aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-universe http://nl.archive.ubuntu.com/ubuntu/ bionic universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-universe' to download repository contents.
aptly@my-server-pc:~$ aptly mirror update bionic-universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/universe/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/universe/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/universe/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/universe/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/universe/source/Sources.gz...
Building download queue...
Download queue: 157819 items (136.58 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/p/python-libcharmstore/python3-libcharmstore_0.0.3-0ubuntu1_all.deb...
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/g/genometools/python-genometools_1.5.10+ds-2_all.deb...
.....
.....
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/h/haskell-microlens-th/haskell-microlens-th_0.4.1.1.orig.tar.gz...

Mirror `bionic-universe` has been successfully updated.

And we continue with the same commands for the boinic updates. Now bionic updates is 50.92 GiB and bionic universe is 26.73 GiB.

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-updates-main http://nl.archive.ubuntu.com/ubuntu/ bionic-updates main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease...
gpgv: Signature made Mon 06 Jan 2020 07:26:06 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-updates-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-updates-main' to download repository contents.
aptly@srv:~$ aptly mirror update bionic-updates-main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease...
gpgv: Signature made Mon 06 Jan 2020 07:26:06 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/main/source/Sources.gz...
Building download queue...
Download queue: 16286 items (50.92 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/p/pam/libpam0g-dev_1.1.8-3.6ubuntu2.18.04.1_i386.deb...
......
......
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/l/linux-azure/linux-cloud-tools-4.18.0-1020-azure_4.18.0-1020.20~18.04.1_amd64.deb...

Mirror `bionic-updates-main` has been successfully updated.

Then the bionic updates “universe”:

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-updates-universe http://nl.archive.ubuntu.com/ubuntu/ bionic-updates universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease...
gpgv: Signature made Tue 07 Jan 2020 03:12:11 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-updates-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-updates-universe' to download repository contents.
aptly@srv:~$ aptly mirror update bionic-updates-universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease...
gpgv: Signature made Tue 07 Jan 2020 03:12:11 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-updates/universe/source/Sources.gz...
Building download queue...
Download queue: 12123 items (26.73 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/g/gcc-6/libx32go9-dbg_6.5.0-2ubuntu1~18.04_i386.deb...
......
......
Mirror `bionic-updates-universe` has been successfully updated.

And the last one is boinc updates “main” (1.02G) and “universe” (0.5G).

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-security-main http://nl.archive.ubuntu.com/ubuntu/ bionic-security main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/InRelease...
gpgv: Signature made Tue 07 Jan 2020 07:06:11 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-security-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-security-main' to download repository contents.
aptly@srv:~$ aptly mirror update bionic-security-main
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/InRelease...
gpgv: Signature made Tue 07 Jan 2020 07:06:11 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/main/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/main/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/main/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/main/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/main/source/Sources.gz...
Building download queue...
Download queue: 740 items (1.02 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/l/linux-meta/linux-image-virtual-hwe-16.04_4.15.0.72.74_i386.deb...
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/main/c/ceph/python-rados_12.2.12-0ubuntu0.18.04.2_i386.deb..
.....
.....
Mirror `bionic-security-main` has been successfully updated.
aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-security-universe http://nl.archive.ubuntu.com/ubuntu/ bionic-security universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/InRelease...
gpgv: Signature made Wed 08 Jan 2020 03:12:37 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"

Mirror [bionic-security-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb] successfully added.
You can run 'aptly mirror update bionic-security-universe' to download repository contents.
aptly@srv:~$ aptly mirror update bionic-security-universe
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/InRelease...
gpgv: Signature made Wed 08 Jan 2020 03:12:37 AM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Good signature from "Ubuntu Archive Automatic Signing Key (2012) <ftpmaster@ubuntu.com>"
Downloading & parsing package files...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/universe/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/universe/debian-installer/binary-i386/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/universe/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/universe/debian-installer/binary-amd64/Packages.gz...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic-security/universe/source/Sources.gz...
Building download queue...
Download queue: 278 items (0.50 GiB)
Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/u/unbound/unbound-host_1.6.7-1ubuntu2.1_amd64.deb...
......
......

Mirror `bionic-security-universe` has been successfully updated.

Here are listing of our mirrors:

aptly@srv:~$ aptly mirror list
List of mirrors:
 * [bionic-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb]
 * [bionic-security-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb]
 * [bionic-security-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb]
 * [bionic-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb]
 * [bionic-updates-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb]
 * [bionic-updates-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb]

To get more information about mirror, run `aptly mirror show <name>`.

STEP 2) Create snapshots of the mirrors

Now create the snapshots of the synchronized mirrors. These snapshots will be published in the next step. We choose the name of the snapshot as the name of the mirror but you may want to create the snapshots with a date suffix.

aptly@srv:~$ aptly snapshot create bionic-main from mirror bionic-main

Snapshot bionic-main successfully created.
You can run 'aptly publish snapshot bionic-main' to publish snapshot as Debian repository.
aptly@srv:~$ aptly snapshot create bionic-universe from mirror bionic-universe

Snapshot bionic-universe successfully created.
You can run 'aptly publish snapshot bionic-universe' to publish snapshot as Debian repository.
aptly@srv:~$ aptly snapshot create bionic-updates-main from mirror bionic-updates-main

Snapshot bionic-updates-main successfully created.
You can run 'aptly publish snapshot bionic-updates-main' to publish snapshot as Debian repository.
aptly@srv:~$ aptly snapshot create bionic-updates-universe from mirror bionic-updates-universe

Snapshot bionic-updates-universe successfully created.
You can run 'aptly publish snapshot bionic-updates-universe' to publish snapshot as Debian repository.
aptly@srv:~$ aptly snapshot create bionic-security-main from mirror bionic-security-main

Snapshot bionic-security-main successfully created.
You can run 'aptly publish snapshot bionic-security-main' to publish snapshot as Debian repository.
aptly@srv:~$ aptly snapshot create bionic-security-universe from mirror bionic-security-universe

Snapshot bionic-security-universe successfully created.
You can run 'aptly publish snapshot bionic-security-universe' to publish snapshot as Debian repository.

The snapshot listing shows:

aptly@srv:~$ aptly snapshot list
List of snapshots:
 * [bionic-main]: Snapshot from mirror [bionic-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb]
 * [bionic-security-main]: Snapshot from mirror [bionic-security-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb]
 * [bionic-security-universe]: Snapshot from mirror [bionic-security-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-security [src] [udeb]
 * [bionic-universe]: Snapshot from mirror [bionic-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic [src] [udeb]
 * [bionic-updates-main]: Snapshot from mirror [bionic-updates-main]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb]
 * [bionic-updates-universe]: Snapshot from mirror [bionic-updates-universe]: http://nl.archive.ubuntu.com/ubuntu/ bionic-updates [src] [udeb]

To get more information about snapshot, run `aptly snapshot show <name>`.

STEP 3) Publish the snapshots of the official mirrors.

The last step to be able to use “the official mirrors” (in fact, you are going to use their snapshots and if you update them you will continue to use the repositories as they look before the update).
Bionic main and universe:

aptly@srv:~$ aptly publish snapshot -component=main,universe bionic-main bionic-universe ubuntu
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:

Snapshots bionic-main, bionic-universe have been successfully published.
Please setup your webserver to serve directory '$HOME/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
  deb http://your-server/ubuntu/ bionic main universe
  deb-src http://your-server/ubuntu/ bionic main universe
Don't forget to add your GPG key to apt with apt-key.

You can also use `aptly serve` to publish your repositories over HTTP quickly.

Bionic-updates main and universe:

aptly@srv:~$ aptly publish snapshot -component=main,universe bionic-updates-main bionic-updates-universe ubuntu
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:

Snapshots bionic-updates-main, bionic-updates-universe have been successfully published.
Please setup your webserver to serve directory '$HOME/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
  deb http://your-server/ubuntu/ bionic-updates main universe
  deb-src http://your-server/ubuntu/ bionic-updates main universe
Don't forget to add your GPG key to apt with apt-key.

You can also use `aptly serve` to publish your repositories over HTTP quickly.

Bionic-security main and universe:

aptly@srv:~$ aptly publish snapshot -component=main,universe bionic-security-main bionic-security-universe ubuntu
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:

Snapshots bionic-security-main, bionic-security-universe have been successfully published.
Please setup your webserver to serve directory '$HOME/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
  deb http://your-server/ubuntu/ bionic-security main universe
  deb-src http://your-server/ubuntu/ bionic-security main universe
Don't forget to add your GPG key to apt with apt-key.

You can also use `aptly serve` to publish your repositories over HTTP quickly.

STEP 4) Use the your repositories instead the official ones

To use them you first should import the key of your new repository and then add them to the Ubuntu’s configuration files. As mentioned in the beginning, we installed the aptly software following our previous article – Install aptly under Ubuntu 18 LTS with nginx serving the packages and the first steps, so the public key of our aptly server is located in the /srv/aptly/.aptly/public/key.pub accessed from the web from “https://aptly.example.com/key.pub“.
Login in your client machine and import the key:

root@srv2:~# wget http://apt.example.com/key.pub
root@srv2:~# apt-key add ./key.pub
OK

Then replace the official repositories in /etc/apt/sources.list with the new ones:

deb https://aptly.exmaple.com/ubuntu bionic main
deb https://aptly.exmaple.com/ubuntu bionic universe
deb https://aptly.exmaple.com/ubuntu bionic-updates main
deb https://aptly.exmaple.com/ubuntu bionic-updates universe
deb https://aptly.exmaple.com/ubuntu bionic-security main
deb https://aptly.exmaple.com/ubuntu bionic-security universe

And update to download the metafiles and upgrade or install packages. apt program will access only your local repositories.

root@srv2:~# apt update
Get:1 http://aptly.example.com/ubuntu bionic InRelease [22,4 kB]
Get:2 http://aptly.example.com/ubuntu bionic-updates InRelease [22,5 kB]
Get:3 http://aptly.example.com/ubuntu bionic-security InRelease [22,5 kB]
Get:4 http://aptly.example.com/ubuntu bionic/main i386 Packages [1472 kB]
Get:5 http://aptly.example.com/ubuntu bionic/main amd64 Packages [1494 kB]
Get:6 http://aptly.example.com/ubuntu bionic/universe i386 Packages [12,5 MB]        
Get:7 http://aptly.example.com/ubuntu bionic/universe amd64 Packages [12,5 MB]      
Get:8 http://aptly.example.com/ubuntu bionic-updates/main amd64 Packages [1301 kB]
Get:9 http://aptly.example.com/ubuntu bionic-updates/main i386 Packages [957 kB]
Get:10 http://aptly.example.com/ubuntu bionic-updates/universe i386 Packages [1505 kB]
Get:11 http://aptly.example.com/ubuntu bionic-updates/universe amd64 Packages [1568 kB]
Get:12 http://aptly.example.com/ubuntu bionic-security/main amd64 Packages [941 kB]
Get:13 http://aptly.example.com/ubuntu bionic-security/main i386 Packages [633 kB]
Get:14 http://aptly.example.com/ubuntu bionic-security/universe i386 Packages [903 kB]
Get:15 http://aptly.example.com/ubuntu bionic-security/universe amd64 Packages [946 kB]
Fetched 36,8 MB in 32s (1160 kB/s)                                                                                                                                         
Reading package lists... Done
Building dependency tree       
Reading state information... Done
384 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@srv:~# apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  libllvm8 libwayland-egl1 linux-headers-4.15.0-74 linux-headers-4.15.0-74-generic linux-image-4.15.0-74-generic linux-modules-4.15.0-74-generic
  linux-modules-extra-4.15.0-74-generic python3-dateutil xdg-desktop-portal xdg-desktop-portal-gtk
The following packages will be upgraded:
  appstream apt apt-config-icons apt-utils aptdaemon aptdaemon-data apturl apturl-common base-files bash bluez bluez-cups bluez-obexd bolt brltty bsdutils console-setup
  console-setup-linux debconf debconf-i18n deja-dup desktop-file-utils dmsetup dpkg fdisk fonts-liberation fonts-noto-cjk fonts-noto-color-emoji friendly-recovery fwupd
.....
.....
  xserver-xorg xserver-xorg-core xserver-xorg-input-all xserver-xorg-legacy xserver-xorg-video-all xwayland
384 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.
Need to get 385 MB of archives.
After this operation, 467 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://aptly.example.com/ubuntu bionic-updates/main amd64 base-files amd64 10.1ubuntu2.7 [60,3 kB]
Get:2 http://aptly.example.com/ubuntu bionic-updates/main amd64 bash amd64 4.4.18-2ubuntu1.2 [614 kB]
Get:3 http://aptly.example.com/ubuntu bionic-updates/main amd64 bsdutils amd64 1:2.31.1-0.4ubuntu3.4 [60,3 kB]
Get:4 http://aptly.example.com/ubuntu bionic-updates/main amd64 tar amd64 1.29b-2ubuntu0.1 [234 kB]
Get:5 http://aptly.example.com/ubuntu bionic-updates/main amd64 dpkg amd64 1.19.0.5ubuntu2.3 [1136 kB]
Get:6 http://aptly.example.com/ubuntu bionic-updates/main amd64 grep amd64 3.1-2build1 [159 kB]
......
......
Adding boot menu entry for EFI firmware configuration
done
Processing triggers for initramfs-tools (0.130ubuntu3.9) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-74-generic
Processing triggers for libc-bin (2.27-3ubuntu1) ...
root@srv2:~#

You see the accessed URLs for the metafiles and then the downloads are from our local repositories.

Throubleshooting – downloading files

You may encounter some problems, especially during the downloading phase. Some packages might get download error or to be unavailable at the moment, but the setup (mirror update command) will not fail till the end. It just reports an error at the end, printing all the problem files. The solution is simple just execute the command again (aka execute mirror update) and it will try downloading ONLY the problem files and it will successfully download them (in rear situations you may have to change the synchronization mirror):

Downloading http://nl.archive.ubuntu.com/ubuntu/pool/universe/h/haskell-microlens-th/haskell-microlens-th_0.4.1.1.orig.tar.gz...
ERROR: unable to update: download errors:
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/libp/libprotocol-acme-perl/libprotocol-acme-perl_1.01-2_all.deb: Get http:/ubuntu/pool/universe/libp/libprotocol-acme-perl/libprotocol-acme-perl_1.01-2_all.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/p/pyfai/python-pyfai-doc_0.15.0+dfsg1-1_all.deb: Get http:/ubuntu/pool/universe/p/pyfai/python-pyfai-doc_0.15.0%2bdfsg1-1_all.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/b/biniou/libbiniou-ocaml_1.0.12-2build2_amd64.deb: Get http:/ubuntu/pool/universe/b/biniou/libbiniou-ocaml_1.0.12-2build2_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/h/haskell-hsemail/libghc-hsemail-dev_2-1_amd64.deb: Get http:/ubuntu/pool/universe/h/haskell-hsemail/libghc-hsemail-dev_2-1_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/s/sfst/libsfst1-1.4-dev_1.4.7b-1build1_amd64.deb: Get http:/ubuntu/pool/universe/s/sfst/libsfst1-1.4-dev_1.4.7b-1build1_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/j/jmxetric/jmxetric_1.0.6-1.debian.tar.gz: Get http:/ubuntu/pool/universe/j/jmxetric/jmxetric_1.0.6-1.debian.tar.gz: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/j/jmxetric/jmxetric_1.0.6-1.dsc: Get http:/ubuntu/pool/universe/j/jmxetric/jmxetric_1.0.6-1.dsc: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/b/between/between_6+dfsg1-3_amd64.deb: Get http:/ubuntu/pool/universe/b/between/between_6%2bdfsg1-3_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/s/spice-gtk/gir1.2-spiceclientgtk-3.0_0.34-1.1build1_amd64.deb: Get http:/ubuntu/pool/universe/s/spice-gtk/gir1.2-spiceclientgtk-3.0_0.34-1.1build1_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/d/dataquay/libdataquay-dev_0.9.1-1_amd64.deb: Get http:/ubuntu/pool/universe/d/dataquay/libdataquay-dev_0.9.1-1_amd64.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/n/node-repeat-element/node-repeat-element_1.1.2+github.orig.tar.gz: Get http:/ubuntu/pool/universe/n/node-repeat-element/node-repeat-element_1.1.2%2bgithub.orig.tar.gz: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/e/emboss-explorer/libemboss-acd-perl_2.2.0-9_all.deb: Get http:/ubuntu/pool/universe/e/emboss-explorer/libemboss-acd-perl_2.2.0-9_all.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/n/node-prop-types/node-prop-types_15.6.0+ds-1.dsc: Get http:/ubuntu/pool/universe/n/node-prop-types/node-prop-types_15.6.0%2bds-1.dsc: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/r/ruby-cocoon/ruby-cocoon_1.2.6.orig.tar.gz: Get http:/ubuntu/pool/universe/r/ruby-cocoon/ruby-cocoon_1.2.6.orig.tar.gz: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/g/gcc-7-cross-ports/lib32go11-dbg-ppc64-cross_7.3.0-16ubuntu3cross1_all.deb: Get http:/ubuntu/pool/universe/g/gcc-7-cross-ports/lib32go11-dbg-ppc64-cross_7.3.0-16ubuntu3cross1_all.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/w/webkitgtk/libjavascriptcoregtk-3.0-dev_2.4.11-3ubuntu3_i386.deb: Get http:/ubuntu/pool/universe/w/webkitgtk/libjavascriptcoregtk-3.0-dev_2.4.11-3ubuntu3_i386.deb: dial tcp 213.136.12.213:80: i/o timeout
  http://nl.archive.ubuntu.com/ubuntu/pool/universe/x/xmltooling/libxmltooling-dev_1.6.4-1ubuntu2_i386.deb: Get http:/ubuntu/pool/universe/x/xmltooling/libxmltooling-dev_1.6.4-1ubuntu2_i386.deb: dial tcp 213.136.12.213:80: i/o timeout

The download of multiple file failed with timeout. Just execute mirror update again (with the name of the mirror you encounter the errors before). For example, update mirror with name bionic-main

aptly mirror update bionic-main

It will download only these files, NOT all other (not the other 130G+ files)!

Throubleshooting – create mirror

Another typical mistake is to not import the Ubuntu (or debian if you create debian mirrors) keys, the fillowing error will occur:

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=true -with-udebs=true bionic-main http://nl.archive.ubuntu.com/ubuntu/ bionic main

Looks like your keyring with trusted keys is empty. You might consider importing some keys.
If you're running Debian or Ubuntu, it's a good idea to import current archive keys by running:

  gpg --no-default-keyring --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export | gpg --no-default-keyring --keyring trustedkeys.gpg --import

(for Ubuntu, use /usr/share/keyrings/ubuntu-archive-keyring.gpg)

Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Can't check signature: No public key

Looks like some keys are missing in your trusted keyring, you may consider importing them from keyserver:

gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 3B4FE6ACC0B21F32

Sometimes keys are stored in repository root in file named Release.key, to import such key:

wget -O - https://some.repo/repository/Release.key | gpg --no-default-keyring --keyring trustedkeys.gpg --import

Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/Release...
Downloading http://nl.archive.ubuntu.com/ubuntu/dists/bionic/Release.gpg...
gpgv: Signature made Thu 26 Apr 2018 11:38:40 PM UTC
gpgv:                using RSA key 3B4FE6ACC0B21F32
gpgv: Can't check signature: No public key

Looks like some keys are missing in your trusted keyring, you may consider importing them from keyserver:

gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keys.gnupg.net --recv-keys 3B4FE6ACC0B21F32

Sometimes keys are stored in repository root in file named Release.key, to import such key:

wget -O - https://some.repo/repository/Release.key | gpg --no-default-keyring --keyring trustedkeys.gpg --import

ERROR: unable to fetch mirror: verification of detached signature failed: exit status 2

You should have the Ubuntu repository public key in your trustedkeys keyring of the user, which executes the commands. In our article, we use “aptly” Unix user, so you keys must be imported in its keyring with (the first thing to do in the “STEP 2)” above):

gpg --no-default-keyring --keyring /usr/share/keyrings/ubuntu-archive-keyring.gpg --export | gpg --no-default-keyring --keyring trustedkeys.gpg --import

Note the file is: “/usr/share/keyrings/ubuntu-archive-keyring.gpg” despite what is printed by the error output, it just prints an example, which is for Debian distribution.

Leave a Reply

Your email address will not be published. Required fields are marked *