docker and dind service (.gitlab-ci.yml) with self-signed certificate and x509: certificate signed by unknown authority

When using GitLab and the CI/CD for building docker images you may stumble on such error using the “docker:dind” (dind stands for docker in docker) image:

$ docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $REGISTRY_URL
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://gitlab.ahelpme.com:4567/v2/: x509: certificate signed by unknown authority
ERROR: Job failed: exit code 1

In our case, because “docker build” command needs a docker service to be running and the GitLab runner needs to provide this docker service so docker:dind is our best option! A self-signed certificate could be really difficult to use in such a big platform as GitLab, but no matter whatever might be the reasons to use docker service in a docker container you may need to use a custom registry with a self-signed certificate!

There are two options to use self-signed certificates with docker:

  1. Add the self-signed certificate in “/etc/docker/certs.d/[custom_registry]/ca.crt”. custom_registry must include the port, for example: “/etc/docker/certs.d/gitlab.example.com\:4567/ca.crt” and restart the docker service! This could be difficult when you use GitLab CI/CD and .gitlab-ci.yml
  2. Add “–insecure-registry” in docker configuration and restart. Apperantly it is easier than the first option when using GitLab CI/CD .gitlab-ci.yml.

The solution

In the GitLab CI/CD file .gitlab-ci.yml add two options (entrypoint, command) to the services, which provides the “dind” (docker in docker). The start of your should start with something like:

image: docker:18.09.7
services:
  - name: docker:18.09.7-dind
    entrypoint: ["dockerd-entrypoint.sh"]
    command: ["--insecure-registry", "gitlab.ahelpme.com:4567"]

Of course, replace the “gitlab.ahelpme.com:4567” with your custom docker registry domain.

Real world example – failed job in gitlab-runner

SCREENSHOT 1) The docker service does not allow login in custom registry with self-signed certificate.

main menu
gitlab-runner registry login failed

Here is the failed Job:

Running with gitlab-runner 12.4.1 (05161b14)
  on sixth-runner HHE14eNN
Using Docker executor with image docker:18.09.7 ...
Starting service docker:18.09.7-dind ...
Pulling docker image docker:18.09.7-dind ...
Using docker image sha256:25a1e57c774167d28c44d88fa296f3e1122c6d79e99b98653c899b170393bbd6 for docker:18.09.7-dind ...
Waiting for services to be up and running...
Pulling docker image docker:18.09.7 ...
Using docker image sha256:108a4437ed8ca450ea2c0456e3d08a36a3ce7934275bd9745741dad2e234b353 for docker:18.09.7 ...
Running on runner-HHE14eNN-project-2-concurrent-0 via a9e21e92e2dd...
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/root/ubuntu-project/.git/
From https://gitlab.ahelpme.com/root/ubuntu-project
 * [new ref]         refs/pipelines/37 -> refs/pipelines/37
 * [new tag]         v17               -> v17
Checking out 2fb562e1 as v17...

Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $REGISTRY_URL
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://gitlab.ahelpme.com:4567/v2/: x509: certificate signed by unknown authority
ERROR: Job failed: exit code 1

SCREENSHOT 2) Added the “–insecure-registry” to “.gitlab-ci.yml” istructs the docker service to allow login to custom registry with self-signed certificate.

main menu
gitlab-runner registry login succeeded

After we have added the two options in our .gitlab-ci.yml. You will see “Login Succeeded” after “docker login”

Running with gitlab-runner 12.4.1 (05161b14)
  on sixth-runner HHE14eNN
Using Docker executor with image docker:18.09.7 ...
Starting service docker:18.09.7-dind ...
Pulling docker image docker:18.09.7-dind ...
Using docker image sha256:25a1e57c774167d28c44d88fa296f3e1122c6d79e99b98653c899b170393bbd6 for docker:18.09.7-dind ...
Waiting for services to be up and running...
Pulling docker image docker:18.09.7 ...
Using docker image sha256:108a4437ed8ca450ea2c0456e3d08a36a3ce7934275bd9745741dad2e234b353 for docker:18.09.7 ...
Running on runner-HHE14eNN-project-2-concurrent-0 via a9e21e92e2dd...
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/root/ubuntu-project/.git/
From https://gitlab.ahelpme.com/root/ubuntu-project
 * [new ref]         refs/pipelines/36 -> refs/pipelines/36
 * [new tag]         v16               -> v16
Checking out d7a4e1c8 as v16...

Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $REGISTRY_URL
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Leave a Reply

Your email address will not be published. Required fields are marked *