Using docker-compose may make your life easier only if you use docker images from the official docker registry. And may live forever without knowing the issues when you try to use gitlab-runner with shell executor and docker-compose, which tries to use an image from a private docker registry. The private docker registry is just your gitlab docker containers for your software.
Here is the very error from docker-compose:
pull access denied for applicaton-server, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
To be able to use a private docker repository (i.e. registry) with docker-compose in gitlab-runner you must check the following:
- Make sure the gitlab-runner is the same user as the user running docker-compose in your scripts, because the docker-compose will use the “~/.docker/config.json”, which hosts the login credentials to the docker repositories (including private ones) and is initially created when the before script kicks in and used (mapped) from the programs running from the gitlab-runner executors, as in the very case shell executor. In simple words, if you want to be able to login to a private docker registry within a gitlab runner the “~/.docker/config.json” must exist in the user’s home directory. If the gitlab-runner is running under the root user, the credential file will be located in “/root/.docker/config.json” so the docker-compose ran by the gitlab-runner must be executed from the root user!
The user running the gitlab-runner and the one, which later in your automation code executes docker-compose, MUST be the same.
So be careful with su and sudo in your scripts! Authentication command and the docker-compose must be with the same user.
- run a before script in .gitlab-ci.yml with docker login command using the gitlab CI authentication token (or explicitly with user and password if your environment is such configured) and the URL of the private docker image (including the port and the “/”):
.... before_script: - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN mytest.private-docker-repository.com/ ....
In fact, you may run the above command just before the docker-compose
- Explicitly add the private registry URL (the whole URL to the image – include all the path to the image if any) to the docker image name in docker-compose.xml file.
In your docker-compose.xml include:.... image: mytest.private-docker-repository.com/myapp/applicaton-server:1.1.1 ....Do not rely on the docker-compose will use the private docker registry if you omit its URL like this:
.... image: applicaton-server:1.1.1 ....Probably an additional configuration is required to use it as default docker registry.
An extended output of the error and the problem:
...... Checking out xxx as master... Skipping Git submodules setup $ ./start-jobs.sh 00:12 + docker-compose -p applicaton-server-v2 run --rm --entrypoint 'dockerize -wait tcp://db:3306 -timeout 60s start-server ../main.xml ' applicaton-server Creating network "applicaton-server-v2_default" with the default driver Pulling db (mysql:5.7)... 5.6: Pulling from library/mysql Digest: sha256:xxx Status: Downloaded newer image for mysql:5.7 Creating applicaton-server-v2_db_1 ... Creating applicaton-server-v2_db_1 Pulling applicaton-server (applicaton-server:1.1.1)... pull access denied for applicaton-server, repository does not exist or may require 'docker login': denied: requested access to the resource is denied + finish + docker-compose logs .....
As you can see the “start-jobs.sh” executes “docker-compose” command and it won’t help if you issue “docker login” command just before “docker-compose”! And in fact, the line:
Pulling applicaton-server (applicaton-server:1.1.1)...
It informs for pulling the image, but it tries to pull it from the official docker registry and not the private docker registry (even it is included in the “~/.docker/config.json” and the file is in place).
You must include the URL of the private docker registry in the docker-compose.xml as mentioned above to be sure the docker-compose command will try to pull it from the right place!
Using “docker login” fails, too:
..... Checking out xxx as master... Skipping Git submodules setup $ ./start-jobs.sh 00:04 + docker login Error: Cannot perform an interactive login from a non TTY device + finish + docker-compose logs .....
Running docker login with gitlab CI token from the user script works perfectly (the script authenticates successfully in the private docker registry), but the problem below is that the docker-compose just pulls the image from the default registry, which is the official docker registry, not the private one. Despite there is a successful docker login command to private registry the default registry use used when there is no registry URL in the docker-compose.xml image section.
Checking out xxx as master... Skipping Git submodules setup 00:34 $ ./start-jobs.sh + trap finish EXIT + docker login -u gitlab-ci-token -p [MASKED] mytest.private-docker-repository.com/ WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /home/gitlab-runner/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded + docker-compose -p applicaton-server-v2 run --rm --entrypoint 'dockerize -wait tcp://db:3306 -timeout 60s start-server ../main.xml ' applicaton-server Creating network "applicaton-server-v2_default" with the default driver Pulling db (mysql:5.7)... 5.6: Pulling from library/mysql Digest: sha256:xxx Status: Downloaded newer image for mysql:5.7 Creating applicaton-server-v2_db_1 ... Creating applicaton-server-v2_db_1 Pulling applicaton-server (applicaton-server:latest)... pull access denied for applicaton-server, repository does not exist or may require 'docker login': denied: requested access to the resource is denied + finish + docker-compose logs