This article is a follow up after the Run podman/docker InfluxDB 1.8 container to collect statistics from collectd, where the time series database InfluxDB stores data and by using Grafana in another container it is easy and lightweight enough to visualize the collected data.
Containerizing the Grafana service is simple enough with docker/podman, but there are several tips and steps to consider before doing it. These steps will significantly ease the maintainer’s life, making upgrading, moving to another server, or backup important data really easy – just stop and start another container with the same options except name and container version.
Here are the important points to mind when running Grafana 9 in a docker/podman container:
Keep on reading!
Category: Docker
Run podman/docker InfluxDB 1.8 container to collect statistics from collectd
Yet another article on the topic of the InfluxDB 1.8 and collectd gathering statistics, in continuation of the articles Monitor and analyze with Grafana, influxdb 1.8 and collectd under Ubuntu 22.04 LTS and Monitor and analyze with Grafana, influxdb 1.8 and collectd under CentOS Stream 9. This time, the InfluxDB runs in a container created with podman or docker software.
Here are the important points to mind when running InfluxDB 1.8 in a docker/podman container:
Keep on reading!
Run a docker container with bigger storage
By default, the Docker command-line utility docker runs containers with 10G storage, which in most cases is enough, but if the user wants to just run a specific container with bigger storage there is an option for the docker command:
docker run --storage-opt size=50G
The option size=50G will set the docker container storage for the current only run command!
Run a Ubuntu 22.04 Docker container with 50G root storage:
root@srv ~ # docker run --storage-opt size=50G -it ubuntu:22.04 bash Unable to find image 'ubuntu:22.04' locally 22.04: Pulling from library/ubuntu e96e057aae67: Pull complete Digest: sha256:4b1d0c4a2d2aaf63b37111f34eb9fa89fa1bf53dd6e4ca954d47caebca4005c2 Status: Downloaded newer image for ubuntu:22.04 root@4caab8c61157:/# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/docker-253:0-39459726-2f2d655687e5bd39620a2a083960ac969d8163b806152765a1fc166f0a82d3d9 50G 170M 50G 1% / tmpfs 64M 0 64M 0% /dev tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup shm 64M 0 64M 0% /dev/shm /dev/mapper/map-99f55d81-4132-42d4-9515-33d8cc11d3e2 3.6T 1.5T 2.2T 40% /etc/hosts tmpfs 7.8G 0 7.8G 0% /proc/asound tmpfs 7.8G 0 7.8G 0% /proc/acpi tmpfs 7.8G 0 7.8G 0% /proc/scsi tmpfs
It’s worth mentioning this option “–storage-opt size=50G” is different from the “–storage-opt dm.basesize=50G“, the first one is used as a command argument to the docker command-line utility. The second one is used with the dockerd daemon to change the default Docker behavior from 10G to 50G storage. Note, either option cannot change the storage size of the already started container.
Starting up standalone ClickHouse server with basic configuration in docker
ClickHouse is a powerful column-oriented database written in C, which generates analytical and statistical reports in real-time using SQL statements!
It supports on-the-fly compression of the data, cluster setup of replicas and shards instances over thousands of servers, and multi-master cluster modes.
The ClickHouse is an ideal instrument for weblogs and easy real-time generating reports of the weblogs! Or for storing the data of user behaviour and interactions with web sites or applications.
The easiest way to run a CLickHouse instance is within a docker/podman container. The docker hub hosts official containers image maintained by the ClickHouse developers.
And this article will show how to run a ClickHouse standalone server, how to manage the ClickHouse configuration features, and what obstacles the user may encounter.
Here are some key points:
- Main server configuration file is config.xml (in /etc/clickhouse-server/config.xml) – all server’s settings like listening port, ports, logger, remote access, cluster setup (shards and replicas), system settings (time zone, umask, and more), monitoring, query logs, dictionaries, compressions and so on. Check out the server settings: https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings/
- The main user configuration file is users.xml (in /etc/clickhouse-server/users.xml), which specifies profiles, users, passwords, ACL, quotas, and so on. It also supports SQL-driven user configuration, check out the available settings and users’ options – https://clickhouse.com/docs/en/operations/settings/settings-users/
- By default, there is a root user with administrative privileges without password, which could only connect to the server from the localhost.
- Do not edit the main configuration file(s). Some options may get deprecated and removed and the modified configuration file to become incompatible with the new releases.
- Every configuration setting could be overriden with configuration files in config.d/. A good practice is to have a configuration file per each setting, which overrides the default one in config.xml. For example:
root@srv ~ # ls -al /etc/clickhouse-server/config.d/ total 48 drwxr-xr-x 2 root root 4096 Nov 22 04:40 . drwxr-xr-x 4 root root 4096 Nov 22 04:13 .. -rw-r--r-- 1 root root 343 Sep 16 2021 00-path.xml -rw-r--r-- 1 root root 58 Nov 22 04:40 01-listen.xml -rw-r--r-- 1 root root 145 Feb 3 2020 02-log_to_console.xml
There are three configurations files, which override the default paths (00-path.xml), change the default listen setting (01-listen.xml), and log to console (02-log_to_console.xml). Here is what to expect in 00-path.xml
<yandex> <path replace="replace">/mnt/storage/ClickHouse/var/</path> <tmp_path replace="replace">/mnt/storage/ClickHouse/tmp/</tmp_path> <user_files_path replace="replace">/mnt/storage/ClickHouse/var/user_files/</user_files_path> <format_schema_path replace="replace">/mnt/storage/ClickHouse/format_schemas/</format_schema_path> </yandex>So the default settings in config.xml path, tmp_path, user_files_path and format_schema_path will be replaced with the above values.
To open the ClickHouse for the outer world, i.e. listen to 0.0.0.0 just include a configuration file like 01-listen.xml.<yandex> <listen_host>0.0.0.0</listen_host> </yandex> - When all additional (including user) configuration files are processed and the result is written in preprocessed_configs/ directory in var directory, for example /var/lib/clickhouse/preprocessed_configs/
- The configuration directories are reloaded each 3600 seconds (by default, it could be changed) by the ClickHouse server and on a change in the configuration files new processed ones are generated and in most cases the changes are loaded on-the-fly. Still, there are settings, which require manual restart of the main process. Check out the manual for more details.
- By default, the logger is in the trace log level, which may generate an enormous amount of logging data. So just change the settings to something more production meaningful like warning level (in config.d/04-part_log.xml).
<yandex> <logger> <level>warning</level> </logger> </yandex> - ClickHouse default ports:
- 8123 is the HTTP client port (8443 is the HTTPS). The client can connect with curl or wget or other command-line HTTP(S) clients to manage and insert data in databases and tables.
- 9000 is the native TCP/IP client port (9440 is the TLS enabled port for this service) to manage and insert data in databases and tables.
- 9004 is the MySQL protocol port. ClickHouse supports MySQL wire protocol and it can be enabled by the
<yandex> <mysql_port>9004</mysql_port> </yandex> - 9009 is the port, which ClickHouse uses to exchange data between ClickHouse servers when using cluster setup and replicas/shards.
- There is a flag directory, in which files with special names may instruct ClickHouse to process commands. For example, creating a blank file with the name: /var/lib/clickhouse/flags/force_restore_data will instruct the ClickHouse to begin a restore procedure for the server.
- A good practice is to make backup of the whole configuration directory despite the main configuration file(s) are not changed and in original state.
- The SQL commands, which are supported by CickHouse server: https://clickhouse.com/docs/en/sql-reference/ and https://clickhouse.com/docs/en/sql-reference/statements/
- The basic and fundamental table type is MergeTree, which is designed for inserting a very large amount of data into a table – https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree/
- Bear in mind, ClickHouse supports SQL syntax and some of the SQL statements, but UPDATE and DELETE statements are not supported, just INSERTs! The main idea behind the ClickHouse is not to change the data, but to add only!
- Batch INSERTs are the preferred way of inserting data! In fact, there is a recommendation of 1 INSERT per a second in the ClickHouse manual
Easy install the latest docker-compose with pip3 under Ubuntu
At present, the latest docker-compose version, which could be installed under Ubuntu 18, 20, and 21 is the 1.25 and 1.27 versions. There may be significant changes included in the latest versions and if one wants to install it there are two options:
- Manual installation from docker from the github – https://github.com/docker/compose/releases
- Install it from pip3.
For example, depends_on.service.condition: service_healthy is added with version 1.28. Using this new feature it is fairly easy to implement waiting for a docker container (service) before starting another docker.
Here is how easy it is to install and to have the latest stable docker-compose version, which is 1.29.2 at the writing of this article:
STEP 1) Update and upgrade.
Do this step always before installing new software.
sudo apt update sudo apt upgrade -y
STEP 2) Install pip3 and docker.
pip 3 is the package installer for Python 3. When using docker-compose it is supposed to have the very Docker software, too.
apt install python3-pip docker systemctl start docker
STEP 3) Install docker-compose using pip3.
pip3 install docker-compose
And here is what a version command prints:
root@srv:~# docker-compose version docker-compose version 1.29.2, build unknown docker-py version: 5.0.2 CPython version: 3.8.10 OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
Just to note, installing packages using other programs other than apt may lead to future conflicts!
The whole console output of the pip3 installing docker-compose
root@srv:~# apt update
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1514 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [33.3 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1069 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [575 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [6324 B]
Get:14 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2668 B]
Get:15 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1070 kB]
Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [790 kB]
Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [30.1 kB]
Get:18 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [525 kB]
Fetched 19.0 MB in 1s (16.7 MB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
root@srv:~# apt upgrade -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
root@srv:~# apt install -y python3-pip docker
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
binutils binutils-common binutils-x86-64-linux-gnu build-essential ca-certificates cpp cpp-9 dirmngr dpkg-dev fakeroot file g++ g++-9 gcc gcc-9 gcc-9-base gnupg gnupg-l10n gnupg-utils
gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libasn1-8-heimdal libassuan0 libatomic1
libbinutils libbsd0 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-9-dev
libgdbm-compat4 libgdbm6 libglib2.0-0 libglib2.0-data libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libicu66 libisl22
libitm1 libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common liblocale-gettext-perl liblsan0 libmagic-mgc libmagic1 libmpc3 libmpdec2 libmpfr6 libnpth0 libperl5.30 libpython3-dev
libpython3-stdlib libpython3.8 libpython3.8-dev libpython3.8-minimal libpython3.8-stdlib libquadmath0 libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db
libsqlite3-0 libssl1.1 libstdc++-9-dev libtsan0 libubsan1 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxml2 linux-libc-dev make manpages manpages-dev mime-support
netbase openssl patch perl perl-modules-5.30 pinentry-curses python-pip-whl python3 python3-dev python3-distutils python3-lib2to3 python3-minimal python3-pkg-resources
python3-setuptools python3-wheel python3.8 python3.8-dev python3.8-minimal readline-common shared-mime-info tzdata wmdocker xdg-user-dirs xz-utils zlib1g-dev
Suggested packages:
binutils-doc cpp-doc gcc-9-locales dbus-user-session libpam-systemd pinentry-gnome3 tor debian-keyring g++-multilib g++-9-multilib gcc-9-doc gcc-multilib autoconf automake libtool flex
bison gdb gcc-doc gcc-9-multilib parcimonie xloadimage scdaemon glibc-doc git bzr gdbm-l10n libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap
libsasl2-modules-otp libsasl2-modules-sql libstdc++-9-doc make-doc man-browser ed diffutils-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl libb-debug-perl
liblocale-codes-perl pinentry-doc python3-doc python3-tk python3-venv python-setuptools-doc python3.8-venv python3.8-doc binfmt-support readline-doc
The following NEW packages will be installed:
binutils binutils-common binutils-x86-64-linux-gnu build-essential ca-certificates cpp cpp-9 dirmngr docker dpkg-dev fakeroot file g++ g++-9 gcc gcc-9 gcc-9-base gnupg gnupg-l10n
gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5 libasn1-8-heimdal libassuan0
libatomic1 libbinutils libbsd0 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-9-dev
libgdbm-compat4 libgdbm6 libglib2.0-0 libglib2.0-data libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libicu66 libisl22
libitm1 libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common liblocale-gettext-perl liblsan0 libmagic-mgc libmagic1 libmpc3 libmpdec2 libmpfr6 libnpth0 libperl5.30 libpython3-dev
libpython3-stdlib libpython3.8 libpython3.8-dev libpython3.8-minimal libpython3.8-stdlib libquadmath0 libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db
libsqlite3-0 libssl1.1 libstdc++-9-dev libtsan0 libubsan1 libwind0-heimdal libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 libxml2 linux-libc-dev make manpages manpages-dev mime-support
netbase openssl patch perl perl-modules-5.30 pinentry-curses python-pip-whl python3 python3-dev python3-distutils python3-lib2to3 python3-minimal python3-pip python3-pkg-resources
python3-setuptools python3-wheel python3.8 python3.8-dev python3.8-minimal readline-common shared-mime-info tzdata wmdocker xdg-user-dirs xz-utils zlib1g-dev
.....
.....
0 upgraded, 128 newly installed, 0 to remove and 0 not upgraded.
Need to get 84.6 MB of archives.
After this operation, 370 MB of additional disk space will be used.
Processing triggers for ca-certificates (20210119~20.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root@srv:~# pip3 install docker-compose
Collecting docker-compose
Downloading docker_compose-1.29.2-py2.py3-none-any.whl (114 kB)
|████████████████████████████████| 114 kB 12.4 MB/s
Collecting requests<3,>=2.20.0
Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 355 kB/s
Collecting jsonschema<4,>=2.5.1
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
|████████████████████████████████| 56 kB 3.4 MB/s
Collecting websocket-client<1,>=0.32.0
Downloading websocket_client-0.59.0-py2.py3-none-any.whl (67 kB)
|████████████████████████████████| 67 kB 3.3 MB/s
Collecting texttable<2,>=0.9.0
Downloading texttable-1.6.4-py2.py3-none-any.whl (10 kB)
Collecting PyYAML<6,>=3.10
Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
|████████████████████████████████| 662 kB 76.9 MB/s
Collecting dockerpty<1,>=0.4.1
Downloading dockerpty-0.4.1.tar.gz (13 kB)
Collecting docker[ssh]>=5
Downloading docker-5.0.2-py2.py3-none-any.whl (145 kB)
|████████████████████████████████| 145 kB 119.5 MB/s
Collecting distro<2,>=1.5.0
Downloading distro-1.6.0-py2.py3-none-any.whl (19 kB)
Collecting docopt<1,>=0.6.1
Downloading docopt-0.6.2.tar.gz (25 kB)
Collecting python-dotenv<1,>=0.13.0
Downloading python_dotenv-0.19.0-py2.py3-none-any.whl (17 kB)
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
|████████████████████████████████| 138 kB 141.1 MB/s
Collecting charset-normalizer~=2.0.0; python_version >= "3"
Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
|████████████████████████████████| 145 kB 133.3 MB/s
Collecting idna<4,>=2.5; python_version >= "3"
Downloading idna-3.2-py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 1.6 MB/s
Collecting pyrsistent>=0.14.0
Downloading pyrsistent-0.18.0-cp38-cp38-manylinux1_x86_64.whl (118 kB)
|████████████████████████████████| 118 kB 131.3 MB/s
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from jsonschema<4,>=2.5.1->docker-compose) (45.2.0)
Collecting six>=1.11.0
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting attrs>=17.4.0
Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
|████████████████████████████████| 53 kB 899 kB/s
Collecting paramiko>=2.4.2; extra == "ssh"
Downloading paramiko-2.7.2-py2.py3-none-any.whl (206 kB)
|████████████████████████████████| 206 kB 147.7 MB/s
Collecting cryptography>=2.5
Downloading cryptography-3.4.8-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
|████████████████████████████████| 3.2 MB 147.4 MB/s
Collecting bcrypt>=3.1.3
Downloading bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl (63 kB)
|████████████████████████████████| 63 kB 1.4 MB/s
Collecting pynacl>=1.0.1
Downloading PyNaCl-1.4.0-cp35-abi3-manylinux1_x86_64.whl (961 kB)
|████████████████████████████████| 961 kB 139.4 MB/s
Collecting cffi>=1.12
Downloading cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB)
|████████████████████████████████| 411 kB 84.0 MB/s
Collecting pycparser
Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
|████████████████████████████████| 112 kB 140.9 MB/s
Building wheels for collected packages: dockerpty, docopt
Building wheel for dockerpty (setup.py) ... done
Created wheel for dockerpty: filename=dockerpty-0.4.1-py3-none-any.whl size=16604 sha256=d6f2d3d74bad523b1a308a952176a1db84cb604611235c1a5ae1c936cefe7889
Stored in directory: /root/.cache/pip/wheels/1a/58/0d/9916bf3c72e224e038beb88f669f68b61d2f274df498ff87c6
Building wheel for docopt (setup.py) ... done
Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=f8c389703e63ff7ec3734b240ba8d62c8f8bd99f3b05ccdcb0de1397aa523655
Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c
Successfully built dockerpty docopt
Installing collected packages: urllib3, charset-normalizer, certifi, idna, requests, pyrsistent, six, attrs, jsonschema, websocket-client, texttable, PyYAML, dockerpty, pycparser, cffi, cryptography, bcrypt, pynacl, paramiko, docker, distro, docopt, python-dotenv, docker-compose
Successfully installed PyYAML-5.4.1 attrs-21.2.0 bcrypt-3.2.0 certifi-2021.5.30 cffi-1.14.6 charset-normalizer-2.0.4 cryptography-3.4.8 distro-1.6.0 docker-5.0.2 docker-compose-1.29.2 dockerpty-0.4.1 docopt-0.6.2 idna-3.2 jsonschema-3.2.0 paramiko-2.7.2 pycparser-2.20 pynacl-1.4.0 pyrsistent-0.18.0 python-dotenv-0.19.0 requests-2.26.0 six-1.16.0 texttable-1.6.4 urllib3-1.26.6 websocket-client-0.59.0
edit mysql options in docker (or docker-compose) mysql
Modifying the default options for the docker (podman) MySQL server is essential. The default MySQL options are too conservative and even for simple (automation?) tests the options could be .
For example, modifying only one or two of the default InnoDB configuration options may lead to boosting multiple times faster execution of SQL queries and the related automation tests.
Here are three simple ways to modify the (default or current) MySQL my.cnf configuration options:
- Command-line arguments. All MySQL configuration options could be overriden by passing them in the command line of mysqld binary. The format is:
--variable-name=value
and the variable names could be obtained by
mysqld --verbose --help
and for the live configuration options:
mysqladmin variables
- Options in a additional configuration file, which will be included in the main configuration. The options in /etc/mysql/conf.d/config-file.cnftake precedence.
- Replacing the default my.cnf configuration file – /etc/mysql/my.cnf.
Check out also the official page – https://hub.docker.com/_/mysql.
Under CentOS 8 docker is replaced by podman and just replace the docker with podman in all of the commands below.
OPTION 1) Command-line arguments.
This is the simplest way of modifying the default my.cnf (the one, which comes with the docker image or this in the current docker image file). It is fast and easy to use and change, just a little bit of much writing in the command-line. As mentioned above all MySQL options could be changed by a command-line argument to the mysqld binary. For example:
mysqld --innodb_buffer_pool_size=1024M
It will start MySQL server with variable innodb_buffer_pool_size set to 1G. Translating it to (for multiple options just add them at the end of the command):
-
docker run
root@srv ~ # docker run --name my-mysql -v /var/lib/mysql:/var/lib/mysql \ -e MYSQL_ROOT_PASSWORD=111111 \ -d mysql:8 \ --innodb_buffer_pool_size=1024M \ --innodb_read_io_threads=4 \ --innodb_flush_log_at_trx_commit=2 \ --innodb_flush_method=O_DIRECT 1bb7f415ab03b8bfd76d1cf268454e3c519c52dc383b1eb85024e506f1d04dea root@srv ~ # docker exec -it my-mysql mysqladmin -p111111 variables|grep innodb_buffer_pool_size | innodb_buffer_pool_size | 1073741824
-
docker-compose:
# Docker MySQL arguments example version: '3.1' services: db: image: mysql:8 command: --default-authentication-plugin=mysql_native_password --innodb_buffer_pool_size=1024M --innodb_read_io_threads=4 --innodb_flush_log_at_trx_commit=2 --innodb_flush_method=O_DIRECT restart: always environment: MYSQL_ROOT_PASSWORD: 111111 volumes: - /var/lib/mysql_data:/var/lib/mysql ports: - "3306:3306"Here is how to run it (the above text file should be named docker-compose.yml and the file should be in the current directory when executing the command below):
root@srv ~ # docker-compose up Creating network "docker-compose-mysql_default" with the default driver Creating my-mysql ... done Attaching to my-mysql my-mysql | 2020-06-16 09:45:35+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.20-1debian10 started. my-mysql | 2020-06-16 09:45:35+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql' my-mysql | 2020-06-16 09:45:35+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.20-1debian10 started. my-mysql | 2020-06-16T09:45:36.293747Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release. my-mysql | 2020-06-16T09:45:36.293906Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.20) starting as process 1 my-mysql | 2020-06-16T09:45:36.307654Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. my-mysql | 2020-06-16T09:45:36.942424Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. my-mysql | 2020-06-16T09:45:37.136537Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '::' port: 33060 my-mysql | 2020-06-16T09:45:37.279733Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed. my-mysql | 2020-06-16T09:45:37.306693Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory. my-mysql | 2020-06-16T09:45:37.353358Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.20' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
And check the option:
root@srv ~ # docker exec -it my-mysql mysqladmin -p111111 variables|grep innodb_buffer_pool_size | innodb_buffer_pool_size | 1073741824
OPTION 2) Options in a additional configuration file.
Create a MySQL option file with name config-file.cnf:
[mysqld] innodb_buffer_pool_size=1024M innodb_read_io_threads=4 innodb_flush_log_at_trx_commit=2 innodb_flush_method=O_DIRECT
- docker run
- docker-compose
The source path may not be absolute path.# Docker MySQL arguments example version: '3.1' services: db: container_name: my-mysql image: mysql:8 command: --default-authentication-plugin=mysql_native_password restart: always environment: MYSQL_ROOT_PASSWORD: 111111 volumes: - /var/lib/mysql_data:/var/lib/mysql - ./config-file.cnf:/etc/mysql/conf.d/config-file.cnf ports: - "3306:3306"
The source path must be absolute path!
docker run --name my-mysql \ -v /var/lib/mysql_data:/var/lib/mysql \ -v /etc/mysql/docker-instances/config-file.cnf:/etc/mysql/conf.d/config-file.cnf \ -e MYSQL_ROOT_PASSWORD=111111 \ -d mysql:8
OPTION 3) Replacing the default my.cnf configuration file.
Add the modified options to a my.cnf template file and map it to the container on /etc/mysql/my.cnf. When overwriting the main MySQL option file – my.cnf you may map the whole /etc/mysql directory (just replace /etc/mysql/my.cnf with /etc/mysql below), too. The source file (or directory) may be any file (or directory) not the /etc/mysql/my.cnf (or /etc/mysql)
- docker run:
The source path must be absolute path.docker run --name my-mysql \ -v /var/lib/mysql_data:/var/lib/mysql \ -v /etc/mysql/my.cnf:/etc/mysql/my.cnf \ -e MYSQL_ROOT_PASSWORD=111111 \ --publish 3306:3306 \ -d mysql:8
Note: here a new option “–publish 3306:3306” is included to show how to map the ports out of the container like all the examples with the docker-compose here.
- docker-compose:
The source path may not be absolute path, but the current directory.# Use root/example as user/password credentials version: '3.1' services: db: container_name: my-mysql image: mysql:8 command: --default-authentication-plugin=mysql_native_password restart: always environment: MYSQL_ROOT_PASSWORD: 111111 volumes: - /var/lib/mysql_data:/var/lib/mysql - ./mysql/my.cnf:/etc/mysql/my.cnf ports: - "3306:3306"
Cron missing path – executing docker/podman – adding network: failed to locate iptables
If you have ever happened to execute some complex scripts using the cron system you were inevitable to discover the Linux environment was different than the login or ssh shell. The different environment tends to lead to a missing or different PATH environment! Here is what happens with podman starting a container from a cron script:
time="2020-04-19T20:45:20Z" level=error msg="Error adding network: failed to locate iptables: exec: \"iptables\": executable file not found in $PATH" time="2020-04-19T20:45:20Z" level=error msg="Error while adding pod to CNI network \"podman\": failed to locate iptables: exec: \"iptables\": executable file not found in $PATH" Error: unable to start container "onedrive-cli": error configuring network namespace for container d297cf80db20441d4258a1acc7d810444795d1ca8730ab242d9fe8a13eaa697d: failed to locate iptables: exec: "iptables": executable file not found in $PATH
The iptables executable is missing because the PATH variable is different than the login or ssh shell one. Executing the commands or the script under ssh or login will result in no error and a proper podman (docker) execution!
A similar problem could have happened with another software trying to execute iptables or another tool, which is not found in the cron’s PATH environment because cron’s environment is very limited and
To ensure the PATH is like the user’s (root) environment just source the “profile” or “.bashrc” file of the current user before the execution of the script or in the first lines of it.
This would do the trick.
. /etc/profile
Or user’s custom
. ~/.bashrc
Or the default OS bashrc
. /etc/bashrc
The dot may be replaced by “source”:
source /etc/bashrc
All (environment) variables will be available after the source command.
Here is the difference:
The environment without the sourcing profile/bashrc file:
LANG=en_US.UTF-8 XDG_SESSION_ID=19118 USER=root PWD=/root HOME=/root SHELL=/bin/sh SHLVL=1 LOGNAME=root DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus XDG_RUNTIME_DIR=/run/user/0 PATH=/usr/bin:/bin _=/usr/bin/env
Sourcing the “/etc/profile” file:
LANG=en_US.UTF-8 HISTCONTROL=ignoredups HOSTNAME=srv.example.com XDG_SESSION_ID=19165 USER=root PWD=/root HOME=/root MAIL=/var/spool/mail/root SHELL=/bin/bash SHLVL=1 LOGNAME=root DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus XDG_RUNTIME_DIR=/run/user/0 PATH=/usr/local/sbin:/usr/sbin:/usr/bin:/bin HISTSIZE=1000 LESSOPEN=||/usr/bin/lesspipe.sh %s _=/usr/bin/env
Multiple additional envrinment varibles, which could be important for user’s scripts executed by the cron.
And in CentOS 8 the iptables happens to be in “/usr/sbin/iptables” – a path /usr/sbin not included in the default cron environment PATH variable!
Of course, the PATH environment may be edited in the cron scheduler with crontab (by just setting the PATH with a path) till the next path missing in it and included in the user’s path! It’s just better to ensure the two environments are the same every time by sourcing the environment configuration file such as /etc/profile or user’s bashrc (or the default on in /etc/bashrc?).
docker mysql – Fatal error: Please read “Security” section of the manual to find out how to run mysqld as root!
Pulling the official MySQL image from the docker registry https://hub.docker.com/r/mysql/mysql-server to start a MySQL instance with your configuration file (and MySQL binary files). Adding the “–volume” option for the configuration directory (or file) and MySQL binary files and you stumble on the error:
2019-12-03 01:13:38 0 [Note] mysqld (mysqld 5.6.46-log) starting as process 67 ... 2019-12-03 01:13:38 67 [ERROR] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root! 2019-12-03 01:13:38 67 [ERROR] Aborting 2019-12-03 01:13:38 67 [Note] Binlog end 2019-12-03 01:13:38 67 [Note] mysqld: Shutdown complete
Apparently, the server option is not configured to run properly as a root user and you do not want to run it, but why it keeps insisting to run it as root?
Because of the entry point script will execute only “mysqld” as a command, which expects to have a “user” option in the “[mysqld]” section of your my.cnf configuration file!
Do not miss the user option in my.cnf! This is how the MySQL server will be using the “mysql” username not the root!
user=mysql
Typical error, because it is not so common to include the username in my.cnf configuration file of the mysqld process to run as. If you use the official docker MySQL image to create your configuration file you would not encounter the above error, but if you use an existing (probably old and from non virtualized environment) my.cnf make sure to include the username, which should be used to run the mysqld process as.
Here is our command to execute the container:
docker run --privileged -d -v /mnt/storage/docker/mysql-slave/files:/var/lib/mysql -v /mnt/storage/docker/mysql-slave/etc/my.cnf:/etc/my.cnf mysql/mysql-server:5.6
Build docker image with custom Dockerfile name – docker build requires exactly 1 argument
Docker uses the Dockerfile to build docker images, but what if you want to change the name and (or) the path of this file?
By default “docker build” command uses a file named Dockerfile on the same directory you execute the “docker build“. There is an option to change the path and name of this special file:
-f, --file string Name of the Dockerfile (Default is 'PATH/Dockerfile')
And the “-f” may include path and file name but it is mandatory to specify the path at the end “docker build” usually the current directory (context by the docker terminology) by adding “.” (the dot at the end of the command)
So if you want to build with a docker file mydockerfile in the current directory you must execute:
docker build -f mydockerfile .
If your file is in a sub-directory execute:
docker build -f subdirectory/mydockerfile .
The command will create a docker image in your local repository. Here is the output of the first command:
root@srv:~/docker# docker build -f mydockerfile . Sending build context to Docker daemon 2.048kB Step 1/3 : FROM ubuntu:bionic-20191029 bionic-20191029: Pulling from library/ubuntu 7ddbc47eeb70: Pull complete c1bbdc448b72: Pull complete 8c3b70e39044: Pull complete 45d437916d57: Pull complete Digest: sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d Status: Downloaded newer image for ubuntu:bionic-20191029 ---> 775349758637 Step 2/3 : MAINTAINER test@example.com ---> Running in 5fa42bca749c Removing intermediate container 5fa42bca749c ---> 0a1ffa1728f4 Step 3/3 : RUN apt-get update && apt-get upgrade -y && apt-get install -y git wget ---> Running in 2e35040f247c Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB] Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] ..... ..... Processing triggers for ca-certificates (20180409) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. Removing intermediate container 2e35040f247c ---> 2382809739a4 Successfully built 2382809739a4
Here is the image:
REPOSITORY TAG IMAGE ID CREATED SIZE root@srv:~# docker images <none> <none> 2382809739a4 About a minute ago 186MB
Build command with custom name and registry URL and TAG
root@srv:~# docker build -t gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base:v0.1 -f mydockerfile . Sending build context to Docker daemon 2.048kB Step 1/3 : FROM ubuntu:bionic-20191029 ---> 775349758637 Step 2/3 : MAINTAINER test@example.com ---> Using cache ---> 0a1ffa1728f4 Step 3/3 : RUN apt-get update && apt-get upgrade -y && apt-get install -y git wget ---> Using cache ---> 2382809739a4 Successfully built 2382809739a4 Successfully tagged gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base:v0.1 root@srv:~# docker push gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base:v0.1 The push refers to repository [gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base] 7cebba4bf6c3: Pushed e0b3afb09dc3: Pushed 6c01b5a53aac: Pushed 2c6ac8e5063e: Pushed cc967c529ced: Pushed v0.1: digest: sha256:acf42078bf46e320c402f09c6417a3dae8992ab4f4f685265486063daf30cb13 size: 1364
the registry URL is “gitlab.ahelpme.com:4567” and the project path is “/root/ubuntu-project/” and the name of the image is “ubuntu18-manual-base” with tag “v0.1“. The build command uses the cache from our first build example here (because the docker file is the same).
Typical errors with “-f”
Two errors you may encounter when trying the “-f” to change the name of the default Dockerfile name:
$ docker build -t gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base:v0.1 -f mydockerfile subdirectory/ unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /builds/dev/docker-containers/mydockerfile: no such file or directory $ docker build -t gitlab.ahelpme.com:4567/root/ubuntu-project/ubuntu18-manual-base:v0.1 -f subdirectory/mydockerfile "docker build" requires exactly 1 argument. See 'docker build --help'. Usage: docker build [OPTIONS] PATH | URL | -
First, you might think the -f would take the path and file name and this should be enough, but the errors above appears!
Our example Dockerfile
This is our simple example docker file:
FROM ubuntu:bionic-20191029 MAINTAINER test@example.com RUN apt-get update && apt-get upgrade -y && apt-get install -y git wget
We are using the official docker image from Ubuntu. Always use official docker images!
Docker change the port mapping of an existing container
Unfortunately, it is not possible to change the port mapping (forwarded ports from the hosts to the container) of an existing RUNNING container!
Not only that, but you cannot change the mapped ports (forwarded ports) even when the container is stopped, so think twice when you run or start a container from the image you’ve chosen. Of course, you can always use docker’s commit command, which just creates a new image from you (running, in a sense of changes fro the original image) container and then you can run the new image with new mapped ports!
Still, there is a solution not involving the creation of new docker images and containers, but just to edit manually a configuration file while the Docker service is stopped.
So if you have several docker containers running you should stop all of them! When the Docker service stops, edit the “hostconfig.json” file! Here is the whole procedure:
- Stop the container.
- Stop the Docker container service.
- Edit the container’s file – hostconfig.json (usually in /var/lib/docker/containers/[ID]/hostconfig.json) and add or replace ports.
- Start the Docker container service.
- Start the docker container.
Real World Example
myuser@srv:~# sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a9e21e92e2dd gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 2 days ago Up 33 hours gitlab-runner
5d025e7f93a4 gitlab/gitlab-ce:latest "/assets/wrapper" 3 days ago Up 34 hours (healthy) 0.0.0.0:80->80/tcp, 0.0.0.0:4567->4567/tcp, 0.0.0.0:1022->22/tcp gitlab
myuser@srv:~# sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a9e21e92e2dd gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 2 days ago Up 33 hours gitlab-runner
5d025e7f93a4 gitlab/gitlab-ce:latest "/assets/wrapper" 3 days ago Up 34 hours (healthy) 0.0.0.0:80->80/tcp, 0.0.0.0:4567->4567/tcp, 0.0.0.0:1022->22/tcp gitlab
myuser@srv:~# sudo docker stop gitlab-runner
gitlab-runner
myuser@srv:~# sudo docker stop gitlab
gitlab
myuser@srv:~# sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
myuser@srv:~# systemctl stop docker
myuser@srv:~# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Thu 2019-11-14 21:54:57 UTC; 5s ago
Docs: https://docs.docker.com
Process: 2340 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=0/SUCCESS)
Main PID: 2340 (code=exited, status=0/SUCCESS)
Nov 14 21:54:33 srv dockerd[2340]: time="2019-11-14T21:54:33.308531424Z" level=warning msg="a9e21e92e2dd297a68f68441353fc3bda39d0bb5564b60d402ae651fa80f5c72 cleanu
Nov 14 21:54:46 srv dockerd[2340]: time="2019-11-14T21:54:46.394643530Z" level=info msg="Container 5d025e7f93a45a50dbbaa87c55d7cdbbf6515bbe1d45ff599074f1cdcf320a0c
Nov 14 21:54:46 srv dockerd[2340]: time="2019-11-14T21:54:46.757171067Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete ty
Nov 14 21:54:47 srv dockerd[2340]: time="2019-11-14T21:54:47.031709355Z" level=warning msg="5d025e7f93a45a50dbbaa87c55d7cdbbf6515bbe1d45ff599074f1cdcf320a0c cleanu
Nov 14 21:54:57 srv systemd[1]: Stopping Docker Application Container Engine...
Nov 14 21:54:57 srv dockerd[2340]: time="2019-11-14T21:54:57.439296168Z" level=info msg="Processing signal 'terminated'"
Nov 14 21:54:57 srv dockerd[2340]: time="2019-11-14T21:54:57.447803201Z" level=info msg="Daemon shutdown complete"
Nov 14 21:54:57 srv dockerd[2340]: time="2019-11-14T21:54:57.449422219Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled
Nov 14 21:54:57 srv dockerd[2340]: time="2019-11-14T21:54:57.449576789Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled
Nov 14 21:54:57 srv systemd[1]: Stopped Docker Application Container Engine.
myuser@srv:~# cat /var/lib/docker/containers/5d025e7f93a45a50dbbaa87c55d7cdbbf6515bbe1d45ff599074f1cdcf320a0c/hostconfig.json
{"Binds":["/srv/gitlab/config:/etc/gitlab","/srv/gitlab/logs:/var/log/gitlab","/srv/gitlab/data:/var/opt/gitlab"],"ContainerIDFile":"","LogConfig":{"Type":"json-file","Config":{}},"NetworkMode":"default","PortBindings":{"22/tcp":[{"HostIp":"","HostPort":"1022"}],"4567/tcp":[{"HostIp":"","HostPort":"4567"}],"80/tcp":[{"HostIp":"","HostPort":"80"}]},"RestartPolicy":{"Name":"always","MaximumRetryCount":0},"AutoRemove":false,"VolumeDriver":"","VolumesFrom":null,"CapAdd":null,"CapDrop":null,"Capabilities":null,"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IpcMode":"private","Cgroup":"","Links":null,"OomScoreAdj":0,"PidMode":"","Privileged":false,"PublishAllPorts":false,"ReadonlyRootfs":false,"SecurityOpt":null,"UTSMode":"","UsernsMode":"","ShmSize":67108864,"Runtime":"runc","ConsoleSize":[0,0],"Isolation":"","CpuShares":0,"Memory":0,"NanoCpus":0,"CgroupParent":"","BlkioWeight":0,"BlkioWeightDevice":[],"BlkioDeviceReadBps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteIOps":null,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpusetCpus":"","CpusetMems":"","Devices":[],"DeviceCgroupRules":null,"DeviceRequests":null,"KernelMemory":0,"KernelMemoryTCP":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":null,"OomKillDisable":false,"PidsLimit":null,"Ulimits":null,"CpuCount":0,"CpuPercent":0,"IOMaximumIOps":0,"IOMaximumBandwidth":0,"MaskedPaths":["/proc/asound","/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware"],"ReadonlyPaths":["/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]
myuser@srv:~# nano /var/lib/docker/containers/5d025e7f93a45a50dbbaa87c55d7cdbbf6515bbe1d45ff599074f1cdcf320a0c/hostconfig.json
myuser@srv:~# systemctl start docker
myuser@srv:~# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2019-11-14 22:12:06 UTC; 2s ago
Docs: https://docs.docker.com
Main PID: 4693 (dockerd)
Tasks: 54
CGroup: /system.slice/docker.service
├─4693 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
├─4867 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4567 -container-ip 172.17.0.3 -container-port 4567
├─4881 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.17.0.3 -container-port 443
├─4895 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 1022 -container-ip 172.17.0.3 -container-port 22
└─4907 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.17.0.3 -container-port 80
Nov 14 22:12:04 srv dockerd[4693]: time="2019-11-14T22:12:04.034007956Z" level=warning msg="Your kernel does not support swap memory limit"
Nov 14 22:12:04 srv dockerd[4693]: time="2019-11-14T22:12:04.034062799Z" level=warning msg="Your kernel does not support cgroup rt period"
Nov 14 22:12:04 srv dockerd[4693]: time="2019-11-14T22:12:04.034074070Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Nov 14 22:12:04 srv dockerd[4693]: time="2019-11-14T22:12:04.034361581Z" level=info msg="Loading containers: start."
Nov 14 22:12:04 srv dockerd[4693]: time="2019-11-14T22:12:04.344354207Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Dae
Nov 14 22:12:05 srv dockerd[4693]: time="2019-11-14T22:12:05.916782317Z" level=info msg="Loading containers: done."
Nov 14 22:12:05 srv dockerd[4693]: time="2019-11-14T22:12:05.988204406Z" level=info msg="Docker daemon" commit=9013bf583a graphdriver(s)=overlay2 version=19.03.4
Nov 14 22:12:05 srv dockerd[4693]: time="2019-11-14T22:12:05.988317448Z" level=info msg="Daemon has completed initialization"
Nov 14 22:12:06 srv dockerd[4693]: time="2019-11-14T22:12:06.010801856Z" level=info msg="API listen on /var/run/docker.sock"
Nov 14 22:12:06 srv systemd[1]: Started Docker Application Container Engine.
myuser@srv:~# sudo docker start gitlab-runner
gitlab-runner
myuser@srv:~# sudo docker start gitlab
gitlab
myuser@srv:~# sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a9e21e92e2dd gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 2 days ago Up 19 seconds gitlab-runner
5d025e7f93a4 gitlab/gitlab-ce:latest "/assets/wrapper" 3 days ago Up 19 seconds (health: starting) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4567->4567/tcp, 0.0.0.0:1022->22/tcp gitlab
myuser@srv:~# wget --no-check-certificate https://192.168.0.238/
--2019-11-14 22:13:30-- https://192.168.0.238/
Connecting to 192.168.0.238:443... connected.
WARNING: certificate common name ‘gitlab.ahelpme.com’ doesn't match requested host name ‘192.168.0.238’.
HTTP request sent, awaiting response... 302 Found
Location: https://192.168.0.238/users/sign_in [following]
--2019-11-14 22:13:30-- https://192.168.0.238/users/sign_in
Reusing existing connection to 192.168.0.238:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’
index.html [ <=> ] 12.41K --.-KB/s in 0s
2019-11-14 22:13:31 (134 MB/s) - ‘index.html’ saved [12708]
Change the ports or add more ports in “PortBindings”. The syntax is pretty straightforward just mind the comas, [] and {}.
"PortBindings":{"22/tcp":[{"HostIp":"","HostPort":"1022"}],"4567/tcp":[{"HostIp":"","HostPort":"4567"}],"80/tcp":[{"HostIp":"","HostPort":"80"}]}
Here we change the mapping from “host port 1022 to 22” to “host port 2222 to 22” just replacing the “1022” to “2222”:
"PortBindings":{"22/tcp":[{"HostIp":"","HostPort":"2222"}],"4567/tcp":[{"HostIp":"","HostPort":"4567"}],"80/tcp":[{"HostIp":"","HostPort":"80"}]}
And the second example is in addition to the 2222 change we want to add another mapping “host from 443 to 443” (open the HTTPS), just add new group with the above syntax:
"PortBindings":{"22/tcp":[{"HostIp":"","HostPort":"2222"}],"4567/tcp":[{"HostIp":"","HostPort":"4567"}],"80/tcp":[{"HostIp":"","HostPort":"80"}],"443/tcp":[{"HostIp":"","HostPort":"443"}]}
A note!
Probably there may be an idea not to be easy to add mapped ports when you think one of the main Docker goals is to isolate services per a Docker instance. It sounds strange to have a docker container for one service exporting a number of ports (or a single port) and later why you would need to expose another port? For another service in the same container, but you should use a separate container, not the same one!
But more and more Docker containers are used also to deliver a fine-tuned environment of a whole platform, which provides multiple services in a single docker container. Let’s take an example – GitLab, which offers installation in a Docker container hosting more than 10 services in a single container!