Change the location of container storage in podman (with SELinux enabled)

There two main options to change the location of all the containers’ storages:

• “mount bind” the new location to the default storage directory (look Note 1)
• Change the path of the location in the configuration file /etc/containers/storage.conf

You should stop all your containers though it is not mandatory.

You should stop the containers (if any) and copy the directory, because when reconfigured the storage path podman won’t access the ones in the old path – containers and images!

STEP 1) Change the storage path in the podman configuration file.

If the SELinux has been disabled, which should not be done, it is just a matter of changing a path option in the configuration file /etc/containers/storage.conf

# Primary Read/Write location of container storage
graphroot = "/var/lib/containers/storage"

Change it to whatever path you like. Mostly, it should point to the big storage device. In our case, the big storage is mounted under “/mnt/mystorage/virtual/storage”. Change the options to:

# Primary Read/Write location of container storage
graphroot = "/mnt/mystorage/virtual/storage"

Check the running configuration with:

[root@lsrv1 mystorage]# podman info
host:
  BuildahVersion: 1.12.0-dev
  CgroupVersion: v1
  Conmon:
    package: conmon-2.0.8-1.el7.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.8, commit: f85c8b1ce77b73bcd48b2d802396321217008762'
  Distribution:
    distribution: '"centos"'
    version: "7"
  MemFree: 191963136
  MemTotal: 16563531776
  OCIRuntime:
    name: runc
    package: runc-1.0.0-67.rc10.el7_8.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree: 7857680384
  SwapTotal: 8581541888
  arch: amd64
  cpus: 8
  eventlogger: journald
  hostname: lsrv1
  kernel: 3.10.0-1062.9.1.el7.x86_64
  os: linux
  rootless: false
  uptime: 607h 10m 53.36s (Approximately 25.29 days)
registries:
  blocked: null
  insecure: null
  search:
  - registry.access.redhat.com
  - registry.fedoraproject.org
  - registry.centos.org
  - docker.io
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions: {}
  GraphRoot: /mnt/mystorage/virtual/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 0
  RunRoot: /var/run/containers/storage
  VolumePath: /mnt/mystorage/virtual/storage/volumes

STEP 2*) Add SELinux rules if it is in enforcing mode.

*And only if you are using SELinux in enforcing mode you must transfer the rules for the new location. Just list the SELinux rules, pick only those with the old path, and add the same rules with the new path.
First, check the status of the SELinux with:

[root@srv ~]# getenforce 
Enforcing

Add the rules for the new path:

[root@lsrv1 ~]# semanage fcontext -l|grep "/var/lib/containers"
/var/lib/containers(/.*)?                          all files          system_u:object_r:container_var_lib_t:s0 
/var/lib/containers/home(/.*)?                     all files          system_u:object_r:openshift_var_lib_t:s0 
/var/lib/containers/atomic(/.*)?                   all files          <<None>>
/var/lib/containers/overlay(/.*)?                  all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/overlay2(/.*)?                 all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/overlay-layers(/.*)?           all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/overlay-images(/.*)?           all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/overlay2-layers(/.*)?          all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/overlay2-images(/.*)?          all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/overlay(/.*)?          all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/overlay2(/.*)?         all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/volumes/[^/]*/.*       all files          system_u:object_r:container_file_t:s0 
/var/lib/containers/storage/overlay-layers(/.*)?   all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/overlay-images(/.*)?   all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/overlay2-layers(/.*)?  all files          system_u:object_r:container_share_t:s0 
/var/lib/containers/storage/overlay2-images(/.*)?  all files          system_u:object_r:container_share_t:s0

It’s simple engough – add an equivalent directory path with semanage and then relabel the path.

[root@lsrv1 mystorage]# semanage fcontext -a -e /var/lib/containers /mnt/mystorage/virtual/storage
[root@lsrv1 mystorage]# restorecon -Rv /mnt/mystorage/virtual/storage
restorecon reset /mnt/mystorage/virtual/storage context unconfined_u:object_r:mnt_t:s0->unconfined_u:object_r:container_var_lib_t:s0

Note, the target is not allowed to end with ‘/’.

Note 1 – mount bind

Use mount bind to change the directory to new location.

mount -o bind /mnt/mystorage/virtual/storage /var/lib/containers

And add it to the /etc/fstab to mount it always on boot:

/mnt/mystorage/virtual/storage          /var/lib/containers     bind            bind                            0 0

First, rename (or copy/rsync the content of it to a temporary location) /var/lib/containers and then “mount bind” it and then copy back the original content from temporary location to preseve all the podman data before the change.
Just to note when using SELinux you should check STEP 2) above and add equivalent direcotry, becuase if relabeling occurs on the storage device could overwrite the SELinux context labels.

*Note 2 – podman temporary data

There is one more storage options in the file, which is used for temporary data and it is located in /var/run (links to /run – a tmpfs directory).

# Temporary storage location
runroot = "/var/run/containers/storage"

Apparently, it stores temporary files like container’s specific confguration file and data for running containers. The data does not occupy much space so the temporary storage directory may not be moved to the big storage device. Still, if the temporary storage directory should be moved to another location and SELinux is enabled the STEP 2) must be taken into consideration.
Here is what kind of data to expect:

[root@lsrv1 storage]# ls -altr /var/run/containers/storage/overlay-containers/65deb46b790795505e94f618f2f9b4a381d56a241a5d976dda9ccb1ba5e6021f/userdata/
total 20
drwx------. 3 root root  60 28 May  9,42 ..
-rw-r--r--. 1 root root 179 28 May  9,42 resolv.conf
-rw-r--r--. 1 root root 361 28 May  9,42 hosts
-rw-r--r--. 1 root root  12 28 May  9,42 hostname
-rw-r--r--. 1 root root   0 28 May  9,42 .containerenv
drwxr-xr-x. 3 root root  60 28 May  9,42 run
-rw-r--r--. 1 root root   5 28 May  9,42 conmon.pid
-rw-r--r--. 1 root root   0 28 May  9,42 oci-log
-rw-r--r--. 1 root root   5 28 May  9,42 pidfile