Change the LXC container root folder under CentOS with SELinux

The default LXC container folder in CentOS (all versions – 7,8, Stream 8 and Stream 9) is /var/lib/lxc, which may resides in the root partition. When changing the lxc.rootfs or (the main directory /var/lib/lxc) to another place, the containers may still work without any additional SELinux permissions. Some tools like lxc-attach would definitely stop working with permission errors – lxc_attach_run_shell: 1333 Permission denied – failed to exec shell. This article will show how to use lxc-create and SELinux commands to properly change the LXC container’s rootfs.
For detailed information how to create a LXC container check out – Run LXC CentOS Stream 9 container with bridged network under CentOS Stream 9 or Run LXC Ubuntu 22.04 LTS container with bridged network under CentOS Stream 9.

Create LXC container with not default path

  • Change the rootfs only. To change only the LXC container root filesystem location use “–dir=” lxc-create option:
    lxc-create --template download -n mycontainer2 --dir=/mnt/storage/servers/mycontainer2 -- --dist centos --release 9-Stream --arch amd64
    

    It will place the files under /mnt/storage/servers/mycontainer2, but the configuration will still be located in /var/lib/lxc/mycontainer2/.

    [root@srv ~]# ls -altr /var/lib/lxc/mycontainer2/
    total 16
    drwxr-xr-x. 3 root root 4096 Oct 14 13:42 ..
    drwxr-xr-x. 2 root root 4096 Oct 14 13:42 rootfs
    -rw-r-----. 1 root root  775 Oct 14 13:42 config
    drwxrwx---. 3 root root 4096 Oct 14 13:42 .
    [root@srv ~]# ls -altr /var/lib/lxc/mycontainer2/rootfs/
    total 8
    drwxr-xr-x. 2 root root 4096 Oct 14 13:42 .
    drwxrwx---. 3 root root 4096 Oct 14 13:42 ..
    [root@srv ~]# ls -altr /mnt/storage/servers/mycontainer2/
    total 76
    drwxrwxrwt.  2 root root 4096 Aug  9  2021 tmp
    drwxr-xr-x.  2 root root 4096 Aug  9  2021 srv
    lrwxrwxrwx.  1 root root    8 Aug  9  2021 sbin -> usr/sbin
    drwxr-xr-x.  2 root root 4096 Aug  9  2021 opt
    drwxr-xr-x.  2 root root 4096 Aug  9  2021 mnt
    drwxr-xr-x.  2 root root 4096 Aug  9  2021 media
    lrwxrwxrwx.  1 root root    9 Aug  9  2021 lib64 -> usr/lib64
    lrwxrwxrwx.  1 root root    7 Aug  9  2021 lib -> usr/lib
    drwxr-xr-x.  2 root root 4096 Aug  9  2021 home
    dr-xr-xr-x.  2 root root 4096 Aug  9  2021 boot
    lrwxrwxrwx.  1 root root    7 Aug  9  2021 bin -> usr/bin
    dr-xr-xr-x.  2 root root 4096 Aug  9  2021 afs
    dr-xr-xr-x.  2 root root 4096 Oct 14 07:11 sys
    dr-xr-xr-x.  2 root root 4096 Oct 14 07:11 proc
    drwxr-xr-x. 12 root root 4096 Oct 14 07:11 usr
    drwxr-xr-x.  8 root root 4096 Oct 14 07:11 run
    drwxr-xr-x. 18 root root 4096 Oct 14 07:11 var
    dr-xr-x---.  2 root root 4096 Oct 14 07:12 root
    drwxr-xr-x.  2 root root 4096 Oct 14 07:12 selinux
    drwxr-xr-x. 19 root root 4096 Oct 14 07:15 .
    drwxr-xr-x.  4 root root 4096 Oct 14 13:41 ..
    drwxr-xr-x.  3 root root 4096 Oct 14 13:42 dev
    drwxr-xr-x. 63 root root 4096 Oct 14 13:42 etc
    
  • Change the LXC container path – the folder containing the configuration and the container’s root filesystems use “-P”
    lxc-create -P /mnt/storage/servers/ --template download -n mycontainer -- --dist centos --release 9-Stream --arch amd64
    

    All the LXC container configuration and root filesystem will be placed under /mnt/storage/servers/[container_name], which in the example above is /mnt/storage/servers/mycontainer

    [root@srv ~]# ls -al /mnt/storage/servers/mycontainer
    total 16
    drwxrwx---.  3 root root 4096 Oct 14 13:38 .
    drwxr-xr-x.  4 root root 4096 Oct 14 13:41 ..
    -rw-r-----.  1 root root  780 Oct 14 13:38 config
    drwxr-xr-x. 19 root root 4096 Oct 14 07:15 rootfs
    

It is better to use the “-P” and to change the LXC container location than only the filesystem path. In this case, a good practice is to make a symbolic link in /var/lib/lxc/[container-name] to the new location:

ln -s /mnt/storage/servers/mycontainer /var/lib/lxc/mycontainer

So all LXC tools will continue to work without explicitly adding an option for the new path of this container.

Change the SELinux file context to be container_var_lib_t of the LXC root filesystem

Add the file context container_var_lib_t to the container’s root filesystem path and change the SELinux labels.
First, verify all the needed tools are installed:

dnf install -y policycoreutils-python-utils container-selinux

Then, add a new file context to the path /mnt/storage/servers/mycontainer and run the restorecon to change the SELinux labels to container_var_lib_t

semanage fcontext -a -t container_var_lib_t '/mnt/storage/servers/mycontainer(/.*)?'
restorecon -Rv /mnt/storage/servers/mycontainer

The file context may be shown with:

[root@srv ~]# ls -alZ /mnt/storage/servers/mycontainer
total 16
drwxrwx---.  3 root root unconfined_u:object_r:container_var_lib_t:s0 4096 Oct 14 13:38 .
drwxr-xr-x.  4 root root unconfined_u:object_r:mnt_t:s0               4096 Oct 14 13:41 ..
-rw-r-----.  1 root root unconfined_u:object_r:container_var_lib_t:s0  780 Oct 14 13:38 config
drwxr-xr-x. 19 root root unconfined_u:object_r:container_var_lib_t:s0 4096 Oct 14 07:15 rootfs

Failing to set the proper SELinux labels may result to errors such as lxc_attach_run_shell: 1333 Permission denied – failed to exec shell

collectd nginx plugin: curl_easy_perform failed because of selinux

Enabling the Nginx plugin for collectd under CentOS (or any other system using SELinux) might be confusing for a newbie. Most sources on the Internet would just install collectd-nginx:

yum install -y collectd-nginx

and configure it in the nginx.conf and collectd.conf. Still, the statistics might not work as expected, the collectd may not be able to gather statistics from the Nginx.

SELinux may prevent collectd (plugin) daemon to connect to Nginx and gather statistics from the Nginx stats page.

Checking the collectd log and it reports a problem:
Keep on reading!