Create and export a GlusterFS volume with NFS-Ganesha in CentOS 8

GlusterFS built-in NFS server supports only NFS version 3. GlusterFS offers NFS exports using NFS-Ganesha, which supports NFS version 3 and 4 protocols.
NFS-Ganesha server is a user-mode file sharing server, which offers a GlusterFS plugin to export GlusterFS volumes. In the following article, the NSF-Ganesha and GlusterFS are installed and a simple GlusterFS volume is created and then exported through NFS 3 and 4 version protocols.
The version of the software in this article:

  • CentOS Stream release 8 (25.04.2021)
  • GlusterFS 8.4
  • NFS-Ganesha 3.5

STEP 1) Install GlusterFS.

dnf install -y centos-release-gluster
dnf install -y glusterfs-server

The first line will installs a new repository under the SIG management – https://wiki.centos.org/SpecialInterestGroup/Storage. The second line installs the GlusterFS server.

STEP 2) Install NFS-Ganesha.

dnf install -y centos-release-nfs-ganesha30
dnf install -y nfs-ganesha nfs-ganesha-gluster

The first line again installs a new repository under the SIG management and the second line installs the NFS-Ganesha server with Gluster plugin.

STEP 3) Create GlusterFS volume

Start the GlusterFS server and create a simple 3 replicas volume with:
Start the GlusterFS on all the three nodes and enable the GlusterFS communication between the three nodes using firewall-cmd utility. So execute the following commands:

systemctl start glusterd
firewall-cmd --permanent --new-zone=glusternodes
firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.200
firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.201
firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.202
firewall-cmd --permanent --zone=glusternodes --add-service=glusterfs
firewall-cmd --reload

On the first node create the GlusterFS volume. First, add the glnode2 and glnode3 to the cluster.

gluster peer probe glnode2
gluster peer probe glnode3
gluster volume create VOL1 replica 3 transport tcp glnode1:/mnt/storage/gluster/brick glnode2:/mnt/storage/gluster/brick glnode3:/mnt/storage/gluster/brick
gluster volume start VOL1


glnode1, glnode2, glnode3 are names, which are added to the /etc/hosts (Of course, a FQDN could be used). The /mnt/storage/gluster/brick is the path, where GlusterFS will use to put its files. It may resides on a different (storage) partition.
Check out the detailed article about creating a GlusterFS 3 replicas volume here: glusterfs with localhost (127.0.0.1) nodes on different servers – glusterfs volume with 3 replicas Of course, 127.0.0.1 may not be used. Everything depends on the servers’ hostnames and their DNS records. A similar example, but not using localhost will be:

192.168.0.200 glnode1
192.168.0.201 glnode2
192.168.0.202 glnode3

STEP 4) Export the GlusterFS volume with NFS share using NFS-Ganesha

The configuration file is /etc/ganesha/ganesha.conf. Include the following configuration:

EXPORT
{
    Export_Id = 1;
    Path = "/VOL1";

    FSAL {
        name = GLUSTER;
        hostname = "glnode1";
        volume = "VOL1";
    }

    Access_type = RW;
    Squash = No_root_squash;
    Disable_ACL = TRUE;
    Pseudo = "/VOL1";
    Protocols = "3","4";
    Transports = "UDP","TCP";
    SecType = "sys";

    CLIENT
    {
        Clients = 127.0.0.1;
    }
}

And then start the NFS-Ganesha daemon:

systemctl start nfs-ganesha

Here are some details about the EXPORT options:

  • Export_Id – Unique ID for each export. Any number, ensure it is unique for the whole configuration.
  • Path – The path GlusterFS volume would be exported. Mounting in the clients with IP:”Path”. The path name may not be the original name as of the GlusterFS volume.
  • FSAL – File System Abstraction Layer. The plugin, which will be used to export a file system. In the case, GlusterFS volume to be exported, so Gluster plugin is used. hostname – the hostname or IP, where the GlusterFS is accessible. volume – the name of the GlusterFS volume to export.
  • Access_type – Read only or read write access is exported.
  • Pseudo – NFSv4 pseudo path for this export.
  • Protocols – which NFS protocols to be suported.
  • Transports – which transport protocols to be supported.
  • CLIENT – the section control the access limitation by IP, password and so on. In addtion some of the export options above may be overidden. Multiple CLIENT sections may exist.

More EXPORT options here – https://github.com/phdeniel/nfs-ganesha/blob/master/src/config_samples/export.txt

Mount the share.

Mounting the above share with mount command:

mount -t nfs glnode1:/VOL1 /mnt/VOL1/

And /etc/fstab example:

glnode1:/VOL1 /mnt/VOL1  nfs defaults,hard,intr,noexec,nosuid,_netdev,fsc,vers=4 0 0

The whole output

[root@node1 ~]# dnf install -y centos-release-gluster
Last metadata expiration check: 0:02:42 ago on Sun Apr 25 09:45:25 2021.
Dependencies resolved.
=========================================================================================================
 Package                                  Architecture      Version              Repository         Size
=========================================================================================================
Installing:
 centos-release-gluster8                  noarch            1.0-1.el8            extras            9.3 k
Installing dependencies:
 centos-release-storage-common            noarch            2-2.el8              extras            9.4 k

Transaction Summary
=========================================================================================================
Install  2 Packages

Total download size: 19 k
Installed size: 2.4 k
Downloading Packages:
(1/2): centos-release-gluster8-1.0-1.el8.noarch.rpm                      109 kB/s | 9.3 kB     00:00    
(2/2): centos-release-storage-common-2-2.el8.noarch.rpm                   96 kB/s | 9.4 kB     00:00    
---------------------------------------------------------------------------------------------------------
Total                                                                     39 kB/s |  19 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                 1/1 
  Installing       : centos-release-storage-common-2-2.el8.noarch                                    1/2 
  Installing       : centos-release-gluster8-1.0-1.el8.noarch                                        2/2 
  Running scriptlet: centos-release-gluster8-1.0-1.el8.noarch                                        2/2 
  Verifying        : centos-release-gluster8-1.0-1.el8.noarch                                        1/2 
  Verifying        : centos-release-storage-common-2-2.el8.noarch                                    2/2 

Installed:
  centos-release-gluster8-1.0-1.el8.noarch          centos-release-storage-common-2-2.el8.noarch         

Complete!
[root@node1 ~]# dnf install -y glusterfs-server
CentOS-8 - Gluster 8                                                     129 kB/s |  50 kB     00:00    
Dependencies resolved.
=========================================================================================================
 Package                              Architecture  Version                 Repository              Size
=========================================================================================================
Installing:
 glusterfs-server                     x86_64        8.4-1.el8               centos-gluster8        1.4 M
Installing dependencies:
 attr                                 x86_64        2.4.48-3.el8            baseos                  68 k
 device-mapper-event                  x86_64        8:1.02.175-5.el8        baseos                 269 k
 device-mapper-event-libs             x86_64        8:1.02.175-5.el8        baseos                 269 k
 device-mapper-persistent-data        x86_64        0.8.5-4.el8             baseos                 468 k
 glusterfs                            x86_64        8.4-1.el8               centos-gluster8        689 k
 glusterfs-cli                        x86_64        8.4-1.el8               centos-gluster8        214 k
 glusterfs-client-xlators             x86_64        8.4-1.el8               centos-gluster8        899 k
 glusterfs-fuse                       x86_64        8.4-1.el8               centos-gluster8        171 k
 libaio                               x86_64        0.3.112-1.el8           baseos                  33 k
 libgfapi0                            x86_64        8.4-1.el8               centos-gluster8        125 k
 libgfchangelog0                      x86_64        8.4-1.el8               centos-gluster8         67 k
 libgfrpc0                            x86_64        8.4-1.el8               centos-gluster8         89 k
 libgfxdr0                            x86_64        8.4-1.el8               centos-gluster8         61 k
 libglusterd0                         x86_64        8.4-1.el8               centos-gluster8         45 k
 libglusterfs0                        x86_64        8.4-1.el8               centos-gluster8        350 k
 lvm2                                 x86_64        8:2.03.11-5.el8         baseos                 1.6 M
 lvm2-libs                            x86_64        8:2.03.11-5.el8         baseos                 1.1 M
 psmisc                               x86_64        23.1-5.el8              baseos                 151 k
 python3-pyxattr                      x86_64        0.5.3-18.el8            centos-gluster8         35 k
 rpcbind                              x86_64        1.2.5-8.el8             baseos                  70 k
 userspace-rcu                        x86_64        0.10.1-4.el8            baseos                 101 k

Transaction Summary
=========================================================================================================
Install  22 Packages

Total download size: 8.2 M
Installed size: 24 M
Downloading Packages:
(1/22): glusterfs-cli-8.4-1.el8.x86_64.rpm                               2.0 MB/s | 214 kB     00:00    
(2/22): glusterfs-fuse-8.4-1.el8.x86_64.rpm                              2.4 MB/s | 171 kB     00:00    
(3/22): glusterfs-client-xlators-8.4-1.el8.x86_64.rpm                    3.5 MB/s | 899 kB     00:00    
(4/22): glusterfs-8.4-1.el8.x86_64.rpm                                   2.6 MB/s | 689 kB     00:00    
(5/22): libgfapi0-8.4-1.el8.x86_64.rpm                                   3.9 MB/s | 125 kB     00:00    
(6/22): libgfchangelog0-8.4-1.el8.x86_64.rpm                             2.6 MB/s |  67 kB     00:00    
(7/22): libgfrpc0-8.4-1.el8.x86_64.rpm                                   3.5 MB/s |  89 kB     00:00    
(8/22): libgfxdr0-8.4-1.el8.x86_64.rpm                                   2.5 MB/s |  61 kB     00:00    
(9/22): libglusterd0-8.4-1.el8.x86_64.rpm                                2.7 MB/s |  45 kB     00:00    
(10/22): python3-pyxattr-0.5.3-18.el8.x86_64.rpm                         2.1 MB/s |  35 kB     00:00    
(11/22): libglusterfs0-8.4-1.el8.x86_64.rpm                              3.7 MB/s | 350 kB     00:00    
(12/22): glusterfs-server-8.4-1.el8.x86_64.rpm                           4.7 MB/s | 1.4 MB     00:00    
(13/22): attr-2.4.48-3.el8.x86_64.rpm                                    473 kB/s |  68 kB     00:00    
(14/22): device-mapper-event-1.02.175-5.el8.x86_64.rpm                   1.9 MB/s | 269 kB     00:00    
(15/22): libaio-0.3.112-1.el8.x86_64.rpm                                 971 kB/s |  33 kB     00:00    
(16/22): device-mapper-persistent-data-0.8.5-4.el8.x86_64.rpm            4.3 MB/s | 468 kB     00:00    
(17/22): device-mapper-event-libs-1.02.175-5.el8.x86_64.rpm              2.0 MB/s | 269 kB     00:00    
(18/22): psmisc-23.1-5.el8.x86_64.rpm                                    1.3 MB/s | 151 kB     00:00    
(19/22): rpcbind-1.2.5-8.el8.x86_64.rpm                                  1.0 MB/s |  70 kB     00:00    
(20/22): userspace-rcu-0.10.1-4.el8.x86_64.rpm                           1.6 MB/s | 101 kB     00:00    
(21/22): lvm2-libs-2.03.11-5.el8.x86_64.rpm                              3.5 MB/s | 1.1 MB     00:00    
(22/22): lvm2-2.03.11-5.el8.x86_64.rpm                                   4.3 MB/s | 1.6 MB     00:00    
---------------------------------------------------------------------------------------------------------
Total                                                                    5.0 MB/s | 8.2 MB     00:01     
warning: /var/cache/dnf/centos-gluster8-ae72c2c38de8ee20/packages/glusterfs-8.4-1.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e451e5b5: NOKEY
CentOS-8 - Gluster 8                                                     1.0 MB/s | 1.0 kB     00:00    
Importing GPG key 0xE451E5B5:
 Userid     : "CentOS Storage SIG (http://wiki.centos.org/SpecialInterestGroup/Storage) <security@centos.org>"
 Fingerprint: 7412 9C0B 173B 071A 3775 951A D4A2 E50B E451 E5B5
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                 1/1 
  Installing       : libgfxdr0-8.4-1.el8.x86_64                                                     1/22 
  Running scriptlet: libgfxdr0-8.4-1.el8.x86_64                                                     1/22 
  Installing       : libglusterfs0-8.4-1.el8.x86_64                                                 2/22 
  Running scriptlet: libglusterfs0-8.4-1.el8.x86_64                                                 2/22 
  Installing       : libgfrpc0-8.4-1.el8.x86_64                                                     3/22 
  Running scriptlet: libgfrpc0-8.4-1.el8.x86_64                                                     3/22 
  Installing       : libaio-0.3.112-1.el8.x86_64                                                    4/22 
  Installing       : glusterfs-client-xlators-8.4-1.el8.x86_64                                      5/22 
  Installing       : device-mapper-event-libs-8:1.02.175-5.el8.x86_64                               6/22 
  Running scriptlet: glusterfs-8.4-1.el8.x86_64                                                     7/22 
  Installing       : glusterfs-8.4-1.el8.x86_64                                                     7/22 
  Running scriptlet: glusterfs-8.4-1.el8.x86_64                                                     7/22 
  Installing       : libglusterd0-8.4-1.el8.x86_64                                                  8/22 
  Running scriptlet: libglusterd0-8.4-1.el8.x86_64                                                  8/22 
  Installing       : glusterfs-cli-8.4-1.el8.x86_64                                                 9/22 
  Installing       : device-mapper-event-8:1.02.175-5.el8.x86_64                                   10/22 
  Running scriptlet: device-mapper-event-8:1.02.175-5.el8.x86_64                                   10/22 
  Installing       : lvm2-libs-8:2.03.11-5.el8.x86_64                                              11/22 
  Installing       : libgfapi0-8.4-1.el8.x86_64                                                    12/22 
  Running scriptlet: libgfapi0-8.4-1.el8.x86_64                                                    12/22 
  Installing       : device-mapper-persistent-data-0.8.5-4.el8.x86_64                              13/22 
  Installing       : lvm2-8:2.03.11-5.el8.x86_64                                                   14/22 
  Running scriptlet: lvm2-8:2.03.11-5.el8.x86_64                                                   14/22 
  Installing       : libgfchangelog0-8.4-1.el8.x86_64                                              15/22 
  Running scriptlet: libgfchangelog0-8.4-1.el8.x86_64                                              15/22 
  Installing       : userspace-rcu-0.10.1-4.el8.x86_64                                             16/22 
  Running scriptlet: userspace-rcu-0.10.1-4.el8.x86_64                                             16/22 
  Running scriptlet: rpcbind-1.2.5-8.el8.x86_64                                                    17/22 
  Installing       : rpcbind-1.2.5-8.el8.x86_64                                                    17/22 
  Running scriptlet: rpcbind-1.2.5-8.el8.x86_64                                                    17/22 
  Installing       : psmisc-23.1-5.el8.x86_64                                                      18/22 
  Installing       : attr-2.4.48-3.el8.x86_64                                                      19/22 
  Installing       : glusterfs-fuse-8.4-1.el8.x86_64                                               20/22 
  Installing       : python3-pyxattr-0.5.3-18.el8.x86_64                                           21/22 
  Installing       : glusterfs-server-8.4-1.el8.x86_64                                             22/22 
  Running scriptlet: glusterfs-server-8.4-1.el8.x86_64                                             22/22 
  Verifying        : glusterfs-8.4-1.el8.x86_64                                                     1/22 
  Verifying        : glusterfs-cli-8.4-1.el8.x86_64                                                 2/22 
  Verifying        : glusterfs-client-xlators-8.4-1.el8.x86_64                                      3/22 
  Verifying        : glusterfs-fuse-8.4-1.el8.x86_64                                                4/22 
  Verifying        : glusterfs-server-8.4-1.el8.x86_64                                              5/22 
  Verifying        : libgfapi0-8.4-1.el8.x86_64                                                     6/22 
  Verifying        : libgfchangelog0-8.4-1.el8.x86_64                                               7/22 
  Verifying        : libgfrpc0-8.4-1.el8.x86_64                                                     8/22 
  Verifying        : libgfxdr0-8.4-1.el8.x86_64                                                     9/22 
  Verifying        : libglusterd0-8.4-1.el8.x86_64                                                 10/22 
  Verifying        : libglusterfs0-8.4-1.el8.x86_64                                                11/22 
  Verifying        : python3-pyxattr-0.5.3-18.el8.x86_64                                           12/22 
  Verifying        : attr-2.4.48-3.el8.x86_64                                                      13/22 
  Verifying        : device-mapper-event-8:1.02.175-5.el8.x86_64                                   14/22 
  Verifying        : device-mapper-event-libs-8:1.02.175-5.el8.x86_64                              15/22 
  Verifying        : device-mapper-persistent-data-0.8.5-4.el8.x86_64                              16/22 
  Verifying        : libaio-0.3.112-1.el8.x86_64                                                   17/22 
  Verifying        : lvm2-8:2.03.11-5.el8.x86_64                                                   18/22 
  Verifying        : lvm2-libs-8:2.03.11-5.el8.x86_64                                              19/22 
  Verifying        : psmisc-23.1-5.el8.x86_64                                                      20/22 
  Verifying        : rpcbind-1.2.5-8.el8.x86_64                                                    21/22 
  Verifying        : userspace-rcu-0.10.1-4.el8.x86_64                                             22/22 

Installed:
  attr-2.4.48-3.el8.x86_64                            device-mapper-event-8:1.02.175-5.el8.x86_64        
  device-mapper-event-libs-8:1.02.175-5.el8.x86_64    device-mapper-persistent-data-0.8.5-4.el8.x86_64   
  glusterfs-8.4-1.el8.x86_64                          glusterfs-cli-8.4-1.el8.x86_64                     
  glusterfs-client-xlators-8.4-1.el8.x86_64           glusterfs-fuse-8.4-1.el8.x86_64                    
  glusterfs-server-8.4-1.el8.x86_64                   libaio-0.3.112-1.el8.x86_64                        
  libgfapi0-8.4-1.el8.x86_64                          libgfchangelog0-8.4-1.el8.x86_64                   
  libgfrpc0-8.4-1.el8.x86_64                          libgfxdr0-8.4-1.el8.x86_64                         
  libglusterd0-8.4-1.el8.x86_64                       libglusterfs0-8.4-1.el8.x86_64                     
  lvm2-8:2.03.11-5.el8.x86_64                         lvm2-libs-8:2.03.11-5.el8.x86_64                   
  psmisc-23.1-5.el8.x86_64                            python3-pyxattr-0.5.3-18.el8.x86_64                
  rpcbind-1.2.5-8.el8.x86_64                          userspace-rcu-0.10.1-4.el8.x86_64                  

Complete!
[root@node1 ~]# dnf install -y centos-release-nfs-ganesha30
Last metadata expiration check: 0:00:22 ago on Sun Apr 25 09:48:14 2021.
Dependencies resolved.
=========================================================================================================
 Package                                 Architecture      Version               Repository         Size
=========================================================================================================
Installing:
 centos-release-nfs-ganesha30            noarch            1.0-2.el8             extras            8.6 k

Transaction Summary
=========================================================================================================
Install  1 Package

Total download size: 8.6 k
Installed size: 704  
Downloading Packages:
centos-release-nfs-ganesha30-1.0-2.el8.noarch.rpm                        113 kB/s | 8.6 kB     00:00    
---------------------------------------------------------------------------------------------------------
Total                                                                     22 kB/s | 8.6 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                 1/1 
  Installing       : centos-release-nfs-ganesha30-1.0-2.el8.noarch                                   1/1 
  Verifying        : centos-release-nfs-ganesha30-1.0-2.el8.noarch                                   1/1 

Installed:
  centos-release-nfs-ganesha30-1.0-2.el8.noarch                                                          

Complete!
[root@node1 ~]# dnf install -y nfs-ganesha nfs-ganesha-gluster
CentOS-8 - NFS Ganesha 3                                                  48 kB/s |  21 kB     00:00    
Dependencies resolved.
=========================================================================================================
 Package                        Arch     Version                             Repository             Size
=========================================================================================================
Installing:
 nfs-ganesha                    x86_64   3.5-1.el8                           centos-nfs-ganesha3   708 k
 nfs-ganesha-gluster            x86_64   3.5-1.el8                           centos-nfs-ganesha3    66 k
Installing dependencies:
 avahi-libs                     x86_64   0.7-20.el8                          baseos                 62 k
 checkpolicy                    x86_64   2.9-1.el8                           baseos                348 k
 cups-libs                      x86_64   1:2.2.6-38.el8                      baseos                433 k
 gssproxy                       x86_64   0.8.0-19.el8                        baseos                119 k
 keyutils                       x86_64   1.5.10-6.el8                        baseos                 63 k
 libicu                         x86_64   60.3-2.el8_1                        baseos                8.8 M
 libntirpc                      x86_64   3.4-1.el8                           centos-nfs-ganesha3   136 k
 libverto-libevent              x86_64   0.3.0-5.el8                         baseos                 16 k
 libwbclient                    x86_64   4.13.3-3.el8                        baseos                119 k
 nfs-ganesha-selinux            noarch   3.5-1.el8                           centos-nfs-ganesha3    38 k
 nfs-utils                      x86_64   1:2.3.3-41.el8                      baseos                497 k
 policycoreutils-python-utils   noarch   2.9-14.el8                          baseos                252 k
 python3-audit                  x86_64   3.0-0.17.20191104git1c2f876.el8     baseos                 86 k
 python3-libsemanage            x86_64   2.9-6.el8                           baseos                127 k
 python3-policycoreutils        noarch   2.9-14.el8                          baseos                2.2 M
 python3-pyyaml                 x86_64   3.12-12.el8                         baseos                193 k
 python3-setools                x86_64   4.3.0-2.el8                         baseos                626 k
 quota                          x86_64   1:4.04-12.el8                       baseos                213 k
 quota-nls                      noarch   1:4.04-12.el8                       baseos                 95 k
 samba-client-libs              x86_64   4.13.3-3.el8                        baseos                5.4 M
 samba-common                   noarch   4.13.3-3.el8                        baseos                218 k
 samba-common-libs              x86_64   4.13.3-3.el8                        baseos                171 k

Transaction Summary
=========================================================================================================
Install  24 Packages

Total download size: 21 M
Installed size: 71 M
Downloading Packages:
(1/24): nfs-ganesha-gluster-3.5-1.el8.x86_64.rpm                         1.0 MB/s |  66 kB     00:00    
(2/24): nfs-ganesha-selinux-3.5-1.el8.noarch.rpm                         1.9 MB/s |  38 kB     00:00    
(3/24): libntirpc-3.4-1.el8.x86_64.rpm                                   1.4 MB/s | 136 kB     00:00    
(4/24): nfs-ganesha-3.5-1.el8.x86_64.rpm                                 4.8 MB/s | 708 kB     00:00    
(5/24): avahi-libs-0.7-20.el8.x86_64.rpm                                 462 kB/s |  62 kB     00:00    
(6/24): checkpolicy-2.9-1.el8.x86_64.rpm                                 2.5 MB/s | 348 kB     00:00    
(7/24): keyutils-1.5.10-6.el8.x86_64.rpm                                 2.0 MB/s |  63 kB     00:00    
(8/24): gssproxy-0.8.0-19.el8.x86_64.rpm                                 2.4 MB/s | 119 kB     00:00    
(9/24): cups-libs-2.2.6-38.el8.x86_64.rpm                                2.7 MB/s | 433 kB     00:00    
(10/24): libverto-libevent-0.3.0-5.el8.x86_64.rpm                        368 kB/s |  16 kB     00:00    
(11/24): libwbclient-4.13.3-3.el8.x86_64.rpm                             2.1 MB/s | 119 kB     00:00    
(12/24): policycoreutils-python-utils-2.9-14.el8.noarch.rpm              2.2 MB/s | 252 kB     00:00    
(13/24): nfs-utils-2.3.3-41.el8.x86_64.rpm                               2.8 MB/s | 497 kB     00:00    
(14/24): python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64.rpm        1.3 MB/s |  86 kB     00:00    
(15/24): python3-libsemanage-2.9-6.el8.x86_64.rpm                        1.9 MB/s | 127 kB     00:00    
(16/24): python3-pyyaml-3.12-12.el8.x86_64.rpm                           2.0 MB/s | 193 kB     00:00    
(17/24): python3-setools-4.3.0-2.el8.x86_64.rpm                          2.5 MB/s | 626 kB     00:00    
(18/24): quota-4.04-12.el8.x86_64.rpm                                    1.8 MB/s | 213 kB     00:00    
(19/24): quota-nls-4.04-12.el8.noarch.rpm                                1.5 MB/s |  95 kB     00:00    
(20/24): python3-policycoreutils-2.9-14.el8.noarch.rpm                   2.5 MB/s | 2.2 MB     00:00    
(21/24): samba-common-4.13.3-3.el8.noarch.rpm                            1.9 MB/s | 218 kB     00:00    
(22/24): samba-common-libs-4.13.3-3.el8.x86_64.rpm                       1.9 MB/s | 171 kB     00:00    
(23/24): libicu-60.3-2.el8_1.x86_64.rpm                                  5.0 MB/s | 8.8 MB     00:01    
(24/24): samba-client-libs-4.13.3-3.el8.x86_64.rpm                       3.9 MB/s | 5.4 MB     00:01    
---------------------------------------------------------------------------------------------------------
Total                                                                    7.9 MB/s |  21 MB     00:02     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                 1/1 
  Running scriptlet: samba-common-4.13.3-3.el8.noarch                                               1/24 
  Installing       : samba-common-4.13.3-3.el8.noarch                                               1/24 
  Running scriptlet: samba-common-4.13.3-3.el8.noarch                                               1/24 
  Installing       : avahi-libs-0.7-20.el8.x86_64                                                   2/24 
  Installing       : libntirpc-3.4-1.el8.x86_64                                                     3/24 
  Installing       : cups-libs-1:2.2.6-38.el8.x86_64                                                4/24 
  Installing       : quota-nls-1:4.04-12.el8.noarch                                                 5/24 
  Installing       : quota-1:4.04-12.el8.x86_64                                                     6/24 
  Installing       : python3-setools-4.3.0-2.el8.x86_64                                             7/24 
  Installing       : python3-pyyaml-3.12-12.el8.x86_64                                              8/24 
  Installing       : python3-libsemanage-2.9-6.el8.x86_64                                           9/24 
  Installing       : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64                          10/24 
  Installing       : libverto-libevent-0.3.0-5.el8.x86_64                                          11/24 
  Installing       : gssproxy-0.8.0-19.el8.x86_64                                                  12/24 
  Running scriptlet: gssproxy-0.8.0-19.el8.x86_64                                                  12/24 
  Installing       : libicu-60.3-2.el8_1.x86_64                                                    13/24 
  Running scriptlet: libicu-60.3-2.el8_1.x86_64                                                    13/24 
  Installing       : libwbclient-4.13.3-3.el8.x86_64                                               14/24 
  Installing       : samba-common-libs-4.13.3-3.el8.x86_64                                         15/24 
  Installing       : samba-client-libs-4.13.3-3.el8.x86_64                                         16/24 
  Installing       : keyutils-1.5.10-6.el8.x86_64                                                  17/24 
  Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64                                               18/24 
  Installing       : nfs-utils-1:2.3.3-41.el8.x86_64                                               18/24 
  Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64                                               18/24 
  Installing       : checkpolicy-2.9-1.el8.x86_64                                                  19/24 
  Installing       : python3-policycoreutils-2.9-14.el8.noarch                                     20/24 
  Installing       : policycoreutils-python-utils-2.9-14.el8.noarch                                21/24 
  Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch                                          22/24 
  Installing       : nfs-ganesha-selinux-3.5-1.el8.noarch                                          22/24 
  Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch                                          22/24 
  Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64                                                  23/24 
  Installing       : nfs-ganesha-3.5-1.el8.x86_64                                                  23/24 
  Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64                                                  23/24 
  Installing       : nfs-ganesha-gluster-3.5-1.el8.x86_64                                          24/24 
  Running scriptlet: libwbclient-4.13.3-3.el8.x86_64                                               24/24 
  Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64                                                  24/24 
  Running scriptlet: nfs-ganesha-gluster-3.5-1.el8.x86_64                                          24/24 
  Verifying        : libntirpc-3.4-1.el8.x86_64                                                     1/24 
  Verifying        : nfs-ganesha-3.5-1.el8.x86_64                                                   2/24 
  Verifying        : nfs-ganesha-gluster-3.5-1.el8.x86_64                                           3/24 
  Verifying        : nfs-ganesha-selinux-3.5-1.el8.noarch                                           4/24 
  Verifying        : avahi-libs-0.7-20.el8.x86_64                                                   5/24 
  Verifying        : checkpolicy-2.9-1.el8.x86_64                                                   6/24 
  Verifying        : cups-libs-1:2.2.6-38.el8.x86_64                                                7/24 
  Verifying        : gssproxy-0.8.0-19.el8.x86_64                                                   8/24 
  Verifying        : keyutils-1.5.10-6.el8.x86_64                                                   9/24 
  Verifying        : libicu-60.3-2.el8_1.x86_64                                                    10/24 
  Verifying        : libverto-libevent-0.3.0-5.el8.x86_64                                          11/24 
  Verifying        : libwbclient-4.13.3-3.el8.x86_64                                               12/24 
  Verifying        : nfs-utils-1:2.3.3-41.el8.x86_64                                               13/24 
  Verifying        : policycoreutils-python-utils-2.9-14.el8.noarch                                14/24 
  Verifying        : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64                          15/24 
  Verifying        : python3-libsemanage-2.9-6.el8.x86_64                                          16/24 
  Verifying        : python3-policycoreutils-2.9-14.el8.noarch                                     17/24 
  Verifying        : python3-pyyaml-3.12-12.el8.x86_64                                             18/24 
  Verifying        : python3-setools-4.3.0-2.el8.x86_64                                            19/24 
  Verifying        : quota-1:4.04-12.el8.x86_64                                                    20/24 
  Verifying        : quota-nls-1:4.04-12.el8.noarch                                                21/24 
  Verifying        : samba-client-libs-4.13.3-3.el8.x86_64                                         22/24 
  Verifying        : samba-common-4.13.3-3.el8.noarch                                              23/24 
  Verifying        : samba-common-libs-4.13.3-3.el8.x86_64                                         24/24 

Installed:
  avahi-libs-0.7-20.el8.x86_64                           checkpolicy-2.9-1.el8.x86_64                    
  cups-libs-1:2.2.6-38.el8.x86_64                        gssproxy-0.8.0-19.el8.x86_64                    
  keyutils-1.5.10-6.el8.x86_64                           libicu-60.3-2.el8_1.x86_64                      
  libntirpc-3.4-1.el8.x86_64                             libverto-libevent-0.3.0-5.el8.x86_64            
  libwbclient-4.13.3-3.el8.x86_64                        nfs-ganesha-3.5-1.el8.x86_64                    
  nfs-ganesha-gluster-3.5-1.el8.x86_64                   nfs-ganesha-selinux-3.5-1.el8.noarch            
  nfs-utils-1:2.3.3-41.el8.x86_64                        policycoreutils-python-utils-2.9-14.el8.noarch  
  python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64   python3-libsemanage-2.9-6.el8.x86_64            
  python3-policycoreutils-2.9-14.el8.noarch              python3-pyyaml-3.12-12.el8.x86_64               
  python3-setools-4.3.0-2.el8.x86_64                     quota-1:4.04-12.el8.x86_64                      
  quota-nls-1:4.04-12.el8.noarch                         samba-client-libs-4.13.3-3.el8.x86_64           
  samba-common-4.13.3-3.el8.noarch                       samba-common-libs-4.13.3-3.el8.x86_64           

Complete!
[root@node1 ~]# systemctl start glusterd
[root@node1 ~]# systemctl status glusterd
● glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2021-04-25 09:49:42 EDT; 3s ago
     Docs: man:glusterd(8)
  Process: 12089 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_>
 Main PID: 12090 (glusterd)
    Tasks: 9 (limit: 23669)
   Memory: 3.6M
   CGroup: /system.slice/glusterd.service
           └─12090 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO

Apr 25 09:49:41 node1 systemd[1]: Starting GlusterFS, a clustered file-system server...
Apr 25 09:49:42 node1 systemd[1]: Started GlusterFS, a clustered file-system server.
[root@node1 ~]# firewall-cmd --permanent --new-zone=glusternodes
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.200
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.201
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.202
success
[root@node1 ~]# firewall-cmd --reload
success
[root@node1 ~]# gluster peer probe glnode2
peer probe: success
[root@node1 ~]# gluster peer probe glnode3
peer probe: success
[root@node1 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G   17M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda2        31G  1.6G   28G   6% /
/dev/sda5        63G   53M   60G   1% /mnt/storage
/dev/sda1       976M  151M  759M  17% /boot
tmpfs           374M     0  374M   0% /run/user/0
[root@node1 ~]# mkdir -p /mnt/storage/gluster/brick
[root@node1 ~]# gluster volume create VOL1 replica 3 transport tcp glnode1:/mnt/storage/gluster/brick glnode2:/mnt/storage/gluster/brick glnode3:/mnt/storage/gluster/brick
volume create: VOL1: success: please start the volume to access data
[root@node1 ~]# gluster volume start VOL1

volume start: VOL1: success
[root@node1 ~]# gluster volume info VOL1
 
Volume Name: VOL1
Type: Replicate
Volume ID: 0c622766-d86f-460d-b07b-53ccbe5e1fe3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: glnode1:/mnt/storage/gluster/brick
Brick2: glnode2:/mnt/storage/gluster/brick
Brick3: glnode3:/mnt/storage/gluster/brick
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
[root@node1 ~]# tail -n 3 /etc/hosts 
127.0.0.1 glnode1
192.168.0.201 glnode2
192.168.0.202 glnode3
[root@node1 ~]# tail -n 24 /etc/ganesha/ganesha.conf 
EXPORT
{
    Export_Id = 1;
    Path = "/VOL1";

    FSAL {
        name = GLUSTER;
        hostname = "glnode1";
        volume = "VOL1";
    }

    Access_type = RW;
    Squash = No_root_squash;
    Disable_ACL = TRUE;
    Pseudo = "/VOL1";
    Protocols = "3","4";
    Transports = "UDP","TCP";
    SecType = "sys";

    CLIENT
    {
        Clients = 127.0.0.1;
    }
}
[root@node1 ~]# systemctl start nfs-ganesha
[root@node1 ~]# systemctl status nfs-ganesha
● nfs-ganesha.service - NFS-Ganesha file server
   Loaded: loaded (/usr/lib/systemd/system/nfs-ganesha.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2021-04-25 09:57:20 EDT; 2min 48s ago
     Docs: http://github.com/nfs-ganesha/nfs-ganesha/wiki
  Process: 12261 ExecStart=/bin/bash -c ${NUMACTL} ${NUMAOPTS} /usr/bin/ganesha.nfsd ${OPTIONS} ${EPOCH}>
 Main PID: 12262 (ganesha.nfsd)
    Tasks: 30 (limit: 23669)
   Memory: 41.3M
   CGroup: /system.slice/nfs-ganesha.service
           └─12262 /usr/bin/ganesha.nfsd -L /var/log/ganesha/ganesha.log -f /etc/ganesha/ganesha.conf -N>

Apr 25 09:57:20 node1 systemd[1]: Starting NFS-Ganesha file server...
Apr 25 09:57:20 node1 systemd[1]: Started NFS-Ganesha file server.
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: Unable to determine the NFSv4 domain; Using 'loc>
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: using (default) domain: localdomain
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: Realms list: 'LOCALDOMAIN'
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: loaded plugin /usr/lib64/libnfsidmap/nsswitch.so>
[root@node1 ~]# mkdir /mnt/VOL1
[root@node1 ~]# mount -t nfs glnode1:/VOL1 /mnt/VOL1/
[root@node1 ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G   17M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda2        31G  1.6G   28G   6% /
/dev/sda5        63G   54M   60G   1% /mnt/storage
/dev/sda1       976M  151M  759M  17% /boot
tmpfs           374M     0  374M   0% /run/user/0
glnode1:/VOL1    63G  695M   60G   2% /mnt/VOL1

Leave a Reply

Your email address will not be published.