GlusterFS built-in NFS server supports only NFS version 3. GlusterFS offers NFS exports using NFS-Ganesha, which supports NFS version 3 and 4 protocols.
NFS-Ganesha server is a user-mode file sharing server, which offers a GlusterFS plugin to export GlusterFS volumes. In the following article, the NSF-Ganesha and GlusterFS are installed and a simple GlusterFS volume is created and then exported through NFS 3 and 4 version protocols.
The version of the software in this article:
- CentOS Stream release 8 (25.04.2021)
- GlusterFS 8.4
- NFS-Ganesha 3.5
STEP 1) Install GlusterFS.
dnf install -y centos-release-gluster dnf install -y glusterfs-server
The first line will installs a new repository under the SIG management – https://wiki.centos.org/SpecialInterestGroup/Storage. The second line installs the GlusterFS server.
STEP 2) Install NFS-Ganesha.
dnf install -y centos-release-nfs-ganesha30 dnf install -y nfs-ganesha nfs-ganesha-gluster
The first line again installs a new repository under the SIG management and the second line installs the NFS-Ganesha server with Gluster plugin.
STEP 3) Create GlusterFS volume
Start the GlusterFS server and create a simple 3 replicas volume with:
Start the GlusterFS on all the three nodes and enable the GlusterFS communication between the three nodes using firewall-cmd utility. So execute the following commands:
systemctl start glusterd firewall-cmd --permanent --new-zone=glusternodes firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.200 firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.201 firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.202 firewall-cmd --permanent --zone=glusternodes --add-service=glusterfs firewall-cmd --reload
On the first node create the GlusterFS volume. First, add the glnode2 and glnode3 to the cluster.
gluster peer probe glnode2 gluster peer probe glnode3 gluster volume create VOL1 replica 3 transport tcp glnode1:/mnt/storage/gluster/brick glnode2:/mnt/storage/gluster/brick glnode3:/mnt/storage/gluster/brick gluster volume start VOL1
glnode1, glnode2, glnode3 are names, which are added to the /etc/hosts (Of course, a FQDN could be used). The /mnt/storage/gluster/brick is the path, where GlusterFS will use to put its files. It may resides on a different (storage) partition.
Check out the detailed article about creating a GlusterFS 3 replicas volume here: glusterfs with localhost (127.0.0.1) nodes on different servers – glusterfs volume with 3 replicas Of course, 127.0.0.1 may not be used. Everything depends on the servers’ hostnames and their DNS records. A similar example, but not using localhost will be:
192.168.0.200 glnode1 192.168.0.201 glnode2 192.168.0.202 glnode3
STEP 4) Export the GlusterFS volume with NFS share using NFS-Ganesha
The configuration file is /etc/ganesha/ganesha.conf. Include the following configuration:
EXPORT
{
Export_Id = 1;
Path = "/VOL1";
FSAL {
name = GLUSTER;
hostname = "glnode1";
volume = "VOL1";
}
Access_type = RW;
Squash = No_root_squash;
Disable_ACL = TRUE;
Pseudo = "/VOL1";
Protocols = "3","4";
Transports = "UDP","TCP";
SecType = "sys";
CLIENT
{
Clients = 127.0.0.1;
}
}
And then start the NFS-Ganesha daemon:
systemctl start nfs-ganesha
Here are some details about the EXPORT options:
- Export_Id – Unique ID for each export. Any number, ensure it is unique for the whole configuration.
- Path – The path GlusterFS volume would be exported. Mounting in the clients with IP:”Path”. The path name may not be the original name as of the GlusterFS volume.
- FSAL – File System Abstraction Layer. The plugin, which will be used to export a file system. In the case, GlusterFS volume to be exported, so Gluster plugin is used. hostname – the hostname or IP, where the GlusterFS is accessible. volume – the name of the GlusterFS volume to export.
- Access_type – Read only or read write access is exported.
- Pseudo – NFSv4 pseudo path for this export.
- Protocols – which NFS protocols to be suported.
- Transports – which transport protocols to be supported.
- CLIENT – the section control the access limitation by IP, password and so on. In addtion some of the export options above may be overidden. Multiple CLIENT sections may exist.
More EXPORT options here – https://github.com/phdeniel/nfs-ganesha/blob/master/src/config_samples/export.txt
Mount the share.
Mounting the above share with mount command:
mount -t nfs glnode1:/VOL1 /mnt/VOL1/
And /etc/fstab example:
glnode1:/VOL1 /mnt/VOL1 nfs defaults,hard,intr,noexec,nosuid,_netdev,fsc,vers=4 0 0
The whole output
[root@node1 ~]# dnf install -y centos-release-gluster
Last metadata expiration check: 0:02:42 ago on Sun Apr 25 09:45:25 2021.
Dependencies resolved.
=========================================================================================================
Package Architecture Version Repository Size
=========================================================================================================
Installing:
centos-release-gluster8 noarch 1.0-1.el8 extras 9.3 k
Installing dependencies:
centos-release-storage-common noarch 2-2.el8 extras 9.4 k
Transaction Summary
=========================================================================================================
Install 2 Packages
Total download size: 19 k
Installed size: 2.4 k
Downloading Packages:
(1/2): centos-release-gluster8-1.0-1.el8.noarch.rpm 109 kB/s | 9.3 kB 00:00
(2/2): centos-release-storage-common-2-2.el8.noarch.rpm 96 kB/s | 9.4 kB 00:00
---------------------------------------------------------------------------------------------------------
Total 39 kB/s | 19 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : centos-release-storage-common-2-2.el8.noarch 1/2
Installing : centos-release-gluster8-1.0-1.el8.noarch 2/2
Running scriptlet: centos-release-gluster8-1.0-1.el8.noarch 2/2
Verifying : centos-release-gluster8-1.0-1.el8.noarch 1/2
Verifying : centos-release-storage-common-2-2.el8.noarch 2/2
Installed:
centos-release-gluster8-1.0-1.el8.noarch centos-release-storage-common-2-2.el8.noarch
Complete!
[root@node1 ~]# dnf install -y glusterfs-server
CentOS-8 - Gluster 8 129 kB/s | 50 kB 00:00
Dependencies resolved.
=========================================================================================================
Package Architecture Version Repository Size
=========================================================================================================
Installing:
glusterfs-server x86_64 8.4-1.el8 centos-gluster8 1.4 M
Installing dependencies:
attr x86_64 2.4.48-3.el8 baseos 68 k
device-mapper-event x86_64 8:1.02.175-5.el8 baseos 269 k
device-mapper-event-libs x86_64 8:1.02.175-5.el8 baseos 269 k
device-mapper-persistent-data x86_64 0.8.5-4.el8 baseos 468 k
glusterfs x86_64 8.4-1.el8 centos-gluster8 689 k
glusterfs-cli x86_64 8.4-1.el8 centos-gluster8 214 k
glusterfs-client-xlators x86_64 8.4-1.el8 centos-gluster8 899 k
glusterfs-fuse x86_64 8.4-1.el8 centos-gluster8 171 k
libaio x86_64 0.3.112-1.el8 baseos 33 k
libgfapi0 x86_64 8.4-1.el8 centos-gluster8 125 k
libgfchangelog0 x86_64 8.4-1.el8 centos-gluster8 67 k
libgfrpc0 x86_64 8.4-1.el8 centos-gluster8 89 k
libgfxdr0 x86_64 8.4-1.el8 centos-gluster8 61 k
libglusterd0 x86_64 8.4-1.el8 centos-gluster8 45 k
libglusterfs0 x86_64 8.4-1.el8 centos-gluster8 350 k
lvm2 x86_64 8:2.03.11-5.el8 baseos 1.6 M
lvm2-libs x86_64 8:2.03.11-5.el8 baseos 1.1 M
psmisc x86_64 23.1-5.el8 baseos 151 k
python3-pyxattr x86_64 0.5.3-18.el8 centos-gluster8 35 k
rpcbind x86_64 1.2.5-8.el8 baseos 70 k
userspace-rcu x86_64 0.10.1-4.el8 baseos 101 k
Transaction Summary
=========================================================================================================
Install 22 Packages
Total download size: 8.2 M
Installed size: 24 M
Downloading Packages:
(1/22): glusterfs-cli-8.4-1.el8.x86_64.rpm 2.0 MB/s | 214 kB 00:00
(2/22): glusterfs-fuse-8.4-1.el8.x86_64.rpm 2.4 MB/s | 171 kB 00:00
(3/22): glusterfs-client-xlators-8.4-1.el8.x86_64.rpm 3.5 MB/s | 899 kB 00:00
(4/22): glusterfs-8.4-1.el8.x86_64.rpm 2.6 MB/s | 689 kB 00:00
(5/22): libgfapi0-8.4-1.el8.x86_64.rpm 3.9 MB/s | 125 kB 00:00
(6/22): libgfchangelog0-8.4-1.el8.x86_64.rpm 2.6 MB/s | 67 kB 00:00
(7/22): libgfrpc0-8.4-1.el8.x86_64.rpm 3.5 MB/s | 89 kB 00:00
(8/22): libgfxdr0-8.4-1.el8.x86_64.rpm 2.5 MB/s | 61 kB 00:00
(9/22): libglusterd0-8.4-1.el8.x86_64.rpm 2.7 MB/s | 45 kB 00:00
(10/22): python3-pyxattr-0.5.3-18.el8.x86_64.rpm 2.1 MB/s | 35 kB 00:00
(11/22): libglusterfs0-8.4-1.el8.x86_64.rpm 3.7 MB/s | 350 kB 00:00
(12/22): glusterfs-server-8.4-1.el8.x86_64.rpm 4.7 MB/s | 1.4 MB 00:00
(13/22): attr-2.4.48-3.el8.x86_64.rpm 473 kB/s | 68 kB 00:00
(14/22): device-mapper-event-1.02.175-5.el8.x86_64.rpm 1.9 MB/s | 269 kB 00:00
(15/22): libaio-0.3.112-1.el8.x86_64.rpm 971 kB/s | 33 kB 00:00
(16/22): device-mapper-persistent-data-0.8.5-4.el8.x86_64.rpm 4.3 MB/s | 468 kB 00:00
(17/22): device-mapper-event-libs-1.02.175-5.el8.x86_64.rpm 2.0 MB/s | 269 kB 00:00
(18/22): psmisc-23.1-5.el8.x86_64.rpm 1.3 MB/s | 151 kB 00:00
(19/22): rpcbind-1.2.5-8.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00
(20/22): userspace-rcu-0.10.1-4.el8.x86_64.rpm 1.6 MB/s | 101 kB 00:00
(21/22): lvm2-libs-2.03.11-5.el8.x86_64.rpm 3.5 MB/s | 1.1 MB 00:00
(22/22): lvm2-2.03.11-5.el8.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00
---------------------------------------------------------------------------------------------------------
Total 5.0 MB/s | 8.2 MB 00:01
warning: /var/cache/dnf/centos-gluster8-ae72c2c38de8ee20/packages/glusterfs-8.4-1.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e451e5b5: NOKEY
CentOS-8 - Gluster 8 1.0 MB/s | 1.0 kB 00:00
Importing GPG key 0xE451E5B5:
Userid : "CentOS Storage SIG (http://wiki.centos.org/SpecialInterestGroup/Storage) <security@centos.org>"
Fingerprint: 7412 9C0B 173B 071A 3775 951A D4A2 E50B E451 E5B5
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : libgfxdr0-8.4-1.el8.x86_64 1/22
Running scriptlet: libgfxdr0-8.4-1.el8.x86_64 1/22
Installing : libglusterfs0-8.4-1.el8.x86_64 2/22
Running scriptlet: libglusterfs0-8.4-1.el8.x86_64 2/22
Installing : libgfrpc0-8.4-1.el8.x86_64 3/22
Running scriptlet: libgfrpc0-8.4-1.el8.x86_64 3/22
Installing : libaio-0.3.112-1.el8.x86_64 4/22
Installing : glusterfs-client-xlators-8.4-1.el8.x86_64 5/22
Installing : device-mapper-event-libs-8:1.02.175-5.el8.x86_64 6/22
Running scriptlet: glusterfs-8.4-1.el8.x86_64 7/22
Installing : glusterfs-8.4-1.el8.x86_64 7/22
Running scriptlet: glusterfs-8.4-1.el8.x86_64 7/22
Installing : libglusterd0-8.4-1.el8.x86_64 8/22
Running scriptlet: libglusterd0-8.4-1.el8.x86_64 8/22
Installing : glusterfs-cli-8.4-1.el8.x86_64 9/22
Installing : device-mapper-event-8:1.02.175-5.el8.x86_64 10/22
Running scriptlet: device-mapper-event-8:1.02.175-5.el8.x86_64 10/22
Installing : lvm2-libs-8:2.03.11-5.el8.x86_64 11/22
Installing : libgfapi0-8.4-1.el8.x86_64 12/22
Running scriptlet: libgfapi0-8.4-1.el8.x86_64 12/22
Installing : device-mapper-persistent-data-0.8.5-4.el8.x86_64 13/22
Installing : lvm2-8:2.03.11-5.el8.x86_64 14/22
Running scriptlet: lvm2-8:2.03.11-5.el8.x86_64 14/22
Installing : libgfchangelog0-8.4-1.el8.x86_64 15/22
Running scriptlet: libgfchangelog0-8.4-1.el8.x86_64 15/22
Installing : userspace-rcu-0.10.1-4.el8.x86_64 16/22
Running scriptlet: userspace-rcu-0.10.1-4.el8.x86_64 16/22
Running scriptlet: rpcbind-1.2.5-8.el8.x86_64 17/22
Installing : rpcbind-1.2.5-8.el8.x86_64 17/22
Running scriptlet: rpcbind-1.2.5-8.el8.x86_64 17/22
Installing : psmisc-23.1-5.el8.x86_64 18/22
Installing : attr-2.4.48-3.el8.x86_64 19/22
Installing : glusterfs-fuse-8.4-1.el8.x86_64 20/22
Installing : python3-pyxattr-0.5.3-18.el8.x86_64 21/22
Installing : glusterfs-server-8.4-1.el8.x86_64 22/22
Running scriptlet: glusterfs-server-8.4-1.el8.x86_64 22/22
Verifying : glusterfs-8.4-1.el8.x86_64 1/22
Verifying : glusterfs-cli-8.4-1.el8.x86_64 2/22
Verifying : glusterfs-client-xlators-8.4-1.el8.x86_64 3/22
Verifying : glusterfs-fuse-8.4-1.el8.x86_64 4/22
Verifying : glusterfs-server-8.4-1.el8.x86_64 5/22
Verifying : libgfapi0-8.4-1.el8.x86_64 6/22
Verifying : libgfchangelog0-8.4-1.el8.x86_64 7/22
Verifying : libgfrpc0-8.4-1.el8.x86_64 8/22
Verifying : libgfxdr0-8.4-1.el8.x86_64 9/22
Verifying : libglusterd0-8.4-1.el8.x86_64 10/22
Verifying : libglusterfs0-8.4-1.el8.x86_64 11/22
Verifying : python3-pyxattr-0.5.3-18.el8.x86_64 12/22
Verifying : attr-2.4.48-3.el8.x86_64 13/22
Verifying : device-mapper-event-8:1.02.175-5.el8.x86_64 14/22
Verifying : device-mapper-event-libs-8:1.02.175-5.el8.x86_64 15/22
Verifying : device-mapper-persistent-data-0.8.5-4.el8.x86_64 16/22
Verifying : libaio-0.3.112-1.el8.x86_64 17/22
Verifying : lvm2-8:2.03.11-5.el8.x86_64 18/22
Verifying : lvm2-libs-8:2.03.11-5.el8.x86_64 19/22
Verifying : psmisc-23.1-5.el8.x86_64 20/22
Verifying : rpcbind-1.2.5-8.el8.x86_64 21/22
Verifying : userspace-rcu-0.10.1-4.el8.x86_64 22/22
Installed:
attr-2.4.48-3.el8.x86_64 device-mapper-event-8:1.02.175-5.el8.x86_64
device-mapper-event-libs-8:1.02.175-5.el8.x86_64 device-mapper-persistent-data-0.8.5-4.el8.x86_64
glusterfs-8.4-1.el8.x86_64 glusterfs-cli-8.4-1.el8.x86_64
glusterfs-client-xlators-8.4-1.el8.x86_64 glusterfs-fuse-8.4-1.el8.x86_64
glusterfs-server-8.4-1.el8.x86_64 libaio-0.3.112-1.el8.x86_64
libgfapi0-8.4-1.el8.x86_64 libgfchangelog0-8.4-1.el8.x86_64
libgfrpc0-8.4-1.el8.x86_64 libgfxdr0-8.4-1.el8.x86_64
libglusterd0-8.4-1.el8.x86_64 libglusterfs0-8.4-1.el8.x86_64
lvm2-8:2.03.11-5.el8.x86_64 lvm2-libs-8:2.03.11-5.el8.x86_64
psmisc-23.1-5.el8.x86_64 python3-pyxattr-0.5.3-18.el8.x86_64
rpcbind-1.2.5-8.el8.x86_64 userspace-rcu-0.10.1-4.el8.x86_64
Complete!
[root@node1 ~]# dnf install -y centos-release-nfs-ganesha30
Last metadata expiration check: 0:00:22 ago on Sun Apr 25 09:48:14 2021.
Dependencies resolved.
=========================================================================================================
Package Architecture Version Repository Size
=========================================================================================================
Installing:
centos-release-nfs-ganesha30 noarch 1.0-2.el8 extras 8.6 k
Transaction Summary
=========================================================================================================
Install 1 Package
Total download size: 8.6 k
Installed size: 704
Downloading Packages:
centos-release-nfs-ganesha30-1.0-2.el8.noarch.rpm 113 kB/s | 8.6 kB 00:00
---------------------------------------------------------------------------------------------------------
Total 22 kB/s | 8.6 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : centos-release-nfs-ganesha30-1.0-2.el8.noarch 1/1
Verifying : centos-release-nfs-ganesha30-1.0-2.el8.noarch 1/1
Installed:
centos-release-nfs-ganesha30-1.0-2.el8.noarch
Complete!
[root@node1 ~]# dnf install -y nfs-ganesha nfs-ganesha-gluster
CentOS-8 - NFS Ganesha 3 48 kB/s | 21 kB 00:00
Dependencies resolved.
=========================================================================================================
Package Arch Version Repository Size
=========================================================================================================
Installing:
nfs-ganesha x86_64 3.5-1.el8 centos-nfs-ganesha3 708 k
nfs-ganesha-gluster x86_64 3.5-1.el8 centos-nfs-ganesha3 66 k
Installing dependencies:
avahi-libs x86_64 0.7-20.el8 baseos 62 k
checkpolicy x86_64 2.9-1.el8 baseos 348 k
cups-libs x86_64 1:2.2.6-38.el8 baseos 433 k
gssproxy x86_64 0.8.0-19.el8 baseos 119 k
keyutils x86_64 1.5.10-6.el8 baseos 63 k
libicu x86_64 60.3-2.el8_1 baseos 8.8 M
libntirpc x86_64 3.4-1.el8 centos-nfs-ganesha3 136 k
libverto-libevent x86_64 0.3.0-5.el8 baseos 16 k
libwbclient x86_64 4.13.3-3.el8 baseos 119 k
nfs-ganesha-selinux noarch 3.5-1.el8 centos-nfs-ganesha3 38 k
nfs-utils x86_64 1:2.3.3-41.el8 baseos 497 k
policycoreutils-python-utils noarch 2.9-14.el8 baseos 252 k
python3-audit x86_64 3.0-0.17.20191104git1c2f876.el8 baseos 86 k
python3-libsemanage x86_64 2.9-6.el8 baseos 127 k
python3-policycoreutils noarch 2.9-14.el8 baseos 2.2 M
python3-pyyaml x86_64 3.12-12.el8 baseos 193 k
python3-setools x86_64 4.3.0-2.el8 baseos 626 k
quota x86_64 1:4.04-12.el8 baseos 213 k
quota-nls noarch 1:4.04-12.el8 baseos 95 k
samba-client-libs x86_64 4.13.3-3.el8 baseos 5.4 M
samba-common noarch 4.13.3-3.el8 baseos 218 k
samba-common-libs x86_64 4.13.3-3.el8 baseos 171 k
Transaction Summary
=========================================================================================================
Install 24 Packages
Total download size: 21 M
Installed size: 71 M
Downloading Packages:
(1/24): nfs-ganesha-gluster-3.5-1.el8.x86_64.rpm 1.0 MB/s | 66 kB 00:00
(2/24): nfs-ganesha-selinux-3.5-1.el8.noarch.rpm 1.9 MB/s | 38 kB 00:00
(3/24): libntirpc-3.4-1.el8.x86_64.rpm 1.4 MB/s | 136 kB 00:00
(4/24): nfs-ganesha-3.5-1.el8.x86_64.rpm 4.8 MB/s | 708 kB 00:00
(5/24): avahi-libs-0.7-20.el8.x86_64.rpm 462 kB/s | 62 kB 00:00
(6/24): checkpolicy-2.9-1.el8.x86_64.rpm 2.5 MB/s | 348 kB 00:00
(7/24): keyutils-1.5.10-6.el8.x86_64.rpm 2.0 MB/s | 63 kB 00:00
(8/24): gssproxy-0.8.0-19.el8.x86_64.rpm 2.4 MB/s | 119 kB 00:00
(9/24): cups-libs-2.2.6-38.el8.x86_64.rpm 2.7 MB/s | 433 kB 00:00
(10/24): libverto-libevent-0.3.0-5.el8.x86_64.rpm 368 kB/s | 16 kB 00:00
(11/24): libwbclient-4.13.3-3.el8.x86_64.rpm 2.1 MB/s | 119 kB 00:00
(12/24): policycoreutils-python-utils-2.9-14.el8.noarch.rpm 2.2 MB/s | 252 kB 00:00
(13/24): nfs-utils-2.3.3-41.el8.x86_64.rpm 2.8 MB/s | 497 kB 00:00
(14/24): python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64.rpm 1.3 MB/s | 86 kB 00:00
(15/24): python3-libsemanage-2.9-6.el8.x86_64.rpm 1.9 MB/s | 127 kB 00:00
(16/24): python3-pyyaml-3.12-12.el8.x86_64.rpm 2.0 MB/s | 193 kB 00:00
(17/24): python3-setools-4.3.0-2.el8.x86_64.rpm 2.5 MB/s | 626 kB 00:00
(18/24): quota-4.04-12.el8.x86_64.rpm 1.8 MB/s | 213 kB 00:00
(19/24): quota-nls-4.04-12.el8.noarch.rpm 1.5 MB/s | 95 kB 00:00
(20/24): python3-policycoreutils-2.9-14.el8.noarch.rpm 2.5 MB/s | 2.2 MB 00:00
(21/24): samba-common-4.13.3-3.el8.noarch.rpm 1.9 MB/s | 218 kB 00:00
(22/24): samba-common-libs-4.13.3-3.el8.x86_64.rpm 1.9 MB/s | 171 kB 00:00
(23/24): libicu-60.3-2.el8_1.x86_64.rpm 5.0 MB/s | 8.8 MB 00:01
(24/24): samba-client-libs-4.13.3-3.el8.x86_64.rpm 3.9 MB/s | 5.4 MB 00:01
---------------------------------------------------------------------------------------------------------
Total 7.9 MB/s | 21 MB 00:02
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: samba-common-4.13.3-3.el8.noarch 1/24
Installing : samba-common-4.13.3-3.el8.noarch 1/24
Running scriptlet: samba-common-4.13.3-3.el8.noarch 1/24
Installing : avahi-libs-0.7-20.el8.x86_64 2/24
Installing : libntirpc-3.4-1.el8.x86_64 3/24
Installing : cups-libs-1:2.2.6-38.el8.x86_64 4/24
Installing : quota-nls-1:4.04-12.el8.noarch 5/24
Installing : quota-1:4.04-12.el8.x86_64 6/24
Installing : python3-setools-4.3.0-2.el8.x86_64 7/24
Installing : python3-pyyaml-3.12-12.el8.x86_64 8/24
Installing : python3-libsemanage-2.9-6.el8.x86_64 9/24
Installing : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64 10/24
Installing : libverto-libevent-0.3.0-5.el8.x86_64 11/24
Installing : gssproxy-0.8.0-19.el8.x86_64 12/24
Running scriptlet: gssproxy-0.8.0-19.el8.x86_64 12/24
Installing : libicu-60.3-2.el8_1.x86_64 13/24
Running scriptlet: libicu-60.3-2.el8_1.x86_64 13/24
Installing : libwbclient-4.13.3-3.el8.x86_64 14/24
Installing : samba-common-libs-4.13.3-3.el8.x86_64 15/24
Installing : samba-client-libs-4.13.3-3.el8.x86_64 16/24
Installing : keyutils-1.5.10-6.el8.x86_64 17/24
Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64 18/24
Installing : nfs-utils-1:2.3.3-41.el8.x86_64 18/24
Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64 18/24
Installing : checkpolicy-2.9-1.el8.x86_64 19/24
Installing : python3-policycoreutils-2.9-14.el8.noarch 20/24
Installing : policycoreutils-python-utils-2.9-14.el8.noarch 21/24
Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch 22/24
Installing : nfs-ganesha-selinux-3.5-1.el8.noarch 22/24
Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch 22/24
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 23/24
Installing : nfs-ganesha-3.5-1.el8.x86_64 23/24
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 23/24
Installing : nfs-ganesha-gluster-3.5-1.el8.x86_64 24/24
Running scriptlet: libwbclient-4.13.3-3.el8.x86_64 24/24
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 24/24
Running scriptlet: nfs-ganesha-gluster-3.5-1.el8.x86_64 24/24
Verifying : libntirpc-3.4-1.el8.x86_64 1/24
Verifying : nfs-ganesha-3.5-1.el8.x86_64 2/24
Verifying : nfs-ganesha-gluster-3.5-1.el8.x86_64 3/24
Verifying : nfs-ganesha-selinux-3.5-1.el8.noarch 4/24
Verifying : avahi-libs-0.7-20.el8.x86_64 5/24
Verifying : checkpolicy-2.9-1.el8.x86_64 6/24
Verifying : cups-libs-1:2.2.6-38.el8.x86_64 7/24
Verifying : gssproxy-0.8.0-19.el8.x86_64 8/24
Verifying : keyutils-1.5.10-6.el8.x86_64 9/24
Verifying : libicu-60.3-2.el8_1.x86_64 10/24
Verifying : libverto-libevent-0.3.0-5.el8.x86_64 11/24
Verifying : libwbclient-4.13.3-3.el8.x86_64 12/24
Verifying : nfs-utils-1:2.3.3-41.el8.x86_64 13/24
Verifying : policycoreutils-python-utils-2.9-14.el8.noarch 14/24
Verifying : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64 15/24
Verifying : python3-libsemanage-2.9-6.el8.x86_64 16/24
Verifying : python3-policycoreutils-2.9-14.el8.noarch 17/24
Verifying : python3-pyyaml-3.12-12.el8.x86_64 18/24
Verifying : python3-setools-4.3.0-2.el8.x86_64 19/24
Verifying : quota-1:4.04-12.el8.x86_64 20/24
Verifying : quota-nls-1:4.04-12.el8.noarch 21/24
Verifying : samba-client-libs-4.13.3-3.el8.x86_64 22/24
Verifying : samba-common-4.13.3-3.el8.noarch 23/24
Verifying : samba-common-libs-4.13.3-3.el8.x86_64 24/24
Installed:
avahi-libs-0.7-20.el8.x86_64 checkpolicy-2.9-1.el8.x86_64
cups-libs-1:2.2.6-38.el8.x86_64 gssproxy-0.8.0-19.el8.x86_64
keyutils-1.5.10-6.el8.x86_64 libicu-60.3-2.el8_1.x86_64
libntirpc-3.4-1.el8.x86_64 libverto-libevent-0.3.0-5.el8.x86_64
libwbclient-4.13.3-3.el8.x86_64 nfs-ganesha-3.5-1.el8.x86_64
nfs-ganesha-gluster-3.5-1.el8.x86_64 nfs-ganesha-selinux-3.5-1.el8.noarch
nfs-utils-1:2.3.3-41.el8.x86_64 policycoreutils-python-utils-2.9-14.el8.noarch
python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64 python3-libsemanage-2.9-6.el8.x86_64
python3-policycoreutils-2.9-14.el8.noarch python3-pyyaml-3.12-12.el8.x86_64
python3-setools-4.3.0-2.el8.x86_64 quota-1:4.04-12.el8.x86_64
quota-nls-1:4.04-12.el8.noarch samba-client-libs-4.13.3-3.el8.x86_64
samba-common-4.13.3-3.el8.noarch samba-common-libs-4.13.3-3.el8.x86_64
Complete!
[root@node1 ~]# systemctl start glusterd
[root@node1 ~]# systemctl status glusterd
● glusterd.service - GlusterFS, a clustered file-system server
Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2021-04-25 09:49:42 EDT; 3s ago
Docs: man:glusterd(8)
Process: 12089 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_>
Main PID: 12090 (glusterd)
Tasks: 9 (limit: 23669)
Memory: 3.6M
CGroup: /system.slice/glusterd.service
└─12090 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO
Apr 25 09:49:41 node1 systemd[1]: Starting GlusterFS, a clustered file-system server...
Apr 25 09:49:42 node1 systemd[1]: Started GlusterFS, a clustered file-system server.
[root@node1 ~]# firewall-cmd --permanent --new-zone=glusternodes
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.200
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.201
success
[root@node1 ~]# firewall-cmd --permanent --zone=glusternodes --add-source=192.168.0.202
success
[root@node1 ~]# firewall-cmd --reload
success
[root@node1 ~]# gluster peer probe glnode2
peer probe: success
[root@node1 ~]# gluster peer probe glnode3
peer probe: success
[root@node1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda2 31G 1.6G 28G 6% /
/dev/sda5 63G 53M 60G 1% /mnt/storage
/dev/sda1 976M 151M 759M 17% /boot
tmpfs 374M 0 374M 0% /run/user/0
[root@node1 ~]# mkdir -p /mnt/storage/gluster/brick
[root@node1 ~]# gluster volume create VOL1 replica 3 transport tcp glnode1:/mnt/storage/gluster/brick glnode2:/mnt/storage/gluster/brick glnode3:/mnt/storage/gluster/brick
volume create: VOL1: success: please start the volume to access data
[root@node1 ~]# gluster volume start VOL1
volume start: VOL1: success
[root@node1 ~]# gluster volume info VOL1
Volume Name: VOL1
Type: Replicate
Volume ID: 0c622766-d86f-460d-b07b-53ccbe5e1fe3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: glnode1:/mnt/storage/gluster/brick
Brick2: glnode2:/mnt/storage/gluster/brick
Brick3: glnode3:/mnt/storage/gluster/brick
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
[root@node1 ~]# tail -n 3 /etc/hosts
127.0.0.1 glnode1
192.168.0.201 glnode2
192.168.0.202 glnode3
[root@node1 ~]# tail -n 24 /etc/ganesha/ganesha.conf
EXPORT
{
Export_Id = 1;
Path = "/VOL1";
FSAL {
name = GLUSTER;
hostname = "glnode1";
volume = "VOL1";
}
Access_type = RW;
Squash = No_root_squash;
Disable_ACL = TRUE;
Pseudo = "/VOL1";
Protocols = "3","4";
Transports = "UDP","TCP";
SecType = "sys";
CLIENT
{
Clients = 127.0.0.1;
}
}
[root@node1 ~]# systemctl start nfs-ganesha
[root@node1 ~]# systemctl status nfs-ganesha
● nfs-ganesha.service - NFS-Ganesha file server
Loaded: loaded (/usr/lib/systemd/system/nfs-ganesha.service; disabled; vendor preset: disabled)
Active: active (running) since Sun 2021-04-25 09:57:20 EDT; 2min 48s ago
Docs: http://github.com/nfs-ganesha/nfs-ganesha/wiki
Process: 12261 ExecStart=/bin/bash -c ${NUMACTL} ${NUMAOPTS} /usr/bin/ganesha.nfsd ${OPTIONS} ${EPOCH}>
Main PID: 12262 (ganesha.nfsd)
Tasks: 30 (limit: 23669)
Memory: 41.3M
CGroup: /system.slice/nfs-ganesha.service
└─12262 /usr/bin/ganesha.nfsd -L /var/log/ganesha/ganesha.log -f /etc/ganesha/ganesha.conf -N>
Apr 25 09:57:20 node1 systemd[1]: Starting NFS-Ganesha file server...
Apr 25 09:57:20 node1 systemd[1]: Started NFS-Ganesha file server.
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: Unable to determine the NFSv4 domain; Using 'loc>
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: using (default) domain: localdomain
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: Realms list: 'LOCALDOMAIN'
Apr 25 09:57:20 node1 ganesha.nfsd[12262]: libnfsidmap: loaded plugin /usr/lib64/libnfsidmap/nsswitch.so>
[root@node1 ~]# mkdir /mnt/VOL1
[root@node1 ~]# mount -t nfs glnode1:/VOL1 /mnt/VOL1/
[root@node1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 17M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda2 31G 1.6G 28G 6% /
/dev/sda5 63G 54M 60G 1% /mnt/storage
/dev/sda1 976M 151M 759M 17% /boot
tmpfs 374M 0 374M 0% /run/user/0
glnode1:/VOL1 63G 695M 60G 2% /mnt/VOL1