NFS Ganesha is a user-mode file sharing server, which supports NFS 3 and 4.x versions and 9P. NFS Ganesha has several interesting plugins that support exporting files from the cluster and distributed file systems like Ceph and Glusterfs Exporting a file system with NFS Ganesha is simple enough if you do not use SELinux or SELinux is in permissive mode!
This article is to show how to export a server’s directory using NFS protocol Just to note the NFS-Ganesha is tested and supports ext2/ext3/ext4, xfs, brtfs, zfs file systems as of version 3.5 (check the manual for xfs, brtfs and zfs exports – here are sample configurations for them https://github.com/phdeniel/nfs-ganesha/tree/master/src/config_samples). To be able to export a file directory the VFS Ganesha plugin is used. A clean install of minimal CentOS 8 Stream is used so the installation log may differ significantly from the user’s log but the user will see all the dependencies, which are required for this setup.
STEP 1) Install the repository and NFS-Ganesha software
NFS-Ganesha 3 packages are from the CentOS Storage SIG repository, which is a good repository and may be trusted.
dnf install -y centos-release-nfs-ganesha30 dnf install -y nfs-ganesha nfs-ganesha-vfs
STEP 2) Configuration for exporting a directory.
There are two files under /etc/ganesha/:
ganesha.conf vfs.conf
ganesha.conf includes global configuration and NFS share configuration. Each export path begins with the keyword EXPORT followed by a block ebraced by brackets {}.
vfs.conf includes a simple example for the VFS plugin, but this configuration file is not used by the NFS Ganesha server. It is just a sample file.
Here is a simple configuration, which exports /mnt/storage with Read/Write permissions to a single IP. Just add at the end of the file /etc/ganesha/ganesha.conf contains:
EXPORT
{
Export_Id = 2;
Path = /mnt/storage1;
Pseudo = /mnt/storage1;
Protocols = 3,4;
Access_Type = RW;
Squash = None;
FSAL
{
Name = VFS;
}
CLIENT
{
Clients = 192.168.0.12;
}
}
STEP 3) Start the server and mount the exported directory. Configure the firewall.
Start the server, enable the service to start on boot and then configure the firewall to pass the NFS requests:
systemctl start nfs-ganesha systemctl enable nfs-ganesha firewall-cmd --permanent --zone=public --add-service=nfs firewall-cmd --reload
A more specific firewall rule to pass requests only from the single IP, which is included in the nfs-ganesha configuration (aka 192.168.0.12). Replace the above rule with (if the above rule was added it should be removed first):
firewall-cmd --permanent --zone=public --remove-service=nfs firewall-cmd --permanent --zone=public --add-rich-rule="rule family="ipv4" source address="192.168.0.12" service name="nfs" accept" firewall-cmd --reload
Only client with IP 192.168.0.12 could mount the export directory /mnt/storage1. The CLIENT block could be omitted or may be included multiple times redefining some of the parameters from the current EXPORT block or global configuration.
For more information https://github.com/phdeniel/nfs-ganesha/blob/master/src/config_samples/export.txt The user may check all available parameters for the EXPORT block!
And mounting the directory from client with IP 192.168.0.12:
mount 192.168.0.20:/mnt/storage1 /clients/storage1
/etc/fstab example line:
192.168.0.20:/mnt/storage1 /clients/storage1 nfs defaults,hard,intr,noexec,nosuid,_netdev,fsc,vers=4 0 0
The 192.168.0.20 is the server IP of NFS-Ganesha. The /mnt/storage1 is exported directory by NFS-Ganesha and /clients/storage1 is the directory, under which the NFS share will be mounted on the client machine (could be changed to any directory name).
The whole installation log
[root@srv ~]# dnf install -y centos-release-nfs-ganesha30
CentOS Stream 8 - AppStream 4.8 MB/s | 6.7 MB 00:01
CentOS Stream 8 - BaseOS 1.2 MB/s | 2.3 MB 00:01
CentOS Stream 8 - Extras 33 kB/s | 9.1 kB 00:00
Last metadata expiration check: 0:00:01 ago on Tue Apr 13 09:13:20 2021.
Dependencies resolved.
=============================================================================================================================================================================================
Package Architecture Version Repository Size
=============================================================================================================================================================================================
Installing:
centos-release-nfs-ganesha30 noarch 1.0-2.el8 extras 8.6 k
Installing dependencies:
centos-release-storage-common noarch 2-2.el8 extras 9.4 k
Transaction Summary
=============================================================================================================================================================================================
Install 2 Packages
Total download size: 18 k
Installed size: 2.0 k
Downloading Packages:
(1/2): centos-release-nfs-ganesha30-1.0-2.el8.noarch.rpm 149 kB/s | 8.6 kB 00:00
(2/2): centos-release-storage-common-2-2.el8.noarch.rpm 103 kB/s | 9.4 kB 00:00
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 43 kB/s | 18 kB 00:00
warning: /var/cache/dnf/extras-9705a089504ff150/packages/centos-release-nfs-ganesha30-1.0-2.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
CentOS Stream 8 - Extras 1.6 MB/s | 1.6 kB 00:00
Importing GPG key 0x8483C65D:
Userid : "CentOS (CentOS Official Signing Key) <security@centos.org>"
Fingerprint: 99DB 70FA E1D7 CE22 7FB6 4882 05B5 55B3 8483 C65D
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : centos-release-storage-common-2-2.el8.noarch 1/2
Installing : centos-release-nfs-ganesha30-1.0-2.el8.noarch 2/2
Verifying : centos-release-nfs-ganesha30-1.0-2.el8.noarch 1/2
Verifying : centos-release-storage-common-2-2.el8.noarch 2/2
Installed:
centos-release-nfs-ganesha30-1.0-2.el8.noarch centos-release-storage-common-2-2.el8.noarch
Complete!
[root@srv ~]# dnf install -y nfs-ganesha nfs-ganesha-vfs
Last metadata expiration check: 0:11:01 ago on Tue Apr 13 09:14:00 2021.
Dependencies resolved.
=============================================================================================================================================================================================
Package Architecture Version Repository Size
=============================================================================================================================================================================================
Installing:
nfs-ganesha x86_64 3.5-1.el8 centos-nfs-ganesha3 708 k
nfs-ganesha-vfs x86_64 3.5-1.el8 centos-nfs-ganesha3 69 k
Installing dependencies:
avahi-libs x86_64 0.7-20.el8 baseos 62 k
checkpolicy x86_64 2.9-1.el8 baseos 348 k
cups-libs x86_64 1:2.2.6-38.el8 baseos 433 k
gssproxy x86_64 0.8.0-19.el8 baseos 119 k
keyutils x86_64 1.5.10-6.el8 baseos 63 k
libicu x86_64 60.3-2.el8_1 baseos 8.8 M
libntirpc x86_64 3.4-1.el8 centos-nfs-ganesha3 136 k
libverto-libevent x86_64 0.3.0-5.el8 baseos 16 k
libwbclient x86_64 4.13.3-3.el8 baseos 119 k
nfs-ganesha-selinux noarch 3.5-1.el8 centos-nfs-ganesha3 38 k
nfs-utils x86_64 1:2.3.3-41.el8 baseos 497 k
policycoreutils-python-utils noarch 2.9-14.el8 baseos 252 k
psmisc x86_64 23.1-5.el8 baseos 151 k
python3-audit x86_64 3.0-0.17.20191104git1c2f876.el8 baseos 86 k
python3-libsemanage x86_64 2.9-6.el8 baseos 127 k
python3-policycoreutils noarch 2.9-14.el8 baseos 2.2 M
python3-pyyaml x86_64 3.12-12.el8 baseos 193 k
python3-setools x86_64 4.3.0-2.el8 baseos 626 k
quota x86_64 1:4.04-12.el8 baseos 213 k
quota-nls noarch 1:4.04-12.el8 baseos 95 k
rpcbind x86_64 1.2.5-8.el8 baseos 70 k
samba-client-libs x86_64 4.13.3-3.el8 baseos 5.4 M
samba-common noarch 4.13.3-3.el8 baseos 218 k
samba-common-libs x86_64 4.13.3-3.el8 baseos 171 k
userspace-rcu x86_64 0.10.1-4.el8 baseos 101 k
Transaction Summary
=============================================================================================================================================================================================
Install 27 Packages
Total download size: 21 M
Installed size: 72 M
Downloading Packages:
(1/27): nfs-ganesha-selinux-3.5-1.el8.noarch.rpm 582 kB/s | 38 kB 00:00
(2/27): libntirpc-3.4-1.el8.x86_64.rpm 1.7 MB/s | 136 kB 00:00
(3/27): nfs-ganesha-vfs-3.5-1.el8.x86_64.rpm 1.4 MB/s | 69 kB 00:00
(4/27): nfs-ganesha-3.5-1.el8.x86_64.rpm 3.8 MB/s | 708 kB 00:00
(5/27): cups-libs-2.2.6-38.el8.x86_64.rpm 894 kB/s | 433 kB 00:00
(6/27): avahi-libs-0.7-20.el8.x86_64.rpm 103 kB/s | 62 kB 00:00
(7/27): gssproxy-0.8.0-19.el8.x86_64.rpm 3.0 MB/s | 119 kB 00:00
(8/27): keyutils-1.5.10-6.el8.x86_64.rpm 822 kB/s | 63 kB 00:00
(9/27): libverto-libevent-0.3.0-5.el8.x86_64.rpm 212 kB/s | 16 kB 00:00
(10/27): checkpolicy-2.9-1.el8.x86_64.rpm 446 kB/s | 348 kB 00:00
(11/27): libwbclient-4.13.3-3.el8.x86_64.rpm 897 kB/s | 119 kB 00:00
(12/27): policycoreutils-python-utils-2.9-14.el8.noarch.rpm 1.4 MB/s | 252 kB 00:00
(13/27): nfs-utils-2.3.3-41.el8.x86_64.rpm 1.8 MB/s | 497 kB 00:00
(14/27): psmisc-23.1-5.el8.x86_64.rpm 945 kB/s | 151 kB 00:00
(15/27): python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64.rpm 613 kB/s | 86 kB 00:00
(16/27): python3-libsemanage-2.9-6.el8.x86_64.rpm 2.0 MB/s | 127 kB 00:00
(17/27): python3-pyyaml-3.12-12.el8.x86_64.rpm 2.6 MB/s | 193 kB 00:00
(18/27): python3-setools-4.3.0-2.el8.x86_64.rpm 2.5 MB/s | 626 kB 00:00
(19/27): quota-4.04-12.el8.x86_64.rpm 3.5 MB/s | 213 kB 00:00
(20/27): quota-nls-4.04-12.el8.noarch.rpm 2.5 MB/s | 95 kB 00:00
(21/27): rpcbind-1.2.5-8.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00
(22/27): python3-policycoreutils-2.9-14.el8.noarch.rpm 1.7 MB/s | 2.2 MB 00:01
(23/27): samba-common-4.13.3-3.el8.noarch.rpm 1.3 MB/s | 218 kB 00:00
(24/27): samba-common-libs-4.13.3-3.el8.x86_64.rpm 1.4 MB/s | 171 kB 00:00
(25/27): userspace-rcu-0.10.1-4.el8.x86_64.rpm 1.1 MB/s | 101 kB 00:00
(26/27): samba-client-libs-4.13.3-3.el8.x86_64.rpm 3.7 MB/s | 5.4 MB 00:01
(27/27): libicu-60.3-2.el8_1.x86_64.rpm 2.7 MB/s | 8.8 MB 00:03
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 4.0 MB/s | 21 MB 00:05
warning: /var/cache/dnf/centos-nfs-ganesha3-a441f11834edc8e2/packages/libntirpc-3.4-1.el8.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID e451e5b5: NOKEY
CentOS-8 - NFS Ganesha 3 200 kB/s | 1.0 kB 00:00
Importing GPG key 0xE451E5B5:
Userid : "CentOS Storage SIG (http://wiki.centos.org/SpecialInterestGroup/Storage) <security@centos.org>"
Fingerprint: 7412 9C0B 173B 071A 3775 951A D4A2 E50B E451 E5B5
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : userspace-rcu-0.10.1-4.el8.x86_64 1/27
Running scriptlet: userspace-rcu-0.10.1-4.el8.x86_64 1/27
Installing : libntirpc-3.4-1.el8.x86_64 2/27
Running scriptlet: samba-common-4.13.3-3.el8.noarch 3/27
Installing : samba-common-4.13.3-3.el8.noarch 3/27
Running scriptlet: samba-common-4.13.3-3.el8.noarch 3/27
Running scriptlet: rpcbind-1.2.5-8.el8.x86_64 4/27
Installing : rpcbind-1.2.5-8.el8.x86_64 4/27
Running scriptlet: rpcbind-1.2.5-8.el8.x86_64 4/27
Installing : avahi-libs-0.7-20.el8.x86_64 5/27
Installing : cups-libs-1:2.2.6-38.el8.x86_64 6/27
Installing : quota-nls-1:4.04-12.el8.noarch 7/27
Installing : quota-1:4.04-12.el8.x86_64 8/27
Installing : python3-setools-4.3.0-2.el8.x86_64 9/27
Installing : python3-pyyaml-3.12-12.el8.x86_64 10/27
Installing : python3-libsemanage-2.9-6.el8.x86_64 11/27
Installing : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64 12/27
Installing : psmisc-23.1-5.el8.x86_64 13/27
Installing : libverto-libevent-0.3.0-5.el8.x86_64 14/27
Installing : gssproxy-0.8.0-19.el8.x86_64 15/27
Running scriptlet: gssproxy-0.8.0-19.el8.x86_64 15/27
Installing : libicu-60.3-2.el8_1.x86_64 16/27
Running scriptlet: libicu-60.3-2.el8_1.x86_64 16/27
Installing : libwbclient-4.13.3-3.el8.x86_64 17/27
Installing : samba-common-libs-4.13.3-3.el8.x86_64 18/27
Installing : samba-client-libs-4.13.3-3.el8.x86_64 19/27
Installing : keyutils-1.5.10-6.el8.x86_64 20/27
Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64 21/27
Installing : nfs-utils-1:2.3.3-41.el8.x86_64 21/27
Running scriptlet: nfs-utils-1:2.3.3-41.el8.x86_64 21/27
Installing : checkpolicy-2.9-1.el8.x86_64 22/27
Installing : python3-policycoreutils-2.9-14.el8.noarch 23/27
Installing : policycoreutils-python-utils-2.9-14.el8.noarch 24/27
Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch 25/27
Installing : nfs-ganesha-selinux-3.5-1.el8.noarch 25/27
Running scriptlet: nfs-ganesha-selinux-3.5-1.el8.noarch 25/27
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 26/27
Installing : nfs-ganesha-3.5-1.el8.x86_64 26/27
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 26/27
Installing : nfs-ganesha-vfs-3.5-1.el8.x86_64 27/27
Running scriptlet: libwbclient-4.13.3-3.el8.x86_64 27/27
Running scriptlet: nfs-ganesha-3.5-1.el8.x86_64 27/27
Running scriptlet: nfs-ganesha-vfs-3.5-1.el8.x86_64 27/27
Verifying : libntirpc-3.4-1.el8.x86_64 1/27
Verifying : nfs-ganesha-3.5-1.el8.x86_64 2/27
Verifying : nfs-ganesha-selinux-3.5-1.el8.noarch 3/27
Verifying : nfs-ganesha-vfs-3.5-1.el8.x86_64 4/27
Verifying : avahi-libs-0.7-20.el8.x86_64 5/27
Verifying : checkpolicy-2.9-1.el8.x86_64 6/27
Verifying : cups-libs-1:2.2.6-38.el8.x86_64 7/27
Verifying : gssproxy-0.8.0-19.el8.x86_64 8/27
Verifying : keyutils-1.5.10-6.el8.x86_64 9/27
Verifying : libicu-60.3-2.el8_1.x86_64 10/27
Verifying : libverto-libevent-0.3.0-5.el8.x86_64 11/27
Verifying : libwbclient-4.13.3-3.el8.x86_64 12/27
Verifying : nfs-utils-1:2.3.3-41.el8.x86_64 13/27
Verifying : policycoreutils-python-utils-2.9-14.el8.noarch 14/27
Verifying : psmisc-23.1-5.el8.x86_64 15/27
Verifying : python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64 16/27
Verifying : python3-libsemanage-2.9-6.el8.x86_64 17/27
Verifying : python3-policycoreutils-2.9-14.el8.noarch 18/27
Verifying : python3-pyyaml-3.12-12.el8.x86_64 19/27
Verifying : python3-setools-4.3.0-2.el8.x86_64 20/27
Verifying : quota-1:4.04-12.el8.x86_64 21/27
Verifying : quota-nls-1:4.04-12.el8.noarch 22/27
Verifying : rpcbind-1.2.5-8.el8.x86_64 23/27
Verifying : samba-client-libs-4.13.3-3.el8.x86_64 24/27
Verifying : samba-common-4.13.3-3.el8.noarch 25/27
Verifying : samba-common-libs-4.13.3-3.el8.x86_64 26/27
Verifying : userspace-rcu-0.10.1-4.el8.x86_64 27/27
Installed:
avahi-libs-0.7-20.el8.x86_64 checkpolicy-2.9-1.el8.x86_64 cups-libs-1:2.2.6-38.el8.x86_64 gssproxy-0.8.0-19.el8.x86_64
keyutils-1.5.10-6.el8.x86_64 libicu-60.3-2.el8_1.x86_64 libntirpc-3.4-1.el8.x86_64 libverto-libevent-0.3.0-5.el8.x86_64
libwbclient-4.13.3-3.el8.x86_64 nfs-ganesha-3.5-1.el8.x86_64 nfs-ganesha-selinux-3.5-1.el8.noarch nfs-ganesha-vfs-3.5-1.el8.x86_64
nfs-utils-1:2.3.3-41.el8.x86_64 policycoreutils-python-utils-2.9-14.el8.noarch psmisc-23.1-5.el8.x86_64 python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64
python3-libsemanage-2.9-6.el8.x86_64 python3-policycoreutils-2.9-14.el8.noarch python3-pyyaml-3.12-12.el8.x86_64 python3-setools-4.3.0-2.el8.x86_64
quota-1:4.04-12.el8.x86_64 quota-nls-1:4.04-12.el8.noarch rpcbind-1.2.5-8.el8.x86_64 samba-client-libs-4.13.3-3.el8.x86_64
samba-common-4.13.3-3.el8.noarch samba-common-libs-4.13.3-3.el8.x86_64 userspace-rcu-0.10.1-4.el8.x86_64
Complete!
[root@srv ~]# tail -n 17 /etc/ganesha/ganesha.conf
EXPORT
{
Export_Id = 2;
Path = /mnt/storage1;
Pseudo = /mnt/storage1;
Protocols = 3,4;
Access_Type = RW;
Squash = None;
FSAL
{
Name = VFS;
}
CLIENT
{
Clients = 192.168.0.12;
}
}
[root@srv ~]# systemctl status nfs-ganesha
● nfs-ganesha.service - NFS-Ganesha file server
Loaded: loaded (/usr/lib/systemd/system/nfs-ganesha.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2021-04-13 09:51:01 UTC; 3s ago
Docs: http://github.com/nfs-ganesha/nfs-ganesha/wiki
Process: 3070 ExecStart=/bin/bash -c ${NUMACTL} ${NUMAOPTS} /usr/bin/ganesha.nfsd ${OPTIONS} ${EPOCH} (code=exited, status=0/SUCCESS)
Main PID: 3071 (ganesha.nfsd)
Tasks: 21 (limit: 11409)
Memory: 6.2M
CGroup: /system.slice/nfs-ganesha.service
└─3071 /usr/bin/ganesha.nfsd -L /var/log/ganesha/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_EVENT
Apr 13 09:51:01 srv systemd[1]: Starting NFS-Ganesha file server...
Apr 13 09:51:01 srv systemd[1]: Started NFS-Ganesha file server.
Apr 13 09:51:01 srv ganesha.nfsd[3071]: libnfsidmap: Unable to determine the NFSv4 domain; Using 'localdomain' as the NFSv4 domain which means UIDs will be mapped to the 'Nobody-User' user>
Apr 13 09:51:01 srv ganesha.nfsd[3071]: libnfsidmap: using (default) domain: localdomain
Apr 13 09:51:01 srv ganesha.nfsd[3071]: libnfsidmap: Realms list: 'LOCALDOMAIN'
Apr 13 09:51:01 srv ganesha.nfsd[3071]: libnfsidmap: loaded plugin /usr/lib64/libnfsidmap/nsswitch.so for method nsswitch
[root@srv ~]# systemctl enable nfs-ganesha
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-ganesha.service → /usr/lib/systemd/system/nfs-ganesha.service.
[root@srv ~]# firewall-cmd --permanent --zone=public --add-service=nfs
success
[root@srv ~]# firewall-cmd --reload
success
[root@srv ~]# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: enp0s3
sources:
services: cockpit dhcpv6-client nfs ssh
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules
[root@srv ~]# firewall-cmd --permanent --zone=public --remove-service=nfs
success
[root@srv ~]# firewall-cmd --permanent --zone=public --add-rich-rule="rule family="ipv4" source address="192.168.0.12" service name="nfs" accept"
success
[root@srv ~]# firewall-cmd --reload
success
[root@srv ~]# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: enp0s3
sources:
services: cockpit dhcpv6-client ssh
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
rule family="ipv4" source address="192.168.0.12" service name="nfs" accept