A great tool for caching a network filesystem like NFS mounts is cachefilesd! It is easy to use it and a good deal of stats can be retrieved from the tool. More on how it works here https://www.kernel.org/doc/Documentation/filesystems/caching/fscache.txt
Here are quick steps to cache an NFS mounts (it works with NFS-Ganesha servers, too):
- Install the daemon tool cachefilesd
- Check the configuration file /etc/cachefilesd.conf. In most cases, no need to edit the file! Just check the disk limits if they are good.
- Start the cachefilesd daemon.
- Mount the network directories with “fsc” option. Umount and mount them all if they’ve been already mounted. The fsc is mandatory option to enable file cacheing of a network mount.
- Check stats to see if the file cching is working properly.
The example below is under CentOS 8, but it is almost the same in most Linux distributions.
STEP 1) Install the daemon tool cachefilesd
This is straight forward, just install it with the package manager:
[root@srv ~]# dnf install cachefilesd Last metadata expiration check: 2:33:44 ago on Tue 08 Dec 2020 07:18:01 AM UTC. Dependencies resolved. ============================================================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================================================= Installing: cachefilesd x86_64 0.10.10-4.el8 BaseOS 43 k Transaction Summary ============================================================================================================================================================================================= Install 1 Package Total download size: 43 k Installed size: 71 k Is this ok [y/N]: y Downloading Packages: cachefilesd-0.10.10-4.el8.x86_64.rpm 3.1 MB/s | 43 kB 00:00 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 2.8 MB/s | 43 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : cachefilesd-0.10.10-4.el8.x86_64 1/1 Running scriptlet: cachefilesd-0.10.10-4.el8.x86_64 1/1 Verifying : cachefilesd-0.10.10-4.el8.x86_64 1/1 Installed: cachefilesd-0.10.10-4.el8.x86_64 Complete!
STEP 2) Check the configuration file and tune for your system.
In most cases, the defaults in /etc/cachefilesd.conf are good to start with:
dir /var/cache/fscache tag mycache brun 10% bcull 7% bstop 3% frun 10% fcull 7% fstop 3% # Assuming you're using SELinux with the default security policy included in # this package secctx system_u:system_r:cachefiles_kernel_t:s0
The directory where the cache will reside and the lines with the percentages are for disk space limitation. “brun 10%” means cache can runs freely till the disk space drops below 10%. “bcull 7%” – culling the cache when the free space drops below “7%” and more in the man page (or https://linux.die.net/man/5/cachefilesd.conf).
So if one maintains disk free space below 10% the configuration file should be edited.
STEP 3) Start the cachefilesd daemon.
And enable on boot to start automatically.
[root@srv ~]# systemctl start cachefilesd
[root@srv ~]# systemctl enable cachefilesd
Created symlink /etc/systemd/system/multi-user.target.wants/cachefilesd.service → /usr/lib/systemd/system/cachefilesd.service.
[root@srv ~]# systemctl status cachefilesd
● cachefilesd.service - Local network file caching management daemon
Loaded: loaded (/usr/lib/systemd/system/cachefilesd.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2020-12-08 10:01:24 UTC; 11s ago
Main PID: 29786 (cachefilesd)
Tasks: 1 (limit: 408616)
Memory: 2.5M
CGroup: /system.slice/cachefilesd.service
└─29786 /usr/sbin/cachefilesd -n -f /etc/cachefilesd.conf
Dec 08 10:01:24 srv systemd[1]: Starting Local network file caching management daemon...
Dec 08 10:01:24 srv systemd[1]: Started Local network file caching management daemon.
Dec 08 10:01:24 srv cachefilesd[29786]: About to bind cache
Dec 08 10:01:24 srv cachefilesd[29786]: Bound cache
Dec 08 10:01:24 srv cachefilesd[29786]: Daemon Started
The status command shows the daemon cachefilesd is running. But does it cache?
STEP 4) Mount the network filesystems with option fsc
To make cachefilesd cache a network mount the option fsc must be included in the mount options. Remount may not work correctly, so to be sure a full umount/mount should be executed. Here is an example /etc/fstab file:
192.168.0.1:/mnt/storage /mnt/storage nfs defaults,hard,intr,noexec,nosuid,_netdev,fsc,vers=4 0 0
And then mount with simple command:
mount /mnt/storage
Check whether the mounts if the FS cache is used. FSC must be “yes”.
[root@srv ~]# cat /proc/fs/nfsfs/volumes NV SERVER PORT DEV FSID FSC v4 c0a80001 801 0:41 d4098a2af096148:ec7560388cbe5b83 yes
There is a proc file for cache statistics:
[root@srv ~]# cat /proc/fs/fscache/stats FS-Cache statistics Cookies: idx=49 dat=4385599 spc=0 Objects: alc=43666 nal=0 avl=43666 ded=36002 ChkAux : non=0 ok=12289 upd=0 obs=761 Pages : mrk=24915179 unc=24492585 Acquire: n=4385648 nul=0 noc=0 ok=4385648 nbf=0 oom=0 Lookups: n=43666 neg=31372 pos=12294 crt=31372 tmo=0 Invals : n=1 run=1 Updates: n=0 nul=0 run=1 Relinqs: n=4377930 nul=0 wcr=0 rtr=0 AttrChg: n=0 ok=0 nbf=0 oom=0 run=0 Allocs : n=0 ok=0 wt=0 nbf=0 int=0 Allocs : ops=0 owt=0 abt=0 Retrvls: n=751549 ok=716860 wt=21436 nod=34689 nbf=0 int=0 oom=0 Retrvls: ops=751549 owt=9158 abt=0 Stores : n=550412 ok=550412 agn=0 nbf=0 oom=0 Stores : ops=33238 run=583650 pgs=550412 rxd=550412 olm=0 VmScan : nos=23963352 gon=0 bsy=0 can=0 wt=0 Ops : pend=9160 run=784788 enq=26874960 can=0 rej=0 Ops : ini=1301962 dfr=265 rel=1301962 gc=265 CacheOp: alo=0 luo=0 luc=0 gro=0 CacheOp: inv=0 upo=0 dro=0 pto=0 atc=0 syn=0 CacheOp: rap=0 ras=0 alp=0 als=0 wrp=0 ucp=0 dsp=0 CacheEv: nsp=761 stl=0 rtr=0 cul=0
And here is the cache directory filled with files. If there are no files, the FS cache is not used, probably the mount is not mounted with FSC! Umount and mount the mounts again.
[root@srv ~]# find /var/cache/fscache|head -n 20 /var/cache/fscache /var/cache/fscache/graveyard /var/cache/fscache/cache /var/cache/fscache/cache/@4a /var/cache/fscache/cache/@4a/I03nfs /var/cache/fscache/cache/@4a/I03nfs/@84 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XMW1n8cXgyl60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XZG2n84qfxl60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XQM2n8IQ4El60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XZO2n8Ms7kl60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XvQ2n88dKgl60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XV40o8cREC6b0 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7X3Y1n8MHO_l60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7Xhkwn80f23zb0 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XEQ2n8IuAll60 /var/cache/fscache/cache/@4a/I03nfs/@84/Jc0010800840000cG0400/@68/J1100000000000Q0goaWH946iIn7oUMELrd80000000040000g00Kb000wFe000jt000oG3000000040000g000000000/@8b/EA0g00sg0200n8000000ixBMHyy9gdcUm_O8ewl7XoO2n8A7Bll60 [root@srv ~]# du -d 1 -h /var/cache/fscache 4.0K /var/cache/fscache/graveyard 3.8G /var/cache/fscache/cache 3.8G /var/cache/fscache
There are 3.8G in the cache.
Hello,
If FSC is not “yes” what should I do. Thanks
Mount the network filesystems with option fsc. Use the /etc/fstab entry in STEP 4). First, umount your share, then edit the fstab entry as shown above (aka include the “fsc“) and then mount the share.