Install and use collectd under CentOS Stream 9 using OpsTools

Author:

collectd is a daemon, which collects system data and statistics and it may send them over the network to a back-end data. Under CentOS Stream 9 the collectd package has been removed from the CentOS Stream 9 repositories. Special Interest Groups (SIG), which is a group of CentOS Community, provides and supports multiple small repositories for different Linux utilities and tools. One of the SIG‘s repositories is OpsTools, which provides a package for collectd daemon and multiple collectd modules packed in separate packages.

To use collectd under CentOS Stream 9 a CentOS Community repository should be installed – OpsTools.

Steps to install and use the collectd daemon:

dnf install -y centos-release-opstools
dnf install -y collectd

It’s worth noting that all the Linux distribution, which tries to track and be compatible with Red Hat Enterprise Linux like Rocky Linux will have the same problem with missing collectd package. So the SIG OpsTools repository should be also possible to install under all of these Linux distributions.

Here is the output of installing the above packages under CentOS Stream 9:

[root@srv ~]# dnf install -y centos-release-opstools
Last metadata expiration check: 0:00:23 ago on Wed 22 Jun 2022 02:05:19 PM UTC.
Dependencies resolved.
======================================================================================
 Package                       Architecture Version         Repository           Size
======================================================================================
Installing:
 centos-release-opstools       noarch       1-12.el9s       extras-common       8.4 k

Transaction Summary
======================================================================================
Install  1 Package

Total download size: 8.4 k
Installed size: 1.7 k
Downloading Packages:
centos-release-opstools-1-12.el9s.noarch.rpm           62 kB/s | 8.4 kB     00:00    
--------------------------------------------------------------------------------------
Total                                                  22 kB/s | 8.4 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                              1/1 
  Installing       : centos-release-opstools-1-12.el9s.noarch                     1/1 
  Verifying        : centos-release-opstools-1-12.el9s.noarch                     1/1 

Installed:
  centos-release-opstools-1-12.el9s.noarch                                            

Complete!
[root@srv ~]# dnf install -y collectd
CentOS Stream 9 - OpsTools - collectd                  12 kB/s |  41 kB     00:03    
Dependencies resolved.
======================================================================================
 Package          Architecture   Version                Repository               Size
======================================================================================
Installing:
 collectd         x86_64         5.12.0-7.el9s          centos-opstools         673 k
Installing dependencies:
 yajl             x86_64         2.1.0-20.el9           appstream                38 k

Transaction Summary
======================================================================================
Install  2 Packages

Total download size: 711 k
Installed size: 2.2 M
Downloading Packages:
(1/2): yajl-2.1.0-20.el9.x86_64.rpm                   179 kB/s |  38 kB     00:00    
(2/2): collectd-5.12.0-7.el9s.x86_64.rpm              1.1 MB/s | 673 kB     00:00    
--------------------------------------------------------------------------------------
Total                                                 594 kB/s | 711 kB     00:01     
CentOS Stream 9 - OpsTools - collectd                 1.0 MB/s | 1.0 kB     00:00    
Importing GPG key 0x51BC2A13:
 Userid     : "CentOS OpsTools SIG (https://wiki.centos.org/SpecialInterestGroup/OpsTools) <security@centos.org>"
 Fingerprint: 7872 8176 9AD7 3878 85EE A649 4FD9 5327 51BC 2A13
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-OpsTools
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                              1/1 
  Installing       : yajl-2.1.0-20.el9.x86_64                                     1/2 
  Installing       : collectd-5.12.0-7.el9s.x86_64                                2/2 
  Running scriptlet: collectd-5.12.0-7.el9s.x86_64                                2/2 
  Verifying        : collectd-5.12.0-7.el9s.x86_64                                1/2 
  Verifying        : yajl-2.1.0-20.el9.x86_64                                     2/2 

Installed:
  collectd-5.12.0-7.el9s.x86_64                yajl-2.1.0-20.el9.x86_64               

Complete!

collectd default configuration

The default configuration comments all the lines in /etc/collectd.conf except one, which includes all the files under /etc/collectd.d/. So the logic is to have a separate file for each collectd plugin. By default, there are 5 default configuration files activating 5 plugins:

[root@srv ~]# ls -altr /etc/collectd.d/
total 32
-rw-r--r--.  1 root root   18 Feb  1 09:44 90-default-plugins-syslog.conf
-rw-r--r--.  1 root root   18 Feb  1 09:44 90-default-plugins-memory.conf
-rw-r--r--.  1 root root   16 Feb  1 09:44 90-default-plugins-load.conf
-rw-r--r--.  1 root root   21 Feb  1 09:44 90-default-plugins-interface.conf
-rw-r--r--.  1 root root   15 Feb  1 09:44 90-default-plugins-cpu.conf
drwxr-xr-x.  2 root root 4096 Jun 22 19:48 .
drwxr-xr-x. 91 root root 4096 Jun 22 19:48 ..

Installing only collectd won’t install output plugin, which is responsible for sending the collected metrics and statistics to the back-end. Plugins collect data locally from the system and a plugin or plugins send data remotely to the back-end database like rrd files (locally), influxdb, graphite, prometheus, MySQL and more and then using Grafana or other web software to create charts. The simple setup is to use rrd files locally:

root@srv ~]# dnf install -y collectd-rrdtool
Last metadata expiration check: 5:42:31 ago on Wed 22 Jun 2022 02:06:00 PM UTC.
Dependencies resolved.
======================================================================================
 Package                    Arch       Version              Repository           Size
======================================================================================
Installing:
 collectd-rrdtool           x86_64     5.12.0-7.el9s        centos-opstools      22 k
Installing dependencies:
 cairo                      x86_64     1.17.4-7.el9         appstream           660 k
 dejavu-sans-mono-fonts     noarch     2.37-18.el9          baseos              467 k
 fontconfig                 x86_64     2.14.0-1.el9         appstream           297 k
 freetype                   x86_64     2.10.4-9.el9         baseos              388 k
 fribidi                    x86_64     1.0.10-6.el9.2       appstream            85 k
 graphite2                  x86_64     1.3.14-9.el9         baseos               95 k
 harfbuzz                   x86_64     2.7.4-5.el9          baseos              625 k
 libX11                     x86_64     1.7.0-7.el9          appstream           647 k
 libX11-common              noarch     1.7.0-7.el9          appstream           206 k
 libXau                     x86_64     1.0.9-8.el9          appstream            31 k
 libXext                    x86_64     1.3.4-8.el9          appstream            40 k
 libXft                     x86_64     2.3.3-8.el9          appstream            62 k
 libXrender                 x86_64     0.9.10-16.el9        appstream            28 k
 libdatrie                  x86_64     0.2.13-4.el9         appstream            33 k
 libpng                     x86_64     2:1.6.37-12.el9      baseos              117 k
 libthai                    x86_64     0.1.28-8.el9         appstream           208 k
 libxcb                     x86_64     1.13.1-9.el9         appstream           243 k
 pango                      x86_64     1.48.7-2.el9         appstream           303 k
 pixman                     x86_64     0.40.0-5.el9         appstream           271 k
 rrdtool                    x86_64     1.7.2-21.el9         appstream           540 k
 xml-common                 noarch     0.6.3-58.el9         appstream            32 k

Transaction Summary
======================================================================================
Install  22 Packages

Total size: 5.3 M
Total download size: 3.6 M
Installed size: 14 M
Downloading Packages:
[SKIPPED] collectd-rrdtool-5.12.0-7.el9s.x86_64.rpm: Already downloaded              
[SKIPPED] dejavu-sans-mono-fonts-2.37-18.el9.noarch.rpm: Already downloaded          
[SKIPPED] freetype-2.10.4-9.el9.x86_64.rpm: Already downloaded                       
[SKIPPED] graphite2-1.3.14-9.el9.x86_64.rpm: Already downloaded                      
[SKIPPED] harfbuzz-2.7.4-5.el9.x86_64.rpm: Already downloaded                        
[SKIPPED] libpng-1.6.37-12.el9.x86_64.rpm: Already downloaded                        
(7/22): fribidi-1.0.10-6.el9.2.x86_64.rpm             268 kB/s |  85 kB     00:00    
(8/22): fontconfig-2.14.0-1.el9.x86_64.rpm            501 kB/s | 297 kB     00:00    
(9/22): libX11-1.7.0-7.el9.x86_64.rpm                 845 kB/s | 647 kB     00:00    
(10/22): cairo-1.17.4-7.el9.x86_64.rpm                592 kB/s | 660 kB     00:01    
(11/22): libXau-1.0.9-8.el9.x86_64.rpm                356 kB/s |  31 kB     00:00    
(12/22): libXext-1.3.4-8.el9.x86_64.rpm               445 kB/s |  40 kB     00:00    
(13/22): libXft-2.3.3-8.el9.x86_64.rpm                713 kB/s |  62 kB     00:00    
(14/22): libXrender-0.9.10-16.el9.x86_64.rpm          304 kB/s |  28 kB     00:00    
(15/22): libdatrie-0.2.13-4.el9.x86_64.rpm            372 kB/s |  33 kB     00:00    
(16/22): libX11-common-1.7.0-7.el9.noarch.rpm         266 kB/s | 206 kB     00:00    
(17/22): libthai-0.1.28-8.el9.x86_64.rpm              927 kB/s | 208 kB     00:00    
(18/22): libxcb-1.13.1-9.el9.x86_64.rpm               881 kB/s | 243 kB     00:00    
(19/22): pango-1.48.7-2.el9.x86_64.rpm                903 kB/s | 303 kB     00:00    
(20/22): xml-common-0.6.3-58.el9.noarch.rpm           393 kB/s |  32 kB     00:00    
(21/22): pixman-0.40.0-5.el9.x86_64.rpm               955 kB/s | 271 kB     00:00    
(22/22): rrdtool-1.7.2-21.el9.x86_64.rpm              962 kB/s | 540 kB     00:00    
--------------------------------------------------------------------------------------
Total
.....
.....
  rrdtool-1.7.2-21.el9.x86_64                 xml-common-0.6.3-58.el9.noarch         

Complete!
[root@srv ~]# systemctl start collectd
[root@srv ~]# systemctl status collectd
● collectd.service - Collectd statistics daemon
     Loaded: loaded (/usr/lib/systemd/system/collectd.service; disabled; vendor prese>
     Active: active (running) since Wed 2022-06-22 19:49:28 UTC; 16min ago
       Docs: man:collectd(1)
             man:collectd.conf(5)
   Main PID: 1875 (collectd)
      Tasks: 12 (limit: 23472)
     Memory: 9.5M
        CPU: 125ms
     CGroup: /system.slice/collectd.service
             └─1875 /usr/sbin/collectd

Jun 22 19:49:28 srv systemd[1]: Starting Collectd statistics daemon...
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "cpu" successfully loaded.
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "interface" successfully load>
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "load" successfully loaded.
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "memory" successfully loaded.
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "syslog" successfully loaded.
Jun 22 19:49:28 srv collectd[1875]: plugin_load: plugin "rrdtool" successfully loaded.
Jun 22 19:49:28 srv collectd[1875]: Systemd detected, trying to signal readiness.
Jun 22 19:49:28 srv systemd[1]: Started Collectd statistics daemon.
Jun 22 19:49:28 srv collectd[1875]: Initialization complete, entering read-loop.

The rrd file plugin will install all the dependencies and a plugin file under /etc/collectd.d/.

[root@srv ~]# cat /etc/collectd.d/rrdtool.conf 
LoadPlugin rrdtool
<Plugin rrdtool>
       DataDir "/var/lib/collectd/rrd"
       CacheTimeout 120
       CacheFlush   900
</Plugin>h

And after a while the rrd files appear:

[root@srv ~]# find /var/lib/collectd/rrd/
/var/lib/collectd/rrd/
/var/lib/collectd/rrd/srv
/var/lib/collectd/rrd/srv/load
/var/lib/collectd/rrd/srv/load/load.rrd
/var/lib/collectd/rrd/srv/interface-enp0s3
/var/lib/collectd/rrd/srv/interface-enp0s3/if_dropped.rrd
/var/lib/collectd/rrd/srv/interface-enp0s3/if_octets.rrd
/var/lib/collectd/rrd/srv/interface-enp0s3/if_packets.rrd
/var/lib/collectd/rrd/srv/interface-enp0s3/if_errors.rrd
/var/lib/collectd/rrd/srv/cpu-0
/var/lib/collectd/rrd/srv/cpu-0/cpu-system.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-idle.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-user.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-interrupt.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-nice.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-steal.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-wait.rrd
/var/lib/collectd/rrd/srv/cpu-0/cpu-softirq.rrd
/var/lib/collectd/rrd/srv/cpu-1
/var/lib/collectd/rrd/srv/cpu-1/cpu-system.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-idle.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-user.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-interrupt.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-nice.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-steal.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-wait.rrd
/var/lib/collectd/rrd/srv/cpu-1/cpu-softirq.rrd
/var/lib/collectd/rrd/srv/memory
/var/lib/collectd/rrd/srv/memory/memory-slab_unrecl.rrd
/var/lib/collectd/rrd/srv/memory/memory-used.rrd
/var/lib/collectd/rrd/srv/memory/memory-free.rrd
/var/lib/collectd/rrd/srv/memory/memory-slab_recl.rrd
/var/lib/collectd/rrd/srv/memory/memory-buffered.rrd
/var/lib/collectd/rrd/srv/memory/memory-cached.rrd
/var/lib/collectd/rrd/srv/interface-lo
/var/lib/collectd/rrd/srv/interface-lo/if_dropped.rrd
/var/lib/collectd/rrd/srv/interface-lo/if_octets.rrd
/var/lib/collectd/rrd/srv/interface-lo/if_packets.rrd
/var/lib/collectd/rrd/srv/interface-lo/if_errors.rrd

collectd modules in SIG OpsTools repository

Here are all the available collectd modules in OpsTools – around 55 modules:

[root@srv ~]# dnf search collectd*
Last metadata expiration check: 0:01:59 ago on Wed 22 Jun 2022 02:06:00 PM UTC.
========================= Name & Summary Matched: collectd* ==========================
collectd-libpod-stats.x86_64 : Collectd plugin for monitoring resource usage of
                             : containers managed by libpod
collectd-sensubility.x86_64 : collectd-exec extension enabling collectd to bahave like
                            : sensu-client
collectd-utils.x86_64 : Collectd utilities
============================== Name Matched: collectd* ===============================
collectd.x86_64 : Statistics collection daemon for filling RRD files
collectd-amqp.x86_64 : AMQP 0.9 plugin for collectd
collectd-amqp1.x86_64 : Sends JSON-encoded data to an Advanced Message Queuing Protocol
collectd-apache.x86_64 : Apache plugin for collectd
collectd-ascent.x86_64 : Ascent plugin for collectd
collectd-bind.x86_64 : Bind plugin for collectd
collectd-ceph.x86_64 : Ceph plugin for collectd
collectd-check-uptime.x86_64 : Check for cache events for uptime metric
collectd-chrony.x86_64 : Chrony plugin for collectd
collectd-connectivity.x86_64 : Monitors network interface up/down status via netlink library
collectd-curl.x86_64 : Curl plugin for collectd
collectd-curl_json.x86_64 : Curl JSON plugin for collectd
collectd-curl_xml.x86_64 : Curl XML plugin for collectd
collectd-dbi.x86_64 : DBI plugin for collectd
collectd-disk.x86_64 : Disk plugin for collectd
collectd-dns.x86_64 : DNS traffic analysis plugin for collectd
collectd-drbd.x86_64 : DRBD plugin for collectd
collectd-email.x86_64 : Email plugin for collectd
collectd-hugepages.x86_64 : Number of hugepages on Linux
collectd-infiniband.x86_64 : Collect metrics about infiniband ports
collectd-ipmi.x86_64 : IPMI plugin for collectd
collectd-ipvs.x86_64 : IPVS plugin for collectd
collectd-log_logstash.x86_64 : Logstash plugin for collectd
collectd-logparser.x86_64 : Parse different kinds of logs
collectd-mcelog.x86_64 : Machine Check Exceptions notifications
collectd-mdevents.x86_64 : Get events from RAID arrays in syslog
collectd-mysql.x86_64 : MySQL plugin for collectd
collectd-netlink.x86_64 : Netlink plugin for collectd
collectd-nginx.x86_64 : Nginx plugin for collectd
collectd-notify_email.x86_64 : Notify email plugin for collectd
collectd-openldap.x86_64 : OpenLDAP plugin for collectd
collectd-ovs-events.x86_64 : Open vSwitch events plugin for collectd
collectd-ovs-stats.x86_64 : Open vSwitch statsevents plugin for collectd
collectd-pcie-errors.x86_64 : Read errors from PCI Express Device Status
collectd-ping.x86_64 : Ping plugin for collectd
collectd-pmu.x86_64 : Intel PMU plugin for collectd
collectd-postgresql.x86_64 : PostgreSQL plugin for collectd
collectd-procevent.x86_64 : Plugin that monitors process starts/stops via netlink library
collectd-python.x86_64 : Python plugin for collectd
collectd-rdt.x86_64 : Intel RDT plugin for collectd
collectd-rrdcached.x86_64 : RRDCacheD plugin for collectd
collectd-rrdtool.x86_64 : RRDTool plugin for collectd
collectd-sensors.x86_64 : Libsensors module for collectd
collectd-smart.x86_64 : SMART plugin for collectd
collectd-snmp.x86_64 : SNMP module for collectd
collectd-snmp-agent.x86_64 : SNMP agent module for collectd
collectd-sysevent.x86_64 : Plugin that monitors rsyslog for system events
collectd-turbostat.x86_64 : Turbostat module for collectd
collectd-virt.x86_64 : Libvirt plugin for collectd
collectd-write_http.x86_64 : HTTP output plugin for collectd
collectd-write_kafka.x86_64 : Kafka output plugin for collectd
collectd-write_prometheus.x86_64 : Prometheus output plugin for collectd
collectd-write_sensu.x86_64 : Sensu output plugin for collectd
collectd-write_syslog.x86_64 : syslog output plugin for collectd
collectd-write_tsdb.x86_64 : OpenTSDB output plugin for collectd
collectd-zookeeper.x86_64 : Zookeeper plugin for collectd
============================= Summary Matched: collectd* =============================
libcollectdclient.x86_64 : Collectd client library

Check out collectd-ping plugin to track the network latency – Install and use collectd-ping under CentOS 8 to monitor latency

Leave a Reply

Your email address will not be published. Required fields are marked *