Monitor and analyze with Grafana, influxdb 1.8 and collectd under Ubuntu 22.04 LTS

This is an updated version of the previous version of this topic – Monitor and analyze with Grafana, influxdb 1.8 and collectd under CentOS Stream 9, but this time for Ubuntu 22.04 LTS. The article describes how to build modern analytic and monitoring solutions for system and application performance metrics. A solution, which may host all the server’s metrics and a sophisticated application, allows easy analyses of the data and powerful graphs to visualize the data.
A brief introduction to the main three software used to build the proposed solution:

  1. Grafana – an analytics and a web visualization tool. It supports dashboards, charts, graphs, alerts, and many more.
  2. influxdb – a time series database. Bleeding fast reads and writes and optimized for time.
  3. collectd – a data collection daemon, which obtain metrics from the host it is started and sends the metrics to the database (i.e. influxdb). It has around 170 plugins to collect metrics.

What is the task of each tool:

  1. collectd – gathers metrics and statistics using its plugins every 10 seconds on the host it runs and then sends the data over UDP to the influxdb using a simple text-based protocol.
  2. influxdb – listens on an open UDP port for data coming from multiple collectd instances installed on many different devices. In this case, a Linux server running Ubuntu 22.04 LTS.
  3. Grafana – an analytics and a web visualization tool. A web application, which connects to the InfluxDB and visualizes the time series metrics in graphs organized in dashboards. Graphs for CPU, memory, network, storage usage, and many more.
  4. nginx to enable SSL and proxy in front of the Grafana.

The whole solution uses the Ubuntu 22.04 LTS server edition distro. Installing the Ubuntu 22.04 LTS is a mandatory step to proceed further with this article – Installation of base Ubuntu server 22.04 LTS
The UDP influxdb port should be open per IP basis and web port of the web server (nginx) is up to the purpose of the solution – it can be behind a VPN or openly accessible by Internet.

STEP 1) Install additional repositories for Grafana, InfluxDB and collectd.

collectd is part of the Ubuntu official repositories. Grafana and InfluxDB maintain their official repositories. Here is how to install them.
Add the InfluxDB repository by first, importing the key of the InfluxDB repository and add the URL of the repository in /etc/apt/sources.list.

myuser@srv:~$ sudo curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
echo 'deb https://repos.influxdata.com/debian stable main' > /etc/apt/sources.list.d/influxdata.list

Then, repeated the same procedure with the Grafana repository:

myuser@srv:~$ sudo curl -sL https://packages.grafana.com/gpg.key | sudo apt-key add -
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
echo 'deb https://packages.grafana.com/oss/deb stable main' > /etc/apt/sources.list.d/grafana.list

Execute apt update to include the available file packages from all repositories including the ones:

apt update

Keep on reading!

Add source InfluxDB 1.8 with basic authentication in Grafana using the web interface

This article shows how to add a new source in Grafana with screenshots. The source is InluxDB 1.8 with basic authentication enabled. The main purpose of this article is to give the user knowledge of how to:

  • Enable basic authentication in InfluxDB
  • Create users – administrative and ordinary ones in InfluxDB and give permissions for the database.
  • Add the InfluxDB source in Grafana using web interface. with basic authentication enabled with credentials created in the article.

It is supposed the InfluxDB is installed and running on the loopback 127.0.0.1, at least. If the InfluxDB service is not local for the Grafana service replace the 127.0.0.1 with the appropriate IP and adjust the firewall such that it accepts connections from the Grafana server IP. For installing InfluxDB with detailed information including firewall modifications there is another article here – Monitor and analyze with Grafana, InfluxDB 1.8 and collectd under CentOS Stream 9.
No installation information for InfluxDB or Grafana is included in this article and if they are needed check out the article above.

STEP 1) Create users in InfluxDB.

By default, InfluxDB authentication is disabled and no users are required to access and manage the service and the databases. That’s why, the first thing to do is to create an administrative user, which will manage the databases when the basic authentication will be enabled. At the same time, when creating the administrative user, ordinary users may be created, too.
To connect to the InfluxDB to manage the service the InfluxDB command-line tool influx will be used. influx connects to http://127.0.0.1:8086 – an HTTP interface to access the InfluxDB service.

[root@srv ~]# influx
Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10
> CREATE USER admin WITH PASSWORD 'aiqu8ohth9Cheeshai]c' WITH ALL PRIVILEGES
> SHOW USERS
user  admin
----  -----
admin true
> CREATE USER collectd WITH PASSWORD 'ohg|ahTh9Sa|quoh8zoh'
> GRANT READ ON "collectd" TO "collectd"
> SHOW USERS
user     admin
----     -----
admin    true
collectd false
>

First, the administrative user with admin name is created, and then the ordinary user with the collectd name. For the ordinary user, the access privileges are granted only for READ on the collectd database. It is typical to name the database and the user accessing it with the same name. The format of the GRANT command is the following:

GRANT "[PRIVILEGES]" ON "[database_name]" TO "[user_name]"

READ privileges are enough for Grafana to access the data.
Keep on reading!

Monitor and analyze with Grafana, influxdb 1.8 and collectd under CentOS Stream 9

This article describes how to build a modern analytic and monitoring solutions for system and application performance metrics. A solution, which may host all the server’s metrics and a sophisticated application, allows easy analyses of the data and powerful graphs to visualize the data.
A brief introduction to the main three software used to build the proposed solution:

  1. Grafana – an analytics and a web visualization tool. It supports dashboards, charts, graphs, alerts, and many more.
  2. influxdb – a time series database. Bleeding fast reads and writes and optimized for time.
  3. collectd – a data collection daemon, which obtain metrics from the host it is started and sends the metrics to the database (i.e. influxdb). It has around 170 plugins to collect metrics.

What is the task of each tool:

  1. collectd – gathers metrics and statistics using its plugins every 10 seconds on the host it runs and then sends the data over UDP to the influxdb using a simple text-based protocol.
  2. influxdb – listens on an open UDP port for data coming from multiple collectd instances installed on many different devices. In this case, a Linux server running CentOS Stream 9.
  3. Grafana – an analytics and a web visualization tool. A web application, which connects to the InfluxDB and visualizes the time series metrics in graphs organized in dashboards. Graphs for CPU, memory, network, storage usage, and many more.
  4. nginx to enable SSL and proxy in front of the Grafana.

The whole solution uses the CentOS Stream 9 Linux distro. Installing the CentOS Stream 9 is a mandatory step to proceed further with this article – Network installation of CentOS Stream 9 (20220606.0) – minimal server installation
The UDP influxdb port should be open per IP basis and web port of the web server (nginx) is up to the purpose of the solution – it can be behind a VPN or openly accessible by Internet.

STEP 1) Install additional repositories for Grafana, influxdb and collectd.

Install CentOS official EPEL and OpsTools repositories. EPEL provides additional packages to the base CentOS packages and OpsTools provides collectd and more collectd plugins than the ones included in the built-in repositories.

dnf install -y epel-release centos-release-opstools

Add the InfluxDB repository by creating a file in /etc/yum.repos.d/influxdb.repo

[influxdb]
name = InfluxDB Repository - RHEL $releasever
baseurl = https://repos.influxdata.com/centos/$releasever/$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key

Finally, add the Grafana repository in file /etc/yum.repos.d/grafana.repo

[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

Keep on reading!