This article shows how to make a graph showing a Linux machine’s processes states. This plugin could gather the number of the processes grouped by their state or metadata per the selected process defined in the configuration (metadata includes process state, size of the resident segment size (RSS), system/user time used, and so on.). The purpose of this article is to make a graph with all the processes grouped by their state. Graphs per process data are not included here.
The Linux machine is using collectd to gather the processes statistics and send them to the time series back-end – InfluxDB. Grafana is used to visualize the data stored in the time series back-end InfluxDB and organize the graphs in panels and dashboards. Check out the previous articles on the subject to install and configure such software to collect, store and visualize data – Monitor and analyze with Grafana, influxdb 1.8 and collectd under CentOS Stream 9, Monitor and analyze with Grafana, influxdb 1.8 and collectd under Ubuntu 22.04 LTS and Create graph for Linux CPU usage using Grafana, InfluxDB and collectd
The collectd daemon is used to gather data on the Linux system and to send it to the back-end InfluxDB.
The InfluxQL queries for the Linux Processes plugin are grouped by states
The queries are Grafana generated.
SELECT mean("value") FROM "processes_value" WHERE ("host" = 'srv' AND "type" = 'ps_state') AND time >= now() - 6h and time <= now() GROUP BY time(20s), "type_instance" fill(null)
The mean function will compute the mean if there is more than one value in the database for every 15 minutes (this is the “group by” logic here). Probably the most accurate query might be to use the last() function instead of mean() and to group by time(1s). As mentioned above, the collectd interval is 10 seconds. The function non_negative_derivative is needed to compute the difference between the values.
SCREENSHOT 1) Create a new dashboard, which will contain the Processes statesgraph.
SCREENSHOT 2) Add a new panel in the new dashboard, which will contain the Processes statesgraph.
SCREENSHOT 3) Change the “Data Source” to the collectd (InfluxDB) database and ensure on the right top the graph type is “Time series”.
SCREENSHOT 4) Choose the processes_value from the measurement drop-down list.
There are all measurements in the drop-down list in the database collectd.
SCREENSHOT 5) Select the tag name “host” to limit the query for a selected hostname.
A tag is a key-value pair, which represents the metadata of a measurement record. For example, a measurement record consists of the actual measurement value and some metadata for it such as which did the measurement and where. The server hostname “srv” is the tag value and the tag key is the “host” name of the tag.
SCREENSHOT 6) Select the tag value “srv”.
This setup has only one server, so no other servers’ hostnames are shown.
SCREENSHOT 7) Select the type of measurement.
Yet another measurement metadata. The type of measurement is ps_state, i.e. the processes states.
SCREENSHOT 8) Select ps_state for the tag value to draw processes states in the graph.
There is also another valid value fork_rate, which counts the forks occurrence in the system.
SCREENSHOT 9) Add a group by tag_instance to split the different tags in the graph.
The tag_instance shows a process state, which is one of the follwoing: blocked, paging, running, sleeping, stopped and zombies.
SCREENSHOT 10) To give pretty names to the tags in the graph’s legend add to the ALIAS a variable $tag_[tag_key] and in this case, the tag key is type_instance, so the variable will be $tag_type_instance.
There is an important variable “$__interval“, which may be edited and set to the rate of the original data (if applicable, not really for the processes states) or left as is to be computed each time based on the selected time frame of the graph (6 hours for this example).