Installing single node Elasticsearch 7.16 and Kibana 7.16 behind nginx web server under CentOS 8

This article will show how to install two big software – Elasticsearch to store information and Kibana to visualize the information under CentOS 8. Elasticsearch is ideal to store big data such as logs from user activities or server logs – one central repository for data, which is structured properly and it could be easily accessed and manipulated with various software.
Kibana is used mainly for visualizing the data stored in the Elasticseach server and manage the Elasticsearch service by the web. ste

Here is a simple example: send the web servers logs in Elasticsearch and visual statistical data with Kibana.

Using the rpm repository for the two software is the best option for installation and in future upgrades.

STEP 1) Install the CentOS 8.

How to install CentOS 8 could be found here – How to do a network installation of CentOS 8 (8.0.1950) – minimal server installation.
Or if a container approach is needed, there is a how to with LXC containerRun LXC CentOS 8 container with bridged network under CentOS 8.

STEP 2) Install the Elasticsearch.

This installation and configuration is for single node server setup.
First, create a rpm repository file /etc/yum.repos.d/elasticsearch.repo and fill it with the Elasticsearch repository information:

[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Then import the Elasticsearch GPG key and install the Elasticsearch software:

[root@loganalyzer ~]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@loganalyzer ~]# dnf install elasticsearch
Last metadata expiration check: 0:00:19 ago on 11.12.2021 (Sat) 12:43:24 UTC.
Dependencies resolved.
==========================================================================================================================================
 Package            Architecture             Version                     Repository                                Size
==========================================================================================================================================
Installing:
 elasticsearch      x86_64                   7.16.0-1                    elasticsearch                             327 M

Transaction Summary
=========================================================================================================================================
Install  1 Package

Total download size: 327 M
Installed size: 526 M
Is this ok [y/N]: y
Downloading Packages:
elasticsearch-7.16.0-x86_64.rpm                                                                                 43 MB/s | 327 MB     00:07    
------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                           43 MB/s | 327 MB     00:07     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                     1/1 
  Running scriptlet: elasticsearch-7.16.0-1.x86_64                                                                                                                                       1/1 
Creating elasticsearch group... OK
Creating elasticsearch user... OK

  Installing       : elasticsearch-7.16.0-1.x86_64                                                                                                                                       1/1 
  Running scriptlet: elasticsearch-7.16.0-1.x86_64                                                                                                                                       1/1 
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

Created elasticsearch keystore in /etc/elasticsearch/elasticsearch.keystore

[/usr/lib/tmpfiles.d/elasticsearch.conf:1] Line references path below legacy directory /var/run/, updating /var/run/elasticsearch → /run/elasticsearch; please update the tmpfiles.d/ drop-in file accordingly.

  Verifying        : elasticsearch-7.16.0-1.x86_64                                                                                                                                       1/1 

Installed:
  elasticsearch-7.16.0-1.x86_64                                                                                                                                                              

Complete!

The configuration files are placed in /etc/elasticsearch/:
Keep on reading!

Elasticsearch failed to set password apm_system error in initial setup

A relatively typical error when installing a single node Elastic Elasticsearch software is when the passwords are set:

[root@loganalyzer elasticsearch]# ./bin/elasticsearch-setup-passwords -v auto

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y



Connection failure to: http://192.168.0.4:9200/_security/user/apm_system/_password?pretty failed: Read timed out


ERROR: Failed to set password for user [apm_system].

Such error may prevent the initial password setting of several important passwords and compromise the Elasticsearch software security model. Even including the

discovery.type: single-node

in the /etc/elasticsearch/elasticsearch.yml would lead to such error. The missing option in the configuration /etc/elasticsearch/elasticsearch.yml is:

discovery.seed_hosts: ["node-1"]

By default, this option is commented out and it should be set on initial installation, though it is not required when starting the elasticsearch node (with no security model enabled)!
This is an array with all the servers’ hostnames in the cluster setup. In single-node mode, this option (discovery.seed_hosts) should be set only to the hostname of the single node like in this case “node-1”. This is the hostname of the server. The user must include the user’s current server hostname, not this example name “node-1”!

Setting the right hostname for discovery.seed_hosts in /etc/elasticsearch/elasticsearch.yml would let the user to set all password with the Elasticsearch tool elasticsearch-setup-passwords

The error may occur in a cluster setup with multiple servers, too, if the hosts are not filled in this option – discovery.seed_hosts.
Here is what to expect when executing elasticsearch-setup-passwords (even with some RED indexes):

[root@loganalyzer ~]# cd /usr/share/elasticsearch/
[root@loganalyzer elasticsearch]# ./bin/elasticsearch-setup-passwords -v auto

Your cluster health is currently RED.
This means that some cluster data is unavailable and your cluster is not fully functional.

It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
It is very likely that the password changes will fail when run against an unhealthy cluster.

Do you want to continue with the password setup process [y/N]y

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y


Changed password for user apm_system
PASSWORD apm_system = judakai2Wai9Saiph8ah

Changed password for user kibana_system
PASSWORD kibana_system = eisiadit3CieG4Requie

Changed password for user kibana
PASSWORD kibana = bi3NohquohLoonaizei1

Changed password for user logstash_system
PASSWORD logstash_system = AhC2kue5eeR4eK1LeeZa

Changed password for user beats_system
PASSWORD beats_system = reeyu8ooj8Eebee5ni2c

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = aeshahx9Ohkoph3rai6a

Changed password for user elastic
PASSWORD elastic = beiPhei4xu5iXailocei

No errors and the password are set successfully.