syslog | Any IT here? Help Me!

Rename the hostname in syslog-ng

At present, the syslog-ng (syslog-ng Open Source Edition) does not support to set the hostname of the server in the configuration. syslog-ng uses DNS system to resolve the system name or if it is explicitly switch off (with couple of options) it will use the IP. Sometimes the DNS name of the system may be not valid, for example in containers, or just for better naming purposes it is useful to have just a simple option to set the system’s hostname in the syslog packets.

The easiest and best way to change the system’s hostname in the packets is to use substitution rules with rewrite. In fact, the rewrite rule may operate on soft parts of the macros like MESSAGE, PROGRAM, HOST or user defined macros. The syslog message format and its “fields” could be seen in the RFC5424. It’s worth adding it is possible to replace the whole macro or just part of it.
Here is the configuration to set the system’s hostname:

#substitution rule
rewrite my_host { set("my-server-name", value("HOST")); };

#use the rule before the destination!!!
log { source(src); rewrite(my_host); destination(messages);};

The substitution rule should be used before the destination rule to take effect.

Here is a more complex example to set the system’s hostname only to certain packets:
Keep on reading!

syslog – UDP local to syslog-ng and send remote. Forward syslog to remote server.

After writing an article for the rsyslog daemon about forwarding local UDP logging to a remote server using TCP – UDP local to rsyslog and send remote with TCP and compression this time going to use syslog-ng daemon for those who use it as default in their Linux distribution.
As mentioned in the previous article always use a non-blocking way of writing logs using UDP locally and then transfer (forward) the logs to the centralized log server(s). The example here transfers the web server’s access logs to a remote server. The web server is an Nginx web server.
The goal is to use

UDP for the client program (Nginx in the case) for non-blocking log writes.
TCP between our local machine and the remote syslog server – to be sure not to lose messages on bad connectivity.
local caching for our client machine – not to lose messages if the remote syslog is temporary unreachable.

The configuration and the commands are tested on CentOS 7, CentOS 8, Gentoo and Ubuntu 18 LTS. Check out UDP remote logging here – nginx remote logging to UDP rsyslog server (CentOS 7) to see how to build the server-side part – the syslog server accepting the syslog messages and writing them into files.

STEP 1) Listen for local UDP connections

Configuration file /etc/syslog-ng/syslog-ng.conf

source udp_local {
    network(ip(127.0.0.1) port(514) transport("udp") so_rcvbuf(67108864) log_fetch_limit(1000) max-connections(1000) log-iw-size(1000000));
};

Keep on reading!

syslog – UDP local to rsyslog and send remote with TCP and compression

This article is to show how to log Nginx’s access logs locally using UDP to the local rsyslog daemon, which will send the logs to a remote rsyslog server using TCP and compression. In general, logs could generate a lot of traffic and using UDP over distant locations could result in packet loss respectively logs’ lines loss. The idea here is to log messages locally using UDP (non-blocking way) to a local Syslog server, which will send the stream to a remote central Syslog server using TCP connections to be sure no packets are lost. In addition, we are going to enable local caching (if the remote server is temporary unreachable) and compression between the two Syslog servers.
Our goal is to use

UDP for our client program (Nginx in the case) for non-blocking log writes.
TCP between our local machine and the remote syslog server – to be sure not to lose messages on bad connectivity.
local caching for our client machine – not to lose messages if the remote syslog is temporary unreachable.
compression between the local machine and the remote syslog server.

The configuration and the commands are tested on CentOS 7, CentOS 8 and Ubuntu 18 LTS. Check out UDP remote logging here – nginx remote logging to UDP rsyslog server (CentOS 7).

STEP 1) Configure client’s local rsyslog to accept UDP log messages only if the messages’ tags are “nginx”

Couple of things should be enabled in the local client-size rsyslog daemon:

rsyslog to accept UDP messages. Uncomment or add the following under section “Modules” (probably the first section in the file?) in /etc/rsyslog.conf
```
$ModLoad imudp
$UDPServerRun 514
```
or
```
module(load="imudp")
input(type="imudp" port="514")
```
The first is the old syntax, which is still supported and the second is the new syntax. For simplicity, all of the following configuration will be using the new syntax, because the old one is depricated.

Add a rule to catch the tag containing “nginx” and execute action to forward the messages to the remote server

if ($syslogtag == 'nginx:') then {
action(type="omfwd" target="10.10.10.10" port="10514" protocol="tcp" compression.Mode="single" ZipLevel="9"
queue.filename="forwarding" queue.spoolDirectory="/var/log" queue.size="1000000" queue.type="LinkedList" queue.maxFileSize="1g" queue.SaveOnShutdown="on"
action.resumeRetryCount="-1")
& stop
}

The options are almost self-explanatory, the important ones are there is no retry limit count of reconnecting to the server, there is in-disk cache of maximum 1G if the remote server is unavailable and the compression per message is turned on. More on actions – https://www.rsyslog.com/doc/v8-stable/configuration/actions.html, the forward module – https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html and the queue – https://www.rsyslog.com/doc/v8-stable/rainerscript/queue_parameters.html

And restart the rsyslog:

systemctl restart rsyslog

Keep on reading!