nginx remote logging to UDP rsyslog server (CentOS 7)

This article will present to you all the configuration needed to remotely save access logs of an Nginx web server. All the configuration from the client and server sides is included. The client and the server use CentOS 7 Linux distribution and the configuration could be used under different Linux distribution. Probably only Selinux rules are kind of specific to the CentOS 7 and the firewalld rules are specific for those who use it as a firewall replacing the iptables. Here is the summary of what to expect:

• Client-side – nginx configuration
• Server-side – rsyslog configuration to accept UDP connections
• Server-side – selinux and firewall configuration

The JSON formatted logs may be sent to a Elasticsearch server, for example. Here is how to do it – send access logs in json to Elasticsearch using rsyslog

STEP 1) Client-side – the Nginx configuration.

Nginx configuration is pretty simple just a single line with the log template and the IP (and port if not default 514) of the rsyslog server. For the record, this is the official documentation https://nginx.org/en/docs/syslog.html. In addition it worth mentioning there could be multiple access_log directives in a single section to log simultaneously on different targets (and the templates may be different or the same). So you can set the access log output of a section locally and remotely.
Nginx configuration (probably /etc/nginx/nginx.conf or whatever is the organization of your Nginx configuration files.)

server {
     .....
     access_log      /var/log/nginx/example.com_access.log main;
     access_log      syslog:server=10.10.10.2:514,facility=local7,tag=nginx,severity=info main3;
     .....
}

The “main” and “main3” are just names of the logging templates defined earlier (you may check rsyslog remote logging – prevent local messages to appear to see an interesting Nginx logging template).
The error log also could be remotely logged:

error_log syslog:server=10.10.10.3 debug;

STEP 2) Server-side – rsyslog configuration to accept UDP connections.

Of course, if you have not installed the rsyslog it’s high time you installed it with (for CentOS 7):

yum install -y rsyslog

To enable rsyslog to listen for UDP connections your rsyslog configuration file (/etc/rsyslog.conf) must include the following:

$ModLoad imudp
$UDPServerRun 514

Most of the Linux distributions have these two lines commented so you just need to uncomment them by removing the “#” from the beginning of the lines. If the lines are missing just add them under section “MODULES” (it should be near the first lines of the rsyslog configuration file).
Change the 514 with the number you like for the UDP listening port.
Write the client’s incoming lines of information to a different location and prevent merging with the local log messages – rsyslog remote logging – prevent local messages to appear. Include as a first rule under the rules’ section starting with “RULES” of the rsyslog configuration file (/etc/rsyslog.conf):

# Remote logging
$template HostIPtemp,"/mnt/logging/%FROMHOST-IP%.log"
if ($fromhost-ip != "127.0.0.1" ) then ?HostIPtemp
& stop

Logs only of remote hosts are going to be saved under /mnt/logging/.log.

STEP 3) Server-side SELinux and firewall configuration

First, if you change the default save path of the logging like in our example above and the SELinux is in Enforcing mode you need to add the new path to the SELinux:

semanage fcontext -a -t var_log_t '/mnt/logging(/.*)?'
restorecon -Rv /mnt/logging/

If you use the default path in “/var/log” no need to execute the above two commands.

Second, the firewall. There is no authentication or other security so you must use a firewall to limit the connects to the listening UDP port! CentOS 7 and many modern Linux distributions use the firewalld service. Allow only the IPs, which are going to log lines in the server (NO generic allow to all IPs!!!):

firewall-cmd --permanent --zone=public --add-rich-rule="rule family="ipv4" source address="10.10.10.10" port protocol="udp" port="514" accept"
firewall-cmd --zone=public --add-rich-rule="rule family="ipv4" source address="10.10.10.10" port protocol="udp" port="514" accept"

The rules are the same except the first one is with the argument “–permanent” to use it over reboots (or reloads of the firewalld). The second line enables the rule immediately in the system without the need for reloading the firewalld rules.
If you still use iptables you may use something of the kind:

iptables -I INPUT -s 10.10.10.10 -m udp -p udp --dport 514 -j ACCEPT
iptables -A INPUT -m udp -p udp --dport 514 -j DROP

Replace the 10.10.10.10 with the IP of your choice.

And this is all you need to remotely log the Nginx access log. After starting the rsyslog service you are going to have the access log lines saved under “/mnt/logging/.log”:

[root@srv ~]# ls -altr /mnt/logging/
total 2792
drwxr-xr-x. 7 root root    4096 10 Sep 10,05 ..
drwxr-xr-x. 2 root root    4096 10 Sep 12,17 .
-rw-------. 1 root root 1236789 10 Sep 12,17 10.10.10.50.log
-rw-------. 1 root root 5123544 10 Sep 12,17 10.10.10.51.log
-rw-------. 1 root root 3855671 10 Sep 12,17 10.10.10.52.log
-rw-------. 1 root root 4491485 10 Sep 12,17 10.10.10.53.log