Firewalld and how to preserve the original source IP when forwarding to internal IP

Using firewalld and the forwarding options (IP or port forward) might work not as expected if the default setup is left on the system. Consider the simple example:

main menu
Internet <-> router <-> local network

The purpose is to forward a port to a server in the local network, which should be easy enough. Let the forwarding port be 80 and the server should receive the original source IP. To archive this task the system administrator should do the following on the router with firewalld service. Here is one of the simplest methods:

  • When the router’s external IP/interface and the router’s internal IP/interface are in the same firewalld zone. The zone is named “public” in the CentOS world.

The solution uses the masquerade rule added with a rich rule (–add-rich-rule), not the masquerade option of the zone (–add-masquerade).
The default configuration will assign the external interface and the internal interface, which may be a virtual one, in the same firewalld zone such as “public”. When this happens, activating the masquerade option will break the source IP and it will be replaced by the Netfilter with the internal IP address of the router and the internal server will see all incoming connections on the forwarded port as if they were coming from the internal router IP. All different IPs coming to this port will be replaced with the router’s internal IP and forwarded to the server’s internal.

The router’s external IP/interface and the router’s internal IP/interface are in the same firewalld zone.

This solution is demonstrated with a virtual interface – bridge br0, but it may be a network interface. By default, when the bridge is created, it will be added to the default zone, which is “public” in CentOS world. Use –get-active-zones to check the active zones and the assigned interfaces.

[root@srv ~]# firewall-cmd --get-active-zones
  interfaces: eth0 br0
[root@srv ~]# firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: br0 eth0
  services: cockpit dhcpv6-client http https ssh
  ports: 10022/tcp
  forward: yes
  masquerade: no
  rich rules:

If the options forward and masquerade are activated (i.e. yes on the above output) and a forward rule to an internal local IP (some server IP connected to the bridge br0) is introduced to the firewall, the local server will receive all connection attempts to the forwarded port, but the source IP will be overwritten with the gateway IP of the internal (local network). For example, the bridge br0 has IP and the eth0 has Internet IP Forwarding port to a server behind the bridge br0 with IP
Keep on reading!

List all rules and IPs when using firewalld under CentOS

CentOS 7, CentOS 8, CentOS Stream 8, CentOS Stream 9 use firewalld service for the firewall of the machine. Firewalld service is easily controlled by the command-line tool firewall-cmd, which has a relatively simple syntax. It imposes an abstraction layer over the much more complex Linux kernels backends like iptables, ip6tables, arptables, ebtables, ipsetand, and nftables. The Firewalld organizes the firewall rules with the help of policies and zones, but there is a catch – it is not possible to list all the firewall rules to check whether an IP, network, or ethernet interface takes part in some rule or policy.

main menu
nft list ruleset

Yes, it is possible to list all rules of a firewalld zone (firewall-cmd –list-all), but what if there are multiple zones? First, get all the zone names and then enumerate all the zones with a list command to check for an IP. It is not practical and fast.
There is the other trick, searching through the Firewalld configuration files in the directory /etc/firewalld/, which saves all the permanent rules. But what if there are temporary rules, which are not saved in the configuration?
From CentOS 8.2 (RHEL 8.2) firewalld backend defaults to nftables instead of “iptables”. By using the nftables tools we can list all the rules applied in text or JSON format. It is like a snapshot of all the running rules applied by the Linux kernel firewall infrastructure no matter in which zone and policy. There is no such command with the firewall-cmd tool.

CentOS 8.2 (RHEL 8.2) and above including CentOS Stream 9

There is an export command using the nfs command line tool.

nft list ruleset

Keep on reading!

firewalld and podman (or docker) – no internet in the container and could not resolve host

If you happen to use CentOS 8 you have already discovered that Red Hat (i.e. CentOS) switch to podman, which is a fork of docker. So probably the following fix might help to someone, which does not use CentOS 8 or podman. For now, podman and docker are 99.99% the same.
So creating and starting a container is easy and in most cases one command only, but you may stumble on the error your container could not resolve or could not connect to an IP even there is a ping to the IP!
The service in the container may live a happy life without Internet access but just the mapped ports from the outside world. Still, it may happen to need Internet access, let’s say if an update should be performed.
Here is how to fix podman (docker) missing the Internet access in the container:

  • No ping to the outside world. The chances you are missing
    sysctl -w net.ipv4.ip_forward=1

    And do not forget to make it permanent by adding the “net.ipv4.ip_forward=1” to /etc/sysctl.conf (or a file “.conf” in /etc/sysctl.d/).

  • ping to the outside IP of the container is available, but no connection to any service is available! Probably the NAT is not enabled in your podman docker configuration. In the case with firewalld, at least, you must enable the masquerade option of the public zone
    firewall-cmd --zone=public --add-masquerade
    firewall-cmd --permanent --zone=public --add-masquerade

    The second command with “–permanent” is to make the option permanent over reboots.

The error – Could not resolve host (Name or service not known) despite having servers in /etc/resolv.conf and ping to them!

One may think having IPs in /etc/resolv.conf and ping to them in the container should give the container access to the Internet. But the following error occurs:

[root@srv /]# yum install telnet
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
 * base:
 * extras:
 * updates: [Errno 14] curl#6 - "Could not resolve host:; Unknown error"
Trying other mirror. [Errno 14] curl#6 - "Could not resolve host:; Unknown error"
Trying other mirror.

Exiting on user cancel
[root@srv /]# ^C
[root@srv /]# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=56 time=5.05 ms
64 bytes from icmp_seq=2 ttl=56 time=5.06 ms
--- ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 5.050/5.055/5.061/0.071 ms
[root@srv ~]# cat /etc/resolv.conf 
[root@srv /]# ping
ping: Name or service not known

The error 2 – Can’t connect to despite having ping to the IP!

[root@srv /]# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=56 time=9.15 ms
64 bytes from icmp_seq=2 ttl=56 time=9.16 ms
[root@srv2 /]# mysql -h2.2.2.2 -uroot -p
Enter password: 
ERROR 2003 (HY000): Can't connect to MySQL server on '' (113)
[root@srv2 /]#

Despite having ping the MySQL server on and despite the firewall on allows outside connections the container could not connect to it. And testing other services like HTTP, HTTPS, FTP and so on resulted in “unable to connect“, too. Simply because the NAT (aka masquerade is not enabled in the firewall).

Receive multicast packets on CentOS 7 (and other linux distros)

There are so many web pages and blogs post for multicast traffic under linux and how to enable it, but in most of them something always is missing and if you follow them probably you’ll end up with not working setup and you’ll have to search the Internet again – some do need tuning of the linux kernel variables not to drop packets, some need tuning the firewall to allow protocols.
Here we present a real working example of a server under CentOS 7, our server has two network ports:

  1. eno1 – local unicast traffic with local IP
  2. eno2 – multicast traffic

We have multicast TV streams, which we can use through our second network interface and we want to use ffmpeg to encode the video. We have “Multicast Group:port” for every stream, which is like “IP:PORT” and in our case the port is always the same 5000. Here are the steps you need to do if you want to receive these streams.

STEP 1) Set networking and make the configuration permanent.

In CentOS 7 the network of eno2, set a local IP, in fact it does not matter the exact IP, we used local network.



The important lines are highlighted. Configure the network adapter on boot and set
Second and very important add a static route for the multicast traffic for the network interface, which is supposed to have the multicast streams (in our case “eno2”). Use file:

/etc/sysconfig/network-scripts/route-eno2 dev eno2

STEP 2) Kernel variables tuning

[srv@local ~]# for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 0 > "$i";   done
[srv@local ~]# echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

Turn off “Reverse Path Filtering” for all interfaces (rp_filter) and ping for the multicast address to work properly we need to disable icmp_echo_ignore_broadcasts. By default kernel drops these packets so you need this tuning or your application will not receive a single udp packet.
You could save the above two lines in


and do not forget to set proper access rights:

[srv@local ~]# chmod 755 /etc/rc.local

Or you can use


Add the following lines in it:


STEP 3) Allow UDP traffic (and/or IGMP) if you have firewall

Here the port is the “Multicast Group:port” of the IP your application will join (look the example below)

  1. firewalld – the default with CentOS 7:
    [srv@local ~]# firewall-cmd --new-zone=multicast --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-interface=eno2 --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-protocol=igmp --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-protocol=icmp --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-source= --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-source= --permanent
    [srv@local ~]# firewall-cmd --zone=multicast --add-port=5000/udp --permanent
    [srv@local ~]# firewall-cmd --reload

    We make a new zone for the multicast streams and add our interface, the sources and the port we use. We added IGMP, because in some use cases it is needed (in our it could work without IGMP added)

  2. iptables – if you have disabled firewall and still you wanted to have a firewall:
    # allow multicast addresses
    [srv@local ~]# iptables -A INPUT -p udp -d 5000 -j ACCEPT
    [srv@local ~]# iptables -A INPUT -s -j ACCEPT
    [srv@local ~]# iptables -A INPUT -p igmp -d -j ACCEPT
    [srv@local ~]# iptables -A INPUT -p icmp --icmp-type 0 -j ACCEPT

    Probably it is a good idea to see if you have current rules (with “iptables -L -v -n” or even “iptables-save”) and to see if you should use “-A” (above) or “-I” to insert the rules above the DROP rule(s).

* Example with ffmpeg joining to a multicast group

[srv@local ~]# ffmpeg -i 'udp://'
ffmpeg version 2.8.6 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.0 (Gentoo 5.3.0 p1.0, pie-0.6.5)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-march=native -O2 -msse3 -fomit-frame-pointer -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --enable-nonfree --disable-indev=alsa --disable-indev=oss --disable-outdev=alsa --disable-outdev=oss --enable-version3 --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-doc --disable-gnutls --enable-gpl --enable-hardcoded-tables --enable-iconv --disable-lzma --enable-network --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl --disable-vaapi --disable-vdpau --enable-xlib --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --enable-libcaca --disable-openal --disable-opengl --disable-libv4l2 --disable-libpulse --enable-libopencore-amrwb --enable-libopencore-amrnb --disable-libfdk-aac --enable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --enable-libgsm --disable-libmodplug --disable-libopus --disable-libquvi --disable-librtmp --disable-libssh --disable-libschroedinger --disable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-libbs2b --disable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --disable-libass --enable-libfreetype --disable-libsoxr --enable-pthreads --enable-libvo-aacenc --disable-libvo-amrwbenc --enable-libmp3lame --disable-libaacplus --enable-libfaac --disable-libsnappy --enable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --disable-libx265 --enable-libxvid --enable-x11grab --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=host
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[mpeg2video @ 0xf2fa80] Invalid frame dimensions 0x0.
    Last message repeated 10 times
Input #0, mpegts, from 'udp://':
  Duration: N/A, start: 87846.990933, bitrate: 5659 kb/s
  Program 5 
      service_name    : ?TVtest
      service_provider: ?ss
    Stream #0:0[0x33]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 720x576 [SAR 64:45 DAR 16:9], 5467 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x34](bul): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 192 kb/s
At least one output file must be specified

As you can see to join the multicast group and ffmpeg to start encoding you use “udp://” for input stream parameter. Here ffmpeg joins the group and receives packets successfully.

* Generic example to receive UDP multicast stream

You may use the following example in any linux distro like Ubuntu, CentOS 7, Gentoo, OpenSuse and many others to receive muticast streams

[srv@local ~]# #ifconfig or ip - use one of them
[srv@local ~]# #ifconfig
[srv@local ~]# ifconfig eno1 up
[srv@local ~]# route add -net netmask eno2
[srv@local ~]# #or ip
[srv@local ~]# ip addr add dev eno2
[srv@local ~]# ip link set eno2 up
[srv@local ~]# #allow multicast packets to the server
[srv@local ~]# for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 0 > "$i";   done
[srv@local ~]# echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
[srv@local ~]# #if you have firewall use the following for the iptables OR firewalld after that - use only one of them!
[srv@local ~]# iptables -A INPUT -p udp -d 5000 -j ACCEPT
[srv@local ~]# iptables -A INPUT -s -j ACCEPT
[srv@local ~]# iptables -A INPUT -p igmp -d -j ACCEPT
[srv@local ~]# iptables -A INPUT -p icmp --icmp-type 0 -j ACCEPT
[srv@local ~]# #firewalld setup
[srv@local ~]# firewall-cmd --new-zone=multicast --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-interface=eno2 --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-protocol=igmp --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-protocol=icmp --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-source= --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-source= --permanent
[srv@local ~]# firewall-cmd --zone=multicast --add-port=5000/udp --permanent
[srv@local ~]# firewall-cmd --reload