In addition to the previously presented article on the subject Howto do QEMU full virtualization with bridged networking this one shows how to run a QEMU virtual machine with a bridge networking on the host server configured only by using the NetworkManager cli – nmcli.
It is worth mentioning the bridge interface presented in this article is a local bridge device for the server and no Internet addresses or real (or main or Internet-connected) network cards are bound to it. So no MAC addresses of slaved bridged devices will leave the server.
If a network bridge, which includes the Internet (main) server network device is needed, for example, to set real IPs in a virtual machine, there is another article on the bridge networking subject – Replace current interface configuration with a bridge device using nmcli (NetworkManager)
Summary
STEP 1) Add bridge and TUN/TAP device.
[root@srv ~]# nmcli connection add type bridge ifname br0 con-name br0 ipv4.method manual ipv4.addresses "192.168.0.1/24" Connection 'br0' (ad6878c8-1e06-4af8-a81f-1eb39e761df8) successfully added. [root@srv ~]# nmcli connection up br0 Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) [root@srv ~]# nmcli connection add type tun ifname tap0 con-name tap0 mode tap owner 0 ip4 0.0.0.0/24 Connection 'tap0' (dacee2be-a14b-4cf5-83d4-96d072a96725) successfully added. [root@srv ~]# nmcli con add type bridge-slave ifname tap0 master br0 Connection 'bridge-slave-tap0' (66490382-b239-4eb2-ae1d-ee811e39596c) successfully added. [root@srv ~]# nmcli con NAME UUID TYPE DEVICE System eno1 abf4c85b-57cc-4484-4fa9-b4a71689c359 ethernet eno1 br0 ad6878c8-1e06-4af8-a81f-1eb39e761df8 bridge br0 tap0 dacee2be-a14b-4cf5-83d4-96d072a96725 tun tap0 bridge-slave-tap0 66490382-b239-4eb2-ae1d-ee811e39596c ethernet --
First, a bridge device is added with manual IP. If the IP is skipped the bridge interface br0 would have DHCP enabled by default, which may not be the desired.
More detailed information on how to create and add TUN/TAP device with the NetworkManager here – Create bridge and add TUN/TAP device using NetworkManager nmcli under CentOS 8
STEP 2) Install QEMU.
Install the QEMU virtual tools under CentOS 8 Stream. At present, the QEMU version is 6.2, which is pretty new.
[root@srv ~]# dnf install -y qemu-img qemu-kvm-common qemu-kvm-core Last metadata expiration check: 1:03:59 ago on Thu Apr 14 14:18:51 2022. Dependencies resolved. ============================================================================================================== Package Arch Version Repository Size ============================================================================================================== Installing: qemu-img x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 2.1 M qemu-kvm-common x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 1.0 M qemu-kvm-core x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 3.4 M Installing dependencies: device-mapper-multipath-libs x86_64 0.8.4-22.el8 baseos 325 k edk2-ovmf noarch 20220126gitbb1bba3d77-2.el8 appstream 3.6 M ipxe-roms-qemu noarch 20181214-9.git133f4c47.el8 appstream 1.2 M libfdt x86_64 1.6.0-1.el8 appstream 32 k libpmem x86_64 1.6.1-1.el8 appstream 79 k librdmacm x86_64 37.2-1.el8 baseos 78 k seabios-bin noarch 1.15.0-1.module_el8.6.0+1087+b42c8331 appstream 136 k seavgabios-bin noarch 1.15.0-1.module_el8.6.0+1087+b42c8331 appstream 43 k sgabios-bin noarch 1:0.20170427git-3.module_el8.6.0+983+a7505f3f appstream 13 k userspace-rcu x86_64 0.10.1-4.el8 baseos 101 k Transaction Summary ============================================================================================================== Install 13 Packages Total download size: 12 M Installed size: 44 M Downloading Packages: (1/13): libfdt-1.6.0-1.el8.x86_64.rpm 163 kB/s | 32 kB 00:00 (2/13): libpmem-1.6.1-1.el8.x86_64.rpm 606 kB/s | 79 kB 00:00 (3/13): ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch.rpm 2.8 MB/s | 1.2 MB 00:00 (4/13): edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch.rpm 7.0 MB/s | 3.6 MB 00:00 (5/13): qemu-kvm-common-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 (6/13): qemu-kvm-core-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 14 MB/s | 3.4 MB 00:00 (7/13): seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch.rpm 1.2 MB/s | 136 kB 00:00 (8/13): seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch.rpm 741 kB/s | 43 kB 00:00 (9/13): sgabios-bin-0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch.rpm 179 kB/s | 13 kB 00:00 (10/13): qemu-img-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 2.0 MB/s | 2.1 MB 00:01 (11/13): librdmacm-37.2-1.el8.x86_64.rpm 39 kB/s | 78 kB 00:02 (12/13): device-mapper-multipath-libs-0.8.4-22.el8.x86_64.rpm 152 kB/s | 325 kB 00:02 (13/13): userspace-rcu-0.10.1-4.el8.x86_64.rpm 64 kB/s | 101 kB 00:01 -------------------------------------------------------------------------------------------------------------- Total 3.8 MB/s | 12 MB 00:03 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : userspace-rcu-0.10.1-4.el8.x86_64 1/13 Running scriptlet: userspace-rcu-0.10.1-4.el8.x86_64 1/13 Installing : device-mapper-multipath-libs-0.8.4-22.el8.x86_64 2/13 Running scriptlet: device-mapper-multipath-libs-0.8.4-22.el8.x86_64 2/13 Installing : librdmacm-37.2-1.el8.x86_64 3/13 Running scriptlet: librdmacm-37.2-1.el8.x86_64 3/13 Installing : sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch 4/13 Installing : seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 5/13 Installing : seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 6/13 Installing : qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 7/13 Installing : libpmem-1.6.1-1.el8.x86_64 8/13 Running scriptlet: libpmem-1.6.1-1.el8.x86_64 8/13 Installing : libfdt-1.6.0-1.el8.x86_64 9/13 Running scriptlet: libfdt-1.6.0-1.el8.x86_64 9/13 Installing : ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch 10/13 Installing : qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 11/13 Running scriptlet: qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 11/13 Installing : edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch 12/13 Installing : qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 13/13 Running scriptlet: qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 13/13 Verifying : edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch 1/13 Verifying : ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch 2/13 Verifying : libfdt-1.6.0-1.el8.x86_64 3/13 Verifying : libpmem-1.6.1-1.el8.x86_64 4/13 Verifying : qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 5/13 Verifying : qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 6/13 Verifying : qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 7/13 Verifying : seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 8/13 Verifying : seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 9/13 Verifying : sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch 10/13 Verifying : device-mapper-multipath-libs-0.8.4-22.el8.x86_64 11/13 Verifying : librdmacm-37.2-1.el8.x86_64 12/13 Verifying : userspace-rcu-0.10.1-4.el8.x86_64 13/13 Installed: device-mapper-multipath-libs-0.8.4-22.el8.x86_64 edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch libfdt-1.6.0-1.el8.x86_64 libpmem-1.6.1-1.el8.x86_64 librdmacm-37.2-1.el8.x86_64 qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch userspace-rcu-0.10.1-4.el8.x86_64 Complete!
STEP 3) Create QEMU local disk.
The hard disk file for the QEMU virtual machine will be under path /srv/qemu/test. The last argument is the size of the file.
[root@srv ~]# mkdir -p /srv/qemu/test [root@srv ~]# qemu-img create -f qcow2 /srv/qemu/test/test_harddisk.qcow2 100G Formatting '/srv/qemu/test/test_harddisk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=107374182400 lazy_refcounts=off refcount_bits=16
The hard disk is of 100G size provided to the virtual machine, but the actual size varies because qcow2 grows when the data is written.
STEP 4) Run a QEMU virtual server.
Here is an example command to run a QEMU virtual server. First, add masquerade with firewall-cmd to enable NAT, i.e. Internet in the virtual machine:
[root@srv ~]# firewall-cmd --permanent --add-masquerade success [root@srv ~]# firewall-cmd --reload success
Second, run the QEMU virtual machine. The QEMU KVM executable is /usr/libexec/qemu-kvm:
/usr/libexec/qemu-kvm -cpu host,-sse4.1,-sse4.2 -smp 2,maxcpus=8 -daemonize -vnc 192.168.1.10:1 \ -drive file=/srv/qemu/test/test_harddisk.qcow2,index=0,cache=none,aio=threads,if=virtio \ -cdrom /srv/qemu/test/install-amd64-minimal-20220412T191925Z.iso -boot d -m 8192 \ -net nic,model=virtio,macaddr=e6:2b:b3:40:81:0c -net tap,ifname=tap0,script=no,downscript=no \ -monitor telnet:127.0.0.1:5801,server,nowait -writeconfig /srv/qemu/test/test.qcow2.conf
The command above will :
- “-cpu host” – will expose all supported host CPU features (only supported in KVM mode)
- “-smp 4” – sets 4 processors to the virtual machine
- “-daemonize” – start the command in daemon mode
- “-vnc 192.168.1.10:1” – start a VNC server on this IP:PORT = 192.168.1.10:5901, the IP must present on the server or you can use 0.0.0.0:1 for 0.0.0.0:5901, but in every situation limit the access by a firewall
- -drive file=/srv/qemu/test/test_harddisk.qcow2,index=0,cache=none,aio=threads,if=virtio – set the main hard drive of the system
- “-boot d” – boot from the first hard drive
- “-net nic,model=virtio,macaddr=e6:2b:b3:40:81:0c -net tap,ifname=tap0,script=no,downscript=no” – set the network interface using the tap device created by STEP 1).
- “-m 8192” – set virtual RAM size to megs
- “-monitor telnet:127.0.0.1:5801,server,nowait” – set the management console for the this virtual server, you can connect with:
[root@srv ~]# telnet 127.0.0.1 5801 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. QEMU 6.2.0 monitor - type 'help' for more information (qemu) telnet> quit Connection closed.
When quitting the management console you must NOT exit the console with quite/exit or CTRL+d, becuause it will terminate the virtual server, you must disconnect from the console with “CTRL+]” and then quit the telnet shell. With the console you can hot add/remove CPU, RAM, network cards, pci devices, harddrives, start/stop/shutdown/reset the virtual machine and a lot more.
Interesting additional option to consider:
- “-runas qemu” – run under user, you can run the whole virtual machine from a user created especially for it, no need to run it with root, even it is recommended to run it under unprivileged user
* Boot the virtual server from a virtual CD/DVD
Probably the first time booting you might need to boot from an installation disk, this could be done by the following command:
- “-boot c” – First boot device is now CD/DVD. “c” is for CD, “d” is for disk
- “-cdrom /mnt/storage1/disks/isos/CentOS-7-x86_64-NetInstall-1708.iso” – added the installation disk to the virtual machine
A newer QEMU version may need adding “script=no,downscript=no” to the tap0 interface!
/usr/libexec/qemu-kvm -cpu host,-sse4.1,-sse4.2 -smp 2,maxcpus=8 -daemonize -vnc 192.168.1.10:1 \ -drive file=/srv/qemu/test/test_harddisk.qcow2,index=0,cache=none,aio=threads,if=virtio \ -cdrom /srv/qemu/test/install-amd64-minimal-20220412T191925Z.iso -boot c -m 8192 \ -net nic,model=virtio,macaddr=e6:2b:b3:40:81:0c -net tap,ifname=tap0,script=no,downscript=no \ -monitor telnet:127.0.0.1:5801,server,nowait -writeconfig /srv/qemu/test/test.qcow2.conf