In addition to the previously presented article on the subject Howto do QEMU full virtualization with MacVTap networking this one shows how to run a QEMU virtual machine with a MAcVTap device in bridge mode on the host server configured only by using the NetworkManager cli – nmcli.
It is worth mentioning the MacVTap is a virtual bridge, which will make the host and the guest device show up directly on the host switch. So when using QEMU, the guest virtualized system will be as if it is connected to the host switch with one limitation – the host and guest cannot communicate with each other. The IPs of the host won’t be reachable from the guest, so NAT (masquerade) between the host and guest is not possible with this setup. Still, if the NAT server is on another server or a real IP is planned for the guest, MacVTap is the right functionality to use with the QEMU guest system.
Summary
- Add MacVTap device in bridge mode with name macvtap0.
- Install QEMU.
- Create QEMU local disk.
- Run a QEMU virtual server.
STEP 1) Add MacVTap device in bridge mode with name macvtap0
[root@srv ~]# nmcli connection add type macvlan dev enp0s3 mode bridge tap yes ifname macvtap0 con-name macvtap0 ip4 0.0.0.0/24 Connection 'macvtap0' (7a5ef04c-ea98-4642-ac5d-4239f715f631) successfully added. [root@srv ~]# nmcli con NAME UUID TYPE DEVICE enp0s3 09497bbf-da59-42b7-a72c-d69369760b36 ethernet enp0s3 macvtap0 7a5ef04c-ea98-4642-ac5d-4239f715f631 macvlan macvtap0
First, create a MacVTap device with the name macvtap0 in bridge mode with the network interface enp0s3 a and a connection with the name macvtap0. The IP is set to manual mode.
More detailed information on how to create and add MacVTap device with the NetworkManager here – Create MacVTap device using NetworkManager nmcli under CentOS 8
STEP 2) Install QEMU.
Install the QEMU virtual tools under CentOS 8 Stream. At present, the QEMU version is 6.2, which is pretty new.
[root@srv ~]# dnf install -y qemu-img qemu-kvm-common qemu-kvm-core Last metadata expiration check: 1:03:59 ago on Thu Apr 14 14:18:51 2022. Dependencies resolved. ============================================================================================================== Package Arch Version Repository Size ============================================================================================================== Installing: qemu-img x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 2.1 M qemu-kvm-common x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 1.0 M qemu-kvm-core x86_64 15:6.2.0-5.module_el8.6.0+1087+b42c8331 appstream 3.4 M Installing dependencies: device-mapper-multipath-libs x86_64 0.8.4-22.el8 baseos 325 k edk2-ovmf noarch 20220126gitbb1bba3d77-2.el8 appstream 3.6 M ipxe-roms-qemu noarch 20181214-9.git133f4c47.el8 appstream 1.2 M libfdt x86_64 1.6.0-1.el8 appstream 32 k libpmem x86_64 1.6.1-1.el8 appstream 79 k librdmacm x86_64 37.2-1.el8 baseos 78 k seabios-bin noarch 1.15.0-1.module_el8.6.0+1087+b42c8331 appstream 136 k seavgabios-bin noarch 1.15.0-1.module_el8.6.0+1087+b42c8331 appstream 43 k sgabios-bin noarch 1:0.20170427git-3.module_el8.6.0+983+a7505f3f appstream 13 k userspace-rcu x86_64 0.10.1-4.el8 baseos 101 k Transaction Summary ============================================================================================================== Install 13 Packages Total download size: 12 M Installed size: 44 M Downloading Packages: (1/13): libfdt-1.6.0-1.el8.x86_64.rpm 163 kB/s | 32 kB 00:00 (2/13): libpmem-1.6.1-1.el8.x86_64.rpm 606 kB/s | 79 kB 00:00 (3/13): ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch.rpm 2.8 MB/s | 1.2 MB 00:00 (4/13): edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch.rpm 7.0 MB/s | 3.6 MB 00:00 (5/13): qemu-kvm-common-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 (6/13): qemu-kvm-core-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 14 MB/s | 3.4 MB 00:00 (7/13): seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch.rpm 1.2 MB/s | 136 kB 00:00 (8/13): seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch.rpm 741 kB/s | 43 kB 00:00 (9/13): sgabios-bin-0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch.rpm 179 kB/s | 13 kB 00:00 (10/13): qemu-img-6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64.rpm 2.0 MB/s | 2.1 MB 00:01 (11/13): librdmacm-37.2-1.el8.x86_64.rpm 39 kB/s | 78 kB 00:02 (12/13): device-mapper-multipath-libs-0.8.4-22.el8.x86_64.rpm 152 kB/s | 325 kB 00:02 (13/13): userspace-rcu-0.10.1-4.el8.x86_64.rpm 64 kB/s | 101 kB 00:01 -------------------------------------------------------------------------------------------------------------- Total 3.8 MB/s | 12 MB 00:03 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : userspace-rcu-0.10.1-4.el8.x86_64 1/13 Running scriptlet: userspace-rcu-0.10.1-4.el8.x86_64 1/13 Installing : device-mapper-multipath-libs-0.8.4-22.el8.x86_64 2/13 Running scriptlet: device-mapper-multipath-libs-0.8.4-22.el8.x86_64 2/13 Installing : librdmacm-37.2-1.el8.x86_64 3/13 Running scriptlet: librdmacm-37.2-1.el8.x86_64 3/13 Installing : sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch 4/13 Installing : seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 5/13 Installing : seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 6/13 Installing : qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 7/13 Installing : libpmem-1.6.1-1.el8.x86_64 8/13 Running scriptlet: libpmem-1.6.1-1.el8.x86_64 8/13 Installing : libfdt-1.6.0-1.el8.x86_64 9/13 Running scriptlet: libfdt-1.6.0-1.el8.x86_64 9/13 Installing : ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch 10/13 Installing : qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 11/13 Running scriptlet: qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 11/13 Installing : edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch 12/13 Installing : qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 13/13 Running scriptlet: qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 13/13 Verifying : edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch 1/13 Verifying : ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch 2/13 Verifying : libfdt-1.6.0-1.el8.x86_64 3/13 Verifying : libpmem-1.6.1-1.el8.x86_64 4/13 Verifying : qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 5/13 Verifying : qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 6/13 Verifying : qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 7/13 Verifying : seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 8/13 Verifying : seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch 9/13 Verifying : sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch 10/13 Verifying : device-mapper-multipath-libs-0.8.4-22.el8.x86_64 11/13 Verifying : librdmacm-37.2-1.el8.x86_64 12/13 Verifying : userspace-rcu-0.10.1-4.el8.x86_64 13/13 Installed: device-mapper-multipath-libs-0.8.4-22.el8.x86_64 edk2-ovmf-20220126gitbb1bba3d77-2.el8.noarch ipxe-roms-qemu-20181214-9.git133f4c47.el8.noarch libfdt-1.6.0-1.el8.x86_64 libpmem-1.6.1-1.el8.x86_64 librdmacm-37.2-1.el8.x86_64 qemu-img-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 qemu-kvm-common-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 qemu-kvm-core-15:6.2.0-5.module_el8.6.0+1087+b42c8331.x86_64 seabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch seavgabios-bin-1.15.0-1.module_el8.6.0+1087+b42c8331.noarch sgabios-bin-1:0.20170427git-3.module_el8.6.0+983+a7505f3f.noarch userspace-rcu-0.10.1-4.el8.x86_64 Complete!
STEP 3) Create QEMU local disk.
The hard disk file for the QEMU virtual machine will be under path /srv/qemu/test. The last argument is the size of the file.
[root@srv ~]# mkdir -p /srv/qemu/test [root@srv ~]# qemu-img create -f qcow2 /srv/qemu/test/test_harddisk.qcow2 100G Formatting '/srv/qemu/test/test_harddisk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=107374182400 lazy_refcounts=off refcount_bits=16
The hard disk is of 100G size provided to the virtual machine, but the actual size varies because qcow2 grows when the data is written.
STEP 4) Run a QEMU virtual server.
Here is an example command to run a QEMU virtual server. First, add masquerade with Run the QEMU virtual machine. The QEMU KVM executable is /usr/libexec/qemu-kvm:
/usr/libexec/qemu-kvm -cpu host -smp 4 -daemonize -vnc :1 \ -cdrom /srv/qemu/test/install-amd64-minimal-20220417T171236Z.iso -boot d \ -drive file=/srv/qemu/test/test_harddisk.qcow2,index=0,cache=none,aio=threads,if=virtio \ -net nic,model=virtio,macaddr=$(cat /sys/class/net/macvtap0/address) \ -net tap,fd=3 3<>/dev/tap$(cat /sys/class/net/macvtap0/ifindex),script=no,downscript=no \ -m 2048 -monitor telnet:127.0.0.1:5801,server,nowait -writeconfig /srv/qemu/test/test.conf
The command above will :
- “-cpu host” – will expose all supported host CPU features (only supported in KVM mode)
- “-smp 4” – sets 4 processors to the virtual machine
- “-daemonize” – start the command in daemon mode
- “-vnc :1” – start a VNC server on this IP:PORT = *:5901. Configure the firewall to allow specific IPs to connect to the QEMU VNC server.
- -drive file=/srv/qemu/test/test_harddisk.qcow2,index=0,cache=none,aio=threads,if=virtio – set the main hard drive of the system
- “-boot d” – boot from the first hard drive
- “-net nic,model=virtio,macaddr=$(cat /sys/class/net/macvtap0/address) -net tap,fd=3 3<>/dev/tap$(cat /sys/class/net/macvtap0/ifindex),script=no,downscript=no” – set the network interface using the MacVTap device created by STEP 1).
- “-m 20148” – set virtual RAM size to megs
- “-monitor telnet:127.0.0.1:5801,server,nowait” – set the management console for the this virtual server, you can connect with:
[root@srv ~]# telnet 127.0.0.1 5801 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. QEMU 6.2.0 monitor - type 'help' for more information (qemu) telnet> quit Connection closed.
When quitting the management console you must NOT exit the console with quite/exit or CTRL+d, becuause it will terminate the virtual server, you must disconnect from the console with “CTRL+]” and then quit the telnet shell. With the console you can hot add/remove CPU, RAM, network cards, pci devices, harddrives, start/stop/shutdown/reset the virtual machine and a lot more.
Interesting additional option to consider:
- “-runas qemu” – run under user, you can run the whole virtual machine from a user created especially for it, no need to run it with root, even it is recommended to run it under unprivileged user
* Boot the virtual server from a virtual CD/DVD
Probably the first time booting you might need to boot from an installation disk, this could be done by the following command:
- “-boot c” – First boot device is now CD/DVD. “c” is for CD, “d” is for disk
- “-cdrom /srv/qemu/test/CentOS-7-x86_64-NetInstall-1708.iso” – added the installation disk to the virtual machine
* Firewalld configuration to allow VNC connections.
An important firewall configuration to allow connection to specific IP(s) from the Internet:
[root@srv ~]# firewall-cmd --permanent --zone=FedoraServer --add-rich-rule="rule family="ipv4" source address="192.168.0.10" port protocol="tcp" port="5901" accept" success [root@srv ~]# firewall-cmd --reload success
The only IP, which can access the QEMU VNC is 192.168.0.10.