CaffeineAddict
Well-Known Member
---
What kind of virtual machine?
A special low level setup VM for maximum security and performance that Linux and open source has to offer:
QEMU/KVM + Open vSwitch + nftables + software TPM + secure boot
What's the objective?
- What is KVM?
TL;DR (Hypervisor)
More info on the links below:
- What is QEMU?
TL;DR (Virtual Machine)
More info on the links below:
- What is Open vSwitch?
TL;DR (Virtual Switch)
More info on the link below:
- What is nftables?
TL;DR (Firewall)
More info on the links below:
1. Hardware configuration
Hardware needs to be configured for virtualization in UEFI/BIOS as a first step.
1.1 Virtualization for Intel based systems
UEFI/BIOS settings that control Intel virtualization:
2. Installation of software
NOTE: Names of those packages below are specific to Debian and derivatives, for other distros please find their correct names.
3. User account configuration
QEMU VM works in the context of
For this to take effect log out then wait (by default) 10 seconds before loging back in.
Verify with
4. Configure Open vSwitch
Make sure
If not start and enable it:
4.1 Configure Open vSwitch bridge for virtual network
Verify status of present virtual networks if any.
If there are unwanted networks see point 14. for how to remove them.
This
Copy code below and paste it into a file called
We want to generate a new
First install
Generate a new UUID by running the command below:
Copy
Add network from XML file:
NOTE: Replace
Add virtual switch:
NOTE: Creates transient network if network isn't defined first.
Validate it was created correctly:
Start ovs network and set to auto start at boot:
Verify status:
Verify ovs creates a new interface in your virtual switch
4.2 Permanently configure openvswitch IP configuration
NOTE: We need to use
Create a new
Insert code below into nano editor in the terminal:
Save nano editor with
4.3 Replace networking service
So far we have virtual network set up and we have
We'll use
However if you're configuring this on a server machine then you likely use
Verify
Backup previous config:
Stop networking service:
Start
If you're using WI-FI to connect to internet first restart
Restart
Configured NIC
At this point we have virtual switch operational, it will be used by guest systems to connect to internet.
5. Adding new UEFI virtual machine with secure boot and TPM in virt-manager
First open up QEMU VM manager, you can find it in your DE (Desktop Environment) somewhere in applications menu, if not run
Create a new VM by choosing
Next choose option
In the window that follows uncheck checkbox that says
To get that option just start typing "Generic Linux..." into the box and a pop up window will appear...
In same window click on
NOTE: Adding a pool is required to avoid FS permission issues when reading your ISO from disk.
When you click on
then click on
When you click
Next click
In the next window configure amount of CPU cores to use and amount of RAM to dedicate to VM. (screenshot omited)
Click
Then follows the window prior final one:
In the next window few adjustments are required as follows, first set boot option to
Optionally in the same window configure virtual TPM, choose
Next in same window under
Optionally set this as default for new VM's in
Finally click on
At this point it is possible to get the following error:
If an ISO is located on secondary drive mounted at
if it's owned by
user can read and exectue (search) the directory and thus open the ISO.
For instance (where USER is user directory at which secondary disk is mounted):
What follows next is OS installation which depends on what OS you're installing so we won't cover it here, please consult OS install instructions for further steps.
When that's done proceed with next steps.
6. Enable packet forwarding for IPv4 and IPv6
Packets that are not addressed to local system (host) will be seen from the forward hook...
Read values from
Apply changes (
Check IP forwarding was enabled (output must be 1 for both)
7. Configure nftables firewall for routing
First we ensure
Note that you shouldn't have other firewalls running in same time, only one should be active.
Tested
However this is bare minimum with some debug rules and comments to help understand what it does, it doesn't do any detailed filtering so you should customize it if you want finer control.
To use it for IPv6 simply duplicate rules for IPv6 or use
Copy all code and save as
Next make it executable:
And run it with:
8. Connecting to internet from guest system
Based on configuration so far virtual switch address is
Network address is
Here is sample
This guest above uses
IP configuration is static.
Dynamic configuration is not covered in this guide but is definitely possible by configuring DHCP server on host that listens on our virtual switch.
DHCP server can also provide DNS and other services, however firewall should be updated with additional rules to make traffic more restricted (there are rules for DHCP included but not for local DNS)
NOTE: When ever you modify network configuration in guest system it is mandatory to restart
Therefore when done adjusting guest network run in guest system:
9. Useful virt-manager settings
Allow adjusting screen resolution in
Set this by default for new VM's in:
10. Guest services
To be able to restart, shutdown etc. guest system from
and share clipboard etc.
Drag/drop files (but not directories) works from host to guest only, copy/paste does not work both ways.
Sharing clipboard between host and guest works both ways.
Regardless of guest system, in
10.1 Guest services with Linux guest
In Linux guest system...
NOTE:
If not, it can be fixed with
10.2 Guest services with Windows guest
In Windows guest download and install spice-guest-tools
Start
Continued in next post...
What kind of virtual machine?
A special low level setup VM for maximum security and performance that Linux and open source has to offer:
QEMU/KVM + Open vSwitch + nftables + software TPM + secure boot
What's the objective?
QEMU/KVMis natural choice for virtualization on Linux.Open vSwitchwill let us share internet connection from host to guests, the host will act as router to guests while able to use internet for itself as well.nftableswill let us set up specialized firewall, minimally configured that handles routing and protects both the host and guests.- Software
TPMis optional, the VM will use emulated TPM independent of discrete TPM on host (host doesn't need to have TPM). - Secure boot is optional, the VM will use secure boot.
- What is KVM?
TL;DR (Hypervisor)
Kernel-based Virtual Machine (KVM) is a free and open-source virtualization module in the Linux kernel that allows the kernel to function as a hypervisor
More info on the links below:
- What is QEMU?
TL;DR (Virtual Machine)
It provides a variety of hardware and device models for the virtual machine, enabling it to run different guest operating systems.
QEMU can be used with a Kernel-based Virtual Machine (KVM) to emulate hardware at near-native speeds.
More info on the links below:
- What is Open vSwitch?
TL;DR (Virtual Switch)
Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license.
It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols
More info on the link below:
- What is nftables?
TL;DR (Firewall)
nftables is the successor of iptables, it allows for much more flexible, scalable and performance packet classification.
More info on the links below:
1. Hardware configuration
Hardware needs to be configured for virtualization in UEFI/BIOS as a first step.
1.1 Virtualization for Intel based systems
UEFI/BIOS settings that control Intel virtualization:
VT-xis the actual hardware virtualization assistance - necessary to enable virtualization.VT-dAllows you direct passthrough of devices, such as PCI devices - not needed and not used in this guide.
2. Installation of software
NOTE: Names of those packages below are specific to Debian and derivatives, for other distros please find their correct names.
Bash:
# QEMU GUI
sudo apt install virt-manager
# QEMU VM for x86 arch
sudo apt install qemu-system-x86
# UEFI support for Virtual Machines
sudo apt install ovmf
# Software emulator for the Trusted Platform Module (TPM)
sudo apt install swtpm swtpm-tools
# Open vSwitch (virtual switch)
sudo apt install openvswitch-switch
# Required to mount guest file system on host to enable coping files using CLI
sudo apt install libguestfs-tools
# Required for shared directory between host and guest system
sudo apt install virtiofsd
# Required for multiple guest services such as shared clipboard and screen resize
sudo apt install spice-vdagent
3. User account configuration
QEMU VM works in the context of
libvirt user group, in order for you (your user account) to have privileges to work with VM's you need to add yourself to libvirt group:
Bash:
sudo adduser $(id -un) libvirt
For this to take effect log out then wait (by default) 10 seconds before loging back in.
Verify with
groups command you're added to libvirt group, if not wait longer after logout or reboot system and try again.4. Configure Open vSwitch
Make sure
openvswitch-switch service is both enabled and started:
Bash:
sudo systemctl status openvswitch-switch
If not start and enable it:
Bash:
sudo systemctl start openvswitch-switch
sudo systemctl enable openvswitch-switch
4.1 Configure Open vSwitch bridge for virtual network
Verify status of present virtual networks if any.
If there are unwanted networks see point 14. for how to remove them.
Bash:
virsh net-list --all
sudo virsh net-list --all
This
XML code below is virtual network definition we need to configure:Copy code below and paste it into a file called
ovs-network-bridge.xml, place this file anywhere you want, it's only temporary, you can delete it when we're done.
XML:
<network>
<name>ovs</name>
<uuid>4f3296a9-e179-44e5-9692-c5353f0d4413</uuid>
<forward mode="bridge"></forward>
<bridge name="ovsbr0"></bridge>
<virtualport type="openvswitch"></virtualport>
</network>
We want to generate a new
UUID and paste it into ovs-network-bridge.xml config file shown above.First install
uuid-runtime package:
Bash:
sudo apt install uuid-runtime
Generate a new UUID by running the command below:
Bash:
uuidgen
Copy
uuidgen command output and replace UUID between <uuid></uuid> tags in ovs-network-bridge.xml and then save the file.Add network from XML file:
NOTE: Replace
./ovs-network-bridge.xml below with full path to where you stored your file.
Bash:
sudo virsh net-define ./ovs-network-bridge.xml
Add virtual switch:
NOTE: Creates transient network if network isn't defined first.
Bash:
sudo ovs-vsctl add-br ovsbr0
Validate it was created correctly:
Bash:
sudo ovs-vsctl show
ip link show
Start ovs network and set to auto start at boot:
Bash:
sudo virsh net-start ovs-bridge
sudo virsh net-autostart ovs-bridge
Verify status:
Bash:
sudo virsh net-list --all
Verify ovs creates a new interface in your virtual switch
Bash:
sudo ovs-vsctl show
4.2 Permanently configure openvswitch IP configuration
NOTE: We need to use
systemd-networkd for this, using NetworkManager is not well supported (there is "ovs plugin" but it won't work well so don't bother).Create a new
systemd-networkd configuration for our switch:
Bash:
sudo nano /etc/systemd/network/ovsbr0.network
Insert code below into nano editor in the terminal:
INI:
[Match]
Name=ovsbr0
Type=ether
[Network]
Description=openvswitch
[Network]
Address=172.21.11.1/24
Broadcast=true
Save nano editor with
CTRL+O and close with CTRL+X4.3 Replace networking service
So far we have virtual network set up and we have
systemd-networkd configuration for our vSwitch but it's inactive, it is also necessary to stop networking service if it's running and replace it with systemd-networkd.We'll use
systemd-networkd only for VM's, for regular internet from host we'll be using NetworkManager as usual.However if you're configuring this on a server machine then you likely use
systemd-networkd for all networking, if that's true then this clause is not needed except for restarting systemd-networkd.Verify
networking service is running, this is for informational purpose to be aware of:
Bash:
sudo systemctl status networking
Backup previous config:
Bash:
sudo mv /etc/network/interfaces /etc/network/interfaces.bak
Stop networking service:
Bash:
sudo systemctl stop networking
sudo systemctl disable networking
Start
systemd-networkd:
Bash:
sudo systemctl start systemd-networkd
sudo systemctl enable systemd-networkd
If you're using WI-FI to connect to internet first restart
wpa_supplicant:
Bash:
sudo systemctl restart wpa_supplicant
Restart
NetworkManager:
Bash:
sudo systemctl restart NetworkManager
Configured NIC
ovsbr0 must be reported as "configured" under SETUP column below, to verify run:
Bash:
networkctl list
At this point we have virtual switch operational, it will be used by guest systems to connect to internet.
5. Adding new UEFI virtual machine with secure boot and TPM in virt-manager
First open up QEMU VM manager, you can find it in your DE (Desktop Environment) somewhere in applications menu, if not run
virt-manager in terminal to open it.Create a new VM by choosing
File -> New Virtual Machine menu.Next choose option
Local install media (ISO image or CDROM) and click next, see screenshot:In the window that follows uncheck checkbox that says
Automatically detect from the installation media / source and instead choose Generic Linux 2024 (choose the latest year, currently that option is 2024, but you may have later year).To get that option just start typing "Generic Linux..." into the box and a pop up window will appear...
In same window click on
Browse... button to open ISO search window, then click on + button as shown below to add a new pool:NOTE: Adding a pool is required to avoid FS permission issues when reading your ISO from disk.
When you click on
+ button a new window will pop up, here you name your pool under Name: however you want,then click on
Browse button and locate directory where you have your ISO on disk, and finally click Finish, the choosen directory with ISO will be your new ISO pool.When you click
Finish select your pool, then select an ISO from the pool and finally click on Choose Volume:Next click
Forward to continue with the choose ISO:In the next window configure amount of CPU cores to use and amount of RAM to dedicate to VM. (screenshot omited)
Click
Forward and in the next window choose amount of storage to dedicate to VM.(screenshot omited)Then follows the window prior final one:
- Under
Name:name your VM. - Make sure you check the
Customize configuration before install. - Expand
Network selectiondrop down and make sure to selectovs-bridge ... (OpenVSwitch)that we created in previous steps. - Click
Finishto continue.
In the next window few adjustments are required as follows, first set boot option to
SATA CDROM 1 to be first option (move it up) which is the virtual CDROM where our ISO is loaded:Optionally in the same window configure virtual TPM, choose
Emulated from dropdown:Next in same window under
Overview choose Q35 as Chipset and OVMF_CODE_4M.secboot.fd as firmware (UEFI) for secure boot.virt-manager will make use of /usr/share/OVMF/ files and will store virtual disks in /var/lib/libvirt/imagesOptionally set this as default for new VM's in
virt-manager -> Edit -> Preferences -> New VMFinally click on
Begin Installation at the top of the window to start installing OS from ISO.At this point it is possible to get the following error:
The solution for this is as follows:The emulator may not have search permissions for the path
If an ISO is located on secondary drive mounted at
/media/USER/MOUNT_POINT then USER directoryif it's owned by
root must have read and execute permissions for others so that libvirt-qemuuser can read and exectue (search) the directory and thus open the ISO.
For instance (where USER is user directory at which secondary disk is mounted):
Bash:
sudo chmod o+rx /media/USER
What follows next is OS installation which depends on what OS you're installing so we won't cover it here, please consult OS install instructions for further steps.
When that's done proceed with next steps.
6. Enable packet forwarding for IPv4 and IPv6
Packets that are not addressed to local system (host) will be seen from the forward hook...
Bash:
echo "net.ipv4.ip_forward=1" | sudo tee /etc/sysctl.d/ipv4_forward.conf > /dev/null
echo "net.ipv6.conf.all.forwarding=1" | sudo tee /etc/sysctl.d/ipv6_forward.conf > /dev/null
Read values from
/etc/sysctl.conf to confirm above lines were set:
Bash:
sudo sysctl --load --system
Apply changes (
systemd-sysctl (procps) gathers information about processes via the /proc directory)
Bash:
sudo systemctl restart systemd-sysctl
Check IP forwarding was enabled (output must be 1 for both)
Bash:
sudo sysctl --values net.ipv4.ip_forward
sudo sysctl --values net.ipv6.conf.all.forwarding
7. Configure nftables firewall for routing
First we ensure
nftables firewall is installed and enabled:
Bash:
sudo apt install nftables
sudo systemctl start nftables
sudo systemctl enable nftables
Note that you shouldn't have other firewalls running in same time, only one should be active.
Tested
nftables script to run on host is below, it enables IPv4 routing from/to WAN/LAN for guest systems while maintaining connectivity for host. (it also enables traffic between guests)However this is bare minimum with some debug rules and comments to help understand what it does, it doesn't do any detailed filtering so you should customize it if you want finer control.
To use it for IPv6 simply duplicate rules for IPv6 or use
inet instead of ip, see link to nftables wiki at the top for more info.Copy all code and save as
firewall.nft.Next make it executable:
Bash:
chmod +x ./firewall.nft
And run it with:
Bash:
sudo ./firewall.nft
Bash:
#!/usr/sbin/nft -f
flush ruleset
define virtual_nic = { ovsbr0 }
define virt_gateway_addr_4 = { 172.21.11.1 }
define virt_network_addr_4 = { 172.21.11.0/24 }
#
# IPv4 tables
#
add table filter_4
add table nat_4
#
# IPv4 base chains
#
# input: sees incoming packets that are addressed to and have now been routed to the local system and processes running there
add chain filter_4 input_4 {
# filter = 0
type filter hook input priority filter; policy drop;
}
# forward: sees incoming packets that are not addressed to the local system
add chain filter_4 forward_4 {
# filter = 0
type filter hook forward priority filter; policy drop;
}
# output: sees packets that originated from processes in the local machine
add chain filter_4 output_4 {
# filter = 0
type filter hook output priority filter; policy drop;
}
# postrouting: sees all packets after routing, just before they leave the local system (SNAT)
add chain nat_4 postrouting_nat_4 {
# srcnat = 100
type nat hook postrouting priority srcnat; policy accept;
}
#
# IPv4 regular chains
#
# Virtual IPv4 regular chains
add chain filter_4 virtual_in_4 {
comment "Input virtual traffic"
}
add chain filter_4 virtual_out_4 {
comment "Output virtual traffic"
}
# Virtual IPv4 forward regular chains
add chain filter_4 virtual_fwi_4 {
comment "Forward in virtual IPv4 traffic"
}
add chain filter_4 virtual_fwo_4 {
comment "Forward out virtual IPv4 traffic"
}
add chain filter_4 virtual_fwx_4 {
comment "Forward x virtual IPv4 traffic"
}
# Virtual IPv4 NAT regular Chains
add chain nat_4 virtual_nat_4 {
comment "Postrouting SNAT virtual"
}
#
# Base chain rules
#
# IPv4 Loopback rules (stateless)
add rule filter_4 input_4 meta iiftype loopback accept
add rule filter_4 output_4 meta oiftype loopback accept
# ICMPv4 traffic (stateless)
add rule filter_4 input_4 ip protocol icmp accept
add rule filter_4 output_4 ip protocol icmp accept
# Multicast IPv4 traffic (stateless)
add rule filter_4 input_4 fib daddr type multicast accept
add rule filter_4 output_4 fib daddr type multicast accept
# Broadcast IPv4 traffic (stateless)
add rule filter_4 input_4 fib daddr type broadcast accept
add rule filter_4 output_4 fib daddr type broadcast accept
# Established IPv4 traffic
add rule filter_4 input_4 ct state established accept
add rule filter_4 output_4 ct state established accept
# Related IPv4 traffic
add rule filter_4 input_4 ct state related drop
add rule filter_4 output_4 ct state related drop
# New IPv4 traffic
add rule filter_4 input_4 ct state new drop
add rule filter_4 output_4 ct state new accept
# Invalid IPv4 traffic
add rule filter_4 input_4 ct state invalid log flags all prefix "DROP invalid input_4: " drop
add rule filter_4 output_4 ct state invalid log flags all prefix "DROP invalid output_4: " drop
# Untracked IPv4 traffic
add rule filter_4 input_4 ct state untracked log flags all prefix "DROP untracked input_4: " drop
add rule filter_4 output_4 ct state untracked log flags all prefix "DROP untracked output_4: " drop
# Unprocessed IPv4 traffic
add rule filter_4 input_4 log flags ether prefix "DROP unprocessed input_4: "
add rule filter_4 output_4 log prefix "DROP unprocessed output_4: "
# Virtual IPv4 traffic
add rule filter_4 input_4 meta iifname $virtual_nic goto virtual_in_4
add rule filter_4 output_4 meta oifname $virtual_nic goto virtual_out_4
add rule filter_4 forward_4 meta oifname $virtual_nic goto virtual_fwi_4
add rule filter_4 forward_4 meta iifname $virtual_nic goto virtual_fwo_4
add rule filter_4 forward_4 meta iifname $virtual_nic meta oifname $virtual_nic goto virtual_fwx_4
# Postrouting NAT (SNAT)
add rule nat_4 postrouting_nat_4 meta iifname $virtual_nic jump virtual_nat_4
#
# input/virtual.nft
#
# Virtual subnet input IPv4 traffic
# Subnetwork -> vswitch -> LAN (subnetwork -> vswitch -> WAN in forward chain)
#
# Established traffic
add rule filter_4 virtual_in_4 ct state established ip saddr $virt_network_addr_4 accept
# Related traffic
add rule filter_4 virtual_in_4 ct state related ip saddr $virt_network_addr_4 log prefix "DROP related virtual_in_4: " drop
# DHCP server REQUEST (address renewal) (client 68, server 67)
add rule filter_4 virtual_in_4 ct state new ip saddr $virt_network_addr_4 udp sport 68 udp dport 67 ip daddr $virt_gateway_addr_4 accept
# New traffic
add rule filter_4 virtual_in_4 ct state new ip saddr $virt_network_addr_4 log prefix "DROP new virtual_in_4: " accept
# Invalid traffic
add rule filter_4 virtual_in_4 ct state invalid ip saddr $virt_network_addr_4 log flags all prefix "DROP invalid virtual_in_4: " drop
# Untracked traffic
add rule filter_4 virtual_in_4 ct state untracked ip saddr $virt_network_addr_4 log flags all prefix "DROP untracked virtual_in_4: " drop
#
# Default chain action
#
add rule filter_4 virtual_in_4 log flags ether prefix "DROP default virtual_in_4: " drop
#
# output/virtual.nft
#
#
# Virtual subnet output IPv4 traffic
# LAN -> vswitch -> subnetwork (WAN -> vswitch -> subnetwork in forward chain)
#
# Established traffic
add rule filter_4 virtual_out_4 ct state established ip daddr $virt_network_addr_4 accept
# Related traffic
add rule filter_4 virtual_out_4 ct state related ip daddr $virt_network_addr_4 log flags skuid prefix "DROP related virtual_out_4: " drop
# DHCP server ACKNOWLEDGE (address renewal) (server 67, client 68)
add rule filter_4 virtual_out_4 ip saddr $virt_gateway_addr_4 udp sport 67 udp dport 68 ip daddr $virt_network_addr_4 accept
# DHCP server OFFER/ACKNOWLEDGE (server 67, client 68)
add rule filter_4 virtual_out_4 udp sport 67 udp dport 68 accept
# New traffic
add rule filter_4 virtual_out_4 ct state new ip daddr $virt_network_addr_4 log flags skuid prefix "DROP new virtual_out_4: " accept
# Invalid traffic
add rule filter_4 virtual_out_4 ct state invalid ip daddr $virt_network_addr_4 log flags all prefix "DROP invalid virtual_out_4: " drop
# Untracked traffic
add rule filter_4 virtual_out_4 ct state untracked ip daddr $virt_network_addr_4 log flags all prefix "DROP untracked virtual_out_4: " drop
#
# Default chain action
#
add rule filter_4 virtual_out_4 log flags skuid prefix "DROP default virtual_out_4: " drop
#
# forward/forward_in.nft
#
# Subnetwork input forward IPv4 traffic (oifname)
# WAN -> vswitch > subnetwork
#
# Established traffic
add rule filter_4 virtual_fwi_4 ct state established ip daddr $virt_network_addr_4 accept
# Related traffic
add rule filter_4 virtual_fwi_4 ct state related ip daddr $virt_network_addr_4 log prefix "DROP related virtual_fwi_4: " drop
# New traffic
add rule filter_4 virtual_fwi_4 ct state new log prefix "DROP new virtual_fwi_4: " drop
# Invalid traffic
add rule filter_4 virtual_fwi_4 ct state invalid log flags all prefix "DROP invalid virtual_fwi_4: " drop
# Untracked traffic
add rule filter_4 virtual_fwi_4 ct state untracked log flags all prefix "DROP untracked virtual_fwi_4: " drop
#
# Default chain action
#
add rule filter_4 virtual_fwi_4 log flags ether prefix "DROP default virtual_fwi_4: " drop
#
# forward/forward_out.nft
#
# Subnetwork output forward IPv4 traffic (iifname)
# Subnetwork -> vswitch -> WAN
#
# Established traffic
add rule filter_4 virtual_fwo_4 ct state established ip saddr $virt_network_addr_4 accept
# Related traffic
add rule filter_4 virtual_fwo_4 ct state related ip saddr $virt_network_addr_4 log prefix "DROP related virtual_fwo_4: " drop
# ICMPv4 traffic (stateless)
add rule filter_4 virtual_fwo_4 ip saddr $virt_network_addr_4 ip protocol icmp accept
# New traffic
add rule filter_4 virtual_fwo_4 ct state new ip saddr $virt_network_addr_4 accept
# Invalid IPv4 traffic
add rule filter_4 virtual_fwo_4 ct state invalid log flags all prefix "DROP invalid virtual_fwo_4: " drop
# Untracked IPv4 traffic
add rule filter_4 virtual_fwo_4 ct state untracked log flags all prefix "DROP untracked virtual_fwo_4: " drop
#
# Default chain action
#
add rule filter_4 virtual_fwo_4 log flags ether prefix "DROP default virtual_fwo_4: " drop
#
# forward/forward_x.nft
#
# Subnetwork virtual forward IPv4 traffic (iifname oifname)
# Subnetwork -> vswitch -> subnetwork (forward traffic between virtual machines)
#
# Established traffic
add rule filter_4 virtual_fwx_4 ct state established ip daddr $virt_network_addr_4 accept
# Related traffic
add rule filter_4 virtual_fwx_4 ct state related ip daddr $virt_network_addr_4 log prefix "DROP related virtual_fwx_4: " drop
# New traffic
add rule filter_4 virtual_fwx_4 ct state new accept
# Invalid traffic
add rule filter_4 virtual_fwx_4 ct state invalid log flags all prefix "DROP invalid virtual_fwx_4: " drop
# Untracked traffic
add rule filter_4 virtual_fwx_4 ct state untracked log flags all prefix "DROP untracked virtual_fwx_4: " drop
#
# Default chain action
#
add rule filter_4 virtual_fwx_4 log flags ether prefix "DROP default virtual_fwx_4: " drop
#
# postrouting/virtual.nft
#
# virtual_nat_4
# Subnetwork -> vswitch -> WAN
# Masquerade is a special case of SNAT, where the source address is automagically set to the address of the output interface
#
# Do not masquerade multicast and broadact because it's not routed to WAN
add rule nat_4 virtual_nat_4 ip saddr $virt_network_addr_4 fib daddr type multicast return
add rule nat_4 virtual_nat_4 ip saddr $virt_network_addr_4 fib daddr type broadcast return
# Masquerade all packets originating from virtual network toward the LAN or WAN
add rule nat_4 virtual_nat_4 meta l4proto tcp ip saddr $virt_network_addr_4 ip daddr != $virt_network_addr_4 masquerade to :1024-65535
add rule nat_4 virtual_nat_4 meta l4proto udp ip saddr $virt_network_addr_4 ip daddr != $virt_network_addr_4 masquerade to :1024-65535
add rule nat_4 virtual_nat_4 ip saddr $virt_network_addr_4 ip daddr != $virt_network_addr_4 masquerade
8. Connecting to internet from guest system
Based on configuration so far virtual switch address is
172.21.11.1 which acts as gateway, therefore guest system should use this address as default gateway.Network address is
172.21.11.0/24 therefore guest systems can use address range for static configuration in range 172.21.11.2-254Here is sample
NetworkManager GUI configuration about how to configure network in guest system:This guest above uses
172.21.11.10 as its IP and 1.1.1.1 for DNS, 172.21.11.1 is virtual switch - a gateway that's firewalled with nftables routing rules.IP configuration is static.
Dynamic configuration is not covered in this guide but is definitely possible by configuring DHCP server on host that listens on our virtual switch.
DHCP server can also provide DNS and other services, however firewall should be updated with additional rules to make traffic more restricted (there are rules for DHCP included but not for local DNS)
NOTE: When ever you modify network configuration in guest system it is mandatory to restart
NetworkManager for changes to take effect!Therefore when done adjusting guest network run in guest system:
Bash:
sudo systemctl restart NetworkManager
9. Useful virt-manager settings
Allow adjusting screen resolution in
virt-manager including full screen:View -> Auto Scale Display -> AlwaysView -> Auto Scale Display -> Auto resize VM with window
Set this by default for new VM's in:
virt-manager -> Edit -> Preferences -> New VM -> Console -> Graphical console scaling -> Alwaysvirt-manager -> Edit -> Preferences -> New VM -> Console -> Resize guest with window -> On
10. Guest services
To be able to restart, shutdown etc. guest system from
virt-manager menu and to drag/drop filesand share clipboard etc.
Drag/drop files (but not directories) works from host to guest only, copy/paste does not work both ways.
Sharing clipboard between host and guest works both ways.
Regardless of guest system, in
virt-manager menu enable View -> Scale Display -> Auto resize VM with window10.1 Guest services with Linux guest
In Linux guest system...
NOTE:
qemu-guest-agent may be already be installed by default in some guest systems.
Bash:
sudo apt install qemu-guest-agent
sudo apt install spice-vdagent
systemctl reboot
spice-vdagentd should auto start, check if service is already auto starting:
Bash:
systemctl status spice-vdagent
If not, it can be fixed with
systemctl --user add-wants <some known login target>.target spice-vdagent.service10.2 Guest services with Windows guest
In Windows guest download and install spice-guest-tools
Start
Spice Agent and Spice VDAgent services in services.msc and set to auto startup.Continued in next post...
Last edited:

