Have you ever wanted to spin up VMs that appear as separate device on your LAN? It’s much easier than you think. I’ll show how to configure a network bridge and spin up Rocky Linux VM using cloud-init.
I’m still new to this, so take all information with a grain of salt. This article was written mainly for myself to better understand the topic.
Bridge network
This section is heavily based on:
- Arch Wiki page: systemd-networkd - Network bridge with DHCP; …
Have you ever wanted to spin up VMs that appear as separate device on your LAN? It’s much easier than you think. I’ll show how to configure a network bridge and spin up Rocky Linux VM using cloud-init.
I’m still new to this, so take all information with a grain of salt. This article was written mainly for myself to better understand the topic.
Bridge network
This section is heavily based on:
- Arch Wiki page: systemd-networkd - Network bridge with DHCP;
- Arch Wiki page: NetworkManager;
- RedHat Wiki page: Configuring a network bridge.
In Linux networking, a Bridge network is a virtual network interface that acts like a network switch, forwarding network frames between connected devices. Because it operates at OSI layer 2 connected devices appear as physical devices on the same network. This means each connected device, such as a VM or container, can have its own IP address assigned.
In practice, we need to:
- Create bridge network;
- remove NIC connection;
- Enslave NIC as bridge network device.
I’ll define a bridge “br0” with the IPv4 address 192.168.1.2. My NIC is “enp37s0” and the gateway is 192.168.1.1, but yours may differ.
warning
If you are going to use static IP please make sure it is outside of your router DHCP range.
My DHCP range is 100-250 so I’m okay with 192.168.1.2 address.
Configuring bridge using NetworkManager
# Find NIC name
$ nmcli connection show
NAME UUID TYPE DEVICE
enp37s0 f9a9099d-0e8a-41c0-99ca-c4071997d9a8 ethernet enp37s0
lo deedda43-472f-429a-83a9-73b3d26c1606 loopback lo
# disconnect and delete connection
$ nmcli connection down enp37s0
$ nmcli connection delete enp37s0
# Create bridge
$ nmcli connection add type bridge ifname br0 con-name br0
$ nmcli connection modify br0 \
ipv4.method manual \
ipv4.addresses 192.168.1.2/24 \
ipv4.gateway 192.168.1.1
$ nmcli connection add type bridge-slave ifname enp37s0 master br0 con-name br0-enp37s0
$ nmcli connection modify br0 connection.autoconnect yes
# Bring bridge up
$ nmcli connection up br0
$ nmcli connection up br0-enp37s0
To reverse all this you need to down and delete “br0” and “br0-enp37s0”.
Configuring bridge using systemd-networkd
# Create backup
$ mkdir ~/networkd-backup && sudo mv /etc/systemd/network/* ~/networkd-backup
# Create bridge
$ sudo cat <<EOF > /etc/systemd/network/20-br0.netdev
[NetDev]
Name=br0
Kind=bridge
EOF
$ sudo cat <<EOF > /etc/systemd/network/20-br0-en.network
[Match]
Name=en*
[Network]
Bridge=br0
EOF
$ sudo cat <<EOF > /etc/systemd/network/20-br0.network
[Match]
Name=br0
[Link]
RequireForOnline=routable
[Network]
Address=192.168.1.2/24
Gateway=192.168.1.1
[DHCPv4]
SendHostname=yes
Hostname=home-bridge
EOF
# Restart networkd service
$ systemctl restart systemd-networkd.service
To reverse all this you need to rm “/etc/systemd/network/20-br*” files and mv old configuration from ”~/networkd-backup”.
Verifing configuration
$ ip -br a
lo UNKNOWN 127.0.0.1/8 ::1/128
enp37s0 UP
br0 UP 192.168.1.2/24
Both ethernet and bridge interfaces are up. The bridge has the IP address, while the Ethernet interface does not.
You can verify it using native CLI too:
# networkmanager
# The ethernet interface is connected to br0-enp37s0.
$ nmcli device
DEVICE TYPE STATE CONNECTION
br0 bridge connected br0
enp37s0 ethernet connected br0-enp37s0
lo loopback connected (externally) lo
# systemd-networkd
# The ethernet interface must be **enslaved**, "br0" must be **routable**, and both of them should be **configured**.
$ networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp37s0 ether enslaved configured
3 br0 bridge routable configured
Cloud init
This section is heavily based on:
- Rocky Linux wiki page: Guide to cloud-init on Rocky Linux.
Cloud init allows making templates for virtual machines and provide system configuration on first boot. This useful if you don’t want to configure anything from ground up by hands or write Ansible playbooks for automating base configuration.
I’ll show how to spin up a VM with a static IP and a preconfigured SSH key. For this, we need to install two packages:
After installing these packages, give QEMU access to the bridge network:
$ echo "allow br0" >> /etc/qemu/bridge.conf
Create a directory for all VM files and generate an SSH key pair for the VM:
$ mkdir ~/rocky-10-example
$ cd ~/rocky-10-example
$ ssh-keygen -t ed25519 -C 'your@email.example' -f ~/.ssh/homelab -N ''
OS disk
Cloud init requires a special OS disk called a cloud image. It is a preconfigured OS template with cloud-init tools.
Get Rocky Linux image from the official download page, or fetch it with curl:
$ curl -Lo rocky-10.cloud.x86-64.qcow2 https://dl.rockylinux.org/pub/rocky/10/images/x86_64/Rocky-10-GenericCloud-Base.latest.x86_64.qcow2
This image can be used as is, but it’s more efficient to make a new image based on the original. This saves space and simplifies provisioning new VMs without re-downloading the original file:
$ qemu-img create -f qcow2 -F qcow2 -b rocky-10.cloud.x86-64.qcow2 rocky-10-example.qcow2
Configuration
There are a lot of ways to provide configuration. I’ll use NoCloud - it’s easiest way, since virsh has a CLI argument that can create an ISO with the configuration for us.
The minimal configuration contains two files:
- meta-data - gives information about the cloud where this image spins up and about the image instance. All keys and their definitions can be found on the instance-data page;
- user-data - provides declarative way to describe initial configuration of the system. It can be used to configure users, install packages and more.
The minimal meta-data contains two variables:
- instance-id - unique ID of the instance;
- local-hostname - hostname of the instance.
$ cat <<EOF > meta-data
instance-id: rocky-10-example
local-hostname: rocky-10-example
EOF
The minimal user-data contains nothing, but I’ll set up an SSH key for the root user (rocky):
$ cat <<EOF > user-data
#cloud-config
users:
- name: rocky
ssh_authorized_keys:
- PASTE HERE YOUR SSH PUBLIC KEY
EOF
network-data provides a declarative way to describe VM networking. It can be used to configure a static IP, DNS, and more.
Let’s assign a static IP to this VM:
$ cat <<EOF > network-data
version: 2
ethernets:
ens3:
dhcp4: no
addresses:
- 192.168.1.10/24
gateway4: 192.168.1.1
EOF
warning
If you are going to use static IP please make sure it is outside of your router DHCP range.
My DHCP range is 100-250 so I’m okay with 192.168.1.10 address.
If you don’t provide network-data, the VM will use DHCP to obtain a random IP address. To find it, run:
$ virsh list
Id Name State
----------------------------------
1 rocky-10-example running
$ host rocky-10-example
rocky-10-example has address 192.168.1.222
rocky-10-example has IPv6 address fdc3:8965:6a45::3e2
Launch VM
$ virt-install \
--name rocky-10-example \
--memory 2048 --vcpus 2 \
--network bridge=br0 \
--disk path=rocky-10-example.qcow2,format=qcow2 \
--cloud-init user-data=user-data,meta-data=meta-data,network-config=network-data \
--os-variant rocky-unknown \
--import --noautoconsole
Domain is still running. Installation may be in progress.
You can reconnect to the console to complete the installation process.
| parameter | description |
|---|---|
| —name rocky-10-example | VM name, used internally by libvirt |
| —memory 2048 —vcpus 2 | Memory and CPU configuration |
| —network bridge=br0 | Network binding, replace br0 with your bridge name |
| —disk path=rocky-10-example.qcow2,format=qcow2 | OS disk |
| —cloud-init user-data=user-data,meta-data=meta-data | All cloud init configuration files |
| —os-variant rocky-unknown | On Arch there are no rocky-10 os-variant, so I use unknown |
| —import | Skips installing OS from ISO disk |
| —noautoconsole | Prevents console autoconnect |
Not very fun, but we can check that it’s configured correctly and running.
$ ssh rocky@192.168.1.10 -i ~/.ssh/homelab
The authenticity of host '192.168.1.10 (192.168.1.10)' can't be established.
ED25519 key fingerprint is: SHA256:GUM0J325Vs4QWu3gh25dNH5yTAXrC8weOdhNWSa+xuo
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.1.10' (ED25519) to the list of known hosts.
** WARNING: connection is not using a post-quantum key exchange algorithm.
** This session may be vulnerable to "store now, decrypt later" attacks.
** The server may need to be upgraded. See https://openssh.com/pq.html
[rocky@rocky-10-example ~]$ echo Hello, World!
Hello, World!