Introduction

First of all, the basics are created here. A clean and comprehensible basis in the network configuration helps enormously. Another very practical innovation could be VirtIO-9p (this allows local directories of the host system to be passed on to a VM, similar to NullFS with the jails), but unfortunately it lacks important file system functions that allow, for example, an SQLite database to be stored on it. Therefore, a (very simple) NFS configuration is used here and the need for virtual hard disks is reduced to an absolute minimum. The space requirements (e.g. for photo management) can then grow dynamically without having to specify a fixed hard drive size beforehand.

Goals

The preparation of FreeBSD for the operation of BHYVE VMs is done in a few steps, as only a few packages are required. Important: This guide is explicitly designed to run in coexistence with Bastille Jails. This means that the Storage of dataand also the Network interfaces are configured in such a way that both are based on the same interfaces and logic.

NEW: For the very impatient I have a console only section. There are only commands, no explanations.

Last update:

Basics

FreeBSD

Kernel Module

The ‘bhyve virtual machine monitor’ is already required when booting.

echo 'vmm_load="YES"' >> /boot/loader.conf
kldload vmm

ZFS

As a basis for BHYVE, the desired root directory is created with zfs create -o mountpoint=/usr/local/bhyve/ work/bhyve, in which we will then expand in further articles.

Optional: If the additional data pool is to be used, it can be created with zfs create -o mountpoint=/usr/local/bhyve/data data/byhve.

The directory structure via zfs list | grep bhyve then looks like this:

# zfs list | grep bhyve
data/byhve                             96K  1.75T    96K  /usr/local/bhyve/data
work/bhyve                             96K   893G    96K  /usr/local/bhyve/

BHYVE

Installieren

BHYVE is quickly installed after adapting the package source withpkg install -y tmux vm-bhyve bhyve-firmware grub2-bhyve qemu-tools.

Anpassen

With service vm enable BHYVE is automatically activated at boot time, but with these entries in /etc/rc.conf it only becomes complete:

sysrc vm_dir="zfs:work/bhyve"  # Legt das ZFS Stammverzeichnis fest
sysrc vm_list=""               # Eine Liste der VMs, die beim booten automatisch gestartet werden sollen
sysrc vm_delay="5"             # Eine kurze Pause in Sekunden dazwischen

This prepares the work/bhyve directory the first time with vm init and creates the required directory structure. With vm set console=tmux, TMUX is set as a more user-friendly console instead of the somewhat bulky default. A few sample templates for later can be copied with this cp /usr/local/share/examples/vm-bhyve/* /usr/local/bhyve/.templates/. The /usr/local/bhyve/.templates/config.sample is of particular interest here as further documentation.

The directory structure per ls -l /usr/local/bhyve then looks like this:

drwxr-xr-x  2 root wheel  4 Dec 22 11:44 .config
drwxr-xr-x  2 root wheel  2 Dec 22 11:44 .img
drwxr-xr-x  2 root wheel  2 Dec 22 11:44 .iso
drwxr-xr-x  2 root wheel 20 Dec 22 11:45 .templates
drwxr-xr-x  2 root wheel  2 Dec 22 11:41 data

Network

The BHYVE guests and the host or the network need to be connected to each other, otherwise it gets very lonely. This is made possible by means of a bridge, which the command vm calls a switch. Instead of creating a switch with the integrated function, we use the already existing network configuration here so that BHYVE and Bastille do not interfere with each other and everything does not become unnecessarily complicated.

The existing bridge publicnet0 is therefore made known to the vm configuration as a ‘new’ switch with vm switch create -t manual -b publicnet0 public. The VMs that we create automatically receive new network (tap) interfaces that are connected to this publicnet0 switch. public is just an internal name, an alias. aThrough this, jails and VMs can also communicate with each other.

Before continuing, now is a good time to reboot the system with reboot to see if everything is still working. This completes the preparations and prepares the system for BHYVE.

This describes how files are then exchanged between the host system and the guests.

Console

sysrc cloned_interfaces="bridge0"
sysrc ifconfig_bridge0_name="publicnet0"
sysrc ifconfig_publicnet0="addm igb0 up"
service netif restart
echo 'vmm_load="YES"' >> /boot/loader.conf
kldload vmm
zfs create -o mountpoint=/usr/local/bhyve/ work/bhyve
# zfs create -o mountpoint=/usr/local/bhyve/data data/byhve # Optional
pkg install -y tmux vm-bhyve bhyve-firmware grub2-bhyve qemu-tools
service vm enable
sysrc vm_dir="zfs:work/bhyve"
sysrc vm_list=""
sysrc vm_delay="5"
vm init
vm set console=tmux
cp /usr/local/share/examples/vm-bhyve/* /usr/local/bhyve/.templates/
vm switch create -t manual -b publicnet0 public
reboot

Voilá