Due to problems we encountered with the approach taken in 0.9.2 it will change drastically with 0.9.3. The changes are so far-reaching FiFo 0.9.3 will not recognize jails created with 0.9.2!
Project-FiFo provides experimental support for BSD jails on FreeBSD.
At this point, FiFo depends on FreeBSD-11 +, RCTL, and vnet support.
Many of the steps below are FreeBSD centric, and we highly recommend to contact the FreeBSD community if you have problems or trouble with them as they are a lot more qualified to answer questions regarding BSD specific topics then we are.
- make backups
- don't use experimental features on critical systems
# exact kernel version does not matter!
fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/`freebsd-version -k`/src.txz -o /tmp/src.txz
tar -C / -xzf /tmp/src.txz
cat > FIFOKERNEL <<EOL
options VIMAGE # VNET/Vimage support
options RACCT # Resource containers
options RCTL # same as above
make -j`sysctl -n hw.ncpu` buildkernel KERNCONF=FIFOKERNEL
make -j`sysctl -n hw.ncpu` installkernel KERNCONF=FIFOKERNEL
## enable required modules and enable rcct
cat <<EOF >> /boot/loader.conf
All FiFo components follow a simple rule: mutable data is stored in
/data/<service> best practice is to set up a ZFS filesystem for each of them:
zfs create zroot/data
zfs set mountpoint=/data zroot/data
zfs create zroot/data/sniffle
zfs create zroot/data/snarl
zfs create zroot/data/howl
zfs create zroot/jails
zfs set mountpoint=/zroot/jails zroot/jails
We provide packages for all FiFo components to add the repository please run:
mkdir -p /usr/local/etc/pkg/repos
cat <<EOF > /usr/local/etc/pkg/repos/ProjectFiFo.conf
FiFo has two sets of components, the management system itself and what needs to be installed on each hypervisor, so lets install the hypervisor
pkg install vmadm chunter zlogin
cat <<EOF >> /etc/rc.conf
service zlogin start
service chunter start
Note vmadm does not have to be started, it's run as a service to automatically boot jails on system startup that have the
autoboot attribute set to true.
The management components do not have to live on every hypervisor instead it's enough to install them on one, however for production use where uptime mattes it's recommended to run a cluster for at least 5 nodes.
This component can be run both in a jail on the host.
pkg install fifo-sniffle fifo-snarl fifo-howl fifo-cerberus
cat <<EOF >> /etc/rc.conf
The packages will print some more required steps like allowing howl to use privileged ports like
cat <<EOF >> /etc/sysctl.conf
service sniffle start
service snarl start
service howl start
Cerberus isn't a separate service but rather a set of static files for the UI, so does not need to be started.
vmadm requires a bridge interface to allow VNET interfaces to attach to. If you have multiple networks you can have multiple bridge interfaces. The mapping of a network to its bridge is configured in the vmadm config file.
Please make sure that for all components, when editing the config file you chan change the IP address to the interface you want to use to communicate over.
FiFo provides a vmadm clone for FreeBSD that works with jails. It is API compatible with the SmartOS vmadm in the sense that it provides a subset of its functionality and translates those to native jail equivalents
You might want to configure the config file
/usr/local/etc/vmadm.toml and enable vmadm as a startup services.
Both the pool and networks section might need changing
pool = "zroot/jails"
admin = "bridge0"
FreeBSD and bridges
When using a bridge interface it is important to disable some of the hardware features of the NIC. If this is not done FreeBSD will take them down itself for the first VNET interface is added possibly causing a connection loss for multiple seconds. To avoid this add the following flags
-tso -lro -rxcsum -txcsum -rxcsum6 -txcsum6to the rc.conf bringing up your interface for example:
ifconfig_em0="DHCP -tso -lro -rxcsum -txcsum -rxcsum6 -txcsum6"
Otherwise please follow tine Installing FiFo guide for the setup.
FiFo uses LeoFS as a local dataset cache, we do not provide own FreeBSD packages for it as they provide it themselfs: http://leo-project.net/
An alternative is to disable LeoFS
Updated over 6 years ago