Connect data directories outside the jail

Introduction

Certain directories within a jail can be swapped out to (datasets) "outside". This allows this data to be stored independently of the jail. The goal, similar to Docker, is to depend as little as possible on the jail itself. This way, if the jail is corrupted or deleted for some reason, we are able to restore the previous configuration and data with minimal effort. Also, these directories can then be efficiently and regularly backed up via snapshot, which also simplifies further backups. Not to mention moves or migrations.

Last update:

• 03.03.2024: Minor adjustments
• 23.01.2024: Some tweaking and configuration added to TrueNAS
• 18.08.2023: Initial version

Terminology

• JAILNAME = The name of the jail, ex. gitea
• SOURCEPATH = The directory (DATASET) on the TrueNAS/FreeBSD, ex. /mnt/tank/jails_data/gitea/data
• TARGETPATH = The directory inside the jail, ex. /mnt/tank/iocage/jails/gitea/root/var/db/gitea
• ID = Required user ID within the jail, ex. 211
• ZPOOLNAME = The pool name which contains the data, ex. `tank

Directory structure

Jails are usually stored in /usr/jails or under TrueNAS in /mnt/ZPOOLNAME/iocage/jails with the following directory structure:

└── jails         # Jail main directory
    └── JAILNAME  # Jail Name
        └── root  # / of the Jail

For this purpose, a new dataset named jails_data is now created, which hosts the "jail data" and mirrors the above structure somewhat, so that it remains easily traceable. For example, this could be on a completely different pool /mnt/ZPOOLNAME/jails_data, which may have much more storage space. This is very useful for large data files like pictures or file shares, which have no place in the jail. Also read only if needed.

└── jails_data     # As a counterpart to jails.
    └── JAILNAME   # The same name as in jails, ex. `gitea`
        └── DATASET  # source directories of the outsourced data, ex. `data`

NullFS

But how does that come together? Quite simply via NullFS. NullFS virtually puts the contents of directory A on directory B, even across jail boundaries. Very handy! A key factor is access rights and proper permissions. If there is a user USER with ID 123 in the jail, then the directory outside the jail MUST have the same permissions. Fortunately, the user ID is sufficient for this, so it is not necessary to create the corresponding username in the TrueNAS/FreeBSD host each time as well. Since the directories usually do not yet exist in the jail, they have to be created beforehand. Actually they are created only during the installation of the packages, but this is too late. Because during the installation the content is often already created within the destination directory.

TrueNAS

User creation

• TrueNAS / Accounts / Groups / Add

  GID: ID    # ex. 211
  Name: NAME # ex. gitea

• TrueNAS / Accounts / Users / Add

  Full Name: NAME    # ex. Gitea User
  Username: USERNAME # ex. gitea
  Password: PASSWORD # ex. 123
  User ID: ID        # ex. 211
  Primary Group: ID  # ex. 211
  Shell: nologin     # No login allowed
  Home: /nonexistent # No home directory

Create data dataset

• TrueNAS / Storage / Pools / tank/jails_data/JAILNAME / Add Dataset

  Name: POOLNAME # ex. data

Dataset permissions

• TrueNAS / Jails / Mount Points / tank/jails_data/JAILNAME / Edit Permissions

  User: ID  # ex. 211
  Group: ID # ex. 211

Mount dataset

Practical: The target directories in the jail are created automatically when the mount points are assigned. Also recursive!

• TrueNAS / Jails / JAILNAME / Mount Points / Actions / Add

  data:
  └──Source: SOURCE           # ex. /mnt/tank/jails_data/gitea/data
  └──Destination: DESTINATION # ex. /mnt/tank/iocage/jails/gitea/root/var/db/gitea

So in the jail GITEA the directory /usr/local/etc/gitea points to the actually external directory /mnt/tank/jails_data/gitea/data.
Everything clear? Good!

Console

The jail must be restarted with iocage restart JAILNAME each time to include these paths.
With iocage fstab -e JAILNAME the once set entries can be adjusted or removed afterwards.

zfs create -p ZPOOLNAME/jails_data/JAILNAME/DATASET # create SOURCEPATH (dataset) in jail_data
chown -R ID:ID SOURCEPATH                           # Set user, the naked ID is sufficient, e.g. `211`.
iocage start JAILNAME                               # start jail
iocage exec JAILNAME mkdir -p TARGETPATH            # create directory in jail
iocage fstab -a JAILNAME SOURCEPATH TARGETPATH nullfs rw 0 0 # include directories
iocage restart JAILNAME                             # restart jail

Voilá