Skip to content

Backups

Using a combination of offsite cloud storage for the Docker Apps' config files and dbs; local mirrors for quick recovery; and bare-metal snapshots for catastrophic OS failures.

Cloud - Cloudflare R2

  • Source: /mnt/pool01/homelab/services (Configs, DBs, Scripts)
  • Tool: Kopia (Docker Container)
  • Destination: Cloudflare R2 (S3-Compatible Object Storage)
  • Schedule: Daily
  • Features: Encryption, Deduplication, Versioning (Keep 30 days).

Example

More details at Kopia Setup page here

.kopiaignore

Expand to view current .kopiaignore rules 
# HEAVYWEIGHTS (The 21GB+ Stuff)
# The Docker Engine Root (Images, Overlay2, Containers)
# verified this is ~21GB and can be rebuilt by Docker.
docker-data/

# Large Media Mounts (Just in case they are symlinked)
**/media/
**/movies/
**/tv/
**/downloads/
**/incomplete/
**/games/
**/library/

# JUNK & TEMPORARY FILES
# Global Cache & Logs
**/cache/
**/.cache/
**/logs/
**/log/
**/tmp/
**/*.log
**/*.tmp

# Specific log files not in folders
**/log.txt
**/logs.txt

# --- Specific Pattern Directories ---
# These catch our specific naming conventions
**/*-cache/
**/goaccess/data/**

# Ignore Sonarr/Radarr covers (They are just for Admin UI)
**/MediaCover/
**/mediacover/

# Jellyfin Specifics
**/jellyfin-config/config/cache/
**/jellyfin-config/config/log/
**/jellyfin-config/data/transcodes/
**/jellyfin-config/data/data/subtitles/
**/jellyfin-config/data/data/attachments/
**/jellyfin-config/data/metadata/

# CrowdSec (Downloaded Rules)
# We backup config.yaml/acquis.yaml, but ignore the auto-downloaded rules
**/crowdsec/config/hub/**
**/crowdsec/data/**

# Git (Already on GitHub, no need to backup history twice)
.git/
.gitignore

# OS Junk
.DS_Store
Thumbs.db
lost+found
.Trash-1000

# Ignore extracted subtitle caches
# Keep .srt (text) but ignore .sup/.sub (bitmaps)
*.sup
*.sub
*.idx

Formatting Drive for Backup use

CLI Commands

This is to get my 500GB Crucial SSD as a Linux-native storage environment.

# 1. Wipe existing signatures
# which disk to use obtained via: lsblk
sudo wipefs -a /dev/sda

# 2. Create a fresh GPT Partition Table
sudo parted /dev/sda mklabel gpt

# 3. Create a single partition using 100% of the space
sudo parted /dev/sda mkpart primary ext4 0% 100%

# 4. Format as EXT4 with the label "crucial500"
sudo mkfs.ext4 -L crucial500 /dev/sda1

Establish Permanent Mount Point

Set up mount point at /mnt/crucial500 and configured ownership so the primary user (me) can access it, while ensuring the system handles it correctly during boot.

Add in fstab configuration (/etc/fstab):

# Mount Crucial SSD for Local Backups
# UUID obtained via: lsblk -f | grep sda1
# Options "0 2" ensure the system checks this drive for errors on boot
UUID=<UUID-HERE>  /mnt/crucial500  ext4  defaults,noatime  0  2

Local Mirror Rsync to Mounted SSD

  • Source: /mnt/pool01/homelab/services
  • Tool: rsync driven by systemd timers
  • Destination: /mnt/crucial_ssd/homelab-services-mirror/ (Internal 500GB Crucial SSD)
  • Schedule: Daily at 05:10 AM
  • Features: 1:1 Exact Mirror (deletes files removed from source), fast local restoration. Exclude docker-data (heavy container images/databases) because they are redundant and easily re-downloaded. Only care about configuration files.

Setup Rysnc via Systemd

Systemd components: the service and the timer.

Systemd Service

(/etc/systemd/system/backup-homelab-services.service)

[Unit]
Description=Daily Mirror of Homelab-Services Dir to Crucial SSD
After=network.target local-fs.target

[Service]
Type=oneshot
User=root

ExecStart=/usr/bin/rsync -a --delete --exclude '.git' --exclude-from='/mnt/pool01/homelab/services/.kopiaignore' /mnt/pool01/homelab/services/ /mnt/crucial500/backups/homelab-services-mirror/

[Install]
WantedBy=multi-user.target

Reusing .kopiaignore file

Since we have more or less sorted out the exclusion list for doing the Kopia backups, reusing the same file for this rsync local backup

Systemd Timer

(/etc/systemd/system/backup-homelab-services.timer)

[Unit]
Description=Run Homelab-Services Backup Daily at 5am

[Timer]
OnCalendar=*-*-* 05:10:00
Persistent=true

# Wait up to 10 minutes (600s) after the trigger time to run
# This prevents high load immediately after a reboot
RandomizedDelaySec=600

Unit=backup-homelab-services.service

[Install]
WantedBy=timers.target

Enable & Start:

sudo systemctl daemon-reload
sudo systemctl enable --now backup-homelab-services.service
sudo systemctl enable --now backup-homelab-services.timer

Verification Commands:

  • Check Schedule: systemctl list-timers --all | grep backup
  • Check Logs: journalctl -u backup-homelab-services.service
  • Reload: systemctl daemon-reload

Manual OS Image via Rescuezilla

  • Source: Primary NVMe hosting CachyOS
  • Tool: Rescuezilla (Bootable USB)
  • Destination: External HDD / Alternate Internal Storage
  • Schedule: Monthly (Manual)
  • Features: Complete point-in-time bare-metal restoration (includes boot partitions, LVMs, and kernel states).

Because the host OS holds complex networking, firewall rules, and storage mount configurations, taking a block-level snapshot ensures we can recover from a total drive failure in minutes without reinstalling Arch/CachyOS from scratch.

Workflow Steps

  1. Reboot the server and boot directly from the Rescuezilla USB drive.
  2. Select Backup from the main GUI.
  3. Select Source: Choose the drive and partitions containing the Boot & CachyOS installation.
  4. Select Destination: Choose the external drive or network share.
  5. Compression: Leave at default (gzip) to save space without taking too much time.
  6. Verify: Once completed, safely eject the USB and reboot the server normally.

Graceful Shut Down

Always gracefully shut down the Docker daemon (sudo systemctl stop docker) before rebooting into Rescuezilla to ensure no databases (like Postgres or SQLite) are caught mid-write during the snapshot.