Backup of proxmox configurations #277

New Issue

oysteikt · 2026-01-14T12:20:55+01:00

oysteikt commented

2026-01-14 12:20:55 +01:00

We should keep a continuous backup of the proxmox configurations, especially the XML libvirt files for VMs and disks

oysteikt added the backup label 2026-01-14 12:20:55 +01:00

oysteikt added this to the Kanban project 2026-01-14 12:20:55 +01:00

oysteikt moved this to Medium priority in Kanban on 2026-01-14 12:30:58 +01:00

felixalb commented

2026-01-16 18:54:44 +01:00

Yes, a backup script and a documented restore procedure is important. There are many "PVE Host backup" scripts out there, but many are bad, dangerous, outdated or don't take into account that we are running PVE in a cluster, so I think we have to write our own.

Some interesting notes:

/etc/pve contains most of the important files.
- Note that restoring this is spooky:
  - /dev/fuse on /etc/pve type fuse ..., It's a wack custom filesystem, and a spooky scary skeleton when the host is in a cluster. You cannot write freely to this directory.
- Files like /etc/pve/nodes/HOSTNAME/qemu-server/VMID.conf are important, and they can be safely/freely copied/moved around when restoring, unlike a lot of /etc/pve
/etc/ also has some other important stuff, such as network configurations, SSH host keys, cronjobs, subgid mappings, etc. (hah) that we might want to backup/restore or not
- Many people back up the entirety of /etc
/var/lib/vz contains a lot of ISOs, templates, backup-dumps and is a common place to store cloud-init configs
/var/lib/pve-cluster has some spooky cluster databases that should be backed up

Yes, a backup script and a documented restore procedure is important. There are many "PVE Host backup" scripts out there, but many are bad, dangerous, outdated or don't take into account that we are running PVE in a cluster, so I think we have to write our own. Some interesting notes: - `/etc/pve` contains most of the important files. - Note that restoring this is spooky: - `/dev/fuse on /etc/pve type fuse ...`, It's a wack custom filesystem, and a spooky scary skeleton when the host is in a cluster. You cannot write freely to this directory. - Files like `/etc/pve/nodes/HOSTNAME/qemu-server/VMID.conf` are important, and they can be safely/freely copied/moved around when restoring, unlike a lot of `/etc/pve` - `/etc/` also has some other important stuff, such as network configurations, SSH host keys, cronjobs, subgid mappings, etc. (hah) that we might want to backup/restore or not - Many people back up the entirety of `/etc` - `/var/lib/vz` contains a lot of ISOs, templates, backup-dumps and is a common place to store cloud-init configs - `/var/lib/pve-cluster` has some spooky cluster databases that should be backed up

oysteikt commented

2026-01-16 19:12:12 +01:00

I originally wrote this without automated restoration in mind - mostly to know what is what, had which resources and how it was all configured so that we could read off the old files in the case we had to set it all up again. I suppose automated restoration could be a goal that comes after just doing the backups themselves? Or it might be nice to keep it in mind while working on it.

The isos and templates probably don't need to be backuped, nor the SSH hostkeys and proxmox-vendored cronjobs.

I originally wrote this without automated restoration in mind - mostly to know what is what, had which resources and how it was all configured so that we could read off the old files in the case we had to set it all up again. I suppose automated restoration could be a goal that comes after just doing the backups themselves? Or it might be nice to keep it in mind while working on it. The isos and templates probably don't need to be backuped, nor the SSH hostkeys and proxmox-vendored cronjobs.

felixalb commented

2026-01-16 19:26:18 +01:00

Yes, I also have no intentions of automating the restore procedure, I am just saying that the restore procedure has some gotchas and should be documented, e.g. with a wiki article, before we are in a crisis recovery mode. Planning how to restore is the first step in deciding how to take proper backups.

If a VM has an assigned ISO image in its virtual DVD drive, it will refuse to boot if the file is not present
- Although, "unmounting"/removing the iso is very easy, so we probably don't need the ISOs
idk about templates, if we use the proxmox-provided default templates, we probably don't need them
- I thought we had some homemade/custom NixOS LXC templates, but can't find them, maybe that information is very outdated
idk about SSH hostkeys, but generating new ones are a bit annoying, and might break automated backup jobs that will probably pull over SSH
the cronjobs are a part of the cluster state. When you add a VM disk backup job in the UI, i think it generates entries in /etc/cron.d/vzdump

Yes, I also have no intentions of automating the restore procedure, I am just saying that the restore procedure has some gotchas and should be *documented*, e.g. with a wiki article, before we are in a crisis recovery mode. Planning how to restore is the first step in deciding how to take proper backups. - If a VM has an assigned ISO image in its virtual DVD drive, it will refuse to boot if the file is not present - Although, "unmounting"/removing the iso is very easy, so we probably don't need the ISOs - idk about templates, if we use the proxmox-provided default templates, we probably don't need them - I thought we had some homemade/custom NixOS LXC templates, but can't find them, maybe that information is very outdated - idk about SSH hostkeys, but generating new ones are a bit annoying, and might break automated backup jobs that will probably pull over SSH - the cronjobs are a part of the cluster state. When you add a VM disk backup job in the UI, i think it generates entries in `/etc/cron.d/vzdump`

oysteikt referenced this issue

2026-01-31 03:48:40 +01:00

Backup av "nye" servere #44

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Drift/issues#277