Ildkule: Move monitoring state directories to data volume #88

felixalb · 2024-11-09T15:23:11+01:00

felixalb commented

2024-11-09 15:23:11 +01:00

Ildkule ran out of disk space, so I'm moving the data to an extra volume added in openstack.

HDD performance might be way too slow for Loki and/or Prometheus, but I believe BoltDB will make it work fine
https://github.com/systemd/systemd/issues/25097 and forced use of systemd stateDirectory in /var/lib, hence the bind mounts

Current df -h:

Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G   16G   23G  41% /
/dev/vdb1       246G   20G  213G   9% /data
/dev/vda15      124M   12M  113M  10% /efi

Ildkule ran out of disk space, so I'm moving the data to an extra volume added in openstack. - HDD performance might be way too slow for Loki and/or Prometheus, but I believe BoltDB will make it work fine - https://github.com/systemd/systemd/issues/25097 and forced use of systemd `stateDirectory` in /var/lib, hence the bind mounts Current `df -h`: ``` Filesystem Size Used Avail Use% Mounted on /dev/vda1 40G 16G 23G 41% / /dev/vdb1 246G 20G 213G 9% /data /dev/vda15 124M 12M 113M 10% /efi ```

felixalb added 1 commit 2024-11-09 15:23:11 +01:00

Ildkule: Move monitoring state directories to data volume 11f6ef0572

felixalb added 1 commit 2024-11-09 21:42:47 +01:00

Ildkule: hardware-config: use UUIDs 2d52ebce52

felixalb commented

2024-11-09 21:47:23 +01:00

We could go full systemd and do

diff --git a/hosts/ildkule/services/monitoring/uptime-kuma.nix b/hosts/ildkule/services/monitoring/uptime-kuma.nix
index 9b1c0fc..6fce6c7 100644
--- a/hosts/ildkule/services/monitoring/uptime-kuma.nix
+++ b/hosts/ildkule/services/monitoring/uptime-kuma.nix
@@ -7,6 +7,7 @@ in {
   services.uptime-kuma = {
     enable = true;
     settings = {
+      DATA_DIR = lib.mkForce "/var/lib/uptime-kuma/data";
       PORT = "5059";
       HOST = "127.0.1.2";
     };
@@ -19,8 +20,9 @@ in {
     locations."/".proxyPass = "http://${cfg.settings.HOST}:${cfg.settings.PORT}";
   };

-  fileSystems."/var/lib/uptime-kuma" = {
-    device = stateDir;
-    options = [ "bind" ];
+  systemd.services."uptime-kuma" = {
+    serviceConfig = {
+      BindPaths = "${stateDir}:/var/lib/uptime-kuma/data";
+    };
   };
 }

, but I think it just makes it more difficult to grok and debug, so I prefer bind mounts for now.

We could go full systemd and do ```diff diff --git a/hosts/ildkule/services/monitoring/uptime-kuma.nix b/hosts/ildkule/services/monitoring/uptime-kuma.nix index 9b1c0fc..6fce6c7 100644 --- a/hosts/ildkule/services/monitoring/uptime-kuma.nix +++ b/hosts/ildkule/services/monitoring/uptime-kuma.nix @@ -7,6 +7,7 @@ in { services.uptime-kuma = { enable = true; settings = { + DATA_DIR = lib.mkForce "/var/lib/uptime-kuma/data"; PORT = "5059"; HOST = "127.0.1.2"; }; @@ -19,8 +20,9 @@ in { locations."/".proxyPass = "http://${cfg.settings.HOST}:${cfg.settings.PORT}"; }; - fileSystems."/var/lib/uptime-kuma" = { - device = stateDir; - options = [ "bind" ]; + systemd.services."uptime-kuma" = { + serviceConfig = { + BindPaths = "${stateDir}:/var/lib/uptime-kuma/data"; + }; }; } ``` , but I think it just makes it more difficult to grok and debug, so I prefer bind mounts for now.

👍 1

danio approved these changes 2024-11-09 23:31:00 +01:00

felixalb merged commit 8285d91401 into main

2024-11-10 02:30:11 +01:00

felixalb deleted branch ildkule-data-volume

2024-11-10 02:30:12 +01:00

felixalb referenced this issue from a commit

2024-11-10 02:30:13 +01:00

Merge pull request 'Ildkule: Move monitoring state directories to data volume' (!88) from ildkule-data-volume into main

Sign in to join this conversation.