Aggregate "magic directories" under ~/public (or something equivalent) #144

Open
opened 2024-08-26 17:11:33 +02:00 by oysteikt · 9 comments
Owner

We have a set of magic directories that lets you create different types of content that gets picked up by our systems and hosted in different ways. Currently, these are:

  • ~/web-docs - user web pages. These have been here sinc the dawn of time. Hosted on tom.
  • ~/public_gopher - gopher holes. These are really new. Hosted on isvegg.
  • ~/pvv-photos - gallery photos. These are also relatively new. rsynced from microbel to bekkalokk, and hosted there.

These are all located in the toplevel of every users home directory, and it will become quite the mess when we introduce even more (I already consider it a mess). We should probably aggregate them all under a common directory, where all of the magic stuff goes.

In order to not cause too much breakage, we should leave symlinks. Especially for web-docs, since it's so old. We might consider not doing it for the newer ones.

We have a set of magic directories that lets you create different types of content that gets picked up by our systems and hosted in different ways. Currently, these are: - `~/web-docs` - user web pages. These have been here sinc the dawn of time. Hosted on tom. - `~/public_gopher` - gopher holes. These are really new. Hosted on isvegg. - `~/pvv-photos` - gallery photos. These are also relatively new. rsynced from microbel to bekkalokk, and hosted there. These are all located in the toplevel of every users home directory, and it will become quite the mess when we introduce even more (I already consider it a mess). We should probably aggregate them all under a common directory, where all of the magic stuff goes. In order to not cause too much breakage, we should leave symlinks. Especially for web-docs, since it's so old. We might consider not doing it for the newer ones.
oysteikt added the
services
label 2024-08-26 17:11:33 +02:00
oysteikt added this to the Kanban project 2024-08-26 17:11:33 +02:00
Owner

~/web-docs has been here forever, and I expect people to have these files linked all over the place, scripts with hard-coded paths, scripts that automatically put things in ~/web-docs, etc.
The other ones are quite new and flexible, and I support just moving them to a new location, like under a common ~/public.
I think we can just do this without a lot of planning or notice, as they probably won't break anything (TM), as long as we update the appropriate documentation.

`~/web-docs` has been here forever, and I expect people to have these files linked all over the place, scripts with hard-coded paths, scripts that automatically put things in `~/web-docs`, etc. The other ones are quite new and flexible, and I support just moving them to a new location, like under a common `~/public`. I think we can just do this without a lot of planning or notice, as they probably won't break anything (TM), as long as we update the appropriate documentation.
Owner

At the same time, I would maybe like to add some placeholder directories and README.txt into /etc/skel, to prepare this for new users.

At the same time, I would maybe like to add some placeholder directories and README.txt into /etc/skel, to prepare this for new users.
oysteikt added a new dependency 2024-08-26 22:19:20 +02:00
oysteikt added a new dependency 2024-08-26 22:19:54 +02:00
Owner

A longstanding issue with the current implementation is that the web server needs +x on the entire home directory in order to read ~/web-docs. What stud.ntnu.no does to avoid granting this access is to place the web files completely outside the home directory, and make a symlink inside the home directory for backwards compatibility. This allows the user to not let anything else access their home directory, which significantly reduces the splash zone if someone exploits the web server. I would suggest a similar approach if we're going to make changes to this.

A longstanding issue with the current implementation is that the web server needs `+x` on the entire home directory in order to read `~/web-docs`. What `stud.ntnu.no` does to avoid granting this access is to place the web files completely outside the home directory, and make a symlink inside the home directory for backwards compatibility. This allows the user to not let anything else access their home directory, which significantly reduces the splash zone if someone exploits the web server. I would suggest a similar approach if we're going to make changes to this.
Author
Owner

The way I see it, there are currently two implementation concerns:

  • The webserver should not have access to entire user directories, for security and potentially privacy reasons implied by the dir privileges.
  • The special directories and their content should not be yanked away from the user in case of an NFS failure or equivalent.

Could we find a solution that solves both? Could we create a sort of virtual space for the webserver, where root (or systemd) bindmounts the relevant directories so that anything outside is completely unaddressable by path? Or are there any performance concerns with doing this on top of NFS?

Alternatively, we could set up an OpenBSD box to host user websites, and find a webserver that would let us make use of unveil(2)

The way I see it, there are currently two implementation concerns: - The webserver should not have access to entire user directories, for security and potentially privacy reasons implied by the dir privileges. - The special directories and their content should not be yanked away from the user in case of an NFS failure or equivalent. Could we find a solution that solves both? Could we create a sort of virtual space for the webserver, where root (or systemd) bindmounts the relevant directories so that anything outside is completely unaddressable by path? Or are there any performance concerns with doing this on top of NFS? Alternatively, we could set up an OpenBSD box to host user websites, and find a webserver that would let us make use of [`unveil(2)`](https://man.openbsd.org/unveil.2)
Owner

from my experience with bubblewrap: nfs and namespace bind mounts rarely play well together. This is a shortcoming in the kernel

from my experience with bubblewrap: nfs and namespace bind mounts rarely play well together. This is a shortcoming in the kernel
danio added the
disputed
label 2024-09-12 20:33:55 +02:00
Author
Owner

Two users already exist with ~/public: oyvinlek and andersja. Should we ask them before we just take over this directory name to have a sideeffect? I think oyvinlek's contents of this directory is the most likely to clash with the names of the magic dirs. They also seem to be an active user of our systems.

Two users already exist with `~/public`: oyvinlek and andersja. Should we ask them before we just take over this directory name to have a sideeffect? I think oyvinlek's contents of this directory is the most likely to clash with the names of the magic dirs. They also seem to be an active user of our systems.
Owner

~/pvv-public?

`~/pvv-public`?
Author
Owner

No, we talked with oyvinlek and he changed his dirname. I think the coast is clear

No, we talked with oyvinlek and he changed his dirname. I think the coast is clear
Author
Owner

FTR, microbel's photo gallery script already handles ~/public/pvv-photos

FTR, [microbel's photo gallery script](https://wiki.pvv.ntnu.no/wiki/Drift/Fotogalleri#Skript_p%C3%A5_Maskiner/Microbel) already handles `~/public/pvv-photos`
Sign in to join this conversation.
4 Participants
Notifications
Due Date
No due date set.
Reference: Drift/issues#144
No description provided.