Research whether we can deprecate Tom

h7x4 commented

2023-09-02 22:17:45 +02:00

(Migrated from github.com)

Tom hosts the home directory websites. ~~The machine is physical and quite weak, so moving it shouldn't be a problem powerwise.~~ However, there is a bunch of software installed on the machine, and we do not know anything about the assumptions made by users on their home websites. PHP sites or CGI scripts might assume software to be installed, or just exist in $PATH. We would need to do some research before being able to move this.

EDIT:
This isn't actually a physical machine, it's a VM on Asgore. Reducing priority...

[Tom](https://www.pvv.ntnu.no/pvv/Maskiner/Tom) hosts the home directory websites. <del>The machine is physical and quite weak, so moving it shouldn't be a problem powerwise.</del> However, there is a bunch of software installed on the machine, and we do not know anything about the assumptions made by users on their home websites. PHP sites or CGI scripts might assume software to be installed, or just exist in `$PATH`. We would need to do some research before being able to move this. EDIT: This isn't actually a physical machine, it's a VM on [Asgore](https://www.pvv.ntnu.no/pvv/Maskiner/asgore). Reducing priority...

oysteikt added this to the Kanban project 2024-08-03 22:52:44 +02:00

oysteikt added the servers n' hardware label 2024-08-06 18:14:09 +02:00

oysteikt moved this to Ongoing in Kanban on 2026-01-14 08:59:21 +01:00

oysteikt self-assigned this 2026-01-14 08:59:30 +01:00

oysteikt commented

2026-01-14 09:07:28 +01:00

We'll need to make a tool to test how many websites we break, and try to figure out if it's only due to use of deprecated php or if it's our misconfiguration or missing software.

Maybe we should scan out every single php/cgi file in everyones home directories, and try to query them all, and see if the number of erroneous HTTP responses go up? It's not a perfect measurement, but it might make for a good indicator

We'll need to make a tool to test how many websites we break, and try to figure out if it's only due to use of deprecated php or if it's our misconfiguration or missing software. Maybe we should scan out every single php/cgi file in everyones home directories, and try to query them all, and see if the number of erroneous HTTP responses go up? It's not a perfect measurement, but it might make for a good indicator

oysteikt commented

2026-01-14 17:14:46 +01:00

Update: I created a new VM on blossom named temmie, see the nixos config repo.

SLAGPLAN

create a duplicate domain for testing out temmie, we'll just double host the userwebs for a little while during the testing phase.
create phpfpm pools with php environments that contain packages similar to those present on tom.
use apache httpd to talk to the phpfpm pools
use chroot and bindmounts on the systemd unit to place the nfs automounts in /home/pvv/ inside the chroot. This might have to be apache httpd's sandbox, not sure.
we might be able to use systemd sockets to escape some sandboxes between these pieces.
pro fit?

Update: I created a new VM on `blossom` named `temmie`, see the nixos config repo. ## SLAGPLAN - create a duplicate domain for testing out temmie, we'll just double host the userwebs for a little while during the testing phase. - create phpfpm pools with php environments that contain packages similar to those present on tom. - use apache httpd to talk to the phpfpm pools - use chroot and bindmounts on the systemd unit to place the nfs automounts in /home/pvv/<letter> inside the chroot. This might have to be apache httpd's sandbox, not sure. - we might be able to use systemd sockets to escape some sandboxes between these pieces. - pro fit?

oysteikt commented

2026-01-22 06:03:08 +01:00

Note that there is a custom script at salt/web/apache-logger which splits the logs into files in every user's homedir when such a file exists. Don't forget to add this as well.

Note that there is a custom script at `salt/web/apache-logger` which splits the logs into files in every user's homedir when such a file exists. Don't forget to add this as well.

oysteikt commented

2026-01-30 05:43:20 +01:00

Note, there are quite a few cgi scripts with shebangs like #!/store/bin/perl. Maybe ensure that bin is present in /store as well

EDIT: grepping around, seeing all of the following prefixes:

/store/bin
/store/gnu/bin
/usr/bin
/usr/local/bin
/local/bin
/opt/bin
/opt/local/bin
/sbin
/usr/sbin
/usr/local/sbin
/opt/sbin

These ones are kinda sus:

/var/qmail/bin (for sendmail? used surprisingly often...)
/usr/drwho/local/bin
/usr/local/mlog3/bin
/usr/local/flite/bin
/opt/flite/bin
/opt/csw/bin
/sw/bin
/usr/sfw/bin
/perl/bin
/usr/local/php/bin
/usr/X11R6/bin

These should cover the vast majority of paths used in shebangs and cgi invocations

Note, there are quite a few cgi scripts with shebangs like `#!/store/bin/perl`. Maybe ensure that `bin` is present in `/store` as well EDIT: grepping around, seeing all of the following prefixes: - `/store/bin` - `/store/gnu/bin` - `/usr/bin` - `/usr/local/bin` - `/local/bin` - `/opt/bin` - `/opt/local/bin` - `/sbin` - `/usr/sbin` - `/usr/local/sbin` - `/opt/sbin` These ones are kinda sus: - `/var/qmail/bin` (for sendmail? used surprisingly often...) - `/usr/drwho/local/bin` - `/usr/local/mlog3/bin` - `/usr/local/flite/bin` - `/opt/flite/bin` - `/opt/csw/bin` - `/sw/bin` - `/usr/sfw/bin` - `/perl/bin` - `/usr/local/php/bin` - `/usr/X11R6/bin` These should cover the vast majority of paths used in shebangs and cgi invocations

oysteikt commented

2026-01-30 05:44:53 +01:00

Looking at some of the scripts, it seems essential that the directories are read-write. There is a lot of self-modifying stuff here, and code that generates pdfs and html and whatnot

oysteikt commented

2026-01-30 09:05:49 +01:00

Ran the following commands to get an idea of what kinds of programs are being used in cgi scripts. Note that this is not a complete overview, there are likely many instances of dynamic concatenation of prefix + program as well as just running a program from $PATH. There are likely also false positives, either stuff from comments, from urls (e.g. https://example.com//opt/bin/something), or whatever else. (however I dropped /bin to avoid too many url mismatches).

find \
  /home/pvv/?/* \
  -type f \
  \( \
       -name '*.cgi' \
    -o -name '*.php' \
    -o -name '*.perl' \
    -o -name '*.pl' \
    -o -name '*.sh' \
  \) \
  -exec rg '(/store/bin|/store/gnu/bin|/usr/bin|/usr/local/bin|/local/bin|/opt/bin|/opt/local/bin|/sbin|/usr/sbin|/usr/local/sbin|/opt/sbin|/var/qmail/bin|/usr/drwho/local/bin|/usr/local/mlog3/bin|/usr/local/flite/bin|/opt/flite/bin|/opt/csw/bin|/sw/bin|/usr/sfw/bin|/perl/bin|/usr/local/php/bin|/usr/X11R6/bin)/(?:env\s*)?(\w+(?:/\w+)*)' {} -N -o -r '$1' \; \
  2>/dev/null | tee cgi-progs.txt

cat cgi-progs.list | sort | uniq -c | sort -gr

We get the following list:

Big list

8421 perl
5375 bash
2286 nix
 250 gcc
 242 systemctl
 238 install
 221 bazel
 204 sendmail
 202 jhead
 174 ruby1
 170 clang
 161 sh
 148 file
 138 localedef
  98 python
  97 pkg
  97 php
  94 chkconfig
  84 devcontainer
  84 code
  79 python3
  77 initctl
  59 date
  56 convert
  51 python2
  50 getconf
  44 grep
  44 docker
  43 salt
  42 time
  42 curl
  38 env
  37 sed
  35 ldconfig
  34 ruby
  34 ccache
  31 uname
  28 pip
  28 g
  28 composer
  26 perl5
  26 less
  25 mailx
  24 wget
  24 tar
  24 paxctl
  23 X11
  23 valgrind
  21 mail
  20 more
  20 kasm_server
  20 exiftags
  19 pkg2ng
  19 meson
  18 lndir
  18 finger
  18 diskutil
  18 chmod
  17 test
  17 php5
  17 llvm
  17 ld
  17 gnuplot
  17 find
  17 diff
  17 chacl
  16 timeout
  16 procmail
  16 minikube
  16 kubectl
  16 gzip
  16 awk
  15 yacc
  15 spamd
  15 sleep
  15 service
  14 qemu
  14 passwd
  14 md5
  14 m4
  14 ls
  14 kstat
  14 installkernel
  14 init
  14 git
  14 dscl
  14 aspell
  14 ar
  13 sysctl
  13 nologin
  13 nohup
  13 ninja
  13 lynx
  13 diff3
  13 chown
  13 arch
  12 zipnote
  12 yum
  12 uptime
  12 tail
  12 set
  12 printf
  12 portsnap
  12 grub
  11 zip
  11 sshd
  11 osascript
  11 cmake
  11 cat
  11 as
  10 unzip
  10 tidy
  10 pip3
  10 phpunit
  10 nixos
  10 lockfile
  10 ifconfig
  10 autoconf
   9 x86_64
   9 rclone
   9 protoc
   9 loader
   9 java
   9 id
   9 google
   9 gcov
   9 cc
   8 zsh
   8 zipinfo
   8 X11/Xvfb
   8 uwsgi
   8 true
   8 tac
   8 system
   8 ssh
   8 sccache
   8 pycharm
   8 nuget
   8 modprobe
   8 mapid
   8 locale
   8 iptables
   8 htmldoc
   8 dbus
   8 c
   8 buildifier
   8 apt
   8 aarch64
   7 which
   7 systemd
   7 swapoff
   7 sudo
   7 pip2
   7 nm
   7 myver_
   7 killall
   7 gsed
   7 gets
   7 file_file_g
   7 dot
   7 2to3
   6 xterm
   6 xargs
   6 whois
   6 whoami
   6 validate
   6 update
   6 system_profiler
   6 sparcv9
   6 snmpget
   6 sizer
   6 qmail
   6 pydoc2
   6 pydoc
   6 phpdoc
   6 mysqldump
   6 mapicommd
   6 lpshut
   6 losetup
   6 lilo
   6 keybase
   6 ip6tables
   6 helm
   6 gunzip
   6 gflags_completions
   6 foo
   6 emacs
   6 dpkg
   6 df
   6 biber
   5 xcrun
   5 tr
   5 smail
   5 shorten
   5 security
   5 remsh
   5 rc
   5 psrinfo
   5 pkgutil
   5 pdftotext
   5 oraperl
   5 ntxinfo
   5 mpirun
   5 Mpc
   5 lpd
   5 cp
   5 catdoc
   4 xlockssh
   4 xcodebuild
   4 X11/fvwm2
   4 vifs
   4 touch
   4 tmux
   4 syslogd
   4 swish
   4 strip
   4 speedy
   4 sar
   4 rhash
   4 rcspamd
   4 purge
   4 prtconf
   4 ppp
   4 pp
   4 pdflatex
   4 patchelf
   4 oslevel
   4 openssl
   4 node
   4 nkf
   4 ncftp
   4 mysqld
   4 mrtg
   4 mkfs
   4 loadmap
   4 licensecheck
   4 lcov
   4 jpegtran
   4 insserv
   4 i686
   4 hostinfo
   4 go
   4 ggrep
   4 gdb
   4 ftp
   4 ex
   4 evn
   4 dotnet
   4 dhcpcd
   4 cpp
   4 compress
   4 brew
   4 bazelisk
   3 Xvfb
   3 xdg
   3 wine
   3 vim
   3 terragrunt
   3 swift
   3 svn
   3 supervisord
   3 strace
   3 stat
   3 stap
   3 spamc
   3 safemode
   3 rsync
   3 pwd
   3 pkill
   3 phpenmod
   3 pgrep
   3 pdfinfo
   3 objformat
   3 netpbm
   3 mplayer
   3 md5sum
   3 lpusers
   3 lpsystem
   3 lpstat
   3 lpmove
   3 lpforms
   3 lpfilter
   3 lpadmin
   3 lp
   3 ln
   3 LLVM_PROFILE_FILE
   3 ksh
   3 kodix
   3 kodiremote
   3 kbnm_arm64
   3 kbfsfuse
   3 kakasi
   3 hostname
   3 host
   3 glib
   3 gallery/netpbm
   3 gallery/jhead
   3 gallery
   3 dseditgroup
   3 dladm
   3 curseperl
   3 cmd
   3 clang_tool
   3 chrome
   3 autom4te
   3 autoheader
   3 auto
   3 apparmor_parser
   2 zramctl
   2 ypcat
   2 xz
   2 xxd
   2 Xorg
   2 xmessage
   2 xls2csv
   2 xlock
   2 xinetd
   2 xhost
   2 xclip
   2 xattr
   2 who
   2 waisq
   2 vsdbutil
   2 useradd
   2 usbup
   2 unxz
   2 unlzma
   2 uniq
   2 uncrustify
   2 tsh2eth
   2 transcode
   2 sum
   2 strings
   2 sstty
   2 sparcv9/dtrace
   2 SPAMbaffle
   2 spam
   2 sonar
   2 socat
   2 size
   2 setupAiptekTablet
   2 sensible
   2 script
   2 scp
   2 rwho
   2 rust
   2 run
   2 rssh
   2 rls
   2 ripem
   2 reboot
   2 readelf
   2 ranlib
   2 qmake6
   2 printconf
   2 portserver
   2 pkgconf
   2 ping
   2 perl4
   2 PATH
   2 optipng
   2 open
   2 objdump
   2 objcopy
   2 nodejs
   2 netstat
   2 nano
   2 named
   2 mrtg/get
   2 mount
   2 mold
   2 mlmmj
   2 mktemp
   2 mkpasswd
   2 mkdir
   2 make
   2 lzma
   2 lsb_release
   2 lipo
   2 libbtparse
   2 ldd
   2 lame
   2 kill
   2 kcov
   2 kbnm
   2 ivtv
   2 ipfstat
   2 installer
   2 inetd
   2 igt
   2 identd
   2 iconvconfig
   2 iconv
   2 hexdump
   2 head
   2 hdiutil
   2 ham
   2 gtk
   2 groups
   2 gprof
   2 gpg
   2 gmake
   2 glibtoolize
   2 gfortran
   2 get
   2 gdate
   2 fontforge
   2 fmt
   2 fish
   2 fd
   2 elilo
   2 elfedit
   2 eggs
   2 editor
   2 dwp
   2 dtrace
   2 cygz
   2 cygxslt
   2 cygxml2
   2 cygthai
   2 cygssl
   2 cygpath
   2 cygiconv
   2 cyggpg
   2 cyggcrypt
   2 cygexslt
   2 cygdatrie
   2 cygcrypto
   2 ctest
   2 csh
   2 crosvm
   2 cmp
   2 cmake3
   2 clippy
   2 checkbot
   2 catppt
   2 cargo
   2 bzip2recover
   2 bzip2
   2 bunzip2
   2 build
   2 bindgen
   2 auditwheel
   2 arm
   2 app
   2 apachectl
   2 adjust
   2 addr2line
   1 zunit
   1 xvfb
   1 xmlwf
   1 xlhtml
   1 xkbcomp
   1 wvunpack
   1 wvHtml
   1 wdavdaemon
   1 wc
   1 wavpack
   1 vold
   1 vlc
   1 viz
   1 vimdot
   1 vi
   1 userdbctl
   1 unbound
   1 umount
   1 tt
   1 toast
   1 titrax
   1 tinyproxy
   1 tex
   1 terraform
   1 tclsh
   1 tabnanny
   1 sw_vers
   1 swapon
   1 start
   1 stack
   1 sslyze
   1 sqlite3
   1 sox
   1 sort
   1 shellcheck
   1 shebang
   1 scp2
   1 sam
   1 ruby18
   1 rstartd
   1 rsh
   1 rpmdev
   1 rpm
   1 rm
   1 RETRYEXEC
   1 rar
   1 qmake
   1 qiv
   1 pydoc3
   1 pulumi
   1 pstoimg
   1 ps2pdf
   1 printenv
   1 ppthtml
   1 pprof
   1 postfix
   1 port
   1 policy
   1 phorummail
   1 patch
   1 pargs
   1 pandoc
   1 packer
   1 oggenc
   1 oggdec
   1 ofr
   1 oc
   1 nvim
   1 nu
   1 notify
   1 nawk
   1 mysqlmanager
   1 mv
   1 mt
   1 mppenc
   1 mppdec
   1 mp3info
   1 mkvtoolnix
   1 mkswap
   1 mke2fs
   1 mingetty
   1 mhonarc
   1 melt
   1 mc
   1 mathspic113
   1 macpipe
   1 lupdate6
   1 lupdate
   1 lrelease6
   1 lrelease
   1 link
   1 lesspipe
   1 kubelet
   1 klist
   1 kdenlive_render
   1 jping
   1 JAVAVM_LD_PRELOAD
   1 isainfo
   1 ipconfig
   1 ioscan
   1 ioreg
   1 installbsd
   1 inkscape
   1 in
   1 import
   1 idle3
   1 identify
   1 icewm
   1 i386
   1 hypermail
   1 hub
   1 html2ps
   1 homectl
   1 HEAD
   1 groupadd
   1 gnome
   1 glaxnimate
   1 gimp
   1 ghc
   1 getpcaps
   1 geolite
   1 genso
   1 gedit
   1 gdiff3
   1 fquot
   1 fortune
   1 formail
   1 fluxbox
   1 flac
   1 firefox
   1 ffmpeg
   1 fdfind
   1 fdesetup
   1 exiv2
   1 eximstats
   1 eu
   1 epiphany
   1 entrypoint
   1 enscript
   1 ed
   1 easy_install
   1 dvips
   1 doxygen
   1 docsetutil
   1 dirname
   1 cygfontconfig
   1 create
   1 crash_reporter
   1 cpupower
   1 cpack
   1 config3270
   1 cmucl
   1 chromedriver
   1 chgrp
   1 chfn
   1 busybox
   1 buildapp
   1 bsdtar
   1 bindprocessor
   1 bicep
   1 beep
   1 because
   1 bc
   1 basename
   1 bar
   1 azcopy
   1 aterms
   1 appimagetool
   1 antiword
   1 aliyun
   1 agetty
   1 a2ps
   1 a2enmod
   1 a2enconf

Ran the following commands to get an idea of what kinds of programs are being used in cgi scripts. Note that this is not a complete overview, there are likely many instances of dynamic concatenation of prefix + program as well as just running a program from `$PATH`. There are likely also false positives, either stuff from comments, from urls (e.g. `https://example.com//opt/bin/something`), or whatever else. (however I dropped `/bin` to avoid too many url mismatches). ```bash find \ /home/pvv/?/* \ -type f \ $ \ -name '*.cgi' \ -o -name '*.php' \ -o -name '*.perl' \ -o -name '*.pl' \ -o -name '*.sh' \ $ \ -exec rg '(/store/bin|/store/gnu/bin|/usr/bin|/usr/local/bin|/local/bin|/opt/bin|/opt/local/bin|/sbin|/usr/sbin|/usr/local/sbin|/opt/sbin|/var/qmail/bin|/usr/drwho/local/bin|/usr/local/mlog3/bin|/usr/local/flite/bin|/opt/flite/bin|/opt/csw/bin|/sw/bin|/usr/sfw/bin|/perl/bin|/usr/local/php/bin|/usr/X11R6/bin)/(?:env\s*)?(\w+(?:/\w+)*)' {} -N -o -r '$1' \; \ 2>/dev/null | tee cgi-progs.txt cat cgi-progs.list | sort | uniq -c | sort -gr ``` We get the following list: <details> <summary>Big list</summary> ``` 8421 perl 5375 bash 2286 nix 250 gcc 242 systemctl 238 install 221 bazel 204 sendmail 202 jhead 174 ruby1 170 clang 161 sh 148 file 138 localedef 98 python 97 pkg 97 php 94 chkconfig 84 devcontainer 84 code 79 python3 77 initctl 59 date 56 convert 51 python2 50 getconf 44 grep 44 docker 43 salt 42 time 42 curl 38 env 37 sed 35 ldconfig 34 ruby 34 ccache 31 uname 28 pip 28 g 28 composer 26 perl5 26 less 25 mailx 24 wget 24 tar 24 paxctl 23 X11 23 valgrind 21 mail 20 more 20 kasm_server 20 exiftags 19 pkg2ng 19 meson 18 lndir 18 finger 18 diskutil 18 chmod 17 test 17 php5 17 llvm 17 ld 17 gnuplot 17 find 17 diff 17 chacl 16 timeout 16 procmail 16 minikube 16 kubectl 16 gzip 16 awk 15 yacc 15 spamd 15 sleep 15 service 14 qemu 14 passwd 14 md5 14 m4 14 ls 14 kstat 14 installkernel 14 init 14 git 14 dscl 14 aspell 14 ar 13 sysctl 13 nologin 13 nohup 13 ninja 13 lynx 13 diff3 13 chown 13 arch 12 zipnote 12 yum 12 uptime 12 tail 12 set 12 printf 12 portsnap 12 grub 11 zip 11 sshd 11 osascript 11 cmake 11 cat 11 as 10 unzip 10 tidy 10 pip3 10 phpunit 10 nixos 10 lockfile 10 ifconfig 10 autoconf 9 x86_64 9 rclone 9 protoc 9 loader 9 java 9 id 9 google 9 gcov 9 cc 8 zsh 8 zipinfo 8 X11/Xvfb 8 uwsgi 8 true 8 tac 8 system 8 ssh 8 sccache 8 pycharm 8 nuget 8 modprobe 8 mapid 8 locale 8 iptables 8 htmldoc 8 dbus 8 c 8 buildifier 8 apt 8 aarch64 7 which 7 systemd 7 swapoff 7 sudo 7 pip2 7 nm 7 myver_ 7 killall 7 gsed 7 gets 7 file_file_g 7 dot 7 2to3 6 xterm 6 xargs 6 whois 6 whoami 6 validate 6 update 6 system_profiler 6 sparcv9 6 snmpget 6 sizer 6 qmail 6 pydoc2 6 pydoc 6 phpdoc 6 mysqldump 6 mapicommd 6 lpshut 6 losetup 6 lilo 6 keybase 6 ip6tables 6 helm 6 gunzip 6 gflags_completions 6 foo 6 emacs 6 dpkg 6 df 6 biber 5 xcrun 5 tr 5 smail 5 shorten 5 security 5 remsh 5 rc 5 psrinfo 5 pkgutil 5 pdftotext 5 oraperl 5 ntxinfo 5 mpirun 5 Mpc 5 lpd 5 cp 5 catdoc 4 xlockssh 4 xcodebuild 4 X11/fvwm2 4 vifs 4 touch 4 tmux 4 syslogd 4 swish 4 strip 4 speedy 4 sar 4 rhash 4 rcspamd 4 purge 4 prtconf 4 ppp 4 pp 4 pdflatex 4 patchelf 4 oslevel 4 openssl 4 node 4 nkf 4 ncftp 4 mysqld 4 mrtg 4 mkfs 4 loadmap 4 licensecheck 4 lcov 4 jpegtran 4 insserv 4 i686 4 hostinfo 4 go 4 ggrep 4 gdb 4 ftp 4 ex 4 evn 4 dotnet 4 dhcpcd 4 cpp 4 compress 4 brew 4 bazelisk 3 Xvfb 3 xdg 3 wine 3 vim 3 terragrunt 3 swift 3 svn 3 supervisord 3 strace 3 stat 3 stap 3 spamc 3 safemode 3 rsync 3 pwd 3 pkill 3 phpenmod 3 pgrep 3 pdfinfo 3 objformat 3 netpbm 3 mplayer 3 md5sum 3 lpusers 3 lpsystem 3 lpstat 3 lpmove 3 lpforms 3 lpfilter 3 lpadmin 3 lp 3 ln 3 LLVM_PROFILE_FILE 3 ksh 3 kodix 3 kodiremote 3 kbnm_arm64 3 kbfsfuse 3 kakasi 3 hostname 3 host 3 glib 3 gallery/netpbm 3 gallery/jhead 3 gallery 3 dseditgroup 3 dladm 3 curseperl 3 cmd 3 clang_tool 3 chrome 3 autom4te 3 autoheader 3 auto 3 apparmor_parser 2 zramctl 2 ypcat 2 xz 2 xxd 2 Xorg 2 xmessage 2 xls2csv 2 xlock 2 xinetd 2 xhost 2 xclip 2 xattr 2 who 2 waisq 2 vsdbutil 2 useradd 2 usbup 2 unxz 2 unlzma 2 uniq 2 uncrustify 2 tsh2eth 2 transcode 2 sum 2 strings 2 sstty 2 sparcv9/dtrace 2 SPAMbaffle 2 spam 2 sonar 2 socat 2 size 2 setupAiptekTablet 2 sensible 2 script 2 scp 2 rwho 2 rust 2 run 2 rssh 2 rls 2 ripem 2 reboot 2 readelf 2 ranlib 2 qmake6 2 printconf 2 portserver 2 pkgconf 2 ping 2 perl4 2 PATH 2 optipng 2 open 2 objdump 2 objcopy 2 nodejs 2 netstat 2 nano 2 named 2 mrtg/get 2 mount 2 mold 2 mlmmj 2 mktemp 2 mkpasswd 2 mkdir 2 make 2 lzma 2 lsb_release 2 lipo 2 libbtparse 2 ldd 2 lame 2 kill 2 kcov 2 kbnm 2 ivtv 2 ipfstat 2 installer 2 inetd 2 igt 2 identd 2 iconvconfig 2 iconv 2 hexdump 2 head 2 hdiutil 2 ham 2 gtk 2 groups 2 gprof 2 gpg 2 gmake 2 glibtoolize 2 gfortran 2 get 2 gdate 2 fontforge 2 fmt 2 fish 2 fd 2 elilo 2 elfedit 2 eggs 2 editor 2 dwp 2 dtrace 2 cygz 2 cygxslt 2 cygxml2 2 cygthai 2 cygssl 2 cygpath 2 cygiconv 2 cyggpg 2 cyggcrypt 2 cygexslt 2 cygdatrie 2 cygcrypto 2 ctest 2 csh 2 crosvm 2 cmp 2 cmake3 2 clippy 2 checkbot 2 catppt 2 cargo 2 bzip2recover 2 bzip2 2 bunzip2 2 build 2 bindgen 2 auditwheel 2 arm 2 app 2 apachectl 2 adjust 2 addr2line 1 zunit 1 xvfb 1 xmlwf 1 xlhtml 1 xkbcomp 1 wvunpack 1 wvHtml 1 wdavdaemon 1 wc 1 wavpack 1 vold 1 vlc 1 viz 1 vimdot 1 vi 1 userdbctl 1 unbound 1 umount 1 tt 1 toast 1 titrax 1 tinyproxy 1 tex 1 terraform 1 tclsh 1 tabnanny 1 sw_vers 1 swapon 1 start 1 stack 1 sslyze 1 sqlite3 1 sox 1 sort 1 shellcheck 1 shebang 1 scp2 1 sam 1 ruby18 1 rstartd 1 rsh 1 rpmdev 1 rpm 1 rm 1 RETRYEXEC 1 rar 1 qmake 1 qiv 1 pydoc3 1 pulumi 1 pstoimg 1 ps2pdf 1 printenv 1 ppthtml 1 pprof 1 postfix 1 port 1 policy 1 phorummail 1 patch 1 pargs 1 pandoc 1 packer 1 oggenc 1 oggdec 1 ofr 1 oc 1 nvim 1 nu 1 notify 1 nawk 1 mysqlmanager 1 mv 1 mt 1 mppenc 1 mppdec 1 mp3info 1 mkvtoolnix 1 mkswap 1 mke2fs 1 mingetty 1 mhonarc 1 melt 1 mc 1 mathspic113 1 macpipe 1 lupdate6 1 lupdate 1 lrelease6 1 lrelease 1 link 1 lesspipe 1 kubelet 1 klist 1 kdenlive_render 1 jping 1 JAVAVM_LD_PRELOAD 1 isainfo 1 ipconfig 1 ioscan 1 ioreg 1 installbsd 1 inkscape 1 in 1 import 1 idle3 1 identify 1 icewm 1 i386 1 hypermail 1 hub 1 html2ps 1 homectl 1 HEAD 1 groupadd 1 gnome 1 glaxnimate 1 gimp 1 ghc 1 getpcaps 1 geolite 1 genso 1 gedit 1 gdiff3 1 fquot 1 fortune 1 formail 1 fluxbox 1 flac 1 firefox 1 ffmpeg 1 fdfind 1 fdesetup 1 exiv2 1 eximstats 1 eu 1 epiphany 1 entrypoint 1 enscript 1 ed 1 easy_install 1 dvips 1 doxygen 1 docsetutil 1 dirname 1 cygfontconfig 1 create 1 crash_reporter 1 cpupower 1 cpack 1 config3270 1 cmucl 1 chromedriver 1 chgrp 1 chfn 1 busybox 1 buildapp 1 bsdtar 1 bindprocessor 1 bicep 1 beep 1 because 1 bc 1 basename 1 bar 1 azcopy 1 aterms 1 appimagetool 1 antiword 1 aliyun 1 agetty 1 a2ps 1 a2enmod 1 a2enconf ``` </details>

oysteikt commented

2026-01-31 07:55:13 +01:00

Maybe it would be nice to use the chance to generate a plaintext list of all binaries present, as well as perl, php and python libraries?

oysteikt commented

2026-01-31 11:18:46 +01:00

There is now an initial version up and running at https://temmie.pvv.ntnu.no.

Remaining tasks are:

Performant rendering

Set up phpfpm for more efficient php cgi
Set up fastcgi for more efficient perl cgi
Set up fastcgi? for more efficient python rendering

Fill out missing packages

Fill out more expected python packages
Fill out more expected perl packages
Fill out more expected php packages

Other (somewhat mandatory imo tbh)

Enable suEXEC (do we have this on tom???)
- dynamically generate users with equal usernames as those in salt (they do not need to have the same id, maybe we can utilize tools like userborn or userdbctl, and reserve a range of UIDs for this? Alternatively dynamically allocate users in the PreExecStart of httpd with a private user namespace, and share said namespace with the rendering units).
Run a test querying most websites to see how much we broke (and alternatively how much we fixed).
Install or rewrite the salt/web/apache-logger script to add logs back into people's home directories (maybe don't activate it before tom is dead)
Set up postfix for the sendmail binary
- postfix might have to live in the same user namespace as httpd
Set up logging and metrics and possibly alerts for all sendmailing that is being done from this postfix instance. (I do not trust that none of the hundreds of websites have a buggy websites that lets outsiders get unfettered access to sendmail, this could absolutely tank our mail server reputation)

Optional or future plans

Generate list of packages and document it somewhere
See if we can make the rendering engines sandbox themselves to home directories during script execution.
Do we dare to install python2? It's still available in nixpkgs, maybe if we sandbox it very well?
Install mod_lisp for fun and profit (very optional).
Socket activation and journald adapter whenever apache v2.5 drops
Install a nicer looking file index

There is now an initial version up and running at https://temmie.pvv.ntnu.no. Remaining tasks are: ### Performant rendering - [ ] Set up phpfpm for more efficient php cgi - [ ] Set up fastcgi for more efficient perl cgi - [ ] Set up fastcgi? for more efficient python rendering ### Fill out missing packages - [ ] Fill out more expected python packages - [ ] Fill out more expected perl packages - [ ] Fill out more expected php packages ### Other (somewhat mandatory imo tbh) - [ ] Enable suEXEC (do we have this on tom???) - [ ] dynamically generate users with equal usernames as those in salt (they do not need to have the same id, maybe we can utilize tools like userborn or userdbctl, and reserve a range of UIDs for this? Alternatively dynamically allocate users in the `PreExecStart` of httpd with a private user namespace, and share said namespace with the rendering units). - [ ] Run a test querying most websites to see how much we broke (and alternatively how much we fixed). - [ ] Install or rewrite the `salt/web/apache-logger` script to add logs back into people's home directories (maybe don't activate it before tom is dead) - [ ] Set up postfix for the `sendmail` binary - [ ] postfix might have to live in the same user namespace as httpd - [ ] Set up logging and metrics and possibly alerts for all `sendmail`ing that is being done from this postfix instance. (I do not trust that none of the hundreds of websites have a buggy websites that lets outsiders get unfettered access to `sendmail`, this could absolutely tank our mail server reputation) ### Optional or future plans - [ ] Generate list of packages and document it somewhere - [ ] See if we can make the rendering engines sandbox themselves to home directories during script execution. - [ ] Do we dare to install python2? It's still available in nixpkgs, maybe if we sandbox it very well? - [ ] Install `mod_lisp` for fun and profit (very optional). - [ ] Socket activation and journald adapter whenever apache v2.5 drops - [ ] Install a nicer looking file index

oysteikt referenced this issue

2026-01-31 12:56:53 +01:00

Fix cron warnings for tom #150

oysteikt commented

2026-02-06 08:49:49 +01:00

We should probably consider setting up a path filter for directories such as RCS/.git/.svn/.hg, etc

We should probably consider setting up a path filter for directories such as `RCS`/`.git`/`.svn`/`.hg`, etc