Fix salt dependencies, such as jinja2 (Isvegg++) #124

New Issue

felixalb · 2024-08-16T15:36:22+02:00

felixalb commented

2024-08-16 15:36:22 +02:00

salt-common and salt-minion depends on a lot of python libraries, with poor version management.
This has previously broken several hosts, such as eirin and demiurgen, in passing, and now breaks other hosts, such as isvegg and orchid.

isvegg:~# salt-call state.highstate
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 33, in <module>
    sys.exit(load_entry_point('salt==3002.6', 'console_scripts', 'salt-call')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 437, in salt_call
    import salt.cli.call
  File "/usr/lib/python3/dist-packages/salt/cli/call.py", line 6, in <module>
    import salt.cli.caller
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 15, in <module>
    import salt.minion
  File "/usr/lib/python3/dist-packages/salt/minion.py", line 38, in <module>
    import salt.pillar
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 16, in <module>
    import salt.fileclient
  File "/usr/lib/python3/dist-packages/salt/fileclient.py", line 29, in <module>
    import salt.utils.templates
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 20, in <module>
    import salt.utils.jinja
  File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 28, in <module>
    from jinja2 import BaseLoader, Markup, TemplateNotFound, nodes
ImportError: cannot import name 'Markup' from 'jinja2' (/usr/lib/python3/dist-packages/jinja2/__init__.py)

We could replace salt-minion with a homebrew one, for example bundling specific library versions in a pip virtual environment, rather than relying on compatible libraries in the debian repositories.

Since we already have some custom packages, something like a debian respository with all custom packages would probably be smart.

`salt-common` and `salt-minion` depends on a lot of python libraries, with poor version management. This has previously broken several hosts, such as `eirin` and `demiurgen`, in passing, and now breaks other hosts, such as `isvegg` and `orchid`. ``` isvegg:~# salt-call state.highstate Traceback (most recent call last): File "/usr/bin/salt-call", line 33, in <module> sys.exit(load_entry_point('salt==3002.6', 'console_scripts', 'salt-call')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/salt/scripts.py", line 437, in salt_call import salt.cli.call File "/usr/lib/python3/dist-packages/salt/cli/call.py", line 6, in <module> import salt.cli.caller File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 15, in <module> import salt.minion File "/usr/lib/python3/dist-packages/salt/minion.py", line 38, in <module> import salt.pillar File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 16, in <module> import salt.fileclient File "/usr/lib/python3/dist-packages/salt/fileclient.py", line 29, in <module> import salt.utils.templates File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 20, in <module> import salt.utils.jinja File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 28, in <module> from jinja2 import BaseLoader, Markup, TemplateNotFound, nodes ImportError: cannot import name 'Markup' from 'jinja2' (/usr/lib/python3/dist-packages/jinja2/__init__.py) ``` We could replace salt-minion with a homebrew one, for example bundling specific library versions in a pip virtual environment, rather than relying on compatible libraries in the debian repositories. Since we already have some custom packages, something like a debian respository with all custom packages would probably be smart.

felixalb added the

software

salt

bug

labels 2024-08-16 15:36:22 +02:00

oysteikt added this to the Kanban project 2024-08-16 15:51:37 +02:00

pederbs commented

2024-08-18 03:03:05 +02:00

╰( ͡° ͜ʖ ͡° )つ──☆*:・ﾟ sudo pip install salt

╰( ͡° ͜ʖ ͡° )つ──☆*:・ﾟ `sudo pip install salt`

😕 1

oysteikt commented

2024-08-26 16:40:44 +02:00

Just for the record, the failing packages are the packages from the upstream apt repositories?

Just for the record, the failing packages are the packages from the [upstream apt repositories](https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-by-operating-system/ubuntu.html)?

felixalb commented

2024-08-26 20:20:11 +02:00

Just for the record, the failing packages are the packages from the upstream apt repositories?

hildring:~# apt-cache madison salt-common
salt-common | 3004.2+ds-1 | https://repo.saltproject.io/py3/debian/10/amd64/3004 buster/main amd64 Packages

yes.

> Just for the record, the failing packages are the packages from the [upstream apt repositories](https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-by-operating-system/ubuntu.html)? ``` hildring:~# apt-cache madison salt-common salt-common | 3004.2+ds-1 | https://repo.saltproject.io/py3/debian/10/amd64/3004 buster/main amd64 Packages ``` yes.

oysteikt referenced this issue

2024-12-08 00:52:00 +01:00

Selfhost some debs via gitea #171

oysteikt commented

2024-12-10 16:03:14 +01:00

The new packages from packages.broadcom.com (ref https://saltproject.io/blog/salt-project-package-repo-migration-and-guidance/) are packaged differently than the old upstream ones. For better or for worse, they bundle all necessary deps and put the entire thing in /opt/saltstack. If we point our machines there, everything should probably just work.

When I say they bundle all deps, it goes quite deep. A ton of precompiled shared libs like libkrb5 and liblzma (no openssl though), as well as a vendored python instance.

The new packages from packages.broadcom.com (ref https://saltproject.io/blog/salt-project-package-repo-migration-and-guidance/) are packaged differently than the old upstream ones. For better or for worse, they bundle all necessary deps and put the entire thing in `/opt/saltstack`. If we point our machines there, everything should *probably* just work. When I say they bundle *all deps*, it goes quite deep. A ton of precompiled shared libs like `libkrb5` and `liblzma` (no openssl though), as well as a vendored python instance.

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Drift/issues#124