Drift/pvv-nixos-config
17
5
Fork 1
Code Issues Pull Requests 6 Actions 1 Wiki Activity

fix-openstack-networking #47

Merged
felixalb merged 5 commits from fix-openstack-networking into main 2024-07-31 11:23:00 +02:00
Conversation 2 Commits 5 Files Changed 2 +41 -4
felixalb commented 2024-07-28 20:39:29 +02:00
Owner
Copy Link

Ildkule has been acting up a lot, and the network behavior has been especially unpredictable. With the "default" configuration on all devices, we have usually been able to access the host, however it has had several default routes, several source-addresses, etc.
With these changes, we specify what network interfaces should be world routable, and what interfaces should be local only.

TODO:

  • Remove the Network-ntnu-internal(192.168.11.0/24) default routes from Ildkule
  • Fix the invalid Gateway option on ildkule
  • Make the configuration generic for more hosts
    • Move to values.nix or base.nix
    • Remove hard-coded addresses and comments
  • Add support for IPv6
  • Test on more hosts (bob?)
  • Route 129.241.0.0/16 through Network-ntnu-internal instead of Network-ntnu-global
    • This makes OpenStack firewall rules more logical, as 10.0.0.0/16 is effectively equivalent to 129.241.0.0/16 at NTNU.
    • ... but makes the source-address more unpredictable again, from the perspective of our non-openstack hosts (e.g. NFS-permissions from microbel)
Ildkule has been acting up a lot, and the network behavior has been especially unpredictable. With the "default" configuration on all devices, we have usually been able to access the host, however it has had several default routes, several source-addresses, etc. With these changes, we specify what network interfaces should be world routable, and what interfaces should be local only. TODO: - [x] Remove the Network-ntnu-internal(192.168.11.0/24) default routes from Ildkule - [x] Fix the invalid Gateway option on ildkule - [x] Make the configuration generic for more hosts - [x] Move to values.nix ~~or base.nix~~ - [x] Remove hard-coded addresses and comments - [x] Add support for IPv6 - [ ] Test on more hosts (bob?) - [ ] ~~Route 129.241.0.0/16 through Network-ntnu-internal instead of Network-ntnu-global~~ - This makes OpenStack firewall rules more logical, as 10.0.0.0/16 is effectively equivalent to 129.241.0.0/16 at NTNU. - **... but makes the source-address more unpredictable again, from the perspective of our non-openstack hosts (e.g. NFS-permissions from microbel)**
felixalb commented 2024-07-28 20:39:55 +02:00
Author
Owner
Copy Link

ip route now returns the following, which is much better than before:

default via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024
10.0.0.0/8 dev ens3 proto static scope link
169.254.169.254 via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024
192.168.11.0/24 dev ens3 proto kernel scope link src 192.168.11.133 metric 1024
192.168.11.2 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024
192.168.11.3 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024
192.168.12.0/24 dev ens4 proto kernel scope link src 192.168.12.209 metric 1024
192.168.12.1 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
192.168.12.2 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
192.168.12.3 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
`ip route` now returns the following, which is much better than before: ``` default via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024 10.0.0.0/8 dev ens3 proto static scope link 169.254.169.254 via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024 192.168.11.0/24 dev ens3 proto kernel scope link src 192.168.11.133 metric 1024 192.168.11.2 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024 192.168.11.3 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024 192.168.12.0/24 dev ens4 proto kernel scope link src 192.168.12.209 metric 1024 192.168.12.1 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 192.168.12.2 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 192.168.12.3 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 ```
felixalb changed title from WIP: fix-openstack-networking to fix-openstack-networking 2024-07-29 15:12:55 +02:00
felixalb force-pushed fix-openstack-networking from f1d3dcad84 to 4b3d3709b5 2024-07-29 15:13:26 +02:00 Compare
felixalb added 1 commit 2024-07-29 15:23:00 +02:00
Openstack: fix broken ntnu-internal route 77fa789756
felixalb commented 2024-07-29 15:27:48 +02:00
Author
Owner
Copy Link

I thiiiiink that this is working as intended now. If not, it is at least way better than the old configuration.
Ildkule after rebuilding and rebooting on this branch:

felixalb@ildkule:~/ > ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:ee:81:2d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 10.212.25.209/32 scope global ens3
       valid_lft forever preferred_lft forever
    inet 192.168.11.133/24 metric 1024 brd 192.168.11.255 scope global dynamic ens3
       valid_lft 86324sec preferred_lft 86324sec
    inet6 2001:700:300:6025:f816:3eff:feee:812d/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feee:812d/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:58:f1:e8 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    inet 129.241.153.213/32 scope global ens4
       valid_lft forever preferred_lft forever
    inet 192.168.12.209/24 metric 1024 brd 192.168.12.255 scope global dynamic ens4
       valid_lft 86324sec preferred_lft 86324sec
    inet6 2001:700:300:6026:f816:3eff:fe58:f1e8/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe58:f1e8/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
felixalb@ildkule:~/ > ip r
default via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024
10.0.0.0/8 via 192.168.11.1 dev ens3 proto dhcp metric 1024
169.254.169.254 via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024
192.168.11.0/24 dev ens3 proto kernel scope link src 192.168.11.133 metric 1024
192.168.11.1 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024
192.168.11.2 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024
192.168.11.3 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024
192.168.12.0/24 dev ens4 proto kernel scope link src 192.168.12.209 metric 1024
192.168.12.1 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
192.168.12.2 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
192.168.12.3 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024
felixalb@ildkule:~/ > curl -4 ip.sb
129.241.153.213
felixalb@ildkule:~/ > curl -6 ip.sb
2001:700:300:6026:f816:3eff:fe58:f1e8

Also, after adjusting the Openstack firewall rules; all permutations of SSH and Ping over IPv4 and IPv6 works great.
IMO, this is ready to squash and merge :)

I thiiiiink that this is working as intended now. If not, it is at least way better than the old configuration. Ildkule after rebuilding and rebooting on this branch: ``` felixalb@ildkule:~/ > ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether fa:16:3e:ee:81:2d brd ff:ff:ff:ff:ff:ff altname enp0s3 inet 10.212.25.209/32 scope global ens3 valid_lft forever preferred_lft forever inet 192.168.11.133/24 metric 1024 brd 192.168.11.255 scope global dynamic ens3 valid_lft 86324sec preferred_lft 86324sec inet6 2001:700:300:6025:f816:3eff:feee:812d/128 scope global valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:feee:812d/64 scope link proto kernel_ll valid_lft forever preferred_lft forever 3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether fa:16:3e:58:f1:e8 brd ff:ff:ff:ff:ff:ff altname enp0s4 inet 129.241.153.213/32 scope global ens4 valid_lft forever preferred_lft forever inet 192.168.12.209/24 metric 1024 brd 192.168.12.255 scope global dynamic ens4 valid_lft 86324sec preferred_lft 86324sec inet6 2001:700:300:6026:f816:3eff:fe58:f1e8/128 scope global valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe58:f1e8/64 scope link proto kernel_ll valid_lft forever preferred_lft forever felixalb@ildkule:~/ > ip r default via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024 10.0.0.0/8 via 192.168.11.1 dev ens3 proto dhcp metric 1024 169.254.169.254 via 192.168.12.1 dev ens4 proto dhcp src 192.168.12.209 metric 1024 192.168.11.0/24 dev ens3 proto kernel scope link src 192.168.11.133 metric 1024 192.168.11.1 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024 192.168.11.2 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024 192.168.11.3 dev ens3 proto dhcp scope link src 192.168.11.133 metric 1024 192.168.12.0/24 dev ens4 proto kernel scope link src 192.168.12.209 metric 1024 192.168.12.1 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 192.168.12.2 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 192.168.12.3 dev ens4 proto dhcp scope link src 192.168.12.209 metric 1024 felixalb@ildkule:~/ > curl -4 ip.sb 129.241.153.213 felixalb@ildkule:~/ > curl -6 ip.sb 2001:700:300:6026:f816:3eff:fe58:f1e8 ``` Also, after adjusting the Openstack firewall rules; all permutations of SSH and Ping over IPv4 and IPv6 works great. IMO, this is ready to squash and merge :)
felixalb merged commit 2030d4de39 into main 2024-07-31 11:23:00 +02:00
felixalb deleted branch fix-openstack-networking 2024-07-31 11:23:00 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
art
The issue needs someone to be creative
 backup
big
No, not bug, **big** - this is gonna take a while
 blocked
This issue/PR depends on one or more other issues/PRs
 bug
Something is not working, and it's not the shield hero
 crash report
Report an oopsie
 disputed
kranglefanter
 documentation
Documentation changes required
 duplicate
This issue or pull request already exists
 enhancement
New feature
 good first issue
Get your hands dirty with a new project here
 logging
networking
TTM4100
 nixos
Requires changes to our nixos setup
 question
More information is needed
 salt
Requires changes to our salt setup
 security
Skommel
 servers n' hardware
Regards hardware, servers or VMs
 wontfix
This won't be fixed
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
adriangl albertba alfhj amalieem bjornoka chrisfjo chrivi danio davidk dungby eirikwit felixalb frero jonmro jovre karolds knuta krisleri larshhan oysteikt pederbs root torsteno yorinad
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Drift/pvv-nixos-config#47
Delete Branch "fix-openstack-networking"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?