ISTI-provisioning
/
openstack-infrastructure-terraform
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updates to s2i2s/project-setup/README.md

This commit is contained in:
Andrea Dell'Amico 2026-08-05 18:32:18 +02:00
parent 2d47dab116
commit 2eb8ede3aa
Signed by: adellam
GPG Key ID: 147ABE6CEB9E20FF
1 changed files with 92 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
s2i2s/project-setup/README.md
View File

@ -1,6 +1,6 @@
# S2I2S Project Setup
This Terraform configuration sets up the core infrastructure components for the S2I2S OpenStack project.
This OpenTofu configuration sets up the core infrastructure components for the S2I2S OpenStack project.
## Overview
@ -8,42 +8,84 @@ The project-setup module creates the following resources:
### Virtual Machines
| VM | Purpose | Flavor | Floating IP | DNS Record |
| --- | --- | --- | --- | --- |
| SSH Jump Proxy | Secure SSH gateway for accessing internal VMs | m1.small | Yes | `ssh-jump.s2i2s.cloud.isti.cnr.it` |
| Internal CA | Certificate Authority for internal services | m1.small | No | - |
| HAProxy L7 (x2) | Layer 7 load balancers behind Octavia | m1.medium | No | - |
| Prometheus | Monitoring and metrics collection | m1.medium | Yes | `prometheus.s2i2s.cloud.isti.cnr.it` |
| VM | Purpose | Flavor | Private IP | Boot disk | Floating IP | DNS Record |
| --- | --- | --- | --- | --- | --- | --- |
| SSH Jump Proxy | Secure SSH gateway for accessing internal VMs | m2.small | 10.10.0.5 | 30 GB | Yes | `ssh-jump-proxy.s2i2s.cloud.isti.cnr.it` |
| Internal CA | Certificate Authority for internal services | m1.small | 10.10.0.4 | 10 GB | No | - |
| HAProxy L7 (x2) | Layer 7 load balancers behind Octavia | m1.medium | 10.10.0.11, 10.10.0.12 | 10 GB | No | - |
| Prometheus | Monitoring and metrics collection | m1.medium | 10.10.0.10 | 10 GB | Yes | `prometheus.s2i2s.cloud.isti.cnr.it`, `alertmanager.s2i2s.cloud.isti.cnr.it` (CNAME) |
All VMs run Ubuntu 24.04 and are provisioned with the standard cloud-init user data script.
All VMs run **Ubuntu 24.04** (`Ubuntu-Noble-24.04.img`) and are provisioned with the standard `ubuntu2404.sh` cloud-init user data script. Boot disks are volume-backed with `delete_on_termination = false`.
### Load Balancer
An OVN-based Octavia load balancer (`s2i2s-cloud-l4-load-balancer`) provides L4 load balancing:
- **Provider**: `ovn` (not amphora)
- **VIP**: 10.10.0.20, on the main private subnet
- **Floating IP**: Yes
- **DNS Record**: `octavia-main-lb.s2i2s.cloud.isti.cnr.it`
- **Backend**: HAProxy L7 instances (anti-affinity for HA)
| Listener | Port | Protocol | Health Check |
| --- | --- | --- | --- |
| HTTP | 80 | TCP | HTTP GET `/_haproxy_health_check` |
| HTTPS | 443 | TCP | HTTPS GET `/_haproxy_health_check` |
| Stats | 8880 | TCP | TCP connect |
| Listener | Port | Protocol | Pool method | Health Check |
| --- | --- | --- | --- | --- |
| HTTP | 80 | TCP | SOURCE_IP_PORT | TCP connect |
| HTTPS | 443 | TCP | SOURCE_IP_PORT | TCP connect |
| Stats | 8880 | TCP | SOURCE_IP_PORT | TCP connect |
#### Consequences of the OVN provider
The OVN driver is lighter than amphora (no amphora VMs, uses the main subnet directly), but it
constrains the configuration in ways that are visible throughout this setup:
- **No `allowed_cidrs` on listeners.** Access restrictions must be enforced by security groups
or inside HAProxy itself. This is why the stats port is filtered in the security group below.
- **Only `ROUND_ROBIN` and `SOURCE_IP_PORT`** are available as pool methods; `LEAST_CONNECTIONS`
is not supported.
- **Only `TCP` and `UDP-CONNECT` health monitors**; HTTP/HTTPS monitors are not supported, so all
three pools are probed with a plain TCP connect.
- **The client source IP is preserved.** OVN does not SNAT the traffic to an amphora VIP, so the
HAProxy L7 backends see the real client address rather than an address of the load balancer.
Only health-monitor probes and hairpinned traffic originate from the private subnet.
### Security Groups
| Security Group | Purpose | Use On |
| --- | --- | --- |
| `s2i2s-default-sg` | Default rules: SSH via jump proxy, ICMP, Prometheus node exporter | All VMs |
| `default_for_all` | Default rules: SSH via jump proxy, ICMP, Prometheus node exporter | All VMs |
| `ssh_access_to_the_jump_node` | SSH access from VPN endpoints | SSH Jump Proxy only |
| `debugging_from_jump_node` | Web debugging via SSH tunnels (ports 80, 443, 8100) | VMs needing debug access |
| `traffic_from_the_main_load_balancers` | HTTP/HTTPS from HAProxy L7 (ports 80, 443, 8080, 8888) | Backend web services |
| `traffic_from_main_lb_to_haproxy_l7` | Traffic from Octavia LB to HAProxy | HAProxy L7 VMs |
| `traffic_from_the_main_load_balancers` | HTTP/HTTPS from the HAProxy L7 IPs (ports 80, 443, 8080, 8888) | Backend web services |
| `traffic_from_main_lb_to_haproxy_l7` | Public HTTP/HTTPS plus filtered access to the stats port | HAProxy L7 VMs |
| `public_web_service` | HTTP/HTTPS from anywhere | Public-facing services with floating IP |
| `restricted_web_service` | HTTP from anywhere, HTTPS from VPNs only | Restricted services with floating IP |
| `prometheus_access_from_grafana` | HTTPS access from public Grafana server | Prometheus VM |
> **Do not confuse the two load balancer groups.** `traffic_from_main_lb_to_haproxy_l7` sits on the
> HAProxy L7 VMs themselves and faces the internet. `traffic_from_the_main_load_balancers` sits on
> the backend services *behind* HAProxy: since HAProxy is an L7 proxy that opens new connections
> from its own address, the client IP is **not** preserved on that hop, so its sources stay
> restricted to 10.10.0.11 and 10.10.0.12 and it must never be opened to `0.0.0.0/0`.
#### `traffic_from_main_lb_to_haproxy_l7`
Because the OVN load balancer preserves the client source IP (see above), the public listeners on
the HAProxy L7 VMs cannot be restricted to the private subnet:
| Port | Source | Rationale |
| --- | --- | --- |
| 80 | `0.0.0.0/0` | HTTP arrives with the original client address |
| 443 | `0.0.0.0/0` | HTTPS arrives with the original client address |
| 8880 | main private subnet | Health monitor probes for the stats pool |
| 8880 | D4S VPN 1/2, S2I2S VPN 1/2, InfraScience network | Human access to the HAProxy stats page |
| 8880 | 10.10.0.10/32 and the Prometheus floating IP | Prometheus scrapes the stats, over the private network and through the LB public IP |
| 10000 | 10.10.0.11/32, 10.10.0.12/32 | Peer traffic between the two HAProxy instances |
The stats port is deliberately **not** open to `0.0.0.0/0`: since the OVN provider does not support
`allowed_cidrs` on the listener, this security group is the only place where port 8880 can be
restricted, and client-IP preservation is precisely what makes filtering by real source address
possible. The per-source rules are generated with a `for_each` map in `haproxy.tf`.
### Storage
- **Prometheus Data Volume**: 100 GB SSD (CephSSD) with online resize enabled
@ -78,20 +120,30 @@ An OVN-based Octavia load balancer (`s2i2s-cloud-l4-load-balancer`) provides L4
1. The `main_net_dns_router` configuration must be applied first (creates network, subnet, DNS zone)
2. SSH key must be configured in the OpenStack project
3. OpenStack credentials must be configured (via `clouds.yaml` or environment variables)
4. [OpenTofu](https://opentofu.org/) — this repository is driven with `tofu`, not `terraform`
5. `terraform-provider-openstack/openstack` >= 2.0.0; the configuration is verified against 3.4.0
## Usage
```bash
# Initialize Terraform
terraform init
# Initialize OpenTofu
tofu init
# Review the plan
terraform plan
tofu plan
# Apply the configuration
terraform apply
tofu apply
```
> When re-initialising a directory that was last used with `terraform`, run `tofu init -upgrade`.
> The provider address in `.terraform.lock.hcl` moves from `registry.terraform.io` to
> `registry.opentofu.org`; that is expected.
>
> Provider 3.x tracks a `stateful` attribute on `openstack_networking_secgroup_v2` that 1.53.0 did
> not. The first refresh after upgrading therefore reports `+ stateful = false` as out-of-band
> drift on every security group. It produces no planned change and can be ignored.
## SSH Jump Proxy Configuration
To access VMs in the S2I2S cloud, you must use the SSH jump proxy. Add the following configuration to your `~/.ssh/config` file:
@ -100,7 +152,7 @@ To access VMs in the S2I2S cloud, you must use the SSH jump proxy. Add the follo
# S2I2S SSH Jump Proxy
# Replace <your_username> with your actual username
Host s2i2s-jump
HostName ssh-jump.s2i2s.cloud.isti.cnr.it
HostName ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
User <your_username>
IdentityFile ~/.ssh/your_private_key
ForwardAgent yes
@ -114,32 +166,32 @@ Host s2i2s-jump
Host 10.10.0.*
User <your_username>
ForwardAgent yes
ProxyJump <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ProxyJump <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
# Alternative: named aliases for specific internal hosts
Host s2i2s-prometheus
HostName 10.10.0.10
User <your_username>
ForwardAgent yes
ProxyJump <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ProxyJump <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
Host s2i2s-ca
HostName 10.10.0.4
User <your_username>
ForwardAgent yes
ProxyJump <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ProxyJump <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
Host s2i2s-haproxy-01
HostName 10.10.0.11
User <your_username>
ForwardAgent yes
ProxyJump <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ProxyJump <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
Host s2i2s-haproxy-02
HostName 10.10.0.12
User <your_username>
ForwardAgent yes
ProxyJump <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ProxyJump <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
```
### SSH Usage Examples
@ -155,10 +207,10 @@ ssh 10.10.0.10
ssh s2i2s-prometheus
# Connect without ssh config (replace <your_username>)
ssh -J <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it <your_username>@10.10.0.10
ssh -J <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it <your_username>@10.10.0.10
# Copy a file to an internal VM
scp -J <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it localfile.txt <your_username>@10.10.0.10:/tmp/
scp -J <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it localfile.txt <your_username>@10.10.0.10:/tmp/
# Forward a local port to an internal service
ssh -L 8080:10.10.0.30:80 s2i2s-jump
@ -184,12 +236,12 @@ ssh -L 8080:10.10.0.50:80 s2i2s-jump
ssh -L 8443:10.10.0.50:443 s2i2s-jump
# Without ssh config (replace <your_username>)
ssh -L 8080:10.10.0.50:80 <your_username>@ssh-jump.s2i2s.cloud.isti.cnr.it
ssh -L 8080:10.10.0.50:80 <your_username>@ssh-jump-proxy.s2i2s.cloud.isti.cnr.it
```
## Outputs
The module exports the following outputs for use by other Terraform configurations:
The module exports the following outputs for use by other OpenTofu configurations:
### VM IDs and IPs
@ -219,6 +271,11 @@ The module exports the following outputs for use by other Terraform configuratio
- `main_private_network`, `main_private_subnet`, `main_subnet_network_id`
- `basic_services_ip`, `main_haproxy_l7_ip`
### Image Outputs (re-exported from labs_common_variables)
- `ubuntu_2404`, `ubuntu2404_data_file` - the image and cloud-init script used by every VM here
- `ubuntu_2204`, `ubuntu2204_data_file` - kept for consumers still on 22.04
## File Structure
```text
@ -251,3 +308,8 @@ This module depends on:
- The Prometheus data volume uses CephSSD storage for better I/O performance
- Volumes have `enable_online_resize = true` for live resizing capability
- Security groups are designed to minimize attack surface while allowing necessary traffic flows
- Every instance carries `ignore_changes = [key_pair, user_data, network]` so that rotating the SSH
key or the cloud-init script does not trigger a rebuild
- The main load balancer uses the OVN Octavia provider; see
[Consequences of the OVN provider](#consequences-of-the-ovn-provider) before changing listeners,
pool methods, health monitors or the HAProxy security group