# S2I2S Project Setup This OpenTofu configuration sets up the core infrastructure components for the S2I2S OpenStack project. ## Overview The project-setup module creates the following resources: ### Virtual Machines | VM | Purpose | Flavor | Private IP | Boot disk | Floating IP | DNS Record | | --- | --- | --- | --- | --- | --- | --- | | SSH Jump Proxy | Secure SSH gateway for accessing internal VMs | m2.small | 10.10.0.5 | 30 GB | Yes | `ssh-jump-proxy.s2i2s.cloud.isti.cnr.it` | | Internal CA | Certificate Authority for internal services | m1.small | 10.10.0.4 | 10 GB | No | - | | HAProxy L7 (x2) | Layer 7 load balancers behind Octavia | m1.medium | 10.10.0.11, 10.10.0.12 | 10 GB | No | - | | Prometheus | Monitoring and metrics collection | m1.medium | 10.10.0.10 | 10 GB | Yes | `prometheus.s2i2s.cloud.isti.cnr.it`, `alertmanager.s2i2s.cloud.isti.cnr.it` (CNAME) | All VMs run **Ubuntu 24.04** (`Ubuntu-Noble-24.04.img`) and are provisioned with the standard `ubuntu2404.sh` cloud-init user data script. Boot disks are volume-backed with `delete_on_termination = false`. ### Load Balancer An OVN-based Octavia load balancer (`s2i2s-cloud-l4-load-balancer`) provides L4 load balancing: - **Provider**: `ovn` (not amphora) - **VIP**: 10.10.0.20, on the main private subnet - **Floating IP**: Yes - **DNS Record**: `octavia-main-lb.s2i2s.cloud.isti.cnr.it` - **Backend**: HAProxy L7 instances (anti-affinity for HA) | Listener | Port | Protocol | Pool method | Health Check | | --- | --- | --- | --- | --- | | HTTP | 80 | TCP | SOURCE_IP_PORT | TCP connect | | HTTPS | 443 | TCP | SOURCE_IP_PORT | TCP connect | | Stats | 8880 | TCP | SOURCE_IP_PORT | TCP connect | #### Consequences of the OVN provider The OVN driver is lighter than amphora (no amphora VMs, uses the main subnet directly), but it constrains the configuration in ways that are visible throughout this setup: - **No `allowed_cidrs` on listeners.** Access restrictions must be enforced by security groups or inside HAProxy itself. This is why the stats port is filtered in the security group below. - **Only `ROUND_ROBIN` and `SOURCE_IP_PORT`** are available as pool methods; `LEAST_CONNECTIONS` is not supported. - **Only `TCP` and `UDP-CONNECT` health monitors**; HTTP/HTTPS monitors are not supported, so all three pools are probed with a plain TCP connect. - **The client source IP is preserved.** OVN does not SNAT the traffic to an amphora VIP, so the HAProxy L7 backends see the real client address rather than an address of the load balancer. Only health-monitor probes and hairpinned traffic originate from the private subnet. ### Security Groups | Security Group | Purpose | Use On | | --- | --- | --- | | `default_for_all` | Default rules: SSH via jump proxy, ICMP, Prometheus node exporter | All VMs | | `ssh_access_to_the_jump_node` | SSH access from VPN endpoints | SSH Jump Proxy only | | `debugging_from_jump_node` | Web debugging via SSH tunnels (ports 80, 443, 8100) | VMs needing debug access | | `traffic_from_the_main_load_balancers` | HTTP/HTTPS from the HAProxy L7 IPs (ports 80, 443, 8080, 8888) | Backend web services | | `traffic_from_main_lb_to_haproxy_l7` | Public HTTP/HTTPS plus filtered access to the stats port | HAProxy L7 VMs | | `public_web_service` | HTTP/HTTPS from anywhere | Public-facing services with floating IP | | `restricted_web_service` | HTTP from anywhere, HTTPS from VPNs only | Restricted services with floating IP | | `prometheus_access_from_grafana` | HTTPS access from public Grafana server | Prometheus VM | > **Do not confuse the two load balancer groups.** `traffic_from_main_lb_to_haproxy_l7` sits on the > HAProxy L7 VMs themselves and faces the internet. `traffic_from_the_main_load_balancers` sits on > the backend services *behind* HAProxy: since HAProxy is an L7 proxy that opens new connections > from its own address, the client IP is **not** preserved on that hop, so its sources stay > restricted to 10.10.0.11 and 10.10.0.12 and it must never be opened to `0.0.0.0/0`. #### `traffic_from_main_lb_to_haproxy_l7` Because the OVN load balancer preserves the client source IP (see above), the public listeners on the HAProxy L7 VMs cannot be restricted to the private subnet: | Port | Source | Rationale | | --- | --- | --- | | 80 | `0.0.0.0/0` | HTTP arrives with the original client address | | 443 | `0.0.0.0/0` | HTTPS arrives with the original client address | | 8880 | main private subnet | Health monitor probes for the stats pool | | 8880 | D4S VPN 1/2, S2I2S VPN 1/2, InfraScience network | Human access to the HAProxy stats page | | 8880 | 10.10.0.10/32 and the Prometheus floating IP | Prometheus scrapes the stats, over the private network and through the LB public IP | | 10000 | 10.10.0.11/32, 10.10.0.12/32 | Peer traffic between the two HAProxy instances | The stats port is deliberately **not** open to `0.0.0.0/0`: since the OVN provider does not support `allowed_cidrs` on the listener, this security group is the only place where port 8880 can be restricted, and client-IP preservation is precisely what makes filtering by real source address possible. The per-source rules are generated with a `for_each` map in `haproxy.tf`. ### Storage - **Prometheus Data Volume**: 100 GB SSD (CephSSD) with online resize enabled ## Architecture ```text Internet | +-------------------+-------------------+ | | | [SSH Jump Proxy] [Octavia LB] [Prometheus] | (Floating IP) (Floating IP) | | | +-------+-------+ | | | | [HAProxy L7-01] [HAProxy L7-02] | | | | +-------+-------+ | | +-------------------+ | [Internal Network] | +-------+-------+ | | [Internal CA] [Backend VMs] ``` ## Prerequisites 1. The `main_net_dns_router` configuration must be applied first (creates network, subnet, DNS zone) 2. SSH key must be configured in the OpenStack project 3. OpenStack credentials must be configured (via `clouds.yaml` or environment variables) 4. [OpenTofu](https://opentofu.org/) — this repository is driven with `tofu`, not `terraform` 5. `terraform-provider-openstack/openstack` >= 2.0.0; the configuration is verified against 3.4.0 ## Usage ```bash # Initialize OpenTofu tofu init # Review the plan tofu plan # Apply the configuration tofu apply ``` > When re-initialising a directory that was last used with `terraform`, run `tofu init -upgrade`. > The provider address in `.terraform.lock.hcl` moves from `registry.terraform.io` to > `registry.opentofu.org`; that is expected. > > Provider 3.x tracks a `stateful` attribute on `openstack_networking_secgroup_v2` that 1.53.0 did > not. The first refresh after upgrading therefore reports `+ stateful = false` as out-of-band > drift on every security group. It produces no planned change and can be ignored. ## SSH Jump Proxy Configuration To access VMs in the S2I2S cloud, you must use the SSH jump proxy. Add the following configuration to your `~/.ssh/config` file: ```ssh-config # S2I2S SSH Jump Proxy # Replace with your actual username Host s2i2s-jump HostName ssh-jump-proxy.s2i2s.cloud.isti.cnr.it User IdentityFile ~/.ssh/your_private_key ForwardAgent yes # Keep connection alive ServerAliveInterval 60 ServerAliveCountMax 3 # Pattern match for all S2I2S internal hosts by IP # Matches any IP in the 10.10.0.x range # Usage: ssh 10.10.0.10 Host 10.10.0.* User ForwardAgent yes ProxyJump @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it # Alternative: named aliases for specific internal hosts Host s2i2s-prometheus HostName 10.10.0.10 User ForwardAgent yes ProxyJump @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it Host s2i2s-ca HostName 10.10.0.4 User ForwardAgent yes ProxyJump @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it Host s2i2s-haproxy-01 HostName 10.10.0.11 User ForwardAgent yes ProxyJump @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it Host s2i2s-haproxy-02 HostName 10.10.0.12 User ForwardAgent yes ProxyJump @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it ``` ### SSH Usage Examples ```bash # Connect to the jump proxy directly ssh s2i2s-jump # Connect to an internal VM by IP (using pattern match from ssh config) ssh 10.10.0.10 # Connect to a named internal host (if configured in ssh config) ssh s2i2s-prometheus # Connect without ssh config (replace ) ssh -J @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it @10.10.0.10 # Copy a file to an internal VM scp -J @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it localfile.txt @10.10.0.10:/tmp/ # Forward a local port to an internal service ssh -L 8080:10.10.0.30:80 s2i2s-jump # Create a SOCKS proxy through the jump host ssh -D 1080 s2i2s-jump # Then configure your browser to use SOCKS5 proxy at localhost:1080 ``` ### SSH Debugging via Tunnel For debugging web applications on internal VMs, you can create SSH tunnels: ```bash # Forward local port 8100 to a Tomcat debug port on internal VM # (requires s2i2s-jump defined in ssh config) ssh -L 8100:10.10.0.50:8100 s2i2s-jump # Forward local port 8080 to HTTP on internal VM ssh -L 8080:10.10.0.50:80 s2i2s-jump # Forward local port 8443 to HTTPS on internal VM ssh -L 8443:10.10.0.50:443 s2i2s-jump # Without ssh config (replace ) ssh -L 8080:10.10.0.50:80 @ssh-jump-proxy.s2i2s.cloud.isti.cnr.it ``` ## Outputs The module exports the following outputs for use by other OpenTofu configurations: ### VM IDs and IPs - `ssh_jump_proxy_id`, `ssh_jump_proxy_public_ip`, `ssh_jump_proxy_hostname` - `internal_ca_id` - `main_haproxy_l7_ids` - `prometheus_server_id`, `prometheus_public_ip`, `prometheus_hostname` ### Load Balancer Outputs - `main_loadbalancer_id`, `main_loadbalancer_ip`, `main_loadbalancer_public_ip`, `main_loadbalancer_hostname` ### Security Group Outputs - `default_security_group`, `default_security_group_id`, `default_security_group_name` - `access_to_the_jump_proxy` - `debugging` - `traffic_from_main_haproxy` - `public_web` - `restricted_web` - `main_lb_to_haproxy_l7_security_group` - `prometheus_access_from_grafana` ### Network Outputs (re-exported from main_net_dns_router) - `dns_zone`, `dns_zone_id` - `main_private_network`, `main_private_subnet`, `main_subnet_network_id` - `basic_services_ip`, `main_haproxy_l7_ip` ### Image Outputs (re-exported from labs_common_variables) - `ubuntu_2404`, `ubuntu2404_data_file` - the image and cloud-init script used by every VM here - `ubuntu_2204`, `ubuntu2204_data_file` - kept for consumers still on 22.04 ## File Structure ```text project-setup/ ├── provider.tf # OpenStack provider configuration ├── main.tf # Module references and local variables ├── security-groups.tf # All security group definitions ├── ssh-jump-proxy.tf # SSH jump proxy VM and floating IP ├── internal-ca.tf # Internal CA VM ├── haproxy.tf # HAProxy L7 VMs (pair with anti-affinity) ├── prometheus.tf # Prometheus VM with data volume ├── octavia.tf # OVN-based Octavia load balancer ├── outputs.tf # Output definitions └── README.md # This file ``` ## Dependencies This module depends on: - `../main_net_dns_router` - Network, subnet, router, and DNS zone - `../variables` - Project-specific variables - `../../modules/labs_common_variables` - Common variables (images, flavors, etc.) - `../../modules/ssh-key-ref` - SSH key reference ## Notes - The HAProxy L7 VMs are deployed with anti-affinity to ensure they run on different hypervisors - All VMs use volume-backed boot disks with `delete_on_termination = false` for data persistence - The Prometheus data volume uses CephSSD storage for better I/O performance - Volumes have `enable_online_resize = true` for live resizing capability - Security groups are designed to minimize attack surface while allowing necessary traffic flows - Every instance carries `ignore_changes = [key_pair, user_data, network]` so that rotating the SSH key or the cloud-init script does not trigger a rebuild - The main load balancer uses the OVN Octavia provider; see [Consequences of the OVN provider](#consequences-of-the-ovn-provider) before changing listeners, pool methods, health monitors or the HAProxy security group