forked from ISTI-ansible-roles/ansible-role-mailman
236 lines
10 KiB
Markdown
236 lines
10 KiB
Markdown
# mailman
|
|
|
|
The site-wide antispam defaults are deliberately empty:
|
|
|
|
```yaml
|
|
mailman_antispam_header_checks: []
|
|
mailman_antispam_chain_behaviour: discard
|
|
```
|
|
|
|
Mailing-list owners can manage list-specific header matches and select their
|
|
action (`accept`, `discard`, `hold`, or `reject`) from Postorius. The global
|
|
chain is used only when a matching rule has no explicit action.
|
|
|
|
Existing lists without any header matches can be initialized through the
|
|
Mailman database model by enabling the optional seed operation:
|
|
|
|
```yaml
|
|
mailman_seed_empty_list_header_matches: true
|
|
mailman_empty_list_header_matches:
|
|
- header: 'X-Spam-Flag'
|
|
pattern: '^YES$'
|
|
action: 'discard'
|
|
```
|
|
|
|
The operation is idempotent. A list is modified only when its header-match
|
|
collection is empty; lists with one or more existing rules are left untouched.
|
|
|
|
## Verified weekly restart
|
|
|
|
The optional weekly restart performs separate stop and start operations. It
|
|
waits for systemd, `mailman status`, and the configured LMTP listener to confirm
|
|
that Mailman has stopped before starting it. It then requires the master and
|
|
the LMTP port to become available before considering startup successful:
|
|
|
|
```yaml
|
|
mailman_enable_weekly_verified_restart: true
|
|
mailman_weekly_verified_restart_on_calendar: 'Sun *-*-* 04:00:00'
|
|
mailman_service_stop_timeout: 30
|
|
mailman_weekly_verified_restart_stop_timeout: 120
|
|
mailman_weekly_verified_restart_start_timeout: 120
|
|
```
|
|
|
|
The timer is deliberately not persistent, so a missed run is not executed
|
|
immediately after a server boot. Failures are recorded by systemd in the
|
|
`mailman-verified-restart.service` journal.
|
|
|
|
The managed `mailman.service` uses `KillMode=control-group` and an explicit
|
|
`TimeoutStopSec`. If Mailman's own stop command hangs, systemd terminates the
|
|
whole service cgroup and removes a stale master PID file. The weekly job still
|
|
verifies the real stopped state and does not start a second instance until the
|
|
master and LMTP listener are gone.
|
|
The normal role execution also waits for the configured LMTP port and fails if
|
|
the listener does not become available after starting or restarting Mailman.
|
|
|
|
## Legacy HyperKitty compatibility fixes
|
|
|
|
The optional compatibility patcher addresses failures observed with the
|
|
legacy Python 3.6 web stack: malformed `From`/`Subject` headers, nested
|
|
`message/rfc822` parts without `Content-Transfer-Encoding`, and unreadable
|
|
gettext catalogs. It also normalizes non-ASCII sender addresses and backports
|
|
the byte-oriented delivery used by newer `mailman-hyperkitty` releases, so
|
|
surrogate-escaped message bytes do not pass through Requests as Unicode text.
|
|
Named MIME parts whose content type is missing or empty are extracted as binary
|
|
attachments instead of being appended to the plain-text message body. As an
|
|
additional safeguard, the NUL filtering introduced by django-mailman3 1.3.6 is
|
|
backported for scrubbed text fields only; binary attachments remain unchanged.
|
|
It also backports the HyperKitty 1.3.5 guards for all four asynchronous cache
|
|
tasks that may still reference a mailing list after its archive has been
|
|
deleted. Those obsolete tasks are logged and skipped instead of being retried.
|
|
Enable it only for the known package versions:
|
|
|
|
```yaml
|
|
mailman_enable_legacy_hyperkitty_compatibility_patches: true
|
|
mailman_legacy_hyperkitty_version: '1.3.3'
|
|
mailman_legacy_django_mailman3_version: '1.3.4'
|
|
mailman_legacy_mailman_hyperkitty_version: '1.1.0'
|
|
```
|
|
|
|
The Python source patches are idempotent, compiled before any file is replaced,
|
|
and installed atomically. The task fails without changing any source file if
|
|
the installed versions or expected source fragments do not match. Invalid
|
|
`django_extensions` `.mo` files are preserved beside the original name with a
|
|
`.disabled-by-ansible` suffix, allowing Django to fall back to another locale.
|
|
The standalone HyperKitty qcluster, uWSGI and Mailman services are restarted
|
|
only when a source file or catalog actually changes. Restarting qcluster is
|
|
required because its workers otherwise retain the old Python module in memory.
|
|
After restarting Mailman, the handler requires the configured LMTP listener to
|
|
become available.
|
|
|
|
Disable this option before upgrading HyperKitty, django-mailman3, or
|
|
mailman-hyperkitty, then review whether the compatibility patches are still
|
|
needed with the new versions.
|
|
|
|
The legacy Mailman 3.3.1 decorator is also patched so the optional
|
|
`${hyperkitty_url}` footer placeholder expands to an empty value when the
|
|
list's archive policy is `never` or its HyperKitty archiver is disabled. The
|
|
site-level generic footer can therefore include a permalink only for messages
|
|
that have an active archive. A custom per-list footer still overrides this
|
|
site default.
|
|
|
|
HyperKitty builds `List-Archive`, `Archived-At`, and `${hyperkitty_url}` from
|
|
the Django Site associated with each mail domain. On installations with one
|
|
canonical web frontend, manage that Site explicitly:
|
|
|
|
```yaml
|
|
mailman_manage_postorius_site: true
|
|
mailman_postorius_site_id: 2
|
|
mailman_postorius_site_domain: 'mailman.example.org'
|
|
mailman_postorius_site_name: 'Example Mailman'
|
|
```
|
|
|
|
The helper refuses a domain already assigned to another Django Site and only
|
|
updates the configured Site ID. Existing django-mailman3 mail-domain mappings
|
|
remain attached to that Site. Preview the database change with
|
|
`sudo -u mailman /usr/local/sbin/mailman-configure-django-site --check`.
|
|
|
|
## uWSGI availability safeguards
|
|
|
|
The Mailman web application runs with multiple uWSGI worker processes. A
|
|
request that exceeds the configured timeout causes only its worker to be
|
|
restarted, while the remaining workers continue to serve Postorius and
|
|
HyperKitty. Workers are also recycled after a bounded number of requests or
|
|
when their resident memory exceeds the configured threshold:
|
|
|
|
```yaml
|
|
mailman_postorius_uwsgi_processes: 4
|
|
mailman_postorius_uwsgi_threads: 2
|
|
mailman_postorius_uwsgi_harakiri: 120
|
|
mailman_postorius_uwsgi_max_requests: 1000
|
|
mailman_postorius_uwsgi_reload_on_rss: 512
|
|
```
|
|
|
|
The `harakiri` value should not exceed nginx's `uwsgi_read_timeout`; otherwise
|
|
nginx gives up while the blocked worker remains occupied. `reload-on-rss` is
|
|
expressed in MiB. Changing these values restarts `mailmansuite-uwsgi` through
|
|
the role handler.
|
|
|
|
## HyperKitty archive cleanup
|
|
|
|
Mailman Core stores the archive visibility policy and the activation of each
|
|
archiver as independent per-list settings. Consequently, setting
|
|
`archive_policy` to `never` is not sufficient by itself to stop delivery to
|
|
HyperKitty if that archiver remains enabled. The optional reconciliation timer
|
|
enforces the safe direction every 15 minutes: it disables HyperKitty for lists
|
|
whose policy is `never`, but never enables an archiver and never deletes data:
|
|
|
|
```yaml
|
|
mailman_enable_hyperkitty_archiver_reconciliation: true
|
|
mailman_hyperkitty_archiver_reconciliation_on_active: '5min'
|
|
mailman_hyperkitty_archiver_reconciliation_interval: '15min'
|
|
```
|
|
|
|
A read-only preview of the current inconsistencies is available with:
|
|
|
|
```bash
|
|
sudo -u mailman /usr/local/sbin/mailman-hyperkitty-archiver-reconcile --dry-run
|
|
```
|
|
|
|
Mailman Core also does not delete HyperKitty data when a list is removed or
|
|
when archiving is disabled. The separate cleanup timer removes the
|
|
corresponding HyperKitty `MailingList`, threads and messages in both cases:
|
|
|
|
```yaml
|
|
mailman_enable_hyperkitty_archive_cleanup: true
|
|
mailman_hyperkitty_archive_cleanup_on_calendar: 'Sun *-*-* 03:00:00'
|
|
mailman_hyperkitty_archive_cleanup_deleted_lists: true
|
|
mailman_hyperkitty_archive_cleanup_disabled_lists: true
|
|
mailman_hyperkitty_archive_cleanup_max_deletions: 50
|
|
```
|
|
|
|
Deletion is irreversible. Before changing the database, the cleanup script
|
|
retrieves a complete list from Mailman Core and rechecks each candidate. It
|
|
aborts if Core is unavailable, if Core returns no lists, or if the configured
|
|
maximum number of deletions would be exceeded. HyperKitty deletion signals
|
|
that would enqueue cache rebuilds for the objects being removed are disabled
|
|
only inside the cleanup process. This avoids the known failure in legacy
|
|
HyperKitty releases where those asynchronous jobs try to load a list after it
|
|
has been deleted.
|
|
|
|
The timer is not persistent, so a missed run is not started after boot. A
|
|
read-only preview can be run before the first scheduled execution:
|
|
|
|
```bash
|
|
sudo -u mailman /usr/local/sbin/hyperkitty-archive-cleanup --dry-run
|
|
```
|
|
|
|
The monthly HyperKitty job already configured by this role removes stale
|
|
entries from the full-text search index. Therefore the weekly cleanup does not
|
|
run an additional, expensive full index scan.
|
|
|
|
## Web signup storm protection
|
|
|
|
The optional nginx configuration limits only `POST /accounts/signup/`; all
|
|
other Postorius, HyperKitty and API requests use an empty nginx limit key and
|
|
are not counted. Both a per-client-IP zone and a site-wide zone are used, since
|
|
distributed signup storms can evade a per-IP limit:
|
|
|
|
```yaml
|
|
mailman_enable_signup_rate_limit: true
|
|
mailman_signup_rate_limit_per_ip_rate: '1r/m'
|
|
mailman_signup_rate_limit_per_ip_burst: 2
|
|
mailman_signup_rate_limit_global_rate: '5r/m'
|
|
mailman_signup_rate_limit_global_burst: 10
|
|
mailman_signup_rate_limit_dry_run: true
|
|
```
|
|
|
|
Dry-run mode requires nginx 1.17.1 or newer. It logs requests that would have
|
|
been limited at `notice` level but does not reject them. Review those logs
|
|
before setting `mailman_signup_rate_limit_dry_run` to `false`; enforced requests
|
|
receive HTTP 429. The nginx configuration is validated before it is reloaded.
|
|
|
|
## Expired unverified web-account cleanup
|
|
|
|
The optional timer removes abandoned django-allauth registrations. An account
|
|
is eligible only when it is older than the configured age, has never logged in,
|
|
is neither staff nor superuser, belongs to no group, has no social account, has
|
|
at least one unverified address but no verified address, and has no confirmation
|
|
mail newer than the cutoff:
|
|
|
|
```yaml
|
|
mailman_enable_unverified_account_cleanup: true
|
|
mailman_unverified_account_cleanup_on_calendar: 'Sun *-*-* 03:30:00'
|
|
mailman_unverified_account_cleanup_min_age_days: 7
|
|
mailman_unverified_account_cleanup_max_deletions: 50
|
|
```
|
|
|
|
The script repeats all eligibility tests while holding database locks and
|
|
deletes the complete batch in one transaction. If the candidate count exceeds
|
|
the safety limit, it deletes nothing; it never removes only the first N users.
|
|
The timer is not persistent, so a missed run is not started after boot. Preview
|
|
the candidates before the first execution with:
|
|
|
|
```bash
|
|
sudo -u mailman /usr/local/sbin/mailman-unverified-account-cleanup --dry-run
|
|
```
|