10 KiB

Raw Blame History

mailman

The site-wide antispam defaults are deliberately empty:

mailman_antispam_header_checks: []
mailman_antispam_chain_behaviour: discard

Mailing-list owners can manage list-specific header matches and select their action (accept, discard, hold, or reject) from Postorius. The global chain is used only when a matching rule has no explicit action.

Existing lists without any header matches can be initialized through the Mailman database model by enabling the optional seed operation:

mailman_seed_empty_list_header_matches: true
mailman_empty_list_header_matches:
  - header: 'X-Spam-Flag'
    pattern: '^YES$'
    action: 'discard'

The operation is idempotent. A list is modified only when its header-match collection is empty; lists with one or more existing rules are left untouched.

Verified weekly restart

The optional weekly restart performs separate stop and start operations. It waits for systemd, mailman status, and the configured LMTP listener to confirm that Mailman has stopped before starting it. It then requires the master and the LMTP port to become available before considering startup successful:

mailman_enable_weekly_verified_restart: true
mailman_weekly_verified_restart_on_calendar: 'Sun *-*-* 04:00:00'
mailman_service_stop_timeout: 30
mailman_weekly_verified_restart_stop_timeout: 120
mailman_weekly_verified_restart_start_timeout: 120

The timer is deliberately not persistent, so a missed run is not executed immediately after a server boot. Failures are recorded by systemd in the mailman-verified-restart.service journal.

The managed mailman.service uses KillMode=control-group and an explicit TimeoutStopSec. If Mailman’s own stop command hangs, systemd terminates the whole service cgroup and removes a stale master PID file. The weekly job still verifies the real stopped state and does not start a second instance until the master and LMTP listener are gone. The normal role execution also waits for the configured LMTP port and fails if the listener does not become available after starting or restarting Mailman.

Legacy HyperKitty compatibility fixes

The optional compatibility patcher addresses failures observed with the legacy Python 3.6 web stack: malformed From/Subject headers, nested message/rfc822 parts without Content-Transfer-Encoding, and unreadable gettext catalogs. It also normalizes non-ASCII sender addresses and backports the byte-oriented delivery used by newer mailman-hyperkitty releases, so surrogate-escaped message bytes do not pass through Requests as Unicode text. Named MIME parts whose content type is missing or empty are extracted as binary attachments instead of being appended to the plain-text message body. As an additional safeguard, the NUL filtering introduced by django-mailman3 1.3.6 is backported for scrubbed text fields only; binary attachments remain unchanged. It also backports the HyperKitty 1.3.5 guards for all four asynchronous cache tasks that may still reference a mailing list after its archive has been deleted. Those obsolete tasks are logged and skipped instead of being retried. Enable it only for the known package versions:

mailman_enable_legacy_hyperkitty_compatibility_patches: true
mailman_legacy_hyperkitty_version: '1.3.3'
mailman_legacy_django_mailman3_version: '1.3.4'
mailman_legacy_mailman_hyperkitty_version: '1.1.0'

The Python source patches are idempotent, compiled before any file is replaced, and installed atomically. The task fails without changing any source file if the installed versions or expected source fragments do not match. Invalid django_extensions .mo files are preserved beside the original name with a .disabled-by-ansible suffix, allowing Django to fall back to another locale. The standalone HyperKitty qcluster, uWSGI and Mailman services are restarted only when a source file or catalog actually changes. Restarting qcluster is required because its workers otherwise retain the old Python module in memory. After restarting Mailman, the handler requires the configured LMTP listener to become available.

Disable this option before upgrading HyperKitty, django-mailman3, or mailman-hyperkitty, then review whether the compatibility patches are still needed with the new versions.

The legacy Mailman 3.3.1 decorator is also patched so the optional ${hyperkitty_url} footer placeholder expands to an empty value when the list’s archive policy is never or its HyperKitty archiver is disabled. The site-level generic footer can therefore include a permalink only for messages that have an active archive. A custom per-list footer still overrides this site default.

HyperKitty builds List-Archive, Archived-At, and ${hyperkitty_url} from the Django Site associated with each mail domain. On installations with one canonical web frontend, manage that Site explicitly:

mailman_manage_postorius_site: true
mailman_postorius_site_id: 2
mailman_postorius_site_domain: 'mailman.example.org'
mailman_postorius_site_name: 'Example Mailman'

The helper refuses a domain already assigned to another Django Site and only updates the configured Site ID. Existing django-mailman3 mail-domain mappings remain attached to that Site. Preview the database change with sudo -u mailman /usr/local/sbin/mailman-configure-django-site --check.

uWSGI availability safeguards

The Mailman web application runs with multiple uWSGI worker processes. A request that exceeds the configured timeout causes only its worker to be restarted, while the remaining workers continue to serve Postorius and HyperKitty. Workers are also recycled after a bounded number of requests or when their resident memory exceeds the configured threshold:

mailman_postorius_uwsgi_processes: 4
mailman_postorius_uwsgi_threads: 2
mailman_postorius_uwsgi_harakiri: 120
mailman_postorius_uwsgi_max_requests: 1000
mailman_postorius_uwsgi_reload_on_rss: 512

The harakiri value should not exceed nginx’s uwsgi_read_timeout; otherwise nginx gives up while the blocked worker remains occupied. reload-on-rss is expressed in MiB. Changing these values restarts mailmansuite-uwsgi through the role handler.

HyperKitty archive cleanup

Mailman Core stores the archive visibility policy and the activation of each archiver as independent per-list settings. Consequently, setting archive_policy to never is not sufficient by itself to stop delivery to HyperKitty if that archiver remains enabled. The optional reconciliation timer enforces the safe direction every 15 minutes: it disables HyperKitty for lists whose policy is never, but never enables an archiver and never deletes data:

mailman_enable_hyperkitty_archiver_reconciliation: true
mailman_hyperkitty_archiver_reconciliation_on_active: '5min'
mailman_hyperkitty_archiver_reconciliation_interval: '15min'

A read-only preview of the current inconsistencies is available with:

sudo -u mailman /usr/local/sbin/mailman-hyperkitty-archiver-reconcile --dry-run

Mailman Core also does not delete HyperKitty data when a list is removed or when archiving is disabled. The separate cleanup timer removes the corresponding HyperKitty MailingList, threads and messages in both cases:

mailman_enable_hyperkitty_archive_cleanup: true
mailman_hyperkitty_archive_cleanup_on_calendar: 'Sun *-*-* 03:00:00'
mailman_hyperkitty_archive_cleanup_deleted_lists: true
mailman_hyperkitty_archive_cleanup_disabled_lists: true
mailman_hyperkitty_archive_cleanup_max_deletions: 50

Deletion is irreversible. Before changing the database, the cleanup script retrieves a complete list from Mailman Core and rechecks each candidate. It aborts if Core is unavailable, if Core returns no lists, or if the configured maximum number of deletions would be exceeded. HyperKitty deletion signals that would enqueue cache rebuilds for the objects being removed are disabled only inside the cleanup process. This avoids the known failure in legacy HyperKitty releases where those asynchronous jobs try to load a list after it has been deleted.

The timer is not persistent, so a missed run is not started after boot. A read-only preview can be run before the first scheduled execution:

sudo -u mailman /usr/local/sbin/hyperkitty-archive-cleanup --dry-run

The monthly HyperKitty job already configured by this role removes stale entries from the full-text search index. Therefore the weekly cleanup does not run an additional, expensive full index scan.

The optional nginx configuration limits only POST /accounts/signup/; all other Postorius, HyperKitty and API requests use an empty nginx limit key and are not counted. Both a per-client-IP zone and a site-wide zone are used, since distributed signup storms can evade a per-IP limit:

mailman_enable_signup_rate_limit: true
mailman_signup_rate_limit_per_ip_rate: '1r/m'
mailman_signup_rate_limit_per_ip_burst: 2
mailman_signup_rate_limit_global_rate: '5r/m'
mailman_signup_rate_limit_global_burst: 10
mailman_signup_rate_limit_dry_run: true

Dry-run mode requires nginx 1.17.1 or newer. It logs requests that would have been limited at notice level but does not reject them. Review those logs before setting mailman_signup_rate_limit_dry_run to false; enforced requests receive HTTP 429. The nginx configuration is validated before it is reloaded.

Expired unverified web-account cleanup

The optional timer removes abandoned django-allauth registrations. An account is eligible only when it is older than the configured age, has never logged in, is neither staff nor superuser, belongs to no group, has no social account, has at least one unverified address but no verified address, and has no confirmation mail newer than the cutoff:

mailman_enable_unverified_account_cleanup: true
mailman_unverified_account_cleanup_on_calendar: 'Sun *-*-* 03:30:00'
mailman_unverified_account_cleanup_min_age_days: 7
mailman_unverified_account_cleanup_max_deletions: 50

The script repeats all eligibility tests while holding database locks and deletes the complete batch in one transaction. If the candidate count exceeds the safety limit, it deletes nothing; it never removes only the first N users. The timer is not persistent, so a missed run is not started after boot. Preview the candidates before the first execution with:

sudo -u mailman /usr/local/sbin/mailman-unverified-account-cleanup --dry-run

10 KiB Raw Blame History Unescape Escape