`group_vars/matrix_servers` selects postgres whenever postgres is enabled, which
is the default, so postgres is what essentially every deployment runs. The
scenarios were testing sqlite - a path almost nobody is on.
How little that path is used is not a guess: the mautrix-meta bridges could not
start at all under sqlite, and nobody reported it. Testing the engine users are
actually on is worth more than keeping coverage of the one they are not, so no
scenario is left behind on sqlite.
Four of the eight scenarios have a database and are converted; the other four
have none and are untouched.
molecule-shared/tasks/postgres.yml stands Postgres up on the scenario's network,
with the data directory on a tmpfs since it is thrown away with the container.
The image is pinned at the major the postgres role deploys to new installations
and left to Renovate: when a new major lands, the PR bumping that pin runs every
scenario against it, which is the earliest warning we get that a component does
not cope.
Each scenario gives its database and user names that differ from the role's
defaults, so the component reaching the database proves the role built its
connection string out of them. The assertions moved from "a file appeared at the
path we configured" to "these tables exist", which is strictly stronger: tables
can only appear once the component has resolved the hostname, authenticated with
the credentials the role rendered, and run its migrations to completion.
Costs about 10 seconds per affected scenario (115s to 125s locally for
mautrix-whatsapp), on jobs that run in parallel.
Gotcha worth recording: since Postgres 18 the image puts PGDATA in a versioned
subdirectory and refuses to start if it finds a mount at the old
/var/lib/postgresql/data, so the tmpfs is mounted at /var/lib/postgresql.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGT
They were hard-wrapped at 80 characters, broke mid-parenthesis, and spent lines
restating what the code below them does.
Rewrapped at natural boundaries instead, with the narration dropped and only the
reasons, gotchas and surprises kept. Section dividers stay - they delineate long
plays rather than narrate them.
Comments only; no scenario behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGT
`matrix_bridge_mautrix_discord_bridge_double_puppet_server_map_default` was
built from `matrix_bridge_beeper_linkedin_homeserver_domain` and
`..._homeserver_address` - a different role's variables, copy-pasted along with
the expression.
No user is affected today: group_vars/matrix_servers points both roles'
`homeserver_address` at `matrix_addons_homeserver_client_api_url` and both
domains at `matrix_domain`, so the rendered value is identical either way. It
only diverges for someone who overrides beeper-linkedin's homeserver settings,
where mautrix-discord would silently follow them.
It does not survive mautrix-discord being used without beeper-linkedin's
defaults in scope, which is how the Molecule scenario found it: with only the
one role loaded, the template fails on the undefined name. The scenario's
neutralising override is dropped in the same commit, so the expression is now
exercised rather than bypassed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGT
Scenarios install their Galaxy dependencies with `force: true`, so two roles
running at once re-extract the same collections and roles into ~/.ansible and
pull them out from under each other mid-play. It surfaces as a collection that
was working moments earlier going missing:
the connection plugin 'community.docker.docker' was not found
Found while running five scenarios in parallel, where it cost a run.
ANSIBLE_HOME relocates both `collections/` and `roles/`, so one variable covers
both halves; the scenarios' ANSIBLE_ROLES_PATH workaround now follows it rather
than hardcoding ~/.ansible/roles. Left alone if already set, and unset in CI,
where each role runs in its own job and has nothing to collide with.
Verified by removing var/molecule-ansible-home entirely and running
matrix-alertmanager-receiver from cold: green through idempotence, with the
collections and roles landing under the per-role directory - which also shows
nothing was quietly relying on the shared ~/.ansible being populated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGT
Proves the bridge starts on the configuration this role rendered: the
systemd unit is active and has not been restarting, the appservice port is
open, config.yaml and registration.yaml (parsed, not grepped) carry the
scenario's homeserver address, bot username, tokens, command prefix,
avatar-proxy key, public address and log level, the sqlite database was
created under the role's data path as the role's uid, and the running
container is the version defaults/main.yml pins.
It does not bridge anything. Discord is behind an account, which is where a
scenario stops being a test of this repository.
Two things are specific to this role rather than copied from the
mautrix-whatsapp scenario:
- mautrix-discord is the only bridge here whose validate_config.yml requires
a public address. It is composed from hostname + scheme + path prefix, and
those same three feed the Traefik avatar-proxy labels, so the scenario
picks a non-`/` path prefix and a non-default scheme and asserts the
composed router rule and strip-prefix middleware - both in the label file
and, read back off the running container, as labels Docker accepted.
Traefik labels are left enabled here for that reason, unlike in the
mautrix-whatsapp scenario which asserts their absence.
- matrix_bridge_mautrix_discord_bridge_double_puppet_server_map_default in
the role's defaults references matrix_bridge_beeper_linkedin_homeserver_domain
and matrix_bridge_beeper_linkedin_homeserver_address - variables belonging
to a different role, evidently copy-pasted from
matrix-bridge-beeper-linkedin, and unique to this role among the bridges.
A playbook run has every role's defaults in scope so it resolves silently;
a role scenario has only this role loaded and the template fails on the
undefined name. The scenario neutralises it in its own group_vars rather
than touching the role.
Falsified the crash-loop assertion: pointing the in-container sqlite path at
a directory that does not exist makes the bridge exit on startup, and the run
then fails at "Assert the service is active and has not been restarting" with
"activating after 5 automatic restart(s)". Reverted, and green again since -
including idempotence.
6b4b7647e fixed this for synapse-usage-exporter only. A checkout owned by
a different user makes the git task fail from then on, either with a
permission error or with git's dubious-ownership protection, until
someone removes the directory on the host by hand. It gets into that
state when the matrix user's uid changes (a server migration or a restore
onto a differently numbered user), when an earlier clone ran as another
user, or when someone runs git as root inside the checkout.
Every other role cloning a repository onto the server was open to the
same failure, so ensure the checkout's ownership recursively before
updating it at the remaining 54 sites.
The three matrix-synapse ext clones also gain force=yes. They were the
only on-server clones without it, which left a checkout that an
interrupted run had half-written wedged, instead of repaired on the next
run.
matrix-matrixto used to clone as root into a directory that nothing ever
chowned, unlike every other role. It now becomes the matrix user too.
The Element Web and SchildiChat Web theme checkouts live on the Ansible
controller, where correcting ownership is not ours to do, so they merely
mark the checkout as a safe directory for git.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Part of adopting a uniform naming policy for bridge variables,
where the variable prefix matches the role directory name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part of adopting a uniform naming policy for bridge variables,
where the variable prefix matches the role directory name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: support container_network=host across all roles + systemd templates
Mirror the pattern Slavi introduced for matrix-coturn (aafa8f0) across the
fork: every 'Ensure X container network is created' task gets a
'when: <var> not in ["", "host"]' guard so MDAD does not try to
docker_network create a network literally named 'host' (returns 403,
since host is a pre-defined Docker network).
Mirror the same guard in every systemd unit template that does
'ExecStartPre=docker network connect <addnet> <container>' loops over
matrix_<role>_container_additional_networks: skip the connects when the
container is on host networking (where additional --network attaches
are invalid).
Unblocks DiD setups where MDAD-managed containers share their host's
network namespace (matrix-mdad outer compose service joined to central
postgres/openldap networks) to reach external services on the outer
Docker daemon.
* Simplify container network guards (!= 'host') and fix duplicate when
Guarding on the empty string ('') as well was misleading: systemd unit
templates still render an unconditional --network= flag, so an empty
network value produces a broken docker create command. Only 'host' is
actually supported, so only guard on that. This also matches the
existing convention in the Traefik role
(when: traefik_container_network != 'host').
Also fix a duplicate when key in the meshtastic-relay role, where the
network-creation task already had a when condition - the two are now
combined into a list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Replaces `community.docker.docker_image` with the modern
`docker_image_pull` and `docker_image_build` split modules. Drops the
`ansible_version` compatibility ladder and the now-redundant
`_container_image_force_pull` variable (the new pull module handles
registry refresh natively via `pull: always`). Also registers
`_container_image_build_result` so that a self-build rebuild correctly
triggers a service restart.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/5191.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reformat `{{ { ... }[engine] }}` dict-in-Jinja-expressions across
bot/bridge role defaults for consistent indentation (tabs -> spaces).
Also: fix a missing space in a Jinja `+` expression in matrix-static-files,
and fix indentation in the caddy2-in-container example.
The original PR also renamed `galaxy/<role>` role references to just
`<role>` in setup.yml and jitsi_jvb.yml; those were dropped here because
Ansible does not recurse into subdirectories of `roles/` by default and
no `roles_path` is configured in ansible.cfg, so the rename would break
role resolution.
Co-Authored-By: Slavi Pantaleev <slavi@devture.com>
For each of the 34 roles (3 clients, 9 bots, 22 bridges), this commit:
- Adds `_restart_necessary: false` default variable
- Adds `register:` directives to config/image/systemd tasks
- Computes `_restart_necessary` via set_fact (OR of all .changed results)
- Wires `(_restart_necessary | bool)` in group_vars/matrix_servers
This allows the systemd service manager to skip unnecessary restarts
when running install-* tags and nothing actually changed.
Service roles and complex multi-service roles will follow separately.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These variables track whether a database migration necessitates a service
restart. The new name avoids confusion with the conditional restart
feature introduced in af193043/9accc848/4a8df138, where
devture_systemd_service_manager handles restarting services whose
configuration or image changed. The old _requires_restart name was
ambiguous — it could be mistaken for the systemd_service_manager
mechanism — so _migration_requires_restart makes the purpose explicit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This:
- brings consistency - no more mixing `_name_prefix` and `_registry_prefix`
- adds extensibility - a future patch will allow reconfiguring all registry prefixes for all roles in the playbook
We still have `_docker_` vs `_container_` inconsistencies.
These may be worked on later.
* Add a global config option for Docker network MTU
* Upgrade systemd_docker_base (v1.2.0-0 -> v1.3.0-0)
The new version includes `devture_systemd_docker_base_container_networks_driver_options`
due to 3cc7d12396
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3502
* Switch from passing matrix_playbook_docker_network_mtu to respecting devture_systemd_docker_base_container_networks_driver_options
Related to:
- 3cc7d12396
- https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3502
* Update all roles to versions that respect `devture_systemd_docker_base_container_networks_driver_options`
---------
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
Related to 0241c71a4c
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3270#issuecomment-2143782962
With this change, it should be possible for people to adjust the Docker
dependency from `docker.service` to something else (e.g. `pkg-ContainerManager-dockerd.service`),
or to completely eliminate it by setting `devture_systemd_docker_base_docker_service_name` to an empty string.
This makes it easier for people to use the playbook against a Synology DSM server.
commit cf8637efac
Author: Slavi Pantaleev <slavi@devture.com>
Date: Sun Mar 24 19:14:57 2024 +0200
Make devture_systemd_docker_base_ipv6_enabled automatically reconfigure geerlingguy/ansible-role-docker
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3218
commit dc7af3bc7d
Author: Slavi Pantaleev <slavi@devture.com>
Date: Sun Mar 24 19:10:31 2024 +0200
Replace matrix_ipv6_enabled with devture_systemd_docker_base_ipv6_enabled
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3218
commit 07e900d6a2
Author: Slavi Pantaleev <slavi@devture.com>
Date: Sun Mar 24 19:01:51 2024 +0200
Improve matrix_ipv6_enabled comments
commit 3f03ca7f69
Author: Tilo Spannagel <development@tilosp.de>
Date: Sat Mar 9 19:27:50 2024 +0000
Add setting to enable ipv6
Media didn't work before this patch, likely because this feature is broken:
> N.B. Discord now requires signed expiring download links, which means this solution no longer works. In the future, a more dynamic solution may be implemented where requests go to the bridge and the bridge and the bridge refetches the message if necessary.
Source: https://docs.mau.fi/bridges/go/discord/direct-media.html
Moreover, most users more likely don't want this behavior and would
prefer to keep a complete mirror of the media on Matrix, instead of
going through two 3rd party servers to fetch the media on demand.
The default config for the bridge
(https://github.com/mautrix/discord/blob/main/example-config.yaml)
actually does not enable it.
It seems like 4ed522e8fe
(https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/3133)
lied to us as to what upstream does. Poor PR review lead to this
anti-feature making it into the playbook.
Slavi Pantaleev
renovate[bot]
Kevin Veen-Birkenbach
Matěj Cepl
QEDeD
Benjamin Blacher
HarHarLinks
Suguru Hirahara
David Mehren
gardar