`group_vars/matrix_servers` selects postgres whenever postgres is enabled, which is the default, so postgres is what essentially every deployment runs. The scenarios were testing sqlite - a path almost nobody is on. How little that path is used is not a guess: the mautrix-meta bridges could not start at all under sqlite, and nobody reported it. Testing the engine users are actually on is worth more than keeping coverage of the one they are not, so no scenario is left behind on sqlite. Four of the eight scenarios have a database and are converted; the other four have none and are untouched. molecule-shared/tasks/postgres.yml stands Postgres up on the scenario's network, with the data directory on a tmpfs since it is thrown away with the container. The image is pinned at the major the postgres role deploys to new installations and left to Renovate: when a new major lands, the PR bumping that pin runs every scenario against it, which is the earliest warning we get that a component does not cope. Each scenario gives its database and user names that differ from the role's defaults, so the component reaching the database proves the role built its connection string out of them. The assertions moved from "a file appeared at the path we configured" to "these tables exist", which is strictly stronger: tables can only appear once the component has resolved the hostname, authenticated with the credentials the role rendered, and run its migrations to completion. Costs about 10 seconds per affected scenario (115s to 125s locally for mautrix-whatsapp), on jobs that run in parallel. Gotcha worth recording: since Postgres 18 the image puts PGDATA in a versioned subdirectory and refuses to start if it finds a mount at the old /var/lib/postgresql/data, so the tmpfs is mounted at /var/lib/postgresql. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEH3vxYSQ5SV4N5z61eyGTpull/5575/head
| @@ -91,6 +91,28 @@ own job there, so there is nothing to collide with. | |||||
| The directories are disposable; `var/` is gitignored. Delete `var/molecule-ansible-home/` to force | The directories are disposable; `var/` is gitignored. Delete `var/molecule-ansible-home/` to force | ||||
| a fresh install. | a fresh install. | ||||
| ## Databases | |||||
| Scenarios for roles that have a database run against **Postgres**, not sqlite. | |||||
| That is what `group_vars/matrix_servers` selects whenever postgres is enabled, which is the | |||||
| default, so it is what essentially every deployment runs. sqlite is a path almost nobody is on: | |||||
| a bug that stopped the mautrix-meta bridges from starting at all under sqlite sat unreported | |||||
| for a long time, which says plainly enough whose path is worth testing. | |||||
| `molecule-shared/tasks/postgres.yml` stands one up on the scenario's container network. Include | |||||
| it from `prepare.yml` and point the role at it with its own `_database_engine`, `_database_hostname` | |||||
| and credentials. Give the database and user names that differ from the role's defaults - then the | |||||
| component reaching the database at all proves the role built its connection string out of them. | |||||
| The image is pinned in `molecule-shared/vars.yml` at the major the postgres role deploys to new | |||||
| installations, and Renovate carries it forward. When a new major lands, the PR bumping that pin | |||||
| runs every scenario against it, which is the earliest warning we get that a component does not | |||||
| cope with it. | |||||
| Prefer asserting on the schema the component created over a file on disk: tables can only appear | |||||
| once it has resolved the hostname, authenticated, and run its migrations. | |||||
| ## Reclaiming the disk space | ## Reclaiming the disk space | ||||
| `just molecule-clean` removes what the runs leave under `var/`. | `just molecule-clean` removes what the runs leave under `var/`. | ||||
| @@ -0,0 +1,73 @@ | |||||
| # SPDX-FileCopyrightText: 2026 Slavi Pantaleev | |||||
| # | |||||
| # SPDX-License-Identifier: AGPL-3.0-or-later | |||||
| --- | |||||
| # Stands up Postgres on a container network, for scenarios whose role has a database. | |||||
| # | |||||
| # Include from a scenario's prepare.yml: | |||||
| # | |||||
| # - name: Ensure Postgres is running | |||||
| # ansible.builtin.include_tasks: | |||||
| # file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/postgres.yml" | |||||
| # vars: | |||||
| # molecule_shared_postgres_network: "{{ <role>_container_network }}" | |||||
| # molecule_shared_postgres_database: matrix_<role> | |||||
| # molecule_shared_postgres_username: matrix_<role> | |||||
| # molecule_shared_postgres_password: <something> | |||||
| # | |||||
| # The role should then be pointed at it with its own `_database_engine: postgres` and | |||||
| # `_database_hostname: matrix-postgres-molecule` (or whatever hostname is passed here). | |||||
| # | |||||
| # This is what the playbook does in a real run: `group_vars/matrix_servers` selects postgres | |||||
| # whenever postgres is enabled, which is the default. sqlite is the path almost nobody uses. | |||||
| - name: Ensure a previous Postgres is gone | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - rm | |||||
| - --force | |||||
| - "{{ molecule_shared_postgres_name | default('matrix-postgres-molecule') }}" | |||||
| register: molecule_shared_postgres_removal | |||||
| changed_when: molecule_shared_postgres_removal.rc == 0 | |||||
| failed_when: false | |||||
| - name: Ensure Postgres is running | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - run | |||||
| - --detach | |||||
| - --name={{ molecule_shared_postgres_name | default('matrix-postgres-molecule') }} | |||||
| - --network={{ molecule_shared_postgres_network }} | |||||
| - --network-alias={{ molecule_shared_postgres_hostname | default('matrix-postgres-molecule') }} | |||||
| - --env=POSTGRES_DB={{ molecule_shared_postgres_database }} | |||||
| - --env=POSTGRES_USER={{ molecule_shared_postgres_username }} | |||||
| - --env=POSTGRES_PASSWORD={{ molecule_shared_postgres_password }} | |||||
| # A tmpfs, because the database is thrown away with the container and not writing it | |||||
| # to the overlay filesystem is faster. Mounted at /var/lib/postgresql rather than at | |||||
| # .../data: since 18 the image puts PGDATA in a versioned subdirectory and refuses to | |||||
| # start if it finds a mount at the old path. | |||||
| - --tmpfs=/var/lib/postgresql | |||||
| - "{{ molecule_shared_image_postgres }}" | |||||
| register: molecule_shared_postgres_start | |||||
| changed_when: molecule_shared_postgres_start.rc == 0 | |||||
| # `pg_isready` needs the username: it defaults to the OS user of whoever runs it, which inside | |||||
| # this container is `postgres` and may not be the role the scenario created. | |||||
| - name: Wait for Postgres to accept connections | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - exec | |||||
| - "{{ molecule_shared_postgres_name | default('matrix-postgres-molecule') }}" | |||||
| - pg_isready | |||||
| - --username={{ molecule_shared_postgres_username }} | |||||
| - --dbname={{ molecule_shared_postgres_database }} | |||||
| - --quiet | |||||
| register: molecule_shared_postgres_ready | |||||
| changed_when: false | |||||
| until: molecule_shared_postgres_ready.rc == 0 | |||||
| retries: 30 | |||||
| delay: 2 | |||||
| @@ -12,3 +12,11 @@ molecule_shared_image_curl: "docker.io/curlimages/curl:8.11.1" | |||||
| # Used for small stub services (a fake homeserver, and so on). | # Used for small stub services (a fake homeserver, and so on). | ||||
| # renovate: datasource=docker depName=docker.io/library/python | # renovate: datasource=docker depName=docker.io/library/python | ||||
| molecule_shared_image_python: "docker.io/library/python:3.13-alpine" | molecule_shared_image_python: "docker.io/library/python:3.13-alpine" | ||||
| # Postgres for scenarios whose role has a database. Pinned at the major the postgres role | |||||
| # deploys to new installations, and left to Renovate from there: when a new major lands, | |||||
| # the PR bumping this pin runs every scenario against it, which is the earliest warning we | |||||
| # get that a component does not cope. Users who upgrade Postgres promptly are on that major | |||||
| # well before the postgres role makes it the default for new installations. | |||||
| # renovate: datasource=docker depName=docker.io/library/postgres | |||||
| molecule_shared_image_postgres: "docker.io/library/postgres:18.6-alpine" | |||||
| @@ -61,10 +61,15 @@ provisioner: | |||||
| matrix: | matrix: | ||||
| device_name: Molecule Reminder Bot | device_name: Molecule Reminder Bot | ||||
| # Moved off the role's default name so verify.yml can assert the bot opened the path | |||||
| # the role gave it, with the default name as a negative control. | |||||
| matrix_bot_matrix_reminder_bot_sqlite_database_path_local: /matrix/matrix-reminder-bot/data/molecule-reminders.db | |||||
| matrix_bot_matrix_reminder_bot_sqlite_database_path_in_container: /data/molecule-reminders.db | |||||
| # Postgres, because that is what `group_vars/matrix_servers` selects whenever postgres | |||||
| # is enabled - which is the default, so it is what essentially every real deployment | |||||
| # runs. prepare.yml stands one up. Name and user differ from the role's defaults, so | |||||
| # reaching the database proves the role built its connection string out of these. | |||||
| matrix_bot_matrix_reminder_bot_database_engine: postgres | |||||
| matrix_bot_matrix_reminder_bot_database_hostname: matrix-postgres-molecule | |||||
| matrix_bot_matrix_reminder_bot_database_name: molecule_reminder_bot | |||||
| matrix_bot_matrix_reminder_bot_database_username: molecule_reminder_bot | |||||
| matrix_bot_matrix_reminder_bot_database_password: molecule_pg_password_3c9b02 | |||||
| # verify.yml runs as its own play, where the role's defaults are out of scope, | # verify.yml runs as its own play, where the role's defaults are out of scope, | ||||
| # so the paths it reads are pinned here to match what the role derives. | # so the paths it reads are pinned here to match what the role derives. | ||||
| @@ -74,6 +74,15 @@ | |||||
| # Not an appservice: it logs in with the username and password the role rendered, retrying | # Not an appservice: it logs in with the username and password the role rendered, retrying | ||||
| # every 15 seconds until that succeeds. The stub answers with an access token, which is | # every 15 seconds until that succeeds. The stub answers with an access token, which is | ||||
| # enough to reach the sync loop. Nothing is asserted about the stub itself. | # enough to reach the sync loop. Nothing is asserted about the stub itself. | ||||
| - name: Ensure Postgres is running | |||||
| ansible.builtin.include_tasks: | |||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/postgres.yml" | |||||
| vars: | |||||
| molecule_shared_postgres_network: "{{ matrix_bot_matrix_reminder_bot_container_network }}" | |||||
| molecule_shared_postgres_database: "{{ matrix_bot_matrix_reminder_bot_database_name }}" | |||||
| molecule_shared_postgres_username: "{{ matrix_bot_matrix_reminder_bot_database_username }}" | |||||
| molecule_shared_postgres_password: "{{ matrix_bot_matrix_reminder_bot_database_password }}" | |||||
| - name: Ensure the homeserver stub is running | - name: Ensure the homeserver stub is running | ||||
| ansible.builtin.include_tasks: | ansible.builtin.include_tasks: | ||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | ||||
| @@ -97,9 +97,9 @@ | |||||
| - name: Assert the bot opened the database engine the role selected | - name: Assert the bot opened the database engine the role selected | ||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - "\"Database initialization of type 'sqlite' complete\" in matrix_bot_matrix_reminder_bot_journal.stdout" | |||||
| fail_msg: "The bot did not report a completed SQLite database initialization" | |||||
| success_msg: "The bot initialized the SQLite database the role pointed it at" | |||||
| - "\"Database initialization of type 'postgres' complete\" in matrix_bot_matrix_reminder_bot_journal.stdout" | |||||
| fail_msg: "The bot did not report a completed Postgres database initialization" | |||||
| success_msg: "The bot initialized the Postgres database the role pointed it at" | |||||
| - name: Read the configuration file the role rendered | - name: Read the configuration file the role rendered | ||||
| ansible.builtin.slurp: | ansible.builtin.slurp: | ||||
| @@ -116,7 +116,9 @@ | |||||
| - matrix_bot_matrix_reminder_bot_matrix_homeserver_url in matrix_bot_matrix_reminder_bot_config_rendered | - matrix_bot_matrix_reminder_bot_matrix_homeserver_url in matrix_bot_matrix_reminder_bot_config_rendered | ||||
| - matrix_bot_matrix_reminder_bot_reminders_timezone in matrix_bot_matrix_reminder_bot_config_rendered | - matrix_bot_matrix_reminder_bot_reminders_timezone in matrix_bot_matrix_reminder_bot_config_rendered | ||||
| - matrix_bot_matrix_reminder_bot_command_prefix in matrix_bot_matrix_reminder_bot_config_rendered | - matrix_bot_matrix_reminder_bot_command_prefix in matrix_bot_matrix_reminder_bot_config_rendered | ||||
| - "'sqlite://' + matrix_bot_matrix_reminder_bot_sqlite_database_path_in_container in matrix_bot_matrix_reminder_bot_config_rendered" | |||||
| - matrix_bot_matrix_reminder_bot_database_username in matrix_bot_matrix_reminder_bot_config_rendered | |||||
| - matrix_bot_matrix_reminder_bot_database_name in matrix_bot_matrix_reminder_bot_config_rendered | |||||
| - matrix_bot_matrix_reminder_bot_database_hostname in matrix_bot_matrix_reminder_bot_config_rendered | |||||
| - "'@molecule-allowed:molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered" | - "'@molecule-allowed:molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered" | ||||
| - "'.*:blocked.molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered" | - "'.*:blocked.molecule.local' in matrix_bot_matrix_reminder_bot_config_rendered" | ||||
| fail_msg: "The rendered configuration does not carry the scenario's settings" | fail_msg: "The rendered configuration does not carry the scenario's settings" | ||||
| @@ -138,41 +140,35 @@ | |||||
| vars: | vars: | ||||
| matrix_bot_matrix_reminder_bot_config_rendered: "{{ matrix_bot_matrix_reminder_bot_config_file.content | b64decode }}" | matrix_bot_matrix_reminder_bot_config_rendered: "{{ matrix_bot_matrix_reminder_bot_config_file.content | b64decode }}" | ||||
| # With no HTTP surface, where the database landed is the evidence that the storage | |||||
| # configuration reached the running process and not merely the file on disk. | |||||
| - name: Stat the database at the path the scenario configured | |||||
| ansible.builtin.stat: | |||||
| path: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}" | |||||
| register: matrix_bot_matrix_reminder_bot_database | |||||
| - name: Assert the database landed under the role's data path, owned by the role's user | |||||
| ansible.builtin.assert: | |||||
| that: | |||||
| - matrix_bot_matrix_reminder_bot_database.stat.exists | |||||
| - matrix_bot_matrix_reminder_bot_database.stat.uid == matrix_user_uid | |||||
| - matrix_bot_matrix_reminder_bot_database.stat.gid == matrix_user_gid | |||||
| fail_msg: >- | |||||
| {{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }} is missing or is not | |||||
| owned by {{ matrix_user_uid }}:{{ matrix_user_gid }} | |||||
| success_msg: >- | |||||
| The database is at the configured path, owned by {{ matrix_user_uid }}:{{ matrix_user_gid }} | |||||
| # Negative control for the assertion above: the role's own default database name must NOT | |||||
| # appear, or a file at the configured path would prove nothing. | |||||
| - name: Stat the database name the role would have used by default | |||||
| ansible.builtin.stat: | |||||
| path: "{{ matrix_bot_matrix_reminder_bot_data_path }}/bot.db" | |||||
| register: matrix_bot_matrix_reminder_bot_default_database | |||||
| # With no HTTP surface, the schema in Postgres is the evidence that the storage | |||||
| # configuration reached the running process rather than only the file on disk. The bot can | |||||
| # only have created tables by resolving the hostname, authenticating with the credentials | |||||
| # the role rendered, and running its migrations. | |||||
| - name: List the tables the bot created in Postgres | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - exec | |||||
| - matrix-postgres-molecule | |||||
| - psql | |||||
| - --username={{ matrix_bot_matrix_reminder_bot_database_username }} | |||||
| - --dbname={{ matrix_bot_matrix_reminder_bot_database_name }} | |||||
| - --tuples-only | |||||
| - --no-align | |||||
| - --command=SELECT tablename FROM pg_tables WHERE schemaname = 'public' | |||||
| register: matrix_bot_matrix_reminder_bot_tables | |||||
| changed_when: false | |||||
| - name: Assert the role's default database name was not used | |||||
| - name: Assert the bot created its schema in the database the role pointed it at | |||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - not matrix_bot_matrix_reminder_bot_default_database.stat.exists | |||||
| - matrix_bot_matrix_reminder_bot_tables.rc == 0 | |||||
| - matrix_bot_matrix_reminder_bot_table_names | length > 0 | |||||
| fail_msg: >- | fail_msg: >- | ||||
| {{ matrix_bot_matrix_reminder_bot_data_path }}/bot.db exists as well, so the | |||||
| database at the configured path does not prove the role's storage | |||||
| configuration reached the bot | |||||
| success_msg: "Only the configured database path was used" | |||||
| The bot created no tables in {{ matrix_bot_matrix_reminder_bot_database_name }} | |||||
| success_msg: "The bot created its schema in the database the role pointed it at" | |||||
| vars: | |||||
| matrix_bot_matrix_reminder_bot_table_names: "{{ matrix_bot_matrix_reminder_bot_tables.stdout_lines | select | list }}" | |||||
| # matrix-nio writes its encryption store here once login succeeds, so a populated directory | # matrix-nio writes its encryption store here once login succeeds, so a populated directory | ||||
| # means the bot could use the store path the role created inside a read-only container. | # means the bot could use the store path the role created inside a read-only container. | ||||
| @@ -33,9 +33,15 @@ provisioner: | |||||
| # about it. There is deliberately no Discord on the other side either. | # about it. There is deliberately no Discord on the other side either. | ||||
| matrix_bridge_mautrix_discord_homeserver_address: http://matrix.molecule.local:8008 | matrix_bridge_mautrix_discord_homeserver_address: http://matrix.molecule.local:8008 | ||||
| # sqlite keeps the scenario to one container. Which database engine the bridge can | |||||
| # talk to is not what this proves. | |||||
| matrix_bridge_mautrix_discord_database_engine: sqlite | |||||
| # Postgres, because that is what `group_vars/matrix_servers` selects whenever postgres | |||||
| # is enabled - which is the default, so it is what essentially every real deployment | |||||
| # runs. prepare.yml stands one up. Name and user differ from the role's defaults, so | |||||
| # reaching the database proves the role built its connection string out of these. | |||||
| matrix_bridge_mautrix_discord_database_engine: postgres | |||||
| matrix_bridge_mautrix_discord_database_hostname: matrix-postgres-molecule | |||||
| matrix_bridge_mautrix_discord_database_name: molecule_discord | |||||
| matrix_bridge_mautrix_discord_database_username: molecule_discord | |||||
| matrix_bridge_mautrix_discord_database_password: molecule_pg_password_a17f34 | |||||
| # Here these only have to reach the rendered configuration and the registration. | # Here these only have to reach the rendered configuration and the registration. | ||||
| matrix_bridge_mautrix_discord_appservice_token: molecule_as_token_d15c07 | matrix_bridge_mautrix_discord_appservice_token: molecule_as_token_d15c07 | ||||
| @@ -71,6 +71,15 @@ | |||||
| # The bridge contacts the homeserver as it starts and refuses to run if /whoami does not | # The bridge contacts the homeserver as it starts and refuses to run if /whoami does not | ||||
| # name the bot user it was configured as. See molecule-shared/homeserver-stub.py. | # name the bot user it was configured as. See molecule-shared/homeserver-stub.py. | ||||
| - name: Ensure Postgres is running | |||||
| ansible.builtin.include_tasks: | |||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/postgres.yml" | |||||
| vars: | |||||
| molecule_shared_postgres_network: "{{ matrix_bridge_mautrix_discord_container_network }}" | |||||
| molecule_shared_postgres_database: "{{ matrix_bridge_mautrix_discord_database_name }}" | |||||
| molecule_shared_postgres_username: "{{ matrix_bridge_mautrix_discord_database_username }}" | |||||
| molecule_shared_postgres_password: "{{ matrix_bridge_mautrix_discord_database_password }}" | |||||
| - name: Ensure the homeserver stub is running | - name: Ensure the homeserver stub is running | ||||
| ansible.builtin.include_tasks: | ansible.builtin.include_tasks: | ||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | ||||
| @@ -108,7 +108,7 @@ | |||||
| - mautrix_discord_config.appservice.bot.username == matrix_bridge_mautrix_discord_appservice_bot_username | - mautrix_discord_config.appservice.bot.username == matrix_bridge_mautrix_discord_appservice_bot_username | ||||
| - mautrix_discord_config.appservice.as_token == matrix_bridge_mautrix_discord_appservice_token | - mautrix_discord_config.appservice.as_token == matrix_bridge_mautrix_discord_appservice_token | ||||
| - mautrix_discord_config.appservice.hs_token == matrix_bridge_mautrix_discord_homeserver_token | - mautrix_discord_config.appservice.hs_token == matrix_bridge_mautrix_discord_homeserver_token | ||||
| - mautrix_discord_config.appservice.database.type == 'sqlite3' | |||||
| - mautrix_discord_config.appservice.database.type == 'postgres' | |||||
| - mautrix_discord_config.bridge.command_prefix == matrix_bridge_mautrix_discord_bridge_command_prefix | - mautrix_discord_config.bridge.command_prefix == matrix_bridge_mautrix_discord_bridge_command_prefix | ||||
| - mautrix_discord_config.bridge.avatar_proxy_key == matrix_bridge_mautrix_discord_bridge_avatar_proxy_key | - mautrix_discord_config.bridge.avatar_proxy_key == matrix_bridge_mautrix_discord_bridge_avatar_proxy_key | ||||
| - mautrix_discord_config.bridge.public_address == mautrix_discord_expected_public_address | trim | - mautrix_discord_config.bridge.public_address == mautrix_discord_expected_public_address | trim | ||||
| @@ -139,23 +139,37 @@ | |||||
| mautrix_discord_registration: "{{ mautrix_discord_registration_file.content | b64decode | from_yaml }}" | mautrix_discord_registration: "{{ mautrix_discord_registration_file.content | b64decode | from_yaml }}" | ||||
| mautrix_discord_bot_user_regex: "^@{{ matrix_bridge_mautrix_discord_appservice_bot_username | regex_escape }}:{{ matrix_bridge_mautrix_discord_homeserver_domain | regex_escape }}$" | mautrix_discord_bot_user_regex: "^@{{ matrix_bridge_mautrix_discord_appservice_bot_username | regex_escape }}:{{ matrix_bridge_mautrix_discord_homeserver_domain | regex_escape }}$" | ||||
| # Cheap proof that the data path reached the process and is writable by the uid the role | |||||
| # runs it as. | |||||
| - name: Look for the bridge's sqlite database under the role's data path | |||||
| ansible.builtin.stat: | |||||
| path: "{{ matrix_bridge_mautrix_discord_data_path }}/mautrix-discord.db" | |||||
| register: mautrix_discord_database | |||||
| # Stronger than the file-on-disk check sqlite allowed: the bridge can only have created | |||||
| # tables here by resolving the hostname, authenticating with the credentials the role | |||||
| # rendered, and running its migrations to completion. | |||||
| - name: List the tables the bridge created in Postgres | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - exec | |||||
| - matrix-postgres-molecule | |||||
| - psql | |||||
| - --username={{ matrix_bridge_mautrix_discord_database_username }} | |||||
| - --dbname={{ matrix_bridge_mautrix_discord_database_name }} | |||||
| - --tuples-only | |||||
| - --no-align | |||||
| - --command=SELECT tablename FROM pg_tables WHERE schemaname = 'public' | |||||
| register: mautrix_discord_tables | |||||
| changed_when: false | |||||
| - name: Assert the bridge created its database where the role put its data path | |||||
| - name: Assert the bridge migrated its schema into the database the role pointed it at | |||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - mautrix_discord_database.stat.exists | |||||
| - mautrix_discord_database.stat.uid | int == matrix_user_uid | int | |||||
| - mautrix_discord_tables.rc == 0 | |||||
| - "'version' in mautrix_discord_table_names" | |||||
| - mautrix_discord_table_names | length > 5 | |||||
| fail_msg: >- | fail_msg: >- | ||||
| The bridge did not create its database under | |||||
| {{ matrix_bridge_mautrix_discord_data_path }}, or it is not owned by | |||||
| uid {{ matrix_user_uid }} | |||||
| success_msg: "The bridge created its database under the role's data path, as the role's uid" | |||||
| The bridge did not create its schema in | |||||
| {{ matrix_bridge_mautrix_discord_database_name }} | |||||
| (found {{ mautrix_discord_table_names | length }} table(s)) | |||||
| success_msg: "The bridge migrated its schema into the database the role pointed it at" | |||||
| vars: | |||||
| mautrix_discord_table_names: "{{ mautrix_discord_tables.stdout_lines | select | list }}" | |||||
| - name: Read the image of the running container | - name: Read the image of the running container | ||||
| ansible.builtin.command: | ansible.builtin.command: | ||||
| @@ -34,14 +34,15 @@ provisioner: | |||||
| matrix_bridge_mautrix_meta_messenger_homeserver_address: http://matrix.molecule.local:8008 | matrix_bridge_mautrix_meta_messenger_homeserver_address: http://matrix.molecule.local:8008 | ||||
| matrix_bridge_mautrix_meta_messenger_homeserver_domain: molecule.local | matrix_bridge_mautrix_meta_messenger_homeserver_domain: molecule.local | ||||
| # sqlite keeps the scenario to one container. Which database engine the bridge can | |||||
| # talk to is not what this proves. | |||||
| matrix_bridge_mautrix_meta_messenger_database_engine: sqlite3-fk-wal | |||||
| # The URI the role derives is deliberately NOT overridden here. It used to build | |||||
| # `sqlite:///` + the in-container path, which go-sqlite3 takes as a plain filename | |||||
| # rather than parsing as a URL, so the bridge died at startup. This scenario caught | |||||
| # that, and leaving the role's own value in place is what keeps it caught. | |||||
| # Postgres, because that is what `group_vars/matrix_servers` selects whenever postgres | |||||
| # is enabled - which is the default, so it is what essentially every real deployment | |||||
| # runs. prepare.yml stands one up. Name and user differ from the role's defaults, so | |||||
| # reaching the database proves the role built its connection string out of these. | |||||
| matrix_bridge_mautrix_meta_messenger_database_engine: postgres | |||||
| matrix_bridge_mautrix_meta_messenger_database_hostname: matrix-postgres-molecule | |||||
| matrix_bridge_mautrix_meta_messenger_database_name: molecule_meta_messenger | |||||
| matrix_bridge_mautrix_meta_messenger_database_username: molecule_meta_messenger | |||||
| matrix_bridge_mautrix_meta_messenger_database_password: molecule_pg_password_d24e70 | |||||
| # Here these only have to reach the rendered configuration and the registration. | # Here these only have to reach the rendered configuration and the registration. | ||||
| matrix_bridge_mautrix_meta_messenger_appservice_token: molecule_meta_as_token_5c81de | matrix_bridge_mautrix_meta_messenger_appservice_token: molecule_meta_as_token_5c81de | ||||
| @@ -74,6 +74,15 @@ | |||||
| # The bridge calls /whoami before it will run at all, exiting if the id it gets back is | # The bridge calls /whoami before it will run at all, exiting if the id it gets back is | ||||
| # not the bot user it was configured as. It is not being asked to bridge anything: there | # not the bot user it was configured as. It is not being asked to bridge anything: there | ||||
| # is no Meta account in this scenario, and deliberately never will be. | # is no Meta account in this scenario, and deliberately never will be. | ||||
| - name: Ensure Postgres is running | |||||
| ansible.builtin.include_tasks: | |||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/postgres.yml" | |||||
| vars: | |||||
| molecule_shared_postgres_network: "{{ matrix_bridge_mautrix_meta_messenger_container_network }}" | |||||
| molecule_shared_postgres_database: "{{ matrix_bridge_mautrix_meta_messenger_database_name }}" | |||||
| molecule_shared_postgres_username: "{{ matrix_bridge_mautrix_meta_messenger_database_username }}" | |||||
| molecule_shared_postgres_password: "{{ matrix_bridge_mautrix_meta_messenger_database_password }}" | |||||
| - name: Ensure the homeserver stub is running | - name: Ensure the homeserver stub is running | ||||
| ansible.builtin.include_tasks: | ansible.builtin.include_tasks: | ||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/homeserver-stub.yml" | ||||
| @@ -138,18 +138,17 @@ | |||||
| matrix_bridge_mautrix_meta_messenger_meta_mode=facebook-tor | matrix_bridge_mautrix_meta_messenger_meta_mode=facebook-tor | ||||
| success_msg: "The rendered configuration reflects the Meta mode the scenario selected" | success_msg: "The rendered configuration reflects the Meta mode the scenario selected" | ||||
| # The URI must stay a bare path. go-sqlite3 takes it as a filename, so a | |||||
| # `sqlite:///` prefix makes the bridge die at startup - which is what this | |||||
| # role's default used to do. | |||||
| - name: Assert the configuration points the bridge at the sqlite database the role derived | |||||
| - name: Assert the configuration points the bridge at the database the role derived | |||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - mautrix_meta_messenger_config.database.type == matrix_bridge_mautrix_meta_messenger_database_engine | - mautrix_meta_messenger_config.database.type == matrix_bridge_mautrix_meta_messenger_database_engine | ||||
| - mautrix_meta_messenger_config.database.uri == mautrix_meta_messenger_role_defaults.matrix_bridge_mautrix_meta_messenger_sqlite_database_path_in_container | |||||
| - matrix_bridge_mautrix_meta_messenger_database_username in mautrix_meta_messenger_config.database.uri | |||||
| - matrix_bridge_mautrix_meta_messenger_database_name in mautrix_meta_messenger_config.database.uri | |||||
| - matrix_bridge_mautrix_meta_messenger_database_hostname in mautrix_meta_messenger_config.database.uri | |||||
| fail_msg: >- | fail_msg: >- | ||||
| database.uri is {{ mautrix_meta_messenger_config.database.uri | default('unset') }}, | database.uri is {{ mautrix_meta_messenger_config.database.uri | default('unset') }}, | ||||
| which is not the bare in-container path the role defines | |||||
| success_msg: "The rendered configuration points the bridge at the sqlite database the role derived" | |||||
| which was not built from the scenario's connection settings | |||||
| success_msg: "The rendered configuration points the bridge at the database the role derived" | |||||
| # The public address is what the role builds out of the three exposure | # The public address is what the role builds out of the three exposure | ||||
| # variables; it is the same endpoint the Traefik labels below route to. | # variables; it is the same endpoint the Traefik labels below route to. | ||||
| @@ -201,23 +200,36 @@ | |||||
| mautrix_meta_messenger_messenger_ghost_mxid: "@messenger_1234567890:{{ matrix_bridge_mautrix_meta_messenger_homeserver_domain }}" | mautrix_meta_messenger_messenger_ghost_mxid: "@messenger_1234567890:{{ matrix_bridge_mautrix_meta_messenger_homeserver_domain }}" | ||||
| mautrix_meta_messenger_bot_mxid: "@{{ matrix_bridge_mautrix_meta_messenger_appservice_username }}:{{ matrix_bridge_mautrix_meta_messenger_homeserver_domain }}" | mautrix_meta_messenger_bot_mxid: "@{{ matrix_bridge_mautrix_meta_messenger_appservice_username }}:{{ matrix_bridge_mautrix_meta_messenger_homeserver_domain }}" | ||||
| # Cheap proof that the data path reached the process and is writable by the uid the role | |||||
| # runs the container as. | |||||
| - name: Look for the bridge's sqlite database under the role's data path | |||||
| ansible.builtin.stat: | |||||
| path: "{{ matrix_bridge_mautrix_meta_messenger_data_path }}/mautrix-meta.db" | |||||
| register: mautrix_meta_messenger_database | |||||
| # The bridge can only have created tables here by resolving the hostname, authenticating | |||||
| # with the credentials the role rendered, and running its migrations to completion. | |||||
| - name: List the tables the bridge created in Postgres | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - exec | |||||
| - matrix-postgres-molecule | |||||
| - psql | |||||
| - --username={{ matrix_bridge_mautrix_meta_messenger_database_username }} | |||||
| - --dbname={{ matrix_bridge_mautrix_meta_messenger_database_name }} | |||||
| - --tuples-only | |||||
| - --no-align | |||||
| - --command=SELECT tablename FROM pg_tables WHERE schemaname = 'public' | |||||
| register: mautrix_meta_messenger_tables | |||||
| changed_when: false | |||||
| - name: Assert the bridge created its database where the role put its data path | |||||
| - name: Assert the bridge migrated its schema into the database the role pointed it at | |||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - mautrix_meta_messenger_database.stat.exists | |||||
| - mautrix_meta_messenger_database.stat.uid | int == matrix_user_uid | int | |||||
| - mautrix_meta_messenger_tables.rc == 0 | |||||
| - "'version' in mautrix_meta_messenger_table_names" | |||||
| - mautrix_meta_messenger_table_names | length > 5 | |||||
| fail_msg: >- | fail_msg: >- | ||||
| The bridge did not create its database under | |||||
| {{ matrix_bridge_mautrix_meta_messenger_data_path }}, or it is not owned by | |||||
| uid {{ matrix_user_uid }} | |||||
| success_msg: "The bridge created its database under the role's data path, as the role's uid" | |||||
| The bridge did not create its schema in | |||||
| {{ matrix_bridge_mautrix_meta_messenger_database_name }} | |||||
| (found {{ mautrix_meta_messenger_table_names | length }} table(s)) | |||||
| success_msg: "The bridge migrated its schema into the database the role pointed it at" | |||||
| vars: | |||||
| mautrix_meta_messenger_table_names: "{{ mautrix_meta_messenger_tables.stdout_lines | select | list }}" | |||||
| - name: Read the image of the running container | - name: Read the image of the running container | ||||
| ansible.builtin.command: | ansible.builtin.command: | ||||
| @@ -33,9 +33,18 @@ provisioner: | |||||
| # about it. | # about it. | ||||
| matrix_bridge_mautrix_whatsapp_homeserver_address: http://matrix.molecule.local:8008 | matrix_bridge_mautrix_whatsapp_homeserver_address: http://matrix.molecule.local:8008 | ||||
| # sqlite keeps the scenario to one container. Which database engine the bridge can | |||||
| # talk to is not what this proves. | |||||
| matrix_bridge_mautrix_whatsapp_database_engine: sqlite | |||||
| # Postgres, because that is what `group_vars/matrix_servers` selects whenever postgres | |||||
| # is enabled - which is the default, so it is what essentially every real deployment | |||||
| # runs. prepare.yml stands one up. The password differs from the role's default, so | |||||
| # reaching the database at all means the role's connection string was built from these. | |||||
| # Database name and user differ from the role's defaults, so the bridge reaching the | |||||
| # database at all proves the role built its connection string out of these rather | |||||
| # than out of anything it would have picked on its own. prepare.yml creates them. | |||||
| matrix_bridge_mautrix_whatsapp_database_engine: postgres | |||||
| matrix_bridge_mautrix_whatsapp_database_hostname: matrix-postgres-molecule | |||||
| matrix_bridge_mautrix_whatsapp_database_name: molecule_whatsapp | |||||
| matrix_bridge_mautrix_whatsapp_database_username: molecule_whatsapp | |||||
| matrix_bridge_mautrix_whatsapp_database_password: molecule_pg_password_5e8c21 | |||||
| # Here these only have to reach the rendered configuration and the registration. | # Here these only have to reach the rendered configuration and the registration. | ||||
| matrix_bridge_mautrix_whatsapp_appservice_token: molecule_as_token_4f2a91 | matrix_bridge_mautrix_whatsapp_appservice_token: molecule_as_token_4f2a91 | ||||
| @@ -69,6 +69,15 @@ | |||||
| - mautrix_whatsapp_molecule_network.rc != 0 | - mautrix_whatsapp_molecule_network.rc != 0 | ||||
| - "'already exists' not in mautrix_whatsapp_molecule_network.stderr" | - "'already exists' not in mautrix_whatsapp_molecule_network.stderr" | ||||
| - name: Ensure Postgres is running | |||||
| ansible.builtin.include_tasks: | |||||
| file: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/../../../molecule-shared/tasks/postgres.yml" | |||||
| vars: | |||||
| molecule_shared_postgres_network: "{{ matrix_bridge_mautrix_whatsapp_container_network }}" | |||||
| molecule_shared_postgres_database: "{{ matrix_bridge_mautrix_whatsapp_database_name }}" | |||||
| molecule_shared_postgres_username: "{{ matrix_bridge_mautrix_whatsapp_database_username }}" | |||||
| molecule_shared_postgres_password: "{{ matrix_bridge_mautrix_whatsapp_database_password }}" | |||||
| # The bridge contacts the homeserver as it starts. It is not being asked to bridge | # The bridge contacts the homeserver as it starts. It is not being asked to bridge | ||||
| # anything. See molecule-shared/homeserver-stub.py. | # anything. See molecule-shared/homeserver-stub.py. | ||||
| - name: Ensure the homeserver stub is running | - name: Ensure the homeserver stub is running | ||||
| @@ -117,23 +117,37 @@ | |||||
| vars: | vars: | ||||
| mautrix_whatsapp_registration_rendered: "{{ mautrix_whatsapp_registration_file.content | b64decode }}" | mautrix_whatsapp_registration_rendered: "{{ mautrix_whatsapp_registration_file.content | b64decode }}" | ||||
| # Cheap proof that the data path reached the process and is writable by the uid the role | |||||
| # runs it as. | |||||
| - name: Look for the bridge's sqlite database under the role's data path | |||||
| ansible.builtin.stat: | |||||
| path: "{{ matrix_bridge_mautrix_whatsapp_data_path }}/mautrix-whatsapp.db" | |||||
| register: mautrix_whatsapp_database | |||||
| - name: Assert the bridge created its database where the role put its data path | |||||
| # Stronger than the file-on-disk check sqlite allowed: the bridge can only have created | |||||
| # tables here by resolving the hostname, authenticating with the credentials the role | |||||
| # rendered, and running its migrations to completion. | |||||
| - name: List the tables the bridge created in Postgres | |||||
| ansible.builtin.command: | |||||
| argv: | |||||
| - docker | |||||
| - exec | |||||
| - matrix-postgres-molecule | |||||
| - psql | |||||
| - --username={{ matrix_bridge_mautrix_whatsapp_database_username }} | |||||
| - --dbname={{ matrix_bridge_mautrix_whatsapp_database_name }} | |||||
| - --tuples-only | |||||
| - --no-align | |||||
| - --command=SELECT tablename FROM pg_tables WHERE schemaname = 'public' | |||||
| register: mautrix_whatsapp_tables | |||||
| changed_when: false | |||||
| - name: Assert the bridge migrated its schema into the database the role pointed it at | |||||
| ansible.builtin.assert: | ansible.builtin.assert: | ||||
| that: | that: | ||||
| - mautrix_whatsapp_database.stat.exists | |||||
| - mautrix_whatsapp_database.stat.uid | int == matrix_user_uid | int | |||||
| - mautrix_whatsapp_tables.rc == 0 | |||||
| - "'version' in mautrix_whatsapp_table_names" | |||||
| - mautrix_whatsapp_table_names | length > 5 | |||||
| fail_msg: >- | fail_msg: >- | ||||
| The bridge did not create its database under | |||||
| {{ matrix_bridge_mautrix_whatsapp_data_path }}, or it is not owned by | |||||
| uid {{ matrix_user_uid }} | |||||
| success_msg: "The bridge created its database under the role's data path, as the role's uid" | |||||
| The bridge did not create its schema in | |||||
| {{ matrix_bridge_mautrix_whatsapp_database_name }} | |||||
| (found {{ mautrix_whatsapp_table_names | length }} table(s)) | |||||
| success_msg: "The bridge migrated its schema into the database the role pointed it at" | |||||
| vars: | |||||
| mautrix_whatsapp_table_names: "{{ mautrix_whatsapp_tables.stdout_lines | select | list }}" | |||||
| - name: Read the image of the running container | - name: Read the image of the running container | ||||
| ansible.builtin.command: | ansible.builtin.command: | ||||