Ensure the homeserver accepts matrix-bot-meowlnir's appservice token before provisioning

The homeserver only reads appservice registrations on startup, and the
playbook only restarts it at the very end of a run. On the run which
first enables Meowlnir, bot provisioning therefore talked to a
homeserver still running without Meowlnir's registration, hitting
M_UNKNOWN_TOKEN and hanging indefinitely.

Provisioning now verifies the token upfront and restarts the homeserver
itself when needed, so a single run completes even when Meowlnir was
just enabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs/configuring-playbook-bot-meowlnir.md

@@ -275,6 +275,8 @@ With `management_room_auto_create`, you then have an invitation waiting for you
 
 - Re-running is safe and idempotent, so adding a bot later is a matter of extending the list and running the same command again.
 
+- If the homeserver turns out to be running without Meowlnir's appservice registration (which is the normal state of affairs on the run that first enables Meowlnir), the playbook restarts the homeserver during bot provisioning, so that everything completes in a single run.
+
 - The shortcut commands with the [`just` program](just.md) are also available: `just install-all` or `just setup-all`
 
 ## Usage

group_vars/matrix_servers

@@ -3426,6 +3426,12 @@ matrix_bot_meowlnir_container_additional_networks_auto: |-
 matrix_bot_meowlnir_config_homeserver_address: "{{ matrix_addons_homeserver_client_api_url }}"
 matrix_bot_meowlnir_config_homeserver_domain: "{{ matrix_domain }}"
 
+# Provisioning bots (`ensure-matrix-users-created`) requires the homeserver to accept Meowlnir's appservice registration, which it only reads on startup.
+# On the run which first enables Meowlnir, the homeserver is still running without it, so the role restarts these services to load it.
+# Restarting the main service suffices even with Synapse workers, because their units declare `Requires=matrix-synapse.service` and systemd propagates the restart to them.
+# Left empty for homeserver implementations whose appservice registrations the playbook does not manage (Conduit and its forks), where registration is a manual step anyway.
+matrix_bot_meowlnir_homeserver_restart_systemd_services_list: "{{ ['matrix-' ~ matrix_homeserver_implementation ~ '.service'] if matrix_homeserver_implementation in ['synapse', 'dendrite'] else [] }}"
+
 # Only consulted by bots configured with `management_room_auto_create`, and only as the
 # default for those which do not carry their own `initial_managers` list.
 # May legitimately be empty, in which case such bots fail validation and ask to be told who

roles/custom/matrix-bot-meowlnir/defaults/main.yml

@@ -147,6 +147,19 @@ matrix_bot_meowlnir_bots_start_wait_time_seconds: 15
 # Bounded so that an API which accepts connections but never answers turns into a clear error instead of hanging the playbook (or a human caller) forever.
 matrix_bot_meowlnir_api_request_timeout_seconds: 60
 
+# The systemd services to restart if the homeserver turns out to be running without Meowlnir's appservice registration when bots are about to be provisioned.
+#
+# The homeserver only reads appservice registrations on startup, so on the run which first enables Meowlnir it typically still runs without this one and rejects Meowlnir's appservice token (`M_UNKNOWN_TOKEN`).
+# When that is detected, the services listed here are restarted so that provisioning can proceed within the same run.
+#
+# A playbook may inject the homeserver's own systemd service here for homeserver implementations whose appservice registrations it manages.
+# When the list is empty, no restart is attempted and provisioning fails with instructions instead.
+matrix_bot_meowlnir_homeserver_restart_systemd_services_list: []
+
+# How many times (and how many seconds apart) to re-ask the homeserver whether it accepts Meowlnir's appservice token, while waiting for it to come up.
+matrix_bot_meowlnir_appservice_token_check_retries_count: 30
+matrix_bot_meowlnir_appservice_token_check_retries_delay_seconds: 5
+
 # The name and topic given to management rooms that the playbook creates.
 # Only used by bots with `management_room_auto_create` enabled.
 matrix_bot_meowlnir_management_room_name: Meowlnir management room

roles/custom/matrix-bot-meowlnir/tasks/setup_bots.yml

@@ -21,6 +21,9 @@
     seconds: "{{ matrix_bot_meowlnir_bots_start_wait_time_seconds }}"
   when: matrix_bot_meowlnir_start_result.changed | bool
 
+- name: Ensure the homeserver accepts Meowlnir's appservice token before creating bots
+  ansible.builtin.include_tasks: "{{ role_path }}/tasks/util/ensure_appservice_token_accepted.yml"
+
 - name: Read which bots Meowlnir already has
   ansible.builtin.include_tasks: "{{ role_path }}/tasks/util/read_meowlnir_state.yml"
 

roles/custom/matrix-bot-meowlnir/tasks/util/ensure_appservice_token_accepted.yml

@@ -0,0 +1,94 @@
+# SPDX-FileCopyrightText: 2026 Slavi Pantaleev
+#
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+---
+
+# The homeserver only reads appservice registrations on startup, so on the run which first enables Meowlnir it typically still runs without this one and rejects Meowlnir's appservice token with `M_UNKNOWN_TOKEN`.
+# Meowlnir then cannot register bot users, and the whole provisioning flow below would fail in confusing ways.
+# Restarts are normally left to the systemd service manager at the very end of the playbook, which is too late for the provisioning happening here, so this is checked upfront and the homeserver is restarted early when needed.
+#
+# The checks retry for a while, because the homeserver may still be starting up — an early request fails to connect, or gets a `502` from a reverse proxy in front of the homeserver.
+
+- name: Check whether the homeserver accepts Meowlnir's appservice token
+  block:
+    - name: Ask the homeserver whether it accepts Meowlnir's appservice token
+      ansible.builtin.command:
+        cmd: "{{ matrix_bot_meowlnir_bin_path }}/meowlnir-whoami"
+      register: matrix_bot_meowlnir_whoami_result
+      changed_when: false
+      retries: "{{ matrix_bot_meowlnir_appservice_token_check_retries_count }}"
+      delay: "{{ matrix_bot_meowlnir_appservice_token_check_retries_delay_seconds }}"
+      until: >-
+        matrix_bot_meowlnir_whoami_result.rc == 0
+        and (matrix_bot_meowlnir_whoami_result.stdout_lines | default([]) | length > 0)
+        and (matrix_bot_meowlnir_whoami_result.stdout_lines | last in ['200', '401'])
+  rescue:
+    - name: Fail because the homeserver could not be asked about Meowlnir's appservice token
+      ansible.builtin.fail:
+        msg: >-
+          {{
+            'Could not get a usable answer from the homeserver about Meowlnir\'s appservice token. The last attempt said: '
+            ~ (matrix_bot_meowlnir_whoami_result.stdout | default('') | trim)
+            ~ ' '
+            ~ (matrix_bot_meowlnir_whoami_result.stderr | default('') | trim)
+          }}
+
+- name: Ensure the homeserver picks up Meowlnir's appservice registration
+  when: matrix_bot_meowlnir_whoami_result.stdout_lines | last == '401'
+  block:
+    - name: Fail if the homeserver rejects Meowlnir's appservice token and the playbook does not know how to help
+      ansible.builtin.fail:
+        msg: >-
+          {{
+            'The homeserver rejects Meowlnir\'s appservice token (M_UNKNOWN_TOKEN), meaning it runs without Meowlnir\'s appservice registration, '
+            ~ 'and `matrix_bot_meowlnir_homeserver_restart_systemd_services_list` is empty, so the playbook does not know which service to restart to fix that. '
+            ~ 'If the playbook manages appservice registrations for your homeserver implementation, restart the homeserver (e.g. by running the playbook with `--tags=setup-all,start`) and re-run this tag. '
+            ~ 'Otherwise, register `' ~ matrix_bot_meowlnir_config_path ~ '/registration.yaml` with your homeserver manually, restart it, and re-run this tag.'
+          }}
+      when: matrix_bot_meowlnir_homeserver_restart_systemd_services_list | length == 0
+
+    - name: Ensure the homeserver is restarted, so that it picks up Meowlnir's appservice registration
+      ansible.builtin.service:
+        name: "{{ item }}"
+        state: restarted
+        daemon_reload: true
+      with_items: "{{ matrix_bot_meowlnir_homeserver_restart_systemd_services_list }}"
+
+    # The re-check deliberately registers a differently-named variable.
+    # Registration happens even for skipped tasks, so reusing the name would clobber the result which the surrounding block's `when` keeps being evaluated against.
+    - name: Check whether the homeserver accepts Meowlnir's appservice token after the restart
+      block:
+        - name: Ask the homeserver again whether it accepts Meowlnir's appservice token
+          ansible.builtin.command:
+            cmd: "{{ matrix_bot_meowlnir_bin_path }}/meowlnir-whoami"
+          register: matrix_bot_meowlnir_whoami_recheck_result
+          changed_when: false
+          retries: "{{ matrix_bot_meowlnir_appservice_token_check_retries_count }}"
+          delay: "{{ matrix_bot_meowlnir_appservice_token_check_retries_delay_seconds }}"
+          until: >-
+            matrix_bot_meowlnir_whoami_recheck_result.rc == 0
+            and (matrix_bot_meowlnir_whoami_recheck_result.stdout_lines | default([]) | length > 0)
+            and (matrix_bot_meowlnir_whoami_recheck_result.stdout_lines | last == '200')
+      rescue:
+        - name: Fail because the homeserver still rejects Meowlnir's appservice token after being restarted
+          ansible.builtin.fail:
+            msg: >-
+              {{
+                'The homeserver still rejects Meowlnir\'s appservice token after a restart. The last attempt said: '
+                ~ (matrix_bot_meowlnir_whoami_recheck_result.stdout | default('') | trim)
+                ~ ' '
+                ~ (matrix_bot_meowlnir_whoami_recheck_result.stderr | default('') | trim)
+                ~ ' Check the homeserver\'s logs and make sure it is configured to load Meowlnir\'s appservice registration (`' ~ matrix_bot_meowlnir_config_path ~ '/registration.yaml`).'
+              }}
+
+    # Until the homeserver restart above, Meowlnir had only ever seen its token rejected.
+    # It does recover on its own eventually, but a fresh start is quicker and also clears any state left behind by earlier failed provisioning attempts (a bot creation request which never got to finish keeps its management API busy indefinitely).
+    - name: Ensure matrix-bot-meowlnir is restarted, so that it reconnects to the homeserver cleanly
+      ansible.builtin.service:
+        name: matrix-bot-meowlnir.service
+        state: restarted
+
+    - name: Wait a while, so that Meowlnir can manage to start after reconnecting to the homeserver
+      ansible.builtin.pause:
+        seconds: "{{ matrix_bot_meowlnir_bots_start_wait_time_seconds }}"