Merge branch 'master' of github.com:spantaleev/matrix-docker-ansible-deploy

.envrc

@@ -0,0 +1 @@
+use flake

.github/workflows/matrix.yml

@@ -21,6 +21,6 @@ jobs:
       - name: Check out
         uses: actions/checkout@v3
       - name: Run ansible-lint
-        uses: ansible-community/ansible-lint-action@v6.11.0
+        uses: ansible-community/ansible-lint-action@v6.15.0
         with:
           path: roles/custom

.gitignore

@@ -5,6 +5,7 @@
 /roles/**/files/scratchpad
 .DS_Store
 .python-version
+flake.lock
 
 # ignore roles pulled by ansible-galaxy
 /roles/galaxy/*

docs/configuring-playbook-ldap-auth.md

@@ -8,7 +8,9 @@ If you decide that you'd like to let this playbook install it for you, you need
 
 ```yaml
 matrix_synapse_ext_password_provider_ldap_enabled: true
-matrix_synapse_ext_password_provider_ldap_uri: "ldap://ldap.mydomain.tld:389"
+matrix_synapse_ext_password_provider_ldap_uri: 
+  - "ldap://ldap-01.mydomain.tld:389"
+  - "ldap://ldap-02.mydomain.tld:389"
 matrix_synapse_ext_password_provider_ldap_start_tls: true
 matrix_synapse_ext_password_provider_ldap_base: "ou=users,dc=example,dc=com"
 matrix_synapse_ext_password_provider_ldap_attributes_uid: "uid"

docs/configuring-playbook-synapse.md

@@ -42,8 +42,6 @@ devture_postgres_process_extra_arguments: [
 ]
 ```
 
-**NOTE**: Disabling `matrix-nginx-proxy` (`matrix_nginx_proxy_enabled: false`) (that is, [using your own other webserver](configuring-playbook-own-webserver.md) when running a Synapse worker setup is likely to cause various troubles (see [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090)).
-
 In case any problems occur, make sure to have a look at the [list of synapse issues about workers](https://github.com/matrix-org/synapse/issues?q=workers+in%3Atitle) and your `journalctl --unit 'matrix-*'`.
 
 
@@ -119,3 +117,10 @@ matrix_synapse_container_image_customizations_templates_git_repository_ssh_priva
 
 As mentioned in Synapse's Templates documentation, Synapse will fall back to its own templates if a template is not found in that directory.
 Due to this, it's recommended to only store and maintain template files in your repository if you need to make custom changes. Other files (which you don't need to change), should not be duplicated, so that you don't need to worry about getting out-of-sync with the original Synapse templates.
+
+
+## Monitoring Synapse Metrics with Prometheus and Grafana
+
+This playbook allows you to enable Synapse metrics, which can provide insight into the performance and activity of Synapse.
+
+To enable Synapse metrics see [`configuring-playbook-prometheus-grafana.md`](./configuring-playbook-prometheus-grafana.md)

docs/configuring-playbook.md

@@ -12,7 +12,7 @@ You can then follow these steps inside the playbook directory:
 
 1. copy the sample configuration file (`cp examples/vars.yml inventory/host_vars/matrix.<your-domain>/vars.yml`)
 
-1. edit the configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to your liking. You may also take a look at the various `roles/ROLE_NAME_HERE/defaults/main.yml` files and see if there's something you'd like to copy over and override in your `vars.yml` configuration file.
+1. edit the configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to your liking. You may also take a look at the various `roles/*/ROLE_NAME_HERE/defaults/main.yml` files and see if there's something you'd like to copy over and override in your `vars.yml` configuration file.
 
 1. copy the sample inventory hosts file (`cp examples/hosts inventory/hosts`)
 

docs/importing-synapse-sqlite.md

@@ -3,24 +3,28 @@
 Run this if you'd like to import your database from a previous default installation of Synapse.
 (don't forget to import your `media_store` files as well - see [the importing-synapse-media-store guide](importing-synapse-media-store.md)).
 
-While this playbook always sets up PostgreSQL, by default a Synapse installation would run
-using an SQLite database.
+While this playbook only supports running Synapse in combination with PostgreSQL, a Synapse instance installed manually usually defaults to using an SQLite database.
 
-If you have such a Synapse setup and wish to migrate it here (and over to PostgreSQL), this command is for you.
+If you have such a Synapse setup and wish to migrate it to one managed by the playbook (and over to PostgreSQL), this documentation page is for you.
 
 
 ## Prerequisites
 
-Before doing the actual import, **you need to upload your SQLite database file to the server** (any path is okay).
+Before doing the actual import:
 
+- **ensure you have NOT started Synapse yet**. That is, make sure you have followed the [Installing step](installing.md), but haven't run the playbook's `start` tag yet. If you had started your new Synapse instance, it may have already initialized your Postgres database and importing onto it may not work. In such cases, you may need to clean up the `synapse` database first.
+- **ensure you have uploaded your SQLite database file to the server** (any path is okay)
+- if you're using the integrated Postgres server (**by default, you are** using it, unless you've explicitly switched to [Using an external PostgreSQL server](configuring-playbook-external-postgres.md)), **make sure Postgres is started** by running `just start-group postgres`
 
 ## Importing
 
 Run this command (make sure to replace `<server-path-to-homeserver.db>` with a file path on your server):
 
-	ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_homeserver_db=<server-path-to-homeserver.db>' --tags=import-synapse-sqlite-db
+```sh
+just run-tags import-synapse-sqlite-db --extra-vars=server_path_homeserver_db=<server-path-to-homeserver.db>
+```
 
 **Notes**:
 
-- `<server-path-to-homeserver.db>` must be a file path to a `homeserver.db` **file on the server** (not on your local machine!).
+- `<server-path-to-homeserver.db>` must be replaced with a file path to a `homeserver.db` **file on the server** (not on your local machine!).
 - if the SQLite database is from an older version of Synapse, the **importing procedure may run migrations on it to bring it up to date**. That is, your SQLite database file may get modified and become unusable with your older Synapse version. Keeping a copy of the original is probably wise.

examples/nginx/README.md

@@ -10,7 +10,7 @@ To get started, first follow the [front the integrated reverse-proxy webserver w
 
 ## Using the nginx configuration
 
-Copy the [matrix.conf](matrix.conf) file to your nginx server's filesystem, modify it to your needs and include it your nginx configuration (e.g. `include /path/to/matrix.conf;`).
+Copy the [matrix.conf](matrix.conf) file to your nginx server's filesystem, modify it to your needs and include it in your nginx configuration (e.g. `include /path/to/matrix.conf;`).
 
 This configuration **disables SSL certificate retrieval**, so you will **need to obtain SSL certificates manually** (e.g. by using [certbot](https://certbot.eff.org/)) and set the appropriate path in `matrix.conf`. In the example nginx configuration, a single certificate is used for all subdomains (`matrix.DOMAIN`, `element.DOMAIN`, etc.). For your setup, may wish to change this and use separate `server` blocks and separate certificate files for each host.
 

flake.nix

@@ -0,0 +1,19 @@
+{
+  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
+
+  outputs = { self, nixpkgs, ... }:
+    let
+      pkgs = import nixpkgs { system = "x86_64-linux"; };
+    in
+    {
+      devShell.x86_64-linux = pkgs.mkShell {
+        buildInputs = with pkgs; [
+          just
+          python311Packages.ansible-core
+          python311Packages.passlib
+        ];
+        LC_ALL = "C.UTF-8";
+        LC_CTYPE = "C.UTF-8";
+      };
+    };
+}

group_vars/matrix_servers

@@ -2576,7 +2576,14 @@ matrix_nginx_proxy_container_federation_host_bind_port: "{{ matrix_federation_pu
 matrix_nginx_proxy_trust_forwarded_proto: "{{ matrix_playbook_reverse_proxy_type != 'playbook-managed-nginx' }}"
 matrix_nginx_proxy_x_forwarded_for: "{{ '$remote_addr' if matrix_playbook_reverse_proxy_type == 'playbook-managed-nginx' else '$proxy_add_x_forwarded_for' }}"
 
-matrix_nginx_proxy_container_additional_networks: "{{ [matrix_playbook_reverse_proxyable_services_additional_network] if matrix_playbook_reverse_proxyable_services_additional_network else [] }}"
+matrix_nginx_proxy_container_additional_networks: |
+  {{
+    (
+      ([matrix_playbook_reverse_proxyable_services_additional_network] if matrix_playbook_reverse_proxyable_services_additional_network else [])
+      +
+      ([jitsi_container_network] if jitsi_enabled and matrix_playbook_reverse_proxy_type == 'playbook-managed-nginx' and jitsi_container_network != matrix_nginx_proxy_container_network else [])
+    ) | unique
+  }}
 
 matrix_nginx_proxy_proxy_matrix_client_api_addr_with_container: "{{ 'matrix-corporal:41080' if matrix_corporal_enabled else 'matrix-nginx-proxy:12080' }}"
 matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container: "{{ '127.0.0.1:41080' if matrix_corporal_enabled else '127.0.0.1:12080' }}"

requirements.yml

@@ -1,10 +1,12 @@
+---
+
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-aux.git
   version: v1.0.0-0
   name: aux
 - src: git+https://gitlab.com/etke.cc/roles/backup_borg.git
-  version: v1.2.4-1.7.11-1
+  version: v1.2.4-1.7.12-1
 - src: git+https://github.com/devture/com.devture.ansible.role.container_socket_proxy.git
-  version: v0.1.1-1
+  version: v0.1.1-2
 - src: git+https://github.com/devture/com.devture.ansible.role.docker_sdk_for_python.git
   version: 129c8590e106b83e6f4c259649a613c6279e937a
 - src: git+https://github.com/devture/com.devture.ansible.role.playbook_help.git
@@ -18,13 +20,13 @@
 - src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git
   version: 8e9ec48a09284c84704d7a2dce17da35f181574d
 - src: git+https://github.com/devture/com.devture.ansible.role.systemd_docker_base.git
-  version: 327d2e17f5189ac2480d6012f58cf64a2b46efba
-- src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git
   version: v1.0.0-0
+- src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git
+  version: v1.0.0-1
 - src: git+https://github.com/devture/com.devture.ansible.role.timesync.git
-  version: 3d5bb2976815958cdce3f368fa34fb51554f899b
+  version: v1.0.0-0
 - src: git+https://github.com/devture/com.devture.ansible.role.traefik.git
-  version: v2.9.9-0
+  version: v2.9.10-2
 - src: git+https://github.com/devture/com.devture.ansible.role.traefik_certs_dumper.git
   version: v2.8.1-0
 - src: git+https://gitlab.com/etke.cc/roles/etherpad.git
@@ -33,12 +35,12 @@
   version: 6.1.0
   name: geerlingguy.docker
 - src: git+https://gitlab.com/etke.cc/roles/grafana.git
-  version: v9.4.7-1
+  version: v9.5.1-0
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-jitsi.git
-  version: v8319-6
+  version: v8615-0
   name: jitsi
 - src: git+https://gitlab.com/etke.cc/roles/ntfy.git
-  version: v2.3.1-0
+  version: v2.4.0-0
 - src: git+https://github.com/mother-of-all-self-hosting/ansible-role-prometheus.git
   version: v2.43.0-0
   name: prometheus

roles/custom/matrix-bot-buscarron/defaults/main.yml

@@ -14,6 +14,10 @@ matrix_bot_buscarron_hostname: ''
 # This value must either be `/` or not end with a slash (e.g. `/buscarron`).
 matrix_bot_buscarron_path_prefix: /
 
+# The path at which Buscarron will expose metrics
+# This value must either be `/` or not end with a slash (e.g. `/metrics`).
+matrix_bot_buscarron_metrics_path: /metrics
+
 matrix_bot_buscarron_base_path: "{{ matrix_base_data_path }}/buscarron"
 matrix_bot_buscarron_config_path: "{{ matrix_bot_buscarron_base_path }}/config"
 matrix_bot_buscarron_data_path: "{{ matrix_bot_buscarron_base_path }}/data"
@@ -36,6 +40,15 @@ matrix_bot_buscarron_container_network: matrix-bot-buscarron
 # Use this to expose this container to another reverse proxy, which runs in a different container network.
 matrix_bot_buscarron_container_additional_networks: []
 
+# enable basic auth for metrics
+matrix_bot_buscarron_basicauth_enabled: false
+# temporary file name on the host that runs ansible
+matrix_bot_buscarron_basicauth_file: "/tmp/matrix_bot_buscarron_htpasswd"
+# username
+matrix_bot_buscarron_basicauth_user: ''
+# password
+matrix_bot_buscarron_basicauth_password: ''
+
 # matrix_bot_buscarron_container_labels_traefik_enabled controls whether labels to assist a Traefik reverse-proxy will be attached to the container.
 # See `../templates/labels.j2` for details.
 #
@@ -46,6 +59,8 @@ matrix_bot_buscarron_container_labels_traefik_hostname: "{{ matrix_bot_buscarron
 # The path prefix must either be `/` or not end with a slash (e.g. `/buscarron`).
 matrix_bot_buscarron_container_labels_traefik_path_prefix: "{{ matrix_bot_buscarron_path_prefix }}"
 matrix_bot_buscarron_container_labels_traefik_rule: "Host(`{{ matrix_bot_buscarron_container_labels_traefik_hostname }}`){% if matrix_bot_buscarron_container_labels_traefik_path_prefix != '/' %} && PathPrefix(`{{ matrix_bot_buscarron_container_labels_traefik_path_prefix }}`){% endif %}"
+matrix_bot_buscarron_container_labels_traefik_metrics_path: "{{ matrix_bot_buscarron_metrics_path }}"
+matrix_bot_buscarron_container_labels_traefik_metrics_rule: "Host(`{{ matrix_bot_buscarron_container_labels_traefik_hostname }}`) && Path(`{{ matrix_bot_buscarron_container_labels_traefik_metrics_path }}`)"
 matrix_bot_buscarron_container_labels_traefik_priority: 0
 matrix_bot_buscarron_container_labels_traefik_entrypoints: web-secure
 matrix_bot_buscarron_container_labels_traefik_tls: "{{ matrix_bot_buscarron_container_labels_traefik_entrypoints != 'web' }}"

roles/custom/matrix-bot-buscarron/tasks/setup_install.yml

@@ -40,6 +40,21 @@
     - {path: "{{ matrix_bot_buscarron_docker_src_files_path }}", when: true}
   when: "item.when | bool"
 
+- name: Determine basicauth filename
+  ansible.builtin.set_fact:
+    matrix_bot_buscarron_basicauth_file_tmp: "{{ matrix_bot_buscarron_basicauth_file }}_{{ inventory_hostname }}"
+  when: matrix_bot_buscarron_basicauth_enabled | bool
+
+- name: Generate basic auth file
+  community.general.htpasswd:
+    path: "{{ matrix_bot_buscarron_basicauth_file }}"
+    name: "{{ matrix_bot_buscarron_basicauth_user }}"
+    password: "{{ matrix_bot_buscarron_basicauth_password }}"
+    mode: 0640
+  become: false
+  delegate_to: 127.0.0.1
+  when: matrix_bot_buscarron_basicauth_enabled | bool
+
 - name: Ensure buscarron support files installed
   ansible.builtin.template:
     src: "{{ role_path }}/templates/{{ item }}.j2"
@@ -51,6 +66,14 @@
     - env
     - labels
 
+- name: Ensure temporary basic auth file is removed
+  ansible.builtin.file:
+    path: "{{ matrix_bot_buscarron_basicauth_file }}"
+    state: absent
+  become: false
+  delegate_to: 127.0.0.1
+  when: matrix_bot_buscarron_basicauth_enabled | bool
+
 - name: Ensure buscarron image is pulled
   community.docker.docker_image:
     name: "{{ matrix_bot_buscarron_docker_image }}"

roles/custom/matrix-bot-buscarron/templates/labels.j2

@@ -6,6 +6,7 @@ traefik.docker.network={{ matrix_bot_buscarron_container_labels_traefik_docker_n
 {% endif %}
 
 {% set middlewares = [] %}
+{% set middlewares_metrics = [] %}
 
 {% if matrix_bot_buscarron_container_labels_traefik_path_prefix != '/' %}
 traefik.http.middlewares.matrix-bot-buscarron-slashless-redirect.redirectregex.regex=({{ matrix_bot_buscarron_container_labels_traefik_path_prefix | quote }})$
@@ -18,6 +19,11 @@ traefik.http.middlewares.matrix-bot-buscarron-strip-prefix.stripprefix.prefixes=
 {% set middlewares = middlewares + ['matrix-bot-buscarron-strip-prefix'] %}
 {% endif %}
 
+{% if matrix_bot_buscarron_basicauth_enabled %}
+traefik.http.middlewares.matrix-bot-buscarron-auth.basicauth.users={{ lookup('ansible.builtin.file', matrix_bot_buscarron_basicauth_file) }}
+{% set middlewares_metrics = middlewares + ['matrix-bot-buscarron-auth'] %}
+{% endif %}
+
 {% if matrix_bot_buscarron_container_labels_traefik_additional_response_headers.keys() | length > 0 %}
 {% for name, value in matrix_bot_buscarron_container_labels_traefik_additional_response_headers.items() %}
 traefik.http.middlewares.matrix-bot-buscarron-add-headers.headers.customresponseheaders.{{ name }}={{ value }}
@@ -38,8 +44,23 @@ traefik.http.routers.matrix-bot-buscarron.tls={{ matrix_bot_buscarron_container_
 {% if matrix_bot_buscarron_container_labels_traefik_tls %}
 traefik.http.routers.matrix-bot-buscarron.tls.certResolver={{ matrix_bot_buscarron_container_labels_traefik_tls_certResolver }}
 {% endif %}
-
 traefik.http.services.matrix-bot-buscarron.loadbalancer.server.port=8080
+
+{% if middlewares_metrics | length > 0 %}
+traefik.http.routers.matrix-bot-buscarron-metrics.rule={{ matrix_bot_buscarron_container_labels_traefik_metrics_rule }}
+{% if matrix_bot_buscarron_container_labels_traefik_priority | int > 0 %}
+traefik.http.routers.matrix-bot-buscarron-metrics.priority={{ matrix_bot_buscarron_container_labels_traefik_priority }}
+{% endif %}
+traefik.http.routers.matrix-bot-buscarron-metrics.service=matrix-bot-buscarron
+traefik.http.routers.matrix-bot-buscarron-metrics.middlewares={{ middlewares_metrics | join(',') }}
+traefik.http.routers.matrix-bot-buscarron-metrics.entrypoints={{ matrix_bot_buscarron_container_labels_traefik_entrypoints }}
+traefik.http.routers.matrix-bot-buscarron-metrics.tls={{ matrix_bot_buscarron_container_labels_traefik_tls | to_json }}
+{% if matrix_bot_buscarron_container_labels_traefik_tls %}
+traefik.http.routers.matrix-bot-buscarron-metrics.tls.certResolver={{ matrix_bot_buscarron_container_labels_traefik_tls_certResolver }}
+{% endif %}
+traefik.http.services.matrix-bot-buscarron-metrics.loadbalancer.server.port=8080
+{% endif %}
+
 {% endif %}
 
 {{ matrix_bot_buscarron_container_labels_additional_labels }}

roles/custom/matrix-bot-honoroit/defaults/main.yml

@@ -11,6 +11,10 @@ matrix_bot_honoroit_hostname: ''
 # This value must either be `/` or not end with a slash (e.g. `/honoroit`).
 matrix_bot_honoroit_path_prefix: /
 
+# The path at which honoroit will expose metrics
+# This value must either be `/` or not end with a slash (e.g. `/metrics`).
+matrix_bot_honoroit_metrics_path: /metrics
+
 matrix_bot_honoroit_container_image_self_build: false
 matrix_bot_honoroit_docker_repo: "https://gitlab.com/etke.cc/honoroit.git"
 matrix_bot_honoroit_docker_repo_version: "{{ matrix_bot_honoroit_version }}"
@@ -34,6 +38,15 @@ matrix_bot_honoroit_container_network: matrix-bot-honoroit
 # Use this to expose this container to another reverse proxy, which runs in a different container network.
 matrix_bot_honoroit_container_additional_networks: []
 
+# enable basic auth for metrics
+matrix_bot_honoroit_basicauth_enabled: false
+# temporary file name on the host that runs ansible
+matrix_bot_honoroit_basicauth_file: "/tmp/matrix_bot_honoroit_htpasswd"
+# username
+matrix_bot_honoroit_basicauth_user: ''
+# password
+matrix_bot_honoroit_basicauth_password: ''
+
 # matrix_bot_honoroit_container_labels_traefik_enabled controls whether labels to assist a Traefik reverse-proxy will be attached to the container.
 # See `../templates/labels.j2` for details.
 #
@@ -44,6 +57,8 @@ matrix_bot_honoroit_container_labels_traefik_hostname: "{{ matrix_bot_honoroit_h
 # The path prefix must either be `/` or not end with a slash (e.g. `/honoroit`).
 matrix_bot_honoroit_container_labels_traefik_path_prefix: "{{ matrix_bot_honoroit_path_prefix }}"
 matrix_bot_honoroit_container_labels_traefik_rule: "Host(`{{ matrix_bot_honoroit_container_labels_traefik_hostname }}`){% if matrix_bot_honoroit_container_labels_traefik_path_prefix != '/' %} && PathPrefix(`{{ matrix_bot_honoroit_container_labels_traefik_path_prefix }}`){% endif %}"
+matrix_bot_honoroit_container_labels_traefik_metrics_path: "{{ matrix_bot_honoroit_metrics_path }}"
+matrix_bot_honoroit_container_labels_traefik_metrics_rule: "Host(`{{ matrix_bot_honoroit_container_labels_traefik_hostname }}`) && Path(`{{ matrix_bot_honoroit_container_labels_traefik_metrics_path }}`)"
 matrix_bot_honoroit_container_labels_traefik_priority: 0
 matrix_bot_honoroit_container_labels_traefik_entrypoints: web-secure
 matrix_bot_honoroit_container_labels_traefik_tls: "{{ matrix_bot_honoroit_container_labels_traefik_entrypoints != 'web' }}"

roles/custom/matrix-bot-honoroit/tasks/setup_install.yml

@@ -40,6 +40,21 @@
     - {path: "{{ matrix_bot_honoroit_docker_src_files_path }}", when: true}
   when: "item.when | bool"
 
+- name: Determine basicauth filename
+  ansible.builtin.set_fact:
+    matrix_bot_honoroit_basicauth_file_tmp: "{{ matrix_bot_honoroit_basicauth_file }}_{{ inventory_hostname }}"
+  when: matrix_bot_honoroit_basicauth_enabled | bool
+
+- name: Generate basic auth file
+  community.general.htpasswd:
+    path: "{{ matrix_bot_honoroit_basicauth_file }}"
+    name: "{{ matrix_bot_honoroit_basicauth_user }}"
+    password: "{{ matrix_bot_honoroit_basicauth_password }}"
+    mode: 0640
+  become: false
+  delegate_to: 127.0.0.1
+  when: matrix_bot_honoroit_basicauth_enabled | bool
+
 - name: Ensure honoroit support files installed
   ansible.builtin.template:
     src: "{{ role_path }}/templates/{{ item }}.j2"
@@ -51,6 +66,14 @@
     - env
     - labels
 
+- name: Ensure temporary basic auth file is removed
+  ansible.builtin.file:
+    path: "{{ matrix_bot_honoroit_basicauth_file }}"
+    state: absent
+  become: false
+  delegate_to: 127.0.0.1
+  when: matrix_bot_honoroit_basicauth_enabled | bool
+
 - name: Ensure honoroit image is pulled
   community.docker.docker_image:
     name: "{{ matrix_bot_honoroit_docker_image }}"
@@ -86,6 +109,11 @@
       pull: true
   when: "matrix_bot_honoroit_container_image_self_build | bool"
 
+- name: Ensure honoroit container network is created
+  community.general.docker_network:
+    name: "{{ matrix_bot_honoroit_container_network }}"
+    driver: bridge
+
 - name: Ensure matrix-bot-honoroit.service installed
   ansible.builtin.template:
     src: "{{ role_path }}/templates/systemd/matrix-bot-honoroit.service.j2"

roles/custom/matrix-bot-honoroit/templates/labels.j2

@@ -6,6 +6,7 @@ traefik.docker.network={{ matrix_bot_honoroit_container_labels_traefik_docker_ne
 {% endif %}
 
 {% set middlewares = [] %}
+{% set middlewares_metrics = [] %}
 
 {% if matrix_bot_honoroit_container_labels_traefik_path_prefix != '/' %}
 traefik.http.middlewares.matrix-bot-honoroit-slashless-redirect.redirectregex.regex=({{ matrix_bot_honoroit_container_labels_traefik_path_prefix | quote }})$
@@ -25,6 +26,11 @@ traefik.http.middlewares.matrix-bot-honoroit-add-headers.headers.customresponseh
 {% set middlewares = middlewares + ['matrix-bot-honoroit-add-headers'] %}
 {% endif %}
 
+{% if matrix_bot_honoroit_basicauth_enabled %}
+traefik.http.middlewares.matrix-bot-honoroit-auth.basicauth.users={{ lookup('ansible.builtin.file', matrix_bot_honoroit_basicauth_file) }}
+{% set middlewares_metrics = middlewares + ['matrix-bot-honoroit-auth'] %}
+{% endif %}
+
 traefik.http.routers.matrix-bot-honoroit.rule={{ matrix_bot_honoroit_container_labels_traefik_rule }}
 {% if matrix_bot_honoroit_container_labels_traefik_priority | int > 0 %}
 traefik.http.routers.matrix-bot-honoroit.priority={{ matrix_bot_honoroit_container_labels_traefik_priority }}
@@ -38,8 +44,23 @@ traefik.http.routers.matrix-bot-honoroit.tls={{ matrix_bot_honoroit_container_la
 {% if matrix_bot_honoroit_container_labels_traefik_tls %}
 traefik.http.routers.matrix-bot-honoroit.tls.certResolver={{ matrix_bot_honoroit_container_labels_traefik_tls_certResolver }}
 {% endif %}
-
 traefik.http.services.matrix-bot-honoroit.loadbalancer.server.port=8080
+
+{% if middlewares_metrics | length > 0 %}
+traefik.http.routers.matrix-bot-honoroit-metrics.rule={{ matrix_bot_honoroit_container_labels_traefik_metrics_rule }}
+{% if matrix_bot_honoroit_container_labels_traefik_priority | int > 0 %}
+traefik.http.routers.matrix-bot-honoroit-metrics.priority={{ matrix_bot_honoroit_container_labels_traefik_priority }}
+{% endif %}
+traefik.http.routers.matrix-bot-honoroit-metrics.service=matrix-bot-honoroit
+traefik.http.routers.matrix-bot-honoroit-metrics.middlewares={{ middlewares_metrics | join(',') }}
+traefik.http.routers.matrix-bot-honoroit-metrics.entrypoints={{ matrix_bot_honoroit_container_labels_traefik_entrypoints }}
+traefik.http.routers.matrix-bot-honoroit-metrics.tls={{ matrix_bot_honoroit_container_labels_traefik_tls | to_json }}
+{% if matrix_bot_honoroit_container_labels_traefik_tls %}
+traefik.http.routers.matrix-bot-honoroit-metrics.tls.certResolver={{ matrix_bot_honoroit_container_labels_traefik_tls_certResolver }}
+{% endif %}
+traefik.http.services.matrix-bot-honoroit-metrics.loadbalancer.server.port=8080
+{% endif %}
+
 {% endif %}
 
 {{ matrix_bot_honoroit_container_labels_additional_labels }}

roles/custom/matrix-bot-maubot/defaults/main.yml

@@ -10,7 +10,7 @@ matrix_bot_maubot_docker_src_files_path: "{{ matrix_bot_maubot_base_path }}/dock
 matrix_bot_maubot_docker_repo_version: "{{ 'master' if matrix_bot_maubot_version == 'latest' else matrix_bot_maubot_version }}"
 
 
-matrix_bot_maubot_version: v0.4.0
+matrix_bot_maubot_version: v0.4.1
 matrix_bot_maubot_docker_image: "{{ matrix_bot_maubot_docker_image_name_prefix }}maubot/maubot:{{ matrix_bot_maubot_version }}"
 matrix_bot_maubot_docker_image_name_prefix: "{{ 'localhost/' if matrix_bot_maubot_container_image_self_build else 'dock.mau.dev/' }}"
 matrix_bot_maubot_docker_image_force_pull: "{{ matrix_bot_maubot_docker_image.endswith(':latest') }}"

roles/custom/matrix-bridge-appservice-irc/defaults/main.yml

@@ -11,7 +11,7 @@ matrix_appservice_irc_docker_src_files_path: "{{ matrix_base_data_path }}/appser
 
 # matrix_appservice_irc_version used to contain the full Docker image tag (e.g. `release-X.X.X`).
 # It's a bare version number now. We try to somewhat retain compatibility below.
-matrix_appservice_irc_version: 0.37.1
+matrix_appservice_irc_version: 0.38.0
 matrix_appservice_irc_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/matrix-appservice-irc:{{ matrix_appservice_irc_docker_image_tag }}"
 matrix_appservice_irc_docker_image_tag: "{{ 'latest' if matrix_appservice_irc_version == 'latest' else ('release-' + matrix_appservice_irc_version) }}"
 matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_image.endswith(':latest') }}"

roles/custom/matrix-bridge-hookshot/defaults/main.yml

@@ -10,7 +10,7 @@ matrix_hookshot_container_image_self_build: false
 matrix_hookshot_container_image_self_build_repo: "https://github.com/matrix-org/matrix-hookshot.git"
 matrix_hookshot_container_image_self_build_branch: "{{ 'main' if matrix_hookshot_version == 'latest' else matrix_hookshot_version }}"
 
-matrix_hookshot_version: 3.2.0
+matrix_hookshot_version: 4.0.0
 
 matrix_hookshot_docker_image: "{{ matrix_hookshot_docker_image_name_prefix }}halfshot/matrix-hookshot:{{ matrix_hookshot_version }}"
 matrix_hookshot_docker_image_name_prefix: "{{ 'localhost/' if matrix_hookshot_container_image_self_build else matrix_container_global_registry_prefix }}"

roles/custom/matrix-bridge-mautrix-discord/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_mautrix_discord_container_image_self_build: false
 matrix_mautrix_discord_container_image_self_build_repo: "https://mau.dev/mautrix/discord.git"
 matrix_mautrix_discord_container_image_self_build_branch: "{{ 'main' if matrix_mautrix_discord_version == 'latest' else matrix_mautrix_discord_version }}"
 
-matrix_mautrix_discord_version: v0.2.0
+matrix_mautrix_discord_version: v0.3.0
 # See: https://mau.dev/mautrix/discord/container_registry
 matrix_mautrix_discord_docker_image: "{{ matrix_mautrix_discord_docker_image_name_prefix }}mautrix/discord:{{ matrix_mautrix_discord_version }}"
 matrix_mautrix_discord_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_discord_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-bridge-mautrix-signal/defaults/main.yml

@@ -10,7 +10,7 @@ matrix_mautrix_signal_docker_repo_version: "{{ 'master' if matrix_mautrix_signal
 matrix_mautrix_signal_docker_src_files_path: "{{ matrix_base_data_path }}/mautrix-signal/docker-src"
 
 matrix_mautrix_signal_version: v0.4.2
-matrix_mautrix_signal_daemon_version: 0.23.1
+matrix_mautrix_signal_daemon_version: 0.23.2
 # See: https://mau.dev/mautrix/signal/container_registry
 matrix_mautrix_signal_docker_image: "{{ matrix_mautrix_signal_docker_image_name_prefix }}mautrix/signal:{{ matrix_mautrix_signal_version }}"
 matrix_mautrix_signal_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_signal_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-bridge-mautrix-whatsapp/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_mautrix_whatsapp_container_image_self_build: false
 matrix_mautrix_whatsapp_container_image_self_build_repo: "https://mau.dev/mautrix/whatsapp.git"
 matrix_mautrix_whatsapp_container_image_self_build_branch: "{{ 'master' if matrix_mautrix_whatsapp_version == 'latest' else matrix_mautrix_whatsapp_version }}"
 
-matrix_mautrix_whatsapp_version: v0.8.3
+matrix_mautrix_whatsapp_version: v0.8.4
 # See: https://mau.dev/mautrix/whatsapp/container_registry
 matrix_mautrix_whatsapp_docker_image: "{{ matrix_mautrix_whatsapp_docker_image_name_prefix }}mautrix/whatsapp:{{ matrix_mautrix_whatsapp_version }}"
 matrix_mautrix_whatsapp_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_whatsapp_container_image_self_build else 'dock.mau.dev/' }}"

roles/custom/matrix-client-element/defaults/main.yml

@@ -10,7 +10,7 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
 # - https://github.com/vector-im/element-web/issues/19544
 matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
 
-matrix_client_element_version: v1.11.28
+matrix_client_element_version: v1.11.30
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

roles/custom/matrix-coturn/tasks/setup_install.yml

@@ -73,7 +73,6 @@
     src: "{{ role_path }}/templates/systemd/matrix-coturn.service.j2"
     dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-coturn.service"
     mode: 0644
-  register: matrix_coturn_systemd_service_change_results
 
 # This may be unnecessary when more long-lived certificates are used.
 # We optimize for the common use-case though (short-lived Let's Encrypt certificates).
@@ -83,7 +82,6 @@
     src: "{{ role_path }}/templates/systemd/{{ item }}.j2"
     dest: "{{ devture_systemd_docker_base_systemd_path }}/{{ item }}"
     mode: 0644
-  register: "matrix_coturn_systemd_service_change_results"
   when: "matrix_coturn_tls_enabled | bool"
   with_items:
     - matrix-coturn-reload.service
@@ -94,13 +92,7 @@
   ansible.builtin.file:
     path: "{{ item }}"
     state: absent
-  register: "matrix_coturn_systemd_service_change_results"
   when: "not matrix_coturn_tls_enabled | bool"
   with_items:
     - matrix-coturn-reload.service
     - matrix-coturn-reload.timer
-
-- name: Ensure systemd reloaded if systemd units changed
-  ansible.builtin.service:
-    daemon_reload: true
-  when: "matrix_coturn_systemd_service_change_results.changed"

roles/custom/matrix-synapse-auto-compressor/tasks/main.yml

@@ -1,20 +1,20 @@
 ---
 
-- block:
+- tags:
+    - setup-all
+    - setup-synapse-auto-compressor
+    - install-all
+    - install-synapse-auto-compressor
+  block:
     - when: matrix_synapse_auto_compressor_enabled | bool
       ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml"
 
     - when: matrix_synapse_auto_compressor_enabled | bool
       ansible.builtin.include_tasks: "{{ role_path }}/tasks/install.yml"
-  tags:
+
+- tags:
     - setup-all
     - setup-synapse-auto-compressor
-    - install-all
-    - install-synapse-auto-compressor
-
-- block:
+  block:
     - when: not matrix_synapse_auto_compressor_enabled | bool
       ansible.builtin.include_tasks: "{{ role_path }}/tasks/uninstall.yml"
-  tags:
-    - setup-all
-    - setup-synapse-auto-compressor

roles/custom/matrix-synapse/defaults/main.yml

@@ -4,7 +4,7 @@
 
 matrix_synapse_enabled: true
 
-matrix_synapse_version: v1.80.0
+matrix_synapse_version: v1.82.0
 
 matrix_synapse_username: ''
 matrix_synapse_uid: ''

roles/custom/matrix-synapse/tasks/import_synapse_sqlite_db.yml

@@ -0,0 +1,40 @@
+---
+
+- name: Fail if playbook called incorrectly
+  ansible.builtin.fail:
+    msg: "The `server_path_homeserver_db` variable needs to be provided to this playbook, via --extra-vars"
+  when: "server_path_homeserver_db is not defined or server_path_homeserver_db.startswith('<')"
+
+- name: Check if the provided SQLite homeserver.db file exists
+  ansible.builtin.stat:
+    path: "{{ server_path_homeserver_db }}"
+  register: result_server_path_homeserver_db_stat
+
+- name: Fail if provided SQLite homeserver.db file doesn't exist
+  ansible.builtin.fail:
+    msg: "File cannot be found on the server at {{ server_path_homeserver_db }}"
+  when: "not result_server_path_homeserver_db_stat.stat.exists"
+
+# We don't use the `docker_container` module, because using it with `cap_drop` requires
+# a very recent version, which is not available for a lot of people yet.
+#
+# Also, some old `docker_container` versions were buggy and would leave containers behind
+# on failure, which we had to work around to allow retries (by re-running the playbook).
+- name: Import SQLite database into Postgres
+  ansible.builtin.command:
+    cmd: |
+      docker run
+      --rm
+      --name=matrix-synapse-migrate
+      --log-driver=none
+      --user={{ matrix_user_uid }}:{{ matrix_user_gid }}
+      --cap-drop=ALL
+      --network={{ matrix_synapse_container_network }}
+      --entrypoint=python
+      --mount type=bind,src={{ matrix_synapse_config_dir_path }},dst=/data
+      --mount type=bind,src={{ matrix_synapse_config_dir_path }},dst=/matrix-media-store-parent/media-store
+      --mount type=bind,src={{ server_path_homeserver_db }},dst=/{{ server_path_homeserver_db | basename }}
+      {{ matrix_synapse_docker_image_final }}
+      /usr/local/bin/synapse_port_db --sqlite-database /{{ server_path_homeserver_db | basename }} --postgres-config /data/homeserver.yaml
+  register: matrix_postgres_import_synapse_sqlite_db_result
+  changed_when: matrix_postgres_import_synapse_sqlite_db_result.rc == 0

roles/custom/matrix-synapse/tasks/main.yml

@@ -45,6 +45,12 @@
     - when: matrix_synapse_enabled | bool
       ansible.builtin.include_tasks: "{{ role_path }}/tasks/import_media_store.yml"
 
+- tags:
+    - import-synapse-sqlite-db
+  block:
+    - when: matrix_synapse_enabled | bool
+      ansible.builtin.include_tasks: "{{ role_path }}/tasks/import_synapse_sqlite_db.yml"
+
 - tags:
     - register-user
   block:

roles/custom/matrix-synapse/tasks/rust-synapse-compress-state/compress_room.yml

@@ -9,7 +9,7 @@
       {{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-rust-synapse-compress-state-compress-room
       --user={{ matrix_synapse_uid }}:{{ matrix_synapse_gid }}
       --cap-drop=ALL
-      --network={{ matrix_docker_network }}
+      --network={{ matrix_synapse_container_network }}
       --mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work
       {{ matrix_synapse_rust_synapse_compress_state_docker_image }}
       {{ matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path }} -t -o /work/state-compressor.sql

roles/custom/matrix-synapse/tasks/synapse/workers/util/inject_worker.yml

@@ -61,7 +61,7 @@
 
 # Inject stream writers into the instance map.
 - ansible.builtin.set_fact:
-    matrix_synapse_instance_map: "{{ matrix_synapse_instance_map | combine({matrix_synapse_worker_details.name: {'host': matrix_synapse_worker_details.name, 'port': matrix_synapse_worker_details.replication_port}}) }}"
+    matrix_synapse_instance_map: "{{ matrix_synapse_instance_map | combine({matrix_synapse_worker_details.name: {'host': matrix_synapse_worker_details.name, 'port': matrix_synapse_worker_details.replication_port | int}}) }}"
   when: matrix_synapse_worker_details.type in matrix_synapse_known_instance_map_eligible_worker_types
 
 # Inject pusher instances.

roles/custom/matrix-synapse/templates/synapse/ext/s3-storage-provider/bin/migrate.j2

@@ -7,7 +7,7 @@
 	--mount type=bind,src={{ matrix_synapse_storage_path }},dst=/matrix-media-store-parent,bind-propagation=slave \
 	--mount type=bind,src={{ matrix_synapse_ext_s3_storage_provider_data_path }},dst=/data \
 	--workdir=/data \
-	--network={{ matrix_docker_network }} \
+	--network={{ matrix_synapse_container_network }} \
 	--entrypoint=/bin/bash \
 	{{ matrix_synapse_docker_image_final }} \
 	-c 's3_media_upload update-db $UPDATE_DB_DURATION && s3_media_upload --no-progress check-deleted $MEDIA_PATH && s3_media_upload --no-progress upload $MEDIA_PATH $BUCKET --delete --storage-class $STORAGE_CLASS --endpoint-url $ENDPOINT {% if matrix_synapse_ext_synapse_s3_storage_provider_config_sse_customer_enabled %}--sse-customer-algo $SSE_CUSTOMER_ALGO --sse-customer-key $SSE_CUSTOMER_KEY{% endif %}'

roles/custom/matrix-synapse/templates/synapse/ext/s3-storage-provider/bin/shell.j2

@@ -8,6 +8,6 @@
 	--mount type=bind,src={{ matrix_synapse_storage_path }},dst=/matrix-media-store-parent,bind-propagation=slave \
 	--mount type=bind,src={{ matrix_synapse_ext_s3_storage_provider_data_path }},dst=/data \
 	--workdir=/data \
-	--network={{ matrix_docker_network }} \
+	--network={{ matrix_synapse_container_network }} \
 	--entrypoint=/bin/bash \
 	{{ matrix_synapse_docker_image_final }}

roles/custom/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -1223,12 +1223,12 @@ oembed:
 # This homeserver's ReCAPTCHA public key. Must be specified if
 # enable_registration_captcha is enabled.
 #
-recaptcha_public_key: {{ matrix_synapse_recaptcha_public_key|to_json }}
+recaptcha_public_key: {{ matrix_synapse_recaptcha_public_key|string|to_json }}
 
 # This homeserver's ReCAPTCHA private key. Must be specified if
 # enable_registration_captcha is enabled.
 #
-recaptcha_private_key: {{ matrix_synapse_recaptcha_private_key|to_json }}
+recaptcha_private_key: {{ matrix_synapse_recaptcha_private_key|string|to_json }}
 
 # Uncomment to enable ReCaptcha checks when registering, preventing signup
 # unless a captcha is answered. Requires a valid ReCaptcha
@@ -2515,7 +2515,7 @@ password_providers:
     config:
       enabled: true
       mode: {{ matrix_synapse_ext_password_provider_ldap_mode | string | to_json }}
-      uri: {{ matrix_synapse_ext_password_provider_ldap_uri | string|to_json }}
+      uri: {{ matrix_synapse_ext_password_provider_ldap_uri | to_json }}
       start_tls: {{ matrix_synapse_ext_password_provider_ldap_start_tls|to_json }}
       base: {{ matrix_synapse_ext_password_provider_ldap_base | string|to_json }}
       active_directory: {{ matrix_synapse_ext_password_provider_ldap_active_directory|to_json }}

roles/custom/matrix-synapse/templates/synapse/systemd/matrix-synapse-worker.service.j2

@@ -22,7 +22,7 @@ ExecStartPre={{ devture_systemd_docker_base_host_command_docker }} create \
 			--cap-drop=ALL \
 			--read-only \
 			--tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_synapse_tmp_directory_size_mb }}m \
-			--network={{ matrix_docker_network }} \
+			--network={{ matrix_synapse_container_network }} \
 			{% if matrix_synapse_worker_details.port != 0 %}
 			--health-cmd 'curl -fSs http://localhost:{{ matrix_synapse_worker_details.port }}/health || exit 1' \
 			{% else %}

roles/custom/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2

@@ -38,7 +38,7 @@ ExecStartPre={{ devture_systemd_docker_base_host_command_docker }} create \
 			--cap-drop=ALL \
 			--read-only \
 			--tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_synapse_tmp_directory_size_mb }}m \
-			--network={{ matrix_docker_network }} \
+			--network={{ matrix_synapse_container_network }} \
 			{% if matrix_synapse_container_client_api_host_bind_port %}
 			-p {{ matrix_synapse_container_client_api_host_bind_port }}:{{ matrix_synapse_container_client_api_port }} \
 			{% endif %}

roles/custom/matrix-synapse/vars/main.yml

@@ -166,6 +166,8 @@ matrix_synapse_workers_generic_worker_endpoints:
   - ^/_matrix/client/(api/v1|r0|v3|unstable/.*)/rooms/.*/aliases
   - ^/_matrix/client/(api/v1|r0|v3|unstable)/search$
   - ^/_matrix/client/(r0|v3|unstable)/user/.*/filter(/|$)
+  - ^/_matrix/client/(api/v1|r0|v3|unstable)/directory/room/.*$
+  - ^/_matrix/client/(r0|v3|unstable)/capabilities$
 
   # Encryption requests
   # Note that ^/_matrix/client/(r0|v3|unstable)/keys/upload/ requires `worker_main_http_uri`