Merge branch 'spantaleev:master' into master

docs/configuring-playbook-bridge-hookshot.md

@@ -16,10 +16,29 @@ Refer to the [official instructions](https://matrix-org.github.io/matrix-hooksho
 2. Take special note of the `matrix_hookshot_*_enabled` variables. Services that need no further configuration are enabled by default (GitLab, Generic), while you must first add the required configuration and enable the others (GitHub, Jira, Figma).
 3. If you're setting up the GitHub bridge, you'll need to generate and download a private key file after you created your GitHub app. Copy the contents of that file to the variable `matrix_hookshot_github_private_key` so the playbook can install it for you, or use one of the [other methods](#manage-github-private-key-with-matrix-aux-role) explained below. 
 4. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready. Hookshot can be set up individually using the tag `setup-hookshot`.
-5. Refer to [Hookshot's official instructions](https://matrix-org.github.io/matrix-hookshot/usage.html) to start using the bridge. Note that the different listeners are bound to certain paths (see `matrix_hookshot_matrix_nginx_proxy_configuration` in [init.yml](/roles/matrix-bridge-hookshot/tasks/init.yml)): by default webhooks root is `/hookshot/webhooks/`.
+5. Refer to [Hookshot's official instructions](https://matrix-org.github.io/matrix-hookshot/usage.html) to start using the bridge. **Important:** Note that the different listeners are bound to certain paths which might differe from those assumed by the hookshot documentation, see [URLs for bridges setup](urls-for-bridges-setup) below.
 
 Other configuration options are available via the `matrix_hookshot_configuration_extension_yaml` and `matrix_hookshot_registration_extension_yaml` variables, see the comments in [main.yml](/roles/matrix-bridge-hookshot/defaults/main.yml) for how to use them.
 
+### URLs for bridges setup
+
+All of the following endpoints are reachable on your `matrix.` subdomain (if the feature is enabled).
+
+| Listener | default path | variable | used as |
+|---|---|---|---|
+| webhooks | `/hookshot/webhooks/` | `matrix_hookshot_webhook_endpoint` | GitHub "Webhook URL" |
+| github oauth | `/hookshot/webhooks/oauth` | `matrix_hookshot_github_oauth_endpoint` | GitHub "Callback URL" |
+| jira oauth | `/hookshot/webhooks/jira/oauth` | `matrix_hookshot_jira_oauth_endpoint` | JIRA OAuth |
+| figma endpoint | `/hookshot/webhooks/figma/webhook` | `matrix_hookshot_figma_endpoint` | Figma |
+| provisioning | `/hookshot/v1/` | `matrix_hookshot_provisioning_endpoint` | Dimension [provisioning](#provisioning-api) |
+| appservice | `/hookshot/_matrix/app/` | `matrix_hookshot_appservice_endpoint` | Matrix server |
+| metrics | `/hookshot/metrics/` | `matrix_hookshot_metrics_endpoint` | Prometheus |
+| widgets |  |  | not supported |
+
+See also `matrix_hookshot_matrix_nginx_proxy_configuration` in [init.yml](/roles/matrix-bridge-hookshot/tasks/init.yml).
+
+The different listeners are also reachable *internally* in the docker-network via the container's name (configured by `matrix_hookshot_container_url`) and on different ports (e.g. `matrix_hookshot_appservice_port`). Read [main.yml](/roles/matrix-bridge-hookshot/defaults/main.yml) in detail for more info.
+
 ### Manage GitHub Private Key with matrix-aux role
 
 The GitHub bridge requires you to install a private key file. This can be done in multiple ways:

docs/configuring-playbook-dimension.md

@@ -31,7 +31,7 @@ matrix_dimension_admins:
   - "@user2:{{ matrix_domain }}"
 ```
 
-Admin interface is accessible at `https://dimension.<your-domain>/riot-app/admin` after logging in to element and opening it in any room via "Edit widgets, bridges & bots" and then clicking the "settings"-icon in the upper right corner.
+The admin interface is accessible within Element by accessing it in any room and clicking the cog wheel/settings icon in the top right. Currently, Dimension can be opened in Element by the "Add widgets, bridges, & bots" link in the room information.
 
 ## Access token
 

docs/configuring-playbook-prometheus-grafana.md

@@ -25,6 +25,8 @@ matrix_grafana_default_admin_password: "some_strong_password_chosen_by_you"
 
 By default, a [Grafana](https://grafana.com/) web user-interface will be available at `https://stats.<your-domain>`.
 
+The retention policy of Prometheus metrics is [15 days by default](https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects). Older data gets deleted automatically.
+
 
 ## What does it do?
 

roles/matrix-bot-mjolnir/defaults/main.yml

@@ -4,7 +4,7 @@
 
 matrix_bot_mjolnir_enabled: true
 
-matrix_bot_mjolnir_version: "v1.3.2"
+matrix_bot_mjolnir_version: "v1.4.1"
 
 matrix_bot_mjolnir_container_image_self_build: false
 matrix_bot_mjolnir_container_image_self_build_repo: "https://github.com/matrix-org/mjolnir.git"

roles/matrix-bridge-appservice-irc/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_appservice_irc_container_image_self_build: false
 matrix_appservice_irc_docker_repo: "https://github.com/matrix-org/matrix-appservice-irc.git"
 matrix_appservice_irc_docker_src_files_path: "{{ matrix_base_data_path }}/appservice-irc/docker-src"
 
-matrix_appservice_irc_version: release-0.32.1
+matrix_appservice_irc_version: release-0.33.0
 matrix_appservice_irc_docker_image: "{{ matrix_container_global_registry_prefix }}matrixdotorg/matrix-appservice-irc:{{ matrix_appservice_irc_version }}"
 matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_image.endswith(':latest') }}"
 

roles/matrix-bridge-hookshot/defaults/main.yml

@@ -5,7 +5,7 @@
 
 matrix_hookshot_enabled: true
 
-matrix_hookshot_version: 1.2.0
+matrix_hookshot_version: 1.3.0
 matrix_hookshot_docker_image: "{{ matrix_container_global_registry_prefix }}halfshot/matrix-hookshot:{{ matrix_hookshot_version }}"
 matrix_hookshot_docker_image_force_pull: "{{ matrix_hookshot_docker_image.endswith(':latest') }}"
 

roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2

@@ -106,7 +106,9 @@ bridge:
     # If using this for other servers than the bridge's server,
     # you must also set the URL in the double_puppet_server_map.
     login_shared_secret_map: {{ matrix_mautrix_facebook_bridge_login_shared_secret_map|to_json }}
-    presence: {{ matrix_mautrix_facebook_bridge_presence|to_json }}
+    # Should presence from Facebook be bridged? This doesn't use the same API as the Android app,
+    # so it might be more suspicious to Facebook.
+    presence_from_facebook: {{ matrix_mautrix_facebook_bridge_presence|to_json }}
     # Whether or not to update avatars when syncing all contacts at startup.
     update_avatar_initial_sync: true
     # End-to-bridge encryption support options. These require matrix-nio to be installed with pip

roles/matrix-bridge-mautrix-googlechat/defaults/main.yml

@@ -7,7 +7,7 @@ matrix_mautrix_googlechat_enabled: true
 matrix_mautrix_googlechat_container_image_self_build: false
 matrix_mautrix_googlechat_container_image_self_build_repo: "https://github.com/mautrix/googlechat.git"
 
-matrix_mautrix_googlechat_version: latest
+matrix_mautrix_googlechat_version: v0.3.1
 # See: https://mau.dev/mautrix/googlechat/container_registry
 matrix_mautrix_googlechat_docker_image: "{{ matrix_mautrix_googlechat_docker_image_name_prefix }}mautrix/googlechat:{{ matrix_mautrix_googlechat_version }}"
 matrix_mautrix_googlechat_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_googlechat_container_image_self_build else 'dock.mau.dev/' }}"

roles/matrix-bridge-mautrix-whatsapp/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_mautrix_whatsapp_container_image_self_build: false
 matrix_mautrix_whatsapp_container_image_self_build_repo: "https://mau.dev/mautrix/whatsapp.git"
 matrix_mautrix_whatsapp_container_image_self_build_branch: "{{ 'master' if matrix_mautrix_whatsapp_version == 'latest' else matrix_mautrix_whatsapp_version }}"
 
-matrix_mautrix_whatsapp_version: v0.2.4
+matrix_mautrix_whatsapp_version: v0.3.0
 # See: https://mau.dev/mautrix/whatsapp/container_registry
 matrix_mautrix_whatsapp_docker_image: "{{ matrix_mautrix_whatsapp_docker_image_name_prefix }}mautrix/whatsapp:{{ matrix_mautrix_whatsapp_version }}"
 matrix_mautrix_whatsapp_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_whatsapp_container_image_self_build else 'dock.mau.dev/' }}"

roles/matrix-client-element/defaults/main.yml

@@ -9,7 +9,7 @@ matrix_client_element_container_image_self_build_repo: "https://github.com/vecto
 # - https://github.com/vector-im/element-web/issues/19544
 matrix_client_element_container_image_self_build_low_memory_system_patch_enabled: "{{ ansible_memtotal_mb < 4096 }}"
 
-matrix_client_element_version: v1.10.7
+matrix_client_element_version: v1.10.8
 matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:{{ matrix_client_element_version }}"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else matrix_container_global_registry_prefix }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"

roles/matrix-nginx-proxy/templates/prometheus/external_prometheus.yml.example.j2

@@ -22,7 +22,7 @@ scrape_configs:
       - targets: ['{{ matrix_server_fqn_matrix }}:{{ matrix_nginx_proxy_container_https_host_bind_port if matrix_nginx_proxy_https_enabled else matrix_nginx_proxy_container_http_host_bind_port }}']
         labels:
           job: "master"
-          index: 1
+          index: "0"
 {% for worker in matrix_nginx_proxy_proxy_synapse_workers_enabled_list %}
   - job_name: 'synapse-{{ worker.type }}-{{ worker.instanceId }}'
     metrics_path: /_synapse-worker-{{ worker.type }}-{{ worker.instanceId }}/metrics
@@ -36,5 +36,5 @@ scrape_configs:
       - targets: ['{{ matrix_server_fqn_matrix }}:{{ matrix_nginx_proxy_container_https_host_bind_port if matrix_nginx_proxy_https_enabled else matrix_nginx_proxy_container_http_host_bind_port }}']
         labels:
           job: "{{ worker.type }}"
-          index: {{ worker.instanceId }}
+          index: "{{ worker.instanceId }}"
 {% endfor %}

roles/matrix-synapse/defaults/main.yml

@@ -9,7 +9,7 @@ matrix_synapse_container_image_self_build_repo: "https://github.com/matrix-org/s
 
 matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:{{ matrix_synapse_docker_image_tag }}"
 matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else matrix_container_global_registry_prefix }}"
-matrix_synapse_version: v1.54.0
+matrix_synapse_version: v1.55.2
 matrix_synapse_docker_image_tag: "{{ matrix_synapse_version }}"
 matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"
 
@@ -525,7 +525,7 @@ matrix_synapse_ext_spam_checker_synapse_simple_antispam_config_blocked_homeserve
 # See: https://github.com/matrix-org/mjolnir#synapse-module
 matrix_synapse_ext_spam_checker_mjolnir_antispam_enabled: false
 matrix_synapse_ext_spam_checker_mjolnir_antispam_git_repository_url: "https://github.com/matrix-org/mjolnir"
-matrix_synapse_ext_spam_checker_mjolnir_antispam_git_version: "70f353fbbad0af469b1001080dea194d512b2815"
+matrix_synapse_ext_spam_checker_mjolnir_antispam_git_version: "4008e3f65d3745b9307dd31f1c5aa80c13a61a58"
 matrix_synapse_ext_spam_checker_mjolnir_antispam_config_block_invites: true
 # Flag messages sent by servers/users in the ban lists as spam. Currently
 # this means that spammy messages will appear as empty to users. Default
@@ -620,10 +620,11 @@ matrix_synapse_redaction_retention_period: 7d
 matrix_synapse_user_ips_max_age: 28d
 
 
-matrix_synapse_rust_synapse_compress_state_docker_image: "devture/rust-synapse-compress-state:v0.1.0"
+matrix_synapse_rust_synapse_compress_state_docker_image: "registry.gitlab.com/mb-saces/rust-synapse-compress-state:latest"
 matrix_synapse_rust_synapse_compress_state_docker_image_force_pull: "{{ matrix_synapse_rust_synapse_compress_state_docker_image.endswith(':latest') }}"
 
 matrix_synapse_rust_synapse_compress_state_base_path: "{{ matrix_base_data_path }}/rust-synapse-compress-state"
+matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path: "/usr/local/bin/synapse_compress_state"
 
 
 # Default Synapse configuration template which covers the generic use case.

roles/matrix-synapse/tasks/rust-synapse-compress-state/compress_room.yml

@@ -12,7 +12,7 @@
       --network={{ matrix_docker_network }}
       --mount type=bind,src={{ matrix_synapse_rust_synapse_compress_state_base_path }},dst=/work
       {{ matrix_synapse_rust_synapse_compress_state_docker_image }}
-      /synapse-compress-state -t -o /work/state-compressor.sql
+      {{ matrix_synapse_rust_synapse_compress_state_synapse_compress_state_in_container_path }} -t -o /work/state-compressor.sql
       -p "host={{ matrix_synapse_database_host }} user={{ matrix_synapse_database_user }} password={{ matrix_synapse_database_password }} dbname={{ matrix_synapse_database_database }}"
       -r '{{ room_details.room_id }}'
 

roles/matrix-synapse/templates/synapse/homeserver.yaml.j2

@@ -1977,8 +1977,14 @@ saml2_config:
 #
 #             localpart_template: Jinja2 template for the localpart of the MXID.
 #                 If this is not set, the user will be prompted to choose their
-#                 own username (see 'sso_auth_account_details.html' in the 'sso'
-#                 section of this file).
+#                 own username (see the documentation for the
+#                 'sso_auth_account_details.html' template). This template can
+#                 use the 'localpart_from_email' filter.
+#
+#             confirm_localpart: Whether to prompt the user to validate (or
+#                 change) the generated localpart (see the documentation for the
+#                 'sso_auth_account_details.html' template), instead of
+#                 registering the account right away.
 #
 #             display_name_template: Jinja2 template for the display name to set
 #                 on first login. If unset, no displayname will be set.
@@ -3013,4 +3019,36 @@ redis:
   password: {{ matrix_synapse_redis_password }}
 
 
+## Background Updates ##
+
+# Background updates are database updates that are run in the background in batches.
+# The duration, minimum batch size, default batch size, whether to sleep between batches and if so, how long to
+# sleep can all be configured. This is helpful to speed up or slow down the updates.
+#
+background_updates:
+    # How long in milliseconds to run a batch of background updates for. Defaults to 100. Uncomment and set
+    # a time to change the default.
+    #
+    #background_update_duration_ms: 500
+
+    # Whether to sleep between updates. Defaults to True. Uncomment to change the default.
+    #
+    #sleep_enabled: false
+
+    # If sleeping between updates, how long in milliseconds to sleep for. Defaults to 1000. Uncomment
+    # and set a duration to change the default.
+    #
+    #sleep_duration_ms: 300
+
+    # Minimum size a batch of background updates can be. Must be greater than 0. Defaults to 1. Uncomment and
+    # set a size to change the default.
+    #
+    #min_batch_size: 10
+
+    # The batch size to use for the first iteration of a new background update. The default is 100.
+    # Uncomment and set a size to change the default.
+    #
+    #default_batch_size: 50
+
+
 # vim:ft=yaml