Merge branch 'spantaleev:master' into media_cache_range_requests

CHANGELOG.md

@@ -1,3 +1,31 @@
+# 2022-04-19
+
+## Borg backup support
+
+Thanks to [Aine](https://gitlab.com/etke.cc) of [etke.cc](https://etke.cc/), the playbook can now set up [Borg](https://www.borgbackup.org/) backups with [borgmatic](https://torsion.org/borgmatic/) of your Matrix server.
+
+See our [Setting up borg backup](docs/configuring-playbook-backup-borg.md) documentation to get started.
+
+
+## (Compatibility Break) Upgrading to Synapse v1.57 on setups using workers may require manual action
+
+If you're running a worker setup for Synapse (`matrix_synapse_workers_enabled: true`), the [Synapse v1.57 upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#changes-to-database-schema-for-application-services) say that you may need to take special care when upgrading:
+
+> Synapse v1.57.0 includes a change to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs.
+
+If you're not running an `appservice` worker (`matrix_synapse_workers_preset: little-federation-helper` or `matrix_synapse_workers_appservice_workers_count: 0`), you are probably safe to upgrade as per normal, without taking any special care.
+
+If you are running a setup with an `appservice` worker, or otherwise want to be on the safe side, we recommend the following upgrade path:
+
+0. Pull the latest playbook changes
+1. Stop all services (`ansible-playbook -i inventory/hosts setup.yml --tags=stop`)
+2. Re-run the playbook (`ansible-playbook -i inventory/hosts setup.yml --tags=setup-all`)
+3. Start Postgres (`systemctl start matrix-postgres` on the server)
+4. Start the main Synapse process (`systemctl start matrix-synapse` on the server)
+5. Wait a while so that Synapse can start and complete the database migrations. You can use `journalctl -fu matrix-synapse` on the server to get a clue. Waiting a few minutes should also be enough.
+6. It should now be safe to start all other services. `ansible-playbook -i inventory/hosts setup.yml --tags=start` will do it for you
+
+
 # 2022-04-14
 
 ## (Compatibility Break) Changes to `docker-src` permissions necessitating manual action

docs/configuring-playbook-backup-borg.md

@@ -8,6 +8,9 @@ You will need a remote server where borg will store the backups. There are hoste
 
 The backup will run based on `matrix_backup_borg_schedule` var (systemd timer calendar), default: 4am every day.
 
+By default, if you're using the integrated Postgres database server (as opposed to [an external Postgres server](configuring-playbook-external-postgres.md)), Borg backups will also include dumps of your Postgres database. An alternative solution for backing up the Postgres database is [postgres backup](configuring-playbook-postgres-backup.md). If you decide to go with another solution, you can disable Postgres-backup support for Borg using the `matrix_backup_borg_postgresql_enabled` variable.
+
+
 ## Prerequisites
 
 1. Create a new SSH key:
@@ -51,6 +54,8 @@ where:
 * PASSPHRASE - passphrase used for encrypting backups, you may generate it with `pwgen -s 64 1` or use any password manager
 * PRIVATE KEY - the content of the **private** part of the SSH key you created before
 
+To backup without encryption, add `matrix_backup_borg_encryption: 'none'` to your vars. This will also enable the `matrix_backup_borg_unknown_unencrypted_repo_access_is_ok` variable.
+
 `matrix_backup_borg_location_source_directories` defines the list of directories to back up: it's set to `{{ matrix_base_data_path }}` by default, which is the base directory for every service's data, such as Synapse, Postgres and the bridges. You might want to exclude certain directories or file patterns from the backup using the `matrix_backup_borg_location_exclude_patterns` variable.
 
 Check the `roles/matrix-backup-borg/defaults/main.yml` file for the full list of available options.

docs/configuring-playbook-postgres-backup.md

@@ -2,6 +2,9 @@
 
 The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
 
+For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
+
+
 ## Adjusting the playbook configuration
 
 Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:

docs/configuring-playbook.md

@@ -152,6 +152,13 @@ When you're done with all the configuration you'd like to do, continue with [Ins
 - [Setting up Mjolnir](configuring-playbook-bot-mjolnir.md) - a moderation tool/bot (optional)
 
 
+### Backups
+
+- [Setting up borg backup](configuring-playbook-backup-borg.md) - a full Matrix server backup solution, including the Postgres database (optional)
+
+- [Setting up postgres backup](configuring-playbook-postgres-backup.md) - a Postgres-database backup solution (note: does not include other files) (optional)
+
+
 ### Other specialized services
 
 - [Setting up the Sygnal push gateway](configuring-playbook-sygnal.md) (optional)

roles/matrix-backup-borg/defaults/main.yml

@@ -44,12 +44,15 @@ matrix_backup_borg_location_repositories: []
 # exclude following paths:
 matrix_backup_borg_location_exclude_patterns: []
 
-# borg encryption mode, only repokey-* is supported
+# borg encryption mode, only "repokey-*" and "none" are supported
 matrix_backup_borg_encryption: repokey-blake2
 
 # private ssh key used to connect to the borg repo
 matrix_backup_borg_ssh_key_private: ""
 
+# allow unencrypted repo access
+matrix_backup_borg_unknown_unencrypted_repo_access_is_ok: "{{ matrix_backup_borg_encryption == 'none' }}"
+
 # borg ssh command with ssh key
 matrix_backup_borg_storage_ssh_command: ssh -o "StrictHostKeyChecking accept-new" -i /etc/borgmatic.d/sshkey
 

roles/matrix-backup-borg/tasks/init.yml

@@ -1,4 +1,4 @@
 ---
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-backup-borg.service', 'matrix-backup-borg.timer'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-backup-borg.timer'] }}"
   when: matrix_backup_borg_enabled|bool

roles/matrix-backup-borg/tasks/validate_config.yml

@@ -7,4 +7,9 @@
   with_items:
     - "matrix_backup_borg_ssh_key_private"
     - "matrix_backup_borg_location_repositories"
-    - "matrix_backup_borg_storage_encryption_passphrase"
+
+- name: Fail if encryption passphrase is undefined unless repository is unencrypted
+  fail:
+    msg: >-
+      You need to define a required passphrase using the `matrix_backup_borg_storage_encryption_passphrase` variable.
+  when: "matrix_backup_borg_storage_encryption_passphrase == '' and matrix_backup_borg_encryption != 'none'"

roles/matrix-backup-borg/templates/config.yaml.j2

@@ -11,6 +11,7 @@ storage:
   ssh_command: {{ matrix_backup_borg_storage_ssh_command|to_json }}
   archive_name_format: {{ matrix_backup_borg_storage_archive_name_format|to_json }}
   encryption_passphrase: {{ matrix_backup_borg_storage_encryption_passphrase|to_json }}
+  unknown_unencrypted_repo_access_is_ok: {{ matrix_backup_borg_unknown_unencrypted_repo_access_is_ok|to_json }}
 
 retention:
   keep_hourly: {{ matrix_backup_borg_retention_keep_hourly|to_json }}

roles/matrix-base/defaults/main.yml

@@ -71,6 +71,10 @@ matrix_container_global_registry_prefix: "docker.io/"
 matrix_container_retries_count: 10
 matrix_container_retries_delay: 10
 
+# Each get_url will retry on failed attempt 10 times with delay of 10 seconds between each attempt.
+matrix_geturl_retries_count: 10
+matrix_geturl_retries_delay: 10
+
 matrix_user_username: "matrix"
 matrix_user_groupname: "matrix"
 

roles/matrix-bot-honoroit/defaults/main.yml

@@ -8,7 +8,7 @@ matrix_bot_honoroit_container_image_self_build: false
 matrix_bot_honoroit_docker_repo: "https://gitlab.com/etke.cc/honoroit.git"
 matrix_bot_honoroit_docker_src_files_path: "{{ matrix_base_data_path }}/honoroit/docker-src"
 
-matrix_bot_honoroit_version: v0.9.6
+matrix_bot_honoroit_version: v0.9.7
 matrix_bot_honoroit_docker_image: "{{ matrix_bot_honoroit_docker_image_name_prefix }}honoroit:{{ matrix_bot_honoroit_version }}"
 matrix_bot_honoroit_docker_image_name_prefix: "{{ 'localhost/' if matrix_bot_honoroit_container_image_self_build else 'registry.gitlab.com/etke.cc/' }}"
 matrix_bot_honoroit_docker_image_force_pull: "{{ matrix_bot_honoroit_docker_image.endswith(':latest') }}"
@@ -96,6 +96,15 @@ matrix_bot_honoroit_text_prefix_done: ''
 # Text: greetings
 matrix_bot_honoroit_text_greetings: ''
 
+# Text: invite
+matrix_bot_honoroit_text_invite: ''
+
+# Text: join
+matrix_bot_honoroit_text_join: ''
+
+# Text: leave
+matrix_bot_honoroit_text_leave: ''
+
 # Text: error
 matrix_bot_honoroit_text_error: ''
 

roles/matrix-bot-honoroit/templates/env.j2

@@ -11,6 +11,9 @@ HONOROIT_CACHESIZE={{ matrix_bot_honoroit_cachesize }}
 HONOROIT_TEXT_PREFIX_OPEN={{ matrix_bot_honoroit_text_prefix_open }}
 HONOROIT_TEXT_PREFIX_DONE={{ matrix_bot_honoroit_text_prefix_done }}
 HONOROIT_TEXT_GREETINGS={{ matrix_bot_honoroit_text_greetings }}
+HONOROIT_TEXT_INVITE={{ matrix_bot_honoroit_text_invite }}
+HONOROIT_TEXT_JOIN={{ matrix_bot_honoroit_text_join }}
+HONOROIT_TEXT_LEAVE={{ matrix_bot_honoroit_text_leave }}
 HONOROIT_TEXT_ERROR={{ matrix_bot_honoroit_text_error }}
 HONOROIT_TEXT_EMPTYROOM={{ matrix_bot_honoroit_text_emptyroom }}
 HONOROIT_TEXT_DONE={{ matrix_bot_honoroit_text_done }}

roles/matrix-bridge-mautrix-telegram/defaults/main.yml

@@ -14,7 +14,7 @@ matrix_mautrix_telegram_container_image_self_build: false
 matrix_mautrix_telegram_docker_repo: "https://mau.dev/mautrix/telegram.git"
 matrix_mautrix_telegram_docker_src_files_path: "{{ matrix_base_data_path }}/mautrix-telegram/docker-src"
 
-matrix_mautrix_telegram_version: v0.11.2
+matrix_mautrix_telegram_version: v0.11.3
 # See: https://mau.dev/mautrix/telegram/container_registry
 matrix_mautrix_telegram_docker_image: "dock.mau.dev/mautrix/telegram:{{ matrix_mautrix_telegram_version }}"
 matrix_mautrix_telegram_docker_image_force_pull: "{{ matrix_mautrix_telegram_docker_image.endswith(':latest') }}"

roles/matrix-grafana/tasks/setup.yml

@@ -70,6 +70,10 @@
     group: "{{ matrix_user_groupname }}"
   with_items: "{{ matrix_grafana_dashboard_download_urls_all }}"
   when: matrix_grafana_enabled|bool
+  register: result
+  retries: "{{ matrix_geturl_retries_count }}"
+  delay: "{{ matrix_geturl_retries_delay }}"
+  until: result is not failed
 
 - name: Ensure matrix-grafana.service installed
   template:

roles/matrix-prometheus/tasks/setup_install.yml

@@ -32,6 +32,10 @@
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
   when: "matrix_prometheus_scraper_synapse_rules_enabled|bool"
+  register: result
+  retries: "{{ matrix_geturl_retries_count }}"
+  delay: "{{ matrix_geturl_retries_delay }}"
+  until: result is not failed
 
 - name: Ensure prometheus.yml installed
   copy:

roles/matrix-synapse/defaults/main.yml

@@ -9,7 +9,7 @@ matrix_synapse_container_image_self_build_repo: "https://github.com/matrix-org/s
 
 matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:{{ matrix_synapse_docker_image_tag }}"
 matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else matrix_container_global_registry_prefix }}"
-matrix_synapse_version: v1.56.0
+matrix_synapse_version: v1.57.0
 matrix_synapse_docker_image_tag: "{{ matrix_synapse_version }}"
 matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}"
 

roles/matrix-synapse/tasks/ext/encryption-disabler/setup_install.yml

@@ -8,6 +8,10 @@
     mode: 0440
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
+  register: result
+  retries: "{{ matrix_geturl_retries_count }}"
+  delay: "{{ matrix_geturl_retries_delay }}"
+  until: result is not failed
 
 - set_fact:
     matrix_synapse_modules: |

roles/matrix-synapse/tasks/ext/rest-auth/setup_install.yml

@@ -13,6 +13,10 @@
     mode: 0440
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
+  register: result
+  retries: "{{ matrix_geturl_retries_count }}"
+  delay: "{{ matrix_geturl_retries_delay }}"
+  until: result is not failed
 
 - set_fact:
     matrix_synapse_password_providers_enabled: true

roles/matrix-synapse/tasks/ext/shared-secret-auth/setup_install.yml

@@ -18,6 +18,10 @@
     mode: 0440
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
+  register: result
+  retries: "{{ matrix_geturl_retries_count }}"
+  delay: "{{ matrix_geturl_retries_delay }}"
+  until: result is not failed
 
 - set_fact:
     matrix_synapse_modules: |

roles/matrix-synapse/tasks/rust-synapse-compress-state/main.yml

@@ -11,17 +11,17 @@
 
 - name: Set matrix_synapse_rust_synapse_compress_state_find_rooms_command_wait_time, if not provided
   set_fact:
-    matrix_synapse_rust_synapse_compress_state_find_rooms_command_wait_time: 300
+    matrix_synapse_rust_synapse_compress_state_find_rooms_command_wait_time: 1800
   when: "matrix_synapse_rust_synapse_compress_state_find_rooms_command_wait_time|default('') == ''"
 
 - name: Set matrix_synapse_rust_synapse_compress_state_compress_room_time, if not provided
   set_fact:
-    matrix_synapse_rust_synapse_compress_state_compress_room_time: 1800
+    matrix_synapse_rust_synapse_compress_state_compress_room_time: 3600
   when: "matrix_synapse_rust_synapse_compress_state_compress_room_time|default('') == ''"
 
 - name: Set matrix_synapse_rust_synapse_compress_state_psql_import_time, if not provided
   set_fact:
-    matrix_synapse_rust_synapse_compress_state_psql_import_time: 1800
+    matrix_synapse_rust_synapse_compress_state_psql_import_time: 3600
   when: "matrix_synapse_rust_synapse_compress_state_psql_import_time|default('') == ''"
 
 - name: Set matrix_synapse_rust_synapse_compress_state_min_state_groups_required, if not provided