Synapse v1.152 introduces a new `quarantined_media_changes` stream and requires that any worker serving `/_synapse/admin/v1/quarantine_media/` be declared as a writer for that stream. Otherwise quarantining media fails on the worker. Add `quarantined_media_changes` as a web-facing stream writer alongside the other stream-backed APIs and route the admin endpoint via the same explicit writer-or-main model used for `device_lists`, `thread_subscriptions`, etc. The endpoint is removed from `matrix_synapse_workers_media_repository_endpoints` so the old media-repository route does not shadow (or conflict with) the new writer-or-main route. Without that move, the previously-shipping default of routing `/quarantine_media` to the `media_repository` worker would silently break after the v1.152.0 image bump. Default count is 1 in the `one-of-each` and `specialized-workers` presets (matching `device_lists`), and 0 in `little-federation-helper` (which also has no media-repository worker, so falling back to main is fine). Refs: - https://github.com/element-hq/synapse/blob/develop/docs/upgrade.md#upgrading-to-v11520 - https://element-hq.github.io/synapse/latest/workers.html#the-quarantined_media_changes-stream Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>pull/5189/head
| @@ -950,6 +950,7 @@ matrix_synapse_workers_presets: | |||||
| # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | ||||
| # and disabled in upstream Synapse unless explicitly opted in. | # and disabled in upstream Synapse unless explicitly opted in. | ||||
| stream_writer_thread_subscriptions_stream_workers_count: 0 | stream_writer_thread_subscriptions_stream_workers_count: 0 | ||||
| stream_writer_quarantined_media_changes_stream_workers_count: 0 | |||||
| one-of-each: | one-of-each: | ||||
| room_workers_count: 0 | room_workers_count: 0 | ||||
| sync_workers_count: 0 | sync_workers_count: 0 | ||||
| @@ -973,6 +974,7 @@ matrix_synapse_workers_presets: | |||||
| # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | ||||
| # and disabled in upstream Synapse unless explicitly opted in. | # and disabled in upstream Synapse unless explicitly opted in. | ||||
| stream_writer_thread_subscriptions_stream_workers_count: 0 | stream_writer_thread_subscriptions_stream_workers_count: 0 | ||||
| stream_writer_quarantined_media_changes_stream_workers_count: 1 | |||||
| specialized-workers: | specialized-workers: | ||||
| room_workers_count: 1 | room_workers_count: 1 | ||||
| sync_workers_count: 1 | sync_workers_count: 1 | ||||
| @@ -996,6 +998,7 @@ matrix_synapse_workers_presets: | |||||
| # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | # Keep disabled by default: MSC4306/4308 thread subscriptions are unstable | ||||
| # and disabled in upstream Synapse unless explicitly opted in. | # and disabled in upstream Synapse unless explicitly opted in. | ||||
| stream_writer_thread_subscriptions_stream_workers_count: 0 | stream_writer_thread_subscriptions_stream_workers_count: 0 | ||||
| stream_writer_quarantined_media_changes_stream_workers_count: 1 | |||||
| # Controls whether the matrix-synapse container exposes the various worker ports | # Controls whether the matrix-synapse container exposes the various worker ports | ||||
| # (see `port` and `metrics_port` in `matrix_synapse_workers_enabled_list`) outside of the container. | # (see `port` and `metrics_port` in `matrix_synapse_workers_enabled_list`) outside of the container. | ||||
| @@ -1102,6 +1105,10 @@ matrix_synapse_workers_stream_writer_device_lists_stream_workers_count: "{{ matr | |||||
| # More than 1 worker is also supported of this type. | # More than 1 worker is also supported of this type. | ||||
| matrix_synapse_workers_stream_writer_thread_subscriptions_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_thread_subscriptions_stream_workers_count'] }}" | matrix_synapse_workers_stream_writer_thread_subscriptions_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_thread_subscriptions_stream_workers_count'] }}" | ||||
| # matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_workers_count controls how many stream writers that handle the `quarantined_media_changes` stream to spawn. | |||||
| # More than 1 worker is also supported of this type. | |||||
| matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_workers_count: "{{ matrix_synapse_workers_presets[matrix_synapse_workers_preset]['stream_writer_quarantined_media_changes_stream_workers_count'] }}" | |||||
| # A list of stream writer workers to enable. This list is built automatically based on other variables. | # A list of stream writer workers to enable. This list is built automatically based on other variables. | ||||
| # You're encouraged to enable/disable stream writer workers by setting `matrix_synapse_workers_stream_writer_*_stream_workers_count` variables, instead of adjusting this list manually. | # You're encouraged to enable/disable stream writer workers by setting `matrix_synapse_workers_stream_writer_*_stream_workers_count` variables, instead of adjusting this list manually. | ||||
| matrix_synapse_workers_stream_writers: | | matrix_synapse_workers_stream_writers: | | ||||
| @@ -1125,6 +1132,8 @@ matrix_synapse_workers_stream_writers: | | |||||
| ([{'stream': 'device_lists'}] * matrix_synapse_workers_stream_writer_device_lists_stream_workers_count | int) | ([{'stream': 'device_lists'}] * matrix_synapse_workers_stream_writer_device_lists_stream_workers_count | int) | ||||
| + | + | ||||
| ([{'stream': 'thread_subscriptions'}] * matrix_synapse_workers_stream_writer_thread_subscriptions_stream_workers_count | int) | ([{'stream': 'thread_subscriptions'}] * matrix_synapse_workers_stream_writer_thread_subscriptions_stream_workers_count | int) | ||||
| + | |||||
| ([{'stream': 'quarantined_media_changes'}] * matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_workers_count | int) | |||||
| }} | }} | ||||
| matrix_synapse_workers_stream_writers_container_arguments: [] | matrix_synapse_workers_stream_writers_container_arguments: [] | ||||
| @@ -2105,6 +2114,7 @@ matrix_synapse_reverse_proxy_companion_synapse_stream_writer_presence_stream_wor | |||||
| matrix_synapse_reverse_proxy_companion_synapse_stream_writer_push_rules_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_push_rules_stream_worker_client_server_endpoints }}" | matrix_synapse_reverse_proxy_companion_synapse_stream_writer_push_rules_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_push_rules_stream_worker_client_server_endpoints }}" | ||||
| matrix_synapse_reverse_proxy_companion_synapse_stream_writer_device_lists_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_device_lists_stream_worker_client_server_endpoints }}" | matrix_synapse_reverse_proxy_companion_synapse_stream_writer_device_lists_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_device_lists_stream_worker_client_server_endpoints }}" | ||||
| matrix_synapse_reverse_proxy_companion_synapse_stream_writer_thread_subscriptions_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_thread_subscriptions_stream_worker_client_server_endpoints }}" | matrix_synapse_reverse_proxy_companion_synapse_stream_writer_thread_subscriptions_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_thread_subscriptions_stream_worker_client_server_endpoints }}" | ||||
| matrix_synapse_reverse_proxy_companion_synapse_stream_writer_quarantined_media_changes_stream_worker_client_server_locations: "{{ matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_worker_client_server_endpoints }}" | |||||
| matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations: "{{ matrix_synapse_workers_media_repository_endpoints | default([]) }}" | matrix_synapse_reverse_proxy_companion_synapse_media_repository_locations: "{{ matrix_synapse_workers_media_repository_endpoints | default([]) }}" | ||||
| matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints | default([]) }}" | matrix_synapse_reverse_proxy_companion_synapse_user_dir_locations: "{{ matrix_synapse_workers_user_dir_worker_client_server_endpoints | default([]) }}" | ||||
| matrix_synapse_reverse_proxy_companion_client_server_main_override_locations_regex: ^/_matrix/client/(api/v1|r0|v3|unstable)/(account/3pid/|directory/list/room/|rooms/[^/]+/(forget|upgrade|report)|register) | matrix_synapse_reverse_proxy_companion_client_server_main_override_locations_regex: ^/_matrix/client/(api/v1|r0|v3|unstable)/(account/3pid/|directory/list/room/|rooms/[^/]+/(forget|upgrade|report)|register) | ||||
| @@ -13,6 +13,7 @@ | |||||
| {% set stream_writer_push_rules_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'push_rules') | list %} | {% set stream_writer_push_rules_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'push_rules') | list %} | ||||
| {% set stream_writer_device_lists_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'device_lists') | list %} | {% set stream_writer_device_lists_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'device_lists') | list %} | ||||
| {% set stream_writer_thread_subscriptions_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'thread_subscriptions') | list %} | {% set stream_writer_thread_subscriptions_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'thread_subscriptions') | list %} | ||||
| {% set stream_writer_quarantined_media_changes_stream_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'stream_writer') | selectattr('stream_writer_stream', 'equalto', 'quarantined_media_changes') | list %} | |||||
| {% set media_repository_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %} | {% set media_repository_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'media_repository') | list %} | ||||
| {% set user_dir_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %} | {% set user_dir_workers = matrix_synapse_reverse_proxy_companion_synapse_workers_list | selectattr('type', 'equalto', 'user_dir') | list %} | ||||
| {% set stream_writer_client_server_routes = [ | {% set stream_writer_client_server_routes = [ | ||||
| @@ -64,6 +65,12 @@ | |||||
| 'locations': matrix_synapse_reverse_proxy_companion_synapse_stream_writer_thread_subscriptions_stream_worker_client_server_locations, | 'locations': matrix_synapse_reverse_proxy_companion_synapse_stream_writer_thread_subscriptions_stream_worker_client_server_locations, | ||||
| 'upstream': 'stream_writer_thread_subscriptions_stream_workers_upstream', | 'upstream': 'stream_writer_thread_subscriptions_stream_workers_upstream', | ||||
| }, | }, | ||||
| { | |||||
| 'doc_url': 'https://element-hq.github.io/synapse/latest/workers.html#the-quarantined_media_changes-stream', | |||||
| 'workers': stream_writer_quarantined_media_changes_stream_workers, | |||||
| 'locations': matrix_synapse_reverse_proxy_companion_synapse_stream_writer_quarantined_media_changes_stream_worker_client_server_locations, | |||||
| 'upstream': 'stream_writer_quarantined_media_changes_stream_workers_upstream', | |||||
| }, | |||||
| ] %} | ] %} | ||||
| {% macro render_worker_upstream(name, workers, load_balance) %} | {% macro render_worker_upstream(name, workers, load_balance) %} | ||||
| @@ -140,6 +140,13 @@ matrix_synapse_workers_stream_writer_thread_subscriptions_stream_worker_client_s | |||||
| - ^/_matrix/client/unstable/io.element.msc4306/rooms/.*/thread/.*/subscription$ | - ^/_matrix/client/unstable/io.element.msc4306/rooms/.*/thread/.*/subscription$ | ||||
| - ^/_matrix/client/unstable/io.element.msc4308/thread_subscriptions$ | - ^/_matrix/client/unstable/io.element.msc4308/thread_subscriptions$ | ||||
| # matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_worker_client_server_endpoints contains the endpoints serviced by the `quarantined_media_changes` stream writer. | |||||
| # Since Synapse v1.152, deployments which route `/_synapse/admin/v1/quarantine_media/` to a worker (rather than the main process) must declare those workers as | |||||
| # `quarantined_media_changes` stream writers, otherwise quarantining media will not work. | |||||
| # See: https://github.com/element-hq/synapse/blob/develop/docs/upgrade.md#upgrading-to-v11520 | |||||
| matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_worker_client_server_endpoints: | |||||
| - ^/_synapse/admin/v1/quarantine_media/.*$ | |||||
| # matrix_synapse_workers_user_dir_worker_client_server_endpoints contains the endpoints serviced by the `type = user_dir` (`app = generic_worker`) worker. | # matrix_synapse_workers_user_dir_worker_client_server_endpoints contains the endpoints serviced by the `type = user_dir` (`app = generic_worker`) worker. | ||||
| # See: https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory | # See: https://matrix-org.github.io/synapse/latest/workers.html#updating-the-user-directory | ||||
| matrix_synapse_workers_user_dir_worker_client_server_endpoints: | matrix_synapse_workers_user_dir_worker_client_server_endpoints: | ||||
| @@ -148,11 +155,11 @@ matrix_synapse_workers_user_dir_worker_client_server_endpoints: | |||||
| # matrix_synapse_workers_known_stream_writer_stream_types contains the list of stream writer stream types that the playbook recognizes. | # matrix_synapse_workers_known_stream_writer_stream_types contains the list of stream writer stream types that the playbook recognizes. | ||||
| # This is used for validation purposes. If adding support for a new type, besides adding it to this list, | # This is used for validation purposes. If adding support for a new type, besides adding it to this list, | ||||
| # don't forget to actually configure it where appropriate (see worker.yaml.j2`, the nginx proxy configuration, etc). | # don't forget to actually configure it where appropriate (see worker.yaml.j2`, the nginx proxy configuration, etc). | ||||
| matrix_synapse_workers_known_stream_writer_stream_types: ['events', 'typing', 'to_device', 'account_data', 'receipts', 'presence', 'push_rules', 'device_lists', 'thread_subscriptions'] | |||||
| matrix_synapse_workers_known_stream_writer_stream_types: ['events', 'typing', 'to_device', 'account_data', 'receipts', 'presence', 'push_rules', 'device_lists', 'thread_subscriptions', 'quarantined_media_changes'] | |||||
| # matrix_synapse_workers_webserving_stream_writer_types contains a list of stream writer types that serve web (client) requests. | # matrix_synapse_workers_webserving_stream_writer_types contains a list of stream writer types that serve web (client) requests. | ||||
| # Not all stream writers serve web requests. Some just perform background tasks. | # Not all stream writers serve web requests. Some just perform background tasks. | ||||
| matrix_synapse_workers_webserving_stream_writer_types: ['typing', 'to_device', 'account_data', 'receipts', 'presence', 'push_rules', 'device_lists', 'thread_subscriptions'] | |||||
| matrix_synapse_workers_webserving_stream_writer_types: ['typing', 'to_device', 'account_data', 'receipts', 'presence', 'push_rules', 'device_lists', 'thread_subscriptions', 'quarantined_media_changes'] | |||||
| # matrix_synapse_workers_systemd_services_list contains a list of systemd services (one for each worker systemd service which serves web requests). | # matrix_synapse_workers_systemd_services_list contains a list of systemd services (one for each worker systemd service which serves web requests). | ||||
| # This list is built during runtime. | # This list is built during runtime. | ||||
| @@ -338,8 +345,10 @@ matrix_synapse_workers_media_repository_endpoints: | |||||
| - ^/_synapse/admin/v1/room/.*/media.*$ | - ^/_synapse/admin/v1/room/.*/media.*$ | ||||
| - ^/_synapse/admin/v1/user/.*/media.*$ | - ^/_synapse/admin/v1/user/.*/media.*$ | ||||
| - ^/_synapse/admin/v1/media/.*$ | - ^/_synapse/admin/v1/media/.*$ | ||||
| - ^/_synapse/admin/v1/quarantine_media/.*$ | |||||
| - ^/_synapse/admin/v1/users/.*/media$ | - ^/_synapse/admin/v1/users/.*/media$ | ||||
| # `^/_synapse/admin/v1/quarantine_media/.*$` is intentionally not listed here: | |||||
| # Synapse v1.152+ requires its worker to also be a `quarantined_media_changes` stream writer, | |||||
| # so it's now handled by `matrix_synapse_workers_stream_writer_quarantined_media_changes_stream_worker_client_server_endpoints` instead. | |||||
| matrix_synapse_workers_user_dir_endpoints: | matrix_synapse_workers_user_dir_endpoints: | ||||
| # Handles searches in the user directory. It can handle REST endpoints matching | # Handles searches in the user directory. It can handle REST endpoints matching | ||||