Add mxisd Identity Server support

CHANGELOG.md

@@ -1,3 +1,11 @@
+# 2018-08-15
+
+## mxisd Identity Server support
+
+The playbook now sets up an [mxisd](https://github.com/kamax-io/mxisd) Identity Server for you by default.
+Additional details are available in [Adjusting mxisd Identity Server configuration](docs/configuring-playbook-mxisd.md).
+
+
 # 2018-08-14
 
 ## Email-sending support

README.md

@@ -20,6 +20,8 @@ Using this playbook, you can get the following services configured on your serve
 
 - (optional default) a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default
 
+- (optional default) an [mxisd](https://github.com/kamax-io/mxisd) Matrix Identity server
+
 - (optional default) a [Postfix](http://www.postfix.org/) mail server, through which all Matrix services send outgoing email (can be configured to relay through another SMTP server)
 
 - (optional default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](#using-your-own-webserver-instead-of-this-playbooks-nginx-proxy-optional)
@@ -72,6 +74,8 @@ This playbook sets up your server using the following Docker images:
 
 - [avhost/docker-matrix-riot](https://hub.docker.com/r/avhost/docker-matrix-riot/) - the [Riot.im](https://about.riot.im/) web client (optional)
 
+- [kamax/mxisd](https://hub.docker.com/r/kamax/mxisd/) - the [mxisd](https://github.com/kamax-io/mxisd) Matrix Identity server (optional)
+
 - [postgres](https://hub.docker.com/_/postgres/) - the [Postgres](https://www.postgresql.org/) database server (optional)
 
 - [cloudproto/goofys](https://hub.docker.com/r/cloudproto/goofys/) - the [Goofys](https://github.com/kahing/goofys) Amazon [S3](https://aws.amazon.com/s3/) file-system-mounting program (optional)

docs/configuring-dns.md

@@ -12,8 +12,12 @@ The SRV record should look like this:
 - Name: `_matrix._tcp` (use this text as-is)
 - Content: `10 0 8448 matrix.<your-domain>` (replace `<your-domain>` with your own)
 
-Once you've set up this DNS SRV record, you should create 2 other domain names (`matrix.<your-domain>` and `riot.<your-domain>`) and point both of them to your new server's IP address (DNS `A` record or `CNAME` is fine).
+To make the [mxisd](https://github.com/kamax-io/mxisd) Identity Server (which this playbook installs for you) be authoritative for your domain name, set up one more SRV record that looks like this:
+- Name: `_matrix-identity._tcp` (use this text as-is)
+- Content: `10 0 443 matrix.<your-domain>` (replace `<your-domain>` with your own)
 
-This playbook can then install all the services on that new server and you'll be able to join the Matrix network as `@<username>:<your-domain>`, even though everything is installed elsewhere (not on `<your-domain>`).
+Once you've set up these DNS SRV records, you should create 2 other domain names (`matrix.<your-domain>` and `riot.<your-domain>`) and point both of them to your new server's IP address (DNS `A` record or `CNAME` is fine).
+
+This playbook can then install all the services on that new server and you'll be able to join the Matrix network as `@<username>:<your-domain>`.
 
 When ready to proceed, continue with [Configuring this Ansible playbook](configuring-playbook.md).

docs/configuring-playbook-mxisd.md

@@ -0,0 +1,33 @@
+# Adjusting mxisd Identity Server configuration (optional)
+
+By default, this playbook configures an [mxisd](https://github.com/kamax-io/mxisd) Identity Server for you.
+
+This server is private by default, potentially at the expense of user discoverability.
+
+
+## Matrix.org lookup forwarding
+
+To ensure maximum discovery, you can make your identity server also forward lookups to the central matrix.org Identity server (at the cost of potentially leaking all your contacts information).
+
+Enabling this is discouraged and you'd better [learn more](https://github.com/kamax-io/mxisd/blob/master/docs/features/identity.md#lookups) before proceeding.
+
+Enabling matrix.org forwarding can happen with the following configuration:
+
+```yaml
+matrix_mxisd_matrixorg_forwarding_enabled: true
+```
+
+
+## Additional features
+
+What this playbook configures for your is some bare minimum Identity Server functionality, so that you won't need to rely on external 3rd party services.
+
+Still, mxisd can do much more.
+You can refer to the [mxisd website](https://github.com/kamax-io/mxisd) for more details.
+
+You can override the `matrix_mxisd_template_config` variable and use your own custom configuration template.
+
+
+## Troubleshooting
+
+If email address validation emails sent by mxisd are not reaching you, you should look into [Adjusting email-sending settings](configuring-playbook-email.md).

docs/configuring-playbook.md

@@ -29,4 +29,6 @@ When you're done with all the configuration you'd like to do, continue with [Ins
 
 - [Using an external PostgreSQL server](configuring-playbook-external-postgres.md) (optional)
 
+- [Adjusting mxisd Identity Server configuration](configuring-playbook-mxisd.md) (optional)
+
 - [Using your own webserver, instead of this playbook's nginx proxy](configuring-playbook-own-webserver.md) (optional)

roles/matrix-server/defaults/main.yml

@@ -46,6 +46,16 @@ matrix_synapse_macaroon_secret_key: ""
 matrix_synapse_registration_shared_secret: "{{ matrix_synapse_macaroon_secret_key }}"
 matrix_synapse_form_secret: "{{ matrix_synapse_macaroon_secret_key }}"
 
+# These are the identity servers that would be trusted by Synapse if mxisd is NOT enabled
+matrix_synapse_id_servers_public: ['vector.im', 'riot.im', 'matrix.org']
+
+# These are the identity servers that would be trusted by Synapse if mxisd IS enabled
+matrix_synapse_id_servers_own: "['{{ hostname_matrix }}']"
+
+# The final list of identity servers to use for Synapse.
+# The first one would also be used as riot-web's default identity server.
+matrix_synapse_trusted_third_party_id_servers: "{{ matrix_synapse_id_servers_own if matrix_mxisd_enabled else matrix_synapse_id_servers_public }}"
+
 matrix_synapse_max_upload_size_mb: 10
 matrix_synapse_max_log_file_size_mb: 100
 matrix_synapse_max_log_files_count: 10
@@ -58,7 +68,6 @@ matrix_coturn_base_path: "{{ matrix_base_data_path }}/coturn"
 matrix_coturn_config_path: "{{ matrix_coturn_base_path }}/turnserver.conf"
 matrix_scratchpad_dir: "{{ matrix_base_data_path }}/scratchpad"
 
-
 matrix_docker_image_postgres_v9: "postgres:9.6.10-alpine"
 matrix_docker_image_postgres_v10: "postgres:10.5-alpine"
 matrix_docker_image_postgres_latest: "{{ matrix_docker_image_postgres_v10 }}"
@@ -70,6 +79,7 @@ matrix_docker_image_s3fs: "xueshanf/s3fs:latest"
 matrix_docker_image_goofys: "cloudproto/goofys:latest"
 matrix_docker_image_coturn: "instrumentisto/coturn:4.5.0.7"
 matrix_docker_image_mailer: "panubo/postfix:latest"
+matrix_docker_image_mxisd: "kamax/mxisd:1.1.1"
 
 # The Docker network that all services would be put into
 matrix_docker_network: "matrix"
@@ -84,14 +94,17 @@ matrix_coturn_turn_udp_max_port: 49172
 
 matrix_coturn_turn_external_ip_address: "{{ ansible_host }}"
 
+
 matrix_s3_media_store_enabled: false
 matrix_s3_media_store_bucket_name: "your-bucket-name"
 matrix_s3_media_store_aws_access_key: "your-aws-access-key"
 matrix_s3_media_store_aws_secret_key: "your-aws-secret-key"
 matrix_s3_media_store_region: "eu-central-1"
 
+
 # By default, this playbook sets up a postfix mailer server (running in a container).
 # This is so that Matrix Synapse can send email reminders for unread messages.
+# Other services (like mxisd), however, also use that mailer to send emails through it.
 matrix_mailer_enabled: true
 
 matrix_mailer_sender_address: "matrix@{{ hostname_identity }}"
@@ -102,11 +115,39 @@ matrix_mailer_relay_auth: false
 matrix_mailer_relay_auth_username: ""
 matrix_mailer_relay_auth_password: ""
 
+
+# By default, this playbook installs the mxisd identity server on the same domain as Synapse (`hostname_matrix`).
+# If you wish to use the public identity servers (matrix.org, vector.im, riot.im) instead of your own,
+# you may wish to disable this.
+matrix_mxisd_enabled: true
+
+matrix_mxisd_base_path: "{{ matrix_base_data_path }}/mxisd"
+matrix_mxisd_config_path: "{{ matrix_mxisd_base_path }}/config"
+matrix_mxisd_data_path: "{{ matrix_mxisd_base_path }}/data"
+
+# Your identity server is private by default.
+# To ensure maximum discovery, you can make your identity server
+# also forward lookups to the central matrix.org Identity server
+# (at the cost of potentially leaking all your contacts information).
+# Enabling this is discouraged. Learn more here: https://github.com/kamax-io/mxisd/blob/master/docs/features/identity.md#lookups
+matrix_mxisd_matrixorg_forwarding_enabled: false
+
+# Specifies which template files to use when configuring mxisd.
+# If you'd like to have your own different configuration, feel free to copy and paste
+# the original files into your inventory (e.g. in `inventory/host_vars/<host>/`)
+# and then change the specific host's `vars.yaml` file like this:
+# matrix_mxisd_template_config: "{{ playbook_dir }}/inventory/host_vars/<host>/mxisd.yaml.j2"
+matrix_mxisd_template_config: "{{ role_path }}/templates/mxisd/mxisd.yaml.j2"
+
+
 # By default, this playbook installs the Riot.IM web UI on the `hostname_riot` domain.
 # If you wish to connect to your Matrix server by other means,
 # you may wish to disable this.
 matrix_riot_web_enabled: true
 
+matrix_riot_web_default_identity_server_url: "https://{{ matrix_synapse_trusted_third_party_id_servers[0] }}"
+
+
 # By default, this playbook sets up its own nginx proxy server on port 80/443.
 # This is fine if you're dedicating the whole server to Matrix.
 # But in case that's not the case, you may wish to prevent that

roles/matrix-server/tasks/main.yml

@@ -42,6 +42,11 @@
     - setup-all
     - setup-mailer
 
+- include: tasks/setup_mxisd.yml
+  tags:
+    - setup-all
+    - setup-mxisd
+
 - include: tasks/setup_synapse.yml
   tags:
     - setup-all

roles/matrix-server/tasks/setup_mxisd.yml

@@ -0,0 +1,74 @@
+---
+
+#
+# Tasks related to setting up mxisd
+#
+
+- name: Fail if mailer is not enabled
+  fail:
+    msg: "You need to enable the mailer service (matrix_mailer_enabled) to install mxisd"
+  when: "matrix_mxisd_enabled and not matrix_mailer_enabled"
+
+- name: Ensure mxisd paths exist
+  file:
+    path: "{{ item }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_username }}"
+  with_items:
+    - "{{ matrix_mxisd_config_path }}"
+    - "{{ matrix_mxisd_data_path }}"
+  when: matrix_mxisd_enabled
+
+- name: Ensure mxisd image is pulled
+  docker_image:
+    name: "{{ matrix_docker_image_mxisd }}"
+  when: matrix_mxisd_enabled
+
+- name: Ensure mxisd config installed
+  template:
+    src: "{{ matrix_mxisd_template_config }}"
+    dest: "{{ matrix_mxisd_config_path }}/mxisd.yaml"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_username }}"
+  when: matrix_mxisd_enabled
+
+- name: Ensure matrix-mxisd.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-mxisd.service.j2"
+    dest: "/etc/systemd/system/matrix-mxisd.service"
+    mode: 0644
+  when: matrix_mxisd_enabled
+
+#
+# Tasks related to getting rid of mxisd (if it was previously enabled)
+#
+
+- name: Check existence of matrix-mxisd service
+  stat: path="/etc/systemd/system/matrix-mxisd.service"
+  register: matrix_mxisd_service_stat
+
+- name: Ensure matrix-mxisd is stopped
+  service: name=matrix-mxisd state=stopped daemon_reload=yes
+  register: stopping_result
+  when: "not matrix_mxisd_enabled and matrix_mxisd_service_stat.stat.exists"
+
+- name: Ensure matrix-mxisd.service doesn't exist
+  file:
+    path: "/etc/systemd/system/matrix-mxisd.service"
+    state: absent
+  when: "not matrix_mxisd_enabled and matrix_mxisd_service_stat.stat.exists"
+
+- name: Ensure Matrix mxisd paths don't exist
+  file:
+    path: "{{ matrix_mxisd_base_path }}"
+    state: absent
+  when: "not matrix_mxisd_enabled"
+
+- name: Ensure mxisd Docker image doesn't exist
+  docker_image:
+    name: "{{ matrix_docker_image_mxisd }}"
+    state: absent
+  when: "not matrix_mxisd_enabled"

roles/matrix-server/tasks/start.yml

@@ -15,6 +15,10 @@
   service: name=matrix-mailer enabled=yes state=restarted daemon_reload=yes
   when: matrix_mailer_enabled
 
+- name: Ensure matrix-mxisd autoruns and is restarted
+  service: name=matrix-mxisd enabled=yes state=restarted daemon_reload=yes
+  when: matrix_mxisd_enabled
+
 - name: Ensure matrix-synapse autoruns and is restarted
   service: name=matrix-synapse enabled=yes state=restarted daemon_reload=yes
 

roles/matrix-server/templates/mxisd/mxisd.yaml.j2

@@ -0,0 +1,19 @@
+matrix.domain: {{ hostname_identity }}
+server.name: {{ hostname_matrix }}
+
+key.path: /var/mxisd/sign.key
+
+storage.provider.sqlite.database: /var/mxisd/mxisd.db
+
+threepid.medium.email.identity.from: {{ matrix_mailer_sender_address }}
+threepid.medium.email.connectors.smtp.host: matrix-mailer
+threepid.medium.email.connectors.smtp.port: 587
+threepid.medium.email.connectors.smtp.tls: 0
+
+synapseSql.enabled: true
+synapseSql.type: postgresql
+synapseSql.connection: //{{ matrix_postgres_connection_hostname }}/{{ matrix_postgres_db_name }}?user={{ matrix_postgres_connection_username }}&password={{ matrix_postgres_connection_password }}
+
+{% if matrix_mxisd_matrixorg_forwarding_enabled %}
+forward.servers: ['matrix-org']
+{% endif %}

roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2

@@ -39,6 +39,20 @@ server {
 	ssl_prefer_server_ciphers on;
 	ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
 
+	{% if matrix_mxisd_enabled %}
+	location /_matrix/identity {
+		{% if matrix_nginx_proxy_enabled %}
+			{# Use the embedded DNS resolver in Docker containers to discover the service #}
+			resolver 127.0.0.11 valid=5s;
+			set $backend "matrix-mxisd:8090";
+			proxy_pass http://$backend;
+		{% else %}
+			{# Generic configuration for use outside of our container setup #}
+			proxy_pass http://localhost:8090;
+		{% endif %}
+	}
+	{% endif %}
+
 	location /_matrix {
 		{% if matrix_nginx_proxy_enabled %}
 			{# Use the embedded DNS resolver in Docker containers to discover the service #}

roles/matrix-server/templates/riot-web/config.json.j2

@@ -1,6 +1,6 @@
 {
     "default_hs_url": "https://{{ hostname_matrix }}",
-    "default_is_url": "https://vector.im",
+    "default_is_url": "{{ matrix_riot_web_default_identity_server_url }}",
     "disable_custom_urls": true,
     "disable_guests": false,
     "brand": "Riot",

roles/matrix-server/templates/synapse/homeserver.yaml.j2

@@ -446,9 +446,9 @@ allow_guest_access: False
 # The list of identity servers trusted to verify third party
 # identifiers by this server.
 trusted_third_party_id_servers:
-    - matrix.org
-    - vector.im
-    - riot.im
+{% for server in matrix_synapse_trusted_third_party_id_servers %}
+    - {{ server }}
+{% endfor %}
 
 # Users who register on this homeserver will automatically be joined
 # to these rooms

roles/matrix-server/templates/systemd/matrix-mxisd.service.j2

@@ -0,0 +1,29 @@
+[Unit]
+Description=Matrix mxisd identity server
+After=docker.service
+Requires=docker.service
+{% if not matrix_postgres_use_external %}
+Requires=matrix-postgres.service
+After=matrix-postgres.service
+{% endif %}
+
+[Service]
+Type=simple
+ExecStartPre=-/usr/bin/docker kill matrix-mxisd
+ExecStartPre=-/usr/bin/docker rm matrix-mxisd
+ExecStart=/usr/bin/docker run --rm --name matrix-mxisd \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--network={{ matrix_docker_network }} \
+			{% if not matrix_nginx_proxy_enabled %}
+			-p 127.0.0.1:8090:8090 \
+			{% endif %}
+			-v {{ matrix_mxisd_config_path }}:/etc/mxisd:ro \
+			-v {{ matrix_mxisd_data_path }}:/var/mxisd \
+			{{ matrix_docker_image_mxisd }}
+ExecStop=-/usr/bin/docker kill matrix-mxisd
+ExecStop=-/usr/bin/docker rm matrix-mxisd
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=multi-user.target

roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2

@@ -6,6 +6,9 @@ Wants=matrix-synapse.service
 {% if matrix_riot_web_enabled %}
 Wants=matrix-riot-web.service
 {% endif %}
+{% if matrix_mxisd_enabled %}
+Wants=matrix-mxisd.service
+{% endif %}
 
 [Service]
 Type=simple