Add self-check command

CHANGELOG.md

@@ -1,3 +1,10 @@
+# 2018-10-21
+
+## Self-check maintenance command
+
+The playbook can now [check if services are configured correctly](docs/maintenance-checking-services.md).
+
+
 # 2018-10-05
 
 ## Presence tracking made configurable

docs/README.md

@@ -12,6 +12,8 @@
 
 - [Configuring service discovery via .well-known](configuring-well-known.md)
 
+- [Maintenance / checking if services work](maintenance-checking-services.md)
+
 - [Maintenance / upgrading services](maintenance-upgrading-services.md)
 
 - [Maintenance / upgrading PostgreSQL](maintenance-upgrading-postgres.md)

docs/configuring-well-known.md

@@ -81,4 +81,6 @@ Make sure to:
 
 ## Confirming it works
 
-No matter which method you've used to set up the well-known file, if you've done it correctly you should be able to see a JSON file at a URL like this: `https://matrix.<domain>/.well-known/matrix/client`.
+No matter which method you've used to set up the well-known file, if you've done it correctly you should be able to see a JSON file at a URL like this: `https://matrix.<domain>/.well-known/matrix/client`.
+
+You can also check if everything is configured correctly, by [checking if services work](maintenance-checking-services.md).

docs/installing.md

@@ -33,4 +33,5 @@ ansible-playbook -i inventory/hosts setup.yml --tags=start
 Now that the services are running, you might want to:
 
 - [create your first user account](registering-users.md)
-- or **finalize the installation process** by [Configuring service discovery via .well-known](configuring-well-known.md)
+- or **finalize the installation process** by [Configuring service discovery via .well-known](configuring-well-known.md)
+- or [Check if services work](maintenance-checking-services.md)

docs/maintenance-checking-services.md

@@ -0,0 +1,13 @@
+# Checking if services work
+
+This playbook can perform a check to ensure that you've configured things correctly and that services are running.
+
+To perform the check, run:
+
+```bash
+ansible-playbook -i inventory/hosts setup.yml --tags=self-check
+```
+
+If it's all green, everything is probably running correctly.
+
+Besides this self-check, you can also check your server using the [Federation Tester](https://neo.lain.haus/fed-tester/).

roles/matrix-server/tasks/main.yml

@@ -91,3 +91,9 @@
 - include: tasks/import_media_store.yml
   tags:
     - import-media-store
+
+- include: tasks/self_check.yml
+  delegate_to: 127.0.0.1
+  become: false
+  tags:
+    - self-check

roles/matrix-server/tasks/self_check.yml

@@ -0,0 +1,18 @@
+---
+
+- include: tasks/self_check/self_check_dns.yml
+
+- include: tasks/self_check/self_check_client_api.yml
+
+- include: tasks/self_check/self_check_federation_api.yml
+
+- include: tasks/self_check/self_check_riot_web.yml
+  when: "matrix_riot_web_enabled"
+
+- include: tasks/self_check/self_check_mxisd.yml
+  when: "matrix_mxisd_enabled"
+
+- include: tasks/self_check/self_check_well_known.yml
+
+- include: tasks/self_check/self_check_corporal.yml
+  when: "matrix_corporal_enabled"

roles/matrix-server/tasks/self_check/self_check_client_api.yml

@@ -0,0 +1,20 @@
+---
+
+- set_fact:
+    matrix_client_api_url_endpoint_public: "https://{{ hostname_matrix }}/_matrix/client/versions"
+
+- name: Check Matrix Client API
+  uri:
+    url: "{{ matrix_client_api_url_endpoint_public }}"
+    follow_redirects: false
+  register: result_matrix_client_api
+  ignore_errors: true
+
+- name: Fail if Matrix Client API not working
+  fail:
+    msg: "Failed checking Matrix Client API is up at `{{ hostname_matrix }}` (checked endpoint: `{{ matrix_client_api_url_endpoint_public }}`). Is Synapse running? Is port 443 open in your firewall? Full error: {{ result_matrix_client_api }}"
+  when: "result_matrix_client_api.failed or 'json' not in result_matrix_client_api"
+
+- name: Report working Matrix Client API
+  debug:
+    msg: "The Matrix Client API at `{{ hostname_matrix }}` (checked endpoint: `{{ matrix_client_api_url_endpoint_public }}`) is working"

roles/matrix-server/tasks/self_check/self_check_corporal.yml

@@ -0,0 +1,21 @@
+---
+
+- set_fact:
+    corporal_client_api_url_endpoint_public: "https://{{ hostname_matrix }}/_matrix/client/corporal"
+
+- name: Check Matrix Corporal HTTP gateway
+  uri:
+    url: "{{ corporal_client_api_url_endpoint_public }}"
+    follow_redirects: false
+    return_content: true
+  register: result_corporal_client_api
+  ignore_errors: true
+
+- name: Fail if Matrix Corporal HTTP gateway not working
+  fail:
+    msg: "Failed checking Matrix Corporal is fronting the Matrix Client API at `{{ hostname_matrix }}` (checked endpoint: `{{ corporal_client_api_url_endpoint_public }}`). Is matrix-corporal running? Is port 443 open in your firewall? Full error: {{ result_corporal_client_api }}"
+  when: "result_corporal_client_api.failed or 'Matrix Client-Server API protected by Matrix Corporal' not in result_corporal_client_api.content"
+
+- name: Report working Matrix Corporal HTTP gateway
+  debug:
+    msg: "Matrix Corporal is fronting the Matrix Client API at `{{ hostname_matrix }}` (checked endpoint: `{{ corporal_client_api_url_endpoint_public }}`)"

roles/matrix-server/tasks/self_check/self_check_dns.yml

@@ -0,0 +1,25 @@
+---
+
+- name: Check DNS SRV record
+  shell:
+    cmd: "dig -t srv {{ ('_matrix._tcp.' + hostname_identity + '.')|quote }}"
+  register: result_dig_srv
+  changed_when: false
+  ignore_errors: true
+
+- name: Fail if dig failed
+  fail:
+    msg: "Failed checking DNS SRV record. You likely don't have the `dig` program installed locally. Full error: {{ result_dig_srv }}"
+  when: "result_dig_srv.stderr != ''"
+
+# We expect an answer like this:
+# ;; ANSWER SECTION:
+# _matrix._tcp.DOMAIN. 10800	IN	SRV	10 0 8448 matrix.DOMAIN.
+- name: Fail if DNS SRV record incorrect
+  fail:
+    msg: "It appears the DNS SRV record for {{ hostname_identity }} is not set up correctly. See the 'Configuring DNS' documentation for this playbook. Full DNS answer was: {{ result_dig_srv.stdout }}"
+  when: "('8448 ' + hostname_matrix) not in result_dig_srv.stdout"
+
+- name: Report correct DNS SRV record
+  debug:
+    msg: "The DNS SRV record for {{ hostname_identity }} points to {{ hostname_matrix }}, as expected"

roles/matrix-server/tasks/self_check/self_check_federation_api.yml

@@ -0,0 +1,21 @@
+---
+
+- set_fact:
+    matrix_federation_api_url_endpoint_public: "https://{{ hostname_matrix }}:8448/_matrix/federation/v1/version"
+
+- name: Check Matrix Federation API
+  uri:
+    url: "{{ matrix_federation_api_url_endpoint_public }}"
+    follow_redirects: false
+    validate_certs: false
+  register: result_matrix_federation_api
+  ignore_errors: true
+
+- name: Fail if Matrix Federation API not working
+  fail:
+    msg: "Failed checking Matrix Federation API is up at `{{ hostname_matrix }}` (checked endpoint: `{{ matrix_federation_api_url_endpoint_public }}`). Is Synapse running? Is port 8448 open in your firewall? Full error: {{ result_matrix_federation_api }}"
+  when: "result_matrix_federation_api.failed or 'json' not in result_matrix_federation_api"
+
+- name: Report working Matrix Federation API
+  debug:
+    msg: "The Matrix Federation API at `{{ hostname_matrix }}` (checked endpoint: `{{ matrix_federation_api_url_endpoint_public }}`) is working"

roles/matrix-server/tasks/self_check/self_check_mxisd.yml

@@ -0,0 +1,20 @@
+---
+
+- set_fact:
+    mxisd_url_endpoint_public: "https://{{ hostname_matrix }}/_matrix/identity/api/v1"
+
+- name: Check mxisd Identity Service
+  uri:
+    url: "{{ mxisd_url_endpoint_public }}"
+    follow_redirects: false
+  register: result_mxisd
+  ignore_errors: true
+
+- name: Fail if mxisd Identity Service not working
+  fail:
+    msg: "Failed checking mxisd is up at `{{ hostname_matrix }}` (checked endpoint: `{{ mxisd_url_endpoint_public }}`). Is mxisd running? Is port 443 open in your firewall? Full error: {{ result_mxisd }}"
+  when: "result_mxisd.failed or 'json' not in result_mxisd"
+
+- name: Report working mxisd Identity Service
+  debug:
+    msg: "mxisd at `{{ hostname_matrix }}` is working (checked endpoint: `{{ mxisd_url_endpoint_public }}`)"

roles/matrix-server/tasks/self_check/self_check_riot_web.yml

@@ -0,0 +1,20 @@
+---
+
+- set_fact:
+    riot_web_url_endpoint_public: "https://{{ hostname_riot }}/config.json"
+
+- name: Check riot-web
+  uri:
+    url: "{{ riot_web_url_endpoint_public }}"
+    follow_redirects: false
+  register: result_riot_web
+  ignore_errors: true
+
+- name: Fail if riot-web not working
+  fail:
+    msg: "Failed checking riot-web is up at `{{ hostname_riot }}` (checked endpoint: `{{ riot_web_url_endpoint_public }}`). Is Riot running? Is port 443 open in your firewall? Full error: {{ result_riot_web }}"
+  when: "result_riot_web.failed or 'json' not in result_riot_web"
+
+- name: Report working riot-web
+  debug:
+    msg: "riot-web at `{{ hostname_riot }}` is working (checked endpoint: `{{ riot_web_url_endpoint_public }}`)"

roles/matrix-server/tasks/self_check/self_check_well_known.yml

@@ -0,0 +1,46 @@
+---
+
+- set_fact:
+    well_known_url_matrix: "https://{{ hostname_matrix }}/.well-known/matrix/client"
+    well_known_url_identity: "https://{{ hostname_matrix }}/.well-known/matrix/client"
+
+
+- name: Check .well-known on the matrix hostname
+  uri:
+    url: "{{ well_known_url_matrix }}"
+    follow_redirects: false
+  register: result_well_known_matrix
+  ignore_errors: true
+
+- name: Fail if .well-known not working on the matrix hostname
+  fail:
+    msg: "Failed checking well-known is configured at `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_matrix }}"
+  when: "result_well_known_matrix.failed or 'json' not in result_well_known_matrix"
+
+- name: Report working .well-known on the matrix hostname
+  debug:
+    msg: "well-known is configured at `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`)"
+
+
+- name: Check .well-known on the identity hostname
+  uri:
+    url: "{{ well_known_url_identity }}"
+    follow_redirects: false
+  register: result_well_known_identity
+  ignore_errors: true
+
+- name: Fail if .well-known not working on the identity hostname
+  fail:
+    msg: "Failed checking well-known is configured at `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_identity }}"
+  when: "result_well_known_identity.failed or 'json' not in result_well_known_identity"
+
+- name: Report working .well-known on the identity hostname
+  debug:
+    msg: "well-known is configured at `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`)"
+
+
+# For people who manually copy the well-known file, try to detect if it's outdated
+- name: Fail if well-known is different on matrix hostname and identity hostname
+  fail:
+    msg: "The well-known files at `{{ hostname_matrix }}` and `{{ hostname_identity }}` are different. Perhaps you copied the file manually before and now it's outdated?"
+  when: "result_well_known_matrix.json|to_json != result_well_known_identity.json|to_json"