diff --git a/docs/configuring-playbook-prometheus-nginxlog.md b/docs/configuring-playbook-prometheus-nginxlog.md index 1e8acbd13..832efbd8d 100644 --- a/docs/configuring-playbook-prometheus-nginxlog.md +++ b/docs/configuring-playbook-prometheus-nginxlog.md @@ -3,10 +3,10 @@ It can be useful to have some (visual) insight into NignX logs. This adds [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) to your matrix deployment. -It will provide a prometheus 'metrics' endpoint exposing data from both the `matrix-nginx-proxy` and `matrix-synapse-reverse-proxy-companion` logs and automatically aggregrates the data with prometheus. -Optionally it visualizes the data when [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) is enabled by means of a dedicated dashboard named `NGINX PROXY` +It will provide a prometheus 'metrics' endpoint exposing data from both the `matrix-nginx-proxy` and `matrix-synapse-reverse-proxy-companion` logs and automatically aggregates the data with prometheus. +Optionally it visualizes the data, if [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) is enabled, by means of a dedicated Grafana dashboard named `NGINX PROXY` -You can enable this role by adding the following setting in your configuration file (`inventory/host_vars/matrix./vars.yml`): +You can enable this role by adding the following settings in your configuration file (`inventory/host_vars/matrix./vars.yml`): ```yaml matrix_prometheus_nginxlog_exporter_enabled: true @@ -18,10 +18,10 @@ matrix_prometheus_enabled: true matrix_grafana_enabled: true ``` -x | Prerequisites | var | Description +x | Prerequisites | Variable | Description |:--:|:--:|:--:|:--| **REQUIRED** | `matrix-prometheus`| `matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about. -Optional | [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) | [`matrix_grafana_enabled`](configuring-playbook-prometheus-grafana.md)|[Grafana](https://grafana.com) is the visual component. It shows (on the `stats.` subdomain) the dashboards with the graphs that we're interested in. When enabled the `NGINX PROXY` dashboard is automatically added. +_Optional_ | [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) | [`matrix_grafana_enabled`](configuring-playbook-prometheus-grafana.md)|[Grafana](https://grafana.com) is the visual component. It shows (on the `stats.` subdomain) graphs that we're interested in. When enabled the `NGINX PROXY` dashboard is automatically added. ## Docker Image Compatibility @@ -34,17 +34,14 @@ matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: false matrix_prometheus_nginxlog_exporter_docker_image: path/to/docker/image:tag ``` - ## Security and privacy -Metrics and resulting graphs can contain a lot of information. NginX logs contain information about visitor IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. And you should really not forget to change your Grafana password. - - -## Collecting metrics to an external Prometheus server +Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access. +Please make sure you change the default Grafana password. -The playbook will automatically integrate the metrics into the Prometheus server provided with this playbook. +## Save metrics on an external Prometheus server -The playbook provides a single endpoint (`https://matrix.DOMAIN/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, etc). To enable this `/metrics/*` feature, use `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled`. +The playbook will automatically integrate the metrics into the Prometheus server provided with this playbook. You can choose to save data on an external Prometheus instance. The metrics of this role will be exposed on `https://matrix.DOMAIN/metrics/nginxlog` when setting ```yaml @@ -53,6 +50,7 @@ matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: true # required dependency matrix_nginx_proxy_proxy_matrix_metrics_enabled: true ``` +The playbook can provide a single endpoint (`https://matrix.DOMAIN/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, etc). To enable this `/metrics/*` feature, use `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled`. The following variables may be of interest: diff --git a/roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml b/roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml index 69869e54b..2e85d6298 100644 --- a/roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml +++ b/roles/custom/matrix-prometheus-nginxlog-exporter/defaults/main.yml @@ -1,13 +1,15 @@ --- -# prometheus-nginxlog-exporter exports nginx logs into a prometheus usable format on a `/metrics/ endpoint +# prometheus-nginxlog-exporter exports nginx logs in a prometheus usable format on a `/metrics/ endpoint # See: https://github.com/martin-helmich/prometheus-nginxlog-exporter/ matrix_prometheus_nginxlog_exporter_enabled: true +matrix_prometheus_nginxlog_exporter_version: v1.10.0 + matrix_prometheus_nginxlog_exporter_container_hostname: 'matrix-prometheus-nginxlog-exporter' matrix_prometheus_nginxlog_exporter_container_metrics_port: '4040' matrix_prometheus_nginxlog_exporter_container_syslog_port: '6514' -matrix_prometheus_nginxlog_exporter_version: v1.10.0 +matrix_prometheus_nginxlog_exporter_docker_image_arch: ['amd64', 'arm64'] matrix_prometheus_nginxlog_exporter_docker_image_name_prefix: "{{ 'ghcr.io/martin-helmich/' }}" matrix_prometheus_nginxlog_exporter_docker_image_tag: "{{ 'latest' if matrix_prometheus_nginxlog_exporter_version == 'master' else matrix_prometheus_nginxlog_exporter_version }}-{{ matrix_architecture }}" matrix_prometheus_nginxlog_exporter_docker_image: "{{ matrix_prometheus_nginxlog_exporter_docker_image_name_prefix }}prometheus-nginxlog-exporter/exporter:{{ matrix_prometheus_nginxlog_exporter_docker_image_tag }}" @@ -15,7 +17,7 @@ matrix_prometheus_nginxlog_exporter_docker_image_force_pull: "{{ matrix_promethe matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled: true # Controls whether prometheus-nginxlog-exporter metrics should be proxied (exposed) on `matrix.DOMAIN/metrics/nginxlog`. -# This will only work take effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`. +# This will only take effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`. # See the `matrix-nginx-proxy` role for details about enabling `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. matrix_prometheus_nginxlog_exporter_metrics_proxying_enabled: false diff --git a/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_uninstall.yml b/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_uninstall.yml index acc28ce23..c0d0a8338 100644 --- a/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_uninstall.yml +++ b/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/setup_uninstall.yml @@ -18,7 +18,7 @@ - name: Ensure matrix-prometheus-nginxlog-exporter is stopped ansible.builtin.service: - name: matrix-grafana + name: matrix-prometheus-nginxlog-exporter state: stopped enabled: false daemon_reload: true diff --git a/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/validate_config.yml b/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/validate_config.yml index 4d7c9ab2d..b8423dc8f 100644 --- a/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/validate_config.yml +++ b/roles/custom/matrix-prometheus-nginxlog-exporter/tasks/validate_config.yml @@ -3,7 +3,7 @@ - name: Fail if Prometheus not enabled ansible.builtin.fail: msg: > - You need to enable `matrix_prometheus_enabled` to use Prometheus as data source for nginxlog-exporter. + You need to enable `matrix_prometheus_enabled` as this roles requires Prometheus for data storage. when: "not matrix_prometheus_enabled" - name: Fail if docker image not availble for arch @@ -21,7 +21,7 @@ 'matrix_prometheus_nginxlog_exporter: false' in vars.yml - when: matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled and matrix_architecture not in ['amd64', 'arm64'] + when: matrix_prometheus_nginxlog_exporter_docker_image_arch_check_enabled and matrix_architecture not in matrix_prometheus_nginxlog_exporter_docker_image_arch - name: Fail if nginx-proxy is not set to proxy metrics while prometheus-nginxlog-exporter is diff --git a/roles/custom/matrix-synapse-reverse-proxy-companion/templates/nginx/nginx.conf.j2 b/roles/custom/matrix-synapse-reverse-proxy-companion/templates/nginx/nginx.conf.j2 index b1fa33efc..309bb5301 100644 --- a/roles/custom/matrix-synapse-reverse-proxy-companion/templates/nginx/nginx.conf.j2 +++ b/roles/custom/matrix-synapse-reverse-proxy-companion/templates/nginx/nginx.conf.j2 @@ -54,7 +54,6 @@ http { access_log off; {% endif %} - proxy_connect_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_connect_timeout }}; proxy_send_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_send_timeout }}; proxy_read_timeout {{ matrix_synapse_reverse_proxy_companion_proxy_read_timeout }};