badbl0cks
c87d73435b
This commit significantly improves the gatekeeper system's robustness, monitoring capabilities, and simplifies host header management for backend services.
Key changes include:
**Gatekeeper Health, Management & Resilience:**
- Implemented active health checking for individual gatekeeper containers within the `gatekeeper-manager` service.
- The manager now periodically curls the `/metrics` endpoint of each gatekeeper container.
- Reports health status to a new Gatus `services_gatekeeper` endpoint.
- Automatically attempts to restart the gatekeeper stack if any gatekeeper instance is unhealthy or if the expected number of gatekeepers is not running.
- Refactored the `gatekeeper-manager` shell script for improved state management and signal handling:
- Introduced `STARTED`, `RESTARTING`, `TERMINATING` state flags for more controlled operations.
- Enhanced SIGTERM and SIGHUP handling to gracefully manage gatekeeper lifecycles.
- Added `apk add curl` to ensure `curl` is available in the manager container.
- Renamed the gatekeeper Docker Compose template from `docker-compose_gatekeeper.template.yml` to `gatekeepers.template.yml` and its output to `gatekeepers.yml`.
- Updated `dockergen-gatekeeper` to watch the new template file and notify the correct `gatekeeper-manager` service instance (e.g., `pkmntrade-club-gatekeeper-manager-1`).
- Discover services that should be protected by looking for a `gatekeeper=true` label.
**Host Header Management & `ALLOWED_HOSTS` Simplification:**
- HAProxy configuration (`haproxy.cfg`) now consistently sets the `Host` HTTP header for requests to all backend services (e.g., `pkmntrade.club`, `staging.pkmntrade.club`). This centralizes and standardizes host information.
- Consequently, explicit `ALLOWED_HOSTS` environment variables have been removed from the `web` and `celery` service definitions in `docker-compose_web.yml` and `docker-compose_staging.yml`. Backend Django applications should now rely on the `Host` header set by HAProxy for request validation.
- The `gatekeepers.template.yml` now defines a `TARGET_HOST` environment variable for proxied services (e.g., `web`, `web-staging`). This aligns with the ALLOWED_HOSTS on the target to ensure requests aren't blocked.
**Gatus Monitoring & Configuration Updates:**
- In Gatus configuration (`gatus/config.template.yaml`):
- The "Redis" external service endpoint has been renamed to "Cache" for better clarity and to fit the theme of simple names.
- A new external service endpoint "Gatekeeper" has been added to monitor the overall health reported by the `gatekeeper-manager`.
- Health checks for "Web Worker" endpoints (both main and staging) now include the appropriate `Host` header (e.g., `Host: pkmntrade.club`) to ensure accurate health assessments by Django.
- In `docker-compose_core.yml`, the `curl` commands used by `db-redis-healthcheck` for database and cache health now append `|| true`. This prevents the script from exiting on a curl error (e.g., timeout, connection refused), ensuring that the failure is still reported to Gatus via the `success=false` parameter rather than the script terminating prematurely.
These changes collectively make the gatekeeper system more fault-tolerant, provide better visibility into its status, and streamline the configuration of backend applications by standardizing how they receive host information.
56 lines
1.8 KiB
INI
56 lines
1.8 KiB
INI
# https://docs.haproxy.org/3.1/configuration.html
|
|
global
|
|
log stdout format raw local0 # Send logs to Docker's stdout
|
|
master-worker
|
|
|
|
resolvers docker_resolver
|
|
nameserver docker_dns 127.0.0.11:53 # Docker's internal DNS
|
|
resolve_retries 3
|
|
timeout resolve 1s
|
|
timeout retry 1s
|
|
hold valid 10s
|
|
hold obsolete 30s
|
|
accepted_payload_size 8192 # Optional: Increase if you have many replicas
|
|
|
|
defaults
|
|
mode http
|
|
log global
|
|
timeout client 120s
|
|
timeout connect 120s
|
|
timeout server 120s
|
|
timeout http-request 120s
|
|
option httplog
|
|
|
|
frontend haproxy_entrypoint
|
|
bind :443 ssl crt /certs/crt.pem verify required ca-file /certs/ca.pem
|
|
use_backend %[req.hdr(host),lower,word(1,:)] # strip out port from host
|
|
|
|
frontend checks
|
|
bind :80
|
|
default_backend basic_check
|
|
|
|
backend basic_check
|
|
http-request return status 200 content-type "text/plain" lf-string "OK/HEALTHY"
|
|
|
|
backend pkmntrade.club
|
|
balance leastconn
|
|
http-request set-header Host pkmntrade.club
|
|
server-template gatekeeper-web- 4 gatekeeper-web:8000 check resolvers docker_resolver init-addr libc,none
|
|
|
|
backend staging.pkmntrade.club
|
|
balance leastconn
|
|
http-request set-header Host staging.pkmntrade.club
|
|
server-template gatekeeper-web-staging- 4 gatekeeper-web-staging:8000 check resolvers docker_resolver init-addr libc,none
|
|
|
|
backend feedback.pkmntrade.club
|
|
balance leastconn
|
|
http-request set-header Host feedback.pkmntrade.club
|
|
server-template gatekeeper-feedback- 4 gatekeeper-feedback:8000 check resolvers docker_resolver init-addr libc,none
|
|
|
|
backend health.pkmntrade.club
|
|
balance leastconn
|
|
http-request set-header Host health.pkmntrade.club
|
|
server-template gatekeeper-health- 4 gatekeeper-health:8000 check resolvers docker_resolver init-addr libc,none
|
|
|
|
#EOF - trailing newline required
|
|
|