badblocks/docker-easy-haproxy
1
0
Fork 0
Code Activity

Merge pull request #26 from byjg/stats

Browse source 
Change statistics
This commit is contained in:
Joao M 2022-08-18 15:40:11 -05:00 • committed by GitHub
commit 9ba269845d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
22 changed files with 126 additions and 132 deletions
Download patch file Download diff file Expand all files Collapse all files

3
.gitpod.yml
View file

@ -3,9 +3,6 @@
# and commit this file to your remote git repository to share the goodness with others.
tasks:
- init:
make
- command: |
virtualenv -p /usr/bin/python3 venv
source venv/bin/activate

4
Dockerfile
View file

@ -8,7 +8,7 @@ COPY easymapping /scripts/easymapping/
COPY tests/ /scripts/tests/
COPY assets /
RUN apk add --no-cache haproxy bash python3 py3-pip py-yaml supervisor docker certbot openssl \
RUN apk add --no-cache haproxy bash python3 py3-pip py-yaml docker certbot openssl \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& pip3 install --upgrade pip \
&& pip install -r requirements.txt \
@ -16,4 +16,4 @@ RUN apk add --no-cache haproxy bash python3 py3-pip py-yaml supervisor docker ce
&& openssl dhparam -out /etc/haproxy/dhparam 2048 \
&& openssl dhparam -out /etc/haproxy/dhparam-1024 1024
CMD ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf" ]
CMD ["/bin/bash", "-c", "/scripts/haproxy.sh" ]

48
README.md
View file

@ -45,16 +45,17 @@ The mapping to `/var/run/docker.sock` is necessary to discover the docker contai
The environment variables will setup the HAProxy.
| Environment Variable | Description |
|-------------------------------|-------------------------------------------------------------------------------|
| EASYHAPROXY_DISCOVER | How `haproxy.cfg` will be created: `static`, `docker` or `swarm` |
| EASYHAPROXY_LABEL_PREFIX | (Optional) The key will search for matching resources. Default: `easyhaproxy`. |
| EASYHAPROXY_LETSENCRYPT_EMAIL | (Optional) The email will be used to request the certificate to Letsencrypt |
| Environment Variable | Description |
|-------------------------------|---------------------------------------------------------------------------------------------------------------|
| EASYHAPROXY_DISCOVER | How `haproxy.cfg` will be created: `static`, `docker` or `swarm` |
| EASYHAPROXY_LABEL_PREFIX | (Optional) The key will search for matching resources. Default: `easyhaproxy`. |
| EASYHAPROXY_LETSENCRYPT_EMAIL | (Optional) The email will be used to request the certificate to Letsencrypt |
| EASYHAPROXY_SSL_MODE | (Optional) `STRICT` supports only the most recent TLS version; `DEFAULT` good SSL integration with recent browsers; `LOOSE` supports all old SSL protocols for old browsers (not recommended). |
| HAPROXY_USERNAME | (Optional) The HAProxy username to the statistics. Default: `admin` |
| HAPROXY_PASSWORD | The HAProxy password to the statistics. If not set, it will disable stats. |
| HAPROXY_STATS_PORT | (Optional) The HAProxy port to the statistics. Default: `1936` |
| HAPROXY_CUSTOMERRORS | (Optional) If HAProxy will use custom HTML errors. true/false. Default: false |
| EASYHAPROXY_REFRESH_CONF | (Optional) Check configuration every N seconds. Default: 10 |
| HAPROXY_USERNAME | (Optional) The HAProxy username to the statistics. Default: `admin` |
| HAPROXY_PASSWORD | (Optional) The HAProxy password to the statistics. If not set, statistics will be available with no password |
| HAPROXY_STATS_PORT | (Optional) The HAProxy port to the statistics. Default: `1936`. If set to `false`, disable statistics |
| HAPROXY_CUSTOMERRORS | (Optional) If HAProxy will use custom HTML errors. true/false. Default: `false` |
The environment variable `EASYHAPROXY_DISCOVER` will define where is located your containers (see below for more details):
@ -146,6 +147,33 @@ docker run \
some/myimage
```
### Multiples hosts on the same container
```bash
docker run \
-l easyhaproxy.express.port=80 \
-l easyhaproxy.express.localport=3000 \
-l easyhaproxy.express.host=express.byjg.com.br,admin.byjg.com.br \
.... \
some/myimage
```
If you are using docker-compose you can use this way:
```yaml
version: "3"
services:
mycontainer:
image: some/myimage
labels:
easyhaproxy.express.port: 80
easyhaproxy.express.localport: 3000
easyhaproxy.express.host: >-
express.byjg.com.br,
admin.byjg.com.br
```
### TLS passthrough
Used to pass on SSL termination to a backend. Alternatively, you can enable health-check via SSL on the backend with the optional `health-check` label:
@ -258,7 +286,7 @@ Caveats:
You must expose some ports on the EasyHAProxy container and in the firewall. However, you don't need to expose the other container ports because EasyHAProxy will handle that.
- The ports `80` and `443`.
- If you enable the HAProxy statistics, you must also expose the port defined in `HAPROXY_STATS_PORT` environment variable.
- If you enable the HAProxy statistics, you must also expose the port defined in `HAPROXY_STATS_PORT` environment variable (default 1936). Be aware that statististics are enabled by default with no password.
- Every port defined in `easyhaproxy.[definitions].port` also should be exposed.
e.g.

9
assets/etc/crontabs/root
View file

@ -1,9 +0,0 @@
# do daily/weekly/monthly maintenance
# min hour day month weekday command
*/15 * * * * run-parts /etc/periodic/15min
0 * * * * run-parts /etc/periodic/hourly
0 2 * * * run-parts /etc/periodic/daily
0 3 * * 6 run-parts /etc/periodic/weekly
0 5 1 * * run-parts /etc/periodic/monthly
* * * * * /scripts/haproxy-reload.sh > /proc/$(cat /var/run/supervisord.pid)/fd/1 2>&1

45
assets/etc/supervisord.conf
View file

@ -1,45 +0,0 @@
[unix_http_server]
file=/dev/shm/supervisor.sock ; (the path to the socket file)
[supervisord]
logfile=/var/log/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
user=root ;
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///dev/shm/supervisor.sock ; use a unix:// URL for a unix socket
[program:haproxy]
command = /scripts/haproxy.sh
autostart=true
autorestart=false
priority=5
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:crond]
command=/usr/sbin/crond -f
autostart=true
autorestart=false
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[eventlistener:exit_on_any_fatal]
command=/scripts/exit-event-listener.py
events=PROCESS_STATE_FATAL,PROCESS_STATE_EXITED,PROCESS_STATE_STOPPED

6
assets/scripts/certbot.sh
View file

@ -2,7 +2,7 @@
# Semaphore
if [ -f /tmp/certbot-lock ]; then
echo "[CERTBOT_JOB] Another process is running"
echo "[CERTBOT_JOB] $(date +"$EASYHAPROXY_DATEFORMAT") Another process is running"
exit 0
fi
@ -26,7 +26,7 @@ for domain in $(cat /scripts/letsencrypt_hosts.txt); do
done
if [ -n "$REQUEST_CERTS" ]; then
echo "[CERTBOT_JOB] Requesting certificates for $REQUEST_CERTS"
echo "[CERTBOT_JOB] $(date +"$EASYHAPROXY_DATEFORMAT") Requesting certificates for $REQUEST_CERTS"
certbot certonly \
--standalone \
--preferred-challenges http \
@ -41,7 +41,7 @@ if [ -n "$REQUEST_CERTS" ]; then
fi
if [ -n "$RENEW_CERTS" ]; then
echo "[CERTBOT_JOB] Resquesting renew certificated fort $RENEW_CERTS"
echo "[CERTBOT_JOB] $(date +"$EASYHAPROXY_DATEFORMAT") Resquesting renew certificated fort $RENEW_CERTS"
certbot renew --post-hook "/scripts/certbot_to_haproxy.sh"
fi

23
assets/scripts/exit-event-listener.py
View file

@ -1,23 +0,0 @@
#!/usr/bin/env python
import os
import signal
from supervisor import childutils
def main():
while True:
headers, payload = childutils.listener.wait()
childutils.listener.ok()
events = ['PROCESS_STATE_FATAL', 'PROCESS_STATE_EXITED', 'PROCESS_STATE_STOPPED']
if headers['eventname'] not in events:
continue
print(headers)
print(payload)
os.kill(os.getppid(), signal.SIGTERM)
if __name__ == "__main__":
main()

15
assets/scripts/haproxy-reload.sh
View file

@ -35,7 +35,7 @@ else
RELOAD="false"
else
python3 swarm.py > /etc/haproxy/haproxy.cfg
echo "[CONF_CHECK] New configuration found"
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") New configuration found"
fi
fi
@ -44,24 +44,23 @@ if cmp -s ${CONTROL_FILE} ${CONTROL_FILE}.old ; then
fi
if [[ ! -z "$1" ]]; then
echo "[CONF_CHECK] Initial configuration. Skip certbot."
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") Initial configuration. Skip certbot."
else
/scripts/certbot.sh
fi
# If Certbot reloads successfully will create the file /tmp/force-reload
if [ -f /tmp/force-reload ]; then
echo "[CONF_CHECK] New certificates found..."
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") New certificates found..."
RELOAD="true"
rm /tmp/force-reload
fi
if [[ ! -z "$1" ]]; then
echo "[CONF_CHECK] Start haproxy"
/usr/sbin/haproxy -W -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /var/run/haproxy.sock
fi
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") Starting haproxy..."
/usr/sbin/haproxy -W -f /etc/haproxy/haproxy.cfg $(ls /etc/haproxy/conf.d/*.cfg 2>/dev/null | xargs -I{} echo -f {}) -p /run/haproxy.pid -S /var/run/haproxy.sock &
if [[ "$RELOAD" == "true" ]]; then
echo "[CONF_CHECK] Reloading..."
elif [[ "$RELOAD" == "true" ]]; then
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") Reloading..."
/usr/sbin/haproxy -W -f /etc/haproxy/haproxy.cfg $(ls /etc/haproxy/conf.d/*.cfg 2>/dev/null | xargs -I{} echo -f {}) -p /run/haproxy.pid -x /var/run/haproxy.sock -sf $(cat /run/haproxy.pid) &
fi

14
assets/scripts/haproxy.sh
View file

@ -2,8 +2,18 @@
/usr/sbin/haproxy -v
source /scripts/haproxy-reload.sh initial
if [ -z "$EASYHAPROXY_DATEFORMAT" ]; then
export EASYHAPROXY_DATEFORMAT="%Y-%m-%d %H:%M:%S %Z"
fi
if [ -z "$EASYHAPROXY_REFRESH_CONF" ]; then
export EASYHAPROXY_REFRESH_CONF=10
fi
/scripts/haproxy-reload.sh initial
while true; do
sleep 60
sleep $EASYHAPROXY_REFRESH_CONF
echo "[CONF_CHECK] $(date +"$EASYHAPROXY_DATEFORMAT") - Heartbeat."
/scripts/haproxy-reload.sh
done

15
templates/haproxy.cfg.j2
View file

@ -12,6 +12,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s
@ -26,23 +27,23 @@ defaults
errorfile 504 /etc/haproxy/errors-custom/504.http
{% endif %}
{% if "stats" in data %}
{% set data_stats = data["stats"] | default({}) %}
{% if data_stats["port"] | default(1936) | int > 0 %}
frontend stats
bind *:{{ data["stats"]["port"] | default(1936) }}
bind *:{{ data_stats["port"] | default(1936) }}
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth {{ data["stats"]["username"] }}:{{ data["stats"]["password"] }}
# acl is_proxystats hdr(host) -i some.host.com
# default_backend srv_stats
# use_backend srv_stats if is_proxystats
{% if data_stats["password"] | default("") != "" %}
stats auth {{ data_stats["username"] | default("admin") }}:{{ data_stats["password"] }}
{% endif %}
default_backend srv_stats
backend srv_stats
mode http
server Local 127.0.0.1:{{ data["stats"]["port"] | default(1936) }}
server Local 127.0.0.1:{{ data_stats["port"] | default(1936) }}
{% endif %}
{% for o in data["easymapping"] -%}
{% set mode = o["mode"] or "http" %}

1
tests/expected/no-services.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

4
tests/expected/services-letsencrypt.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s
@ -36,9 +37,6 @@ frontend stats
stats realm Haproxy\ Statistics
stats uri /
stats auth admin:password
# acl is_proxystats hdr(host) -i some.host.com
# default_backend srv_stats
# use_backend srv_stats if is_proxystats
default_backend srv_stats
backend srv_stats

1
tests/expected/services-multi-containers.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

4
tests/expected/services-multiple-hosts.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s
@ -36,9 +37,6 @@ frontend stats
stats realm Haproxy\ Statistics
stats uri /
stats auth joe:s3cr3t
# acl is_proxystats hdr(host) -i some.host.com
# default_backend srv_stats
# use_backend srv_stats if is_proxystats
default_backend srv_stats
backend srv_stats

1
tests/expected/services-redirect-ssl.txt
View file

@ -14,6 +14,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

1
tests/expected/services-tcp.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

1
tests/expected/services.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

13
tests/expected/ssl-loose.txt
View file

@ -14,11 +14,24 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s
timeout server 10m
frontend stats
bind *:1936
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
default_backend srv_stats
backend srv_stats
mode http
server Local 127.0.0.1:1936
backend letsencrypt_backend
mode http

1
tests/expected/ssl-strict.txt
View file

@ -11,6 +11,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s

4
tests/expected/static.txt
View file

@ -16,6 +16,7 @@ global
defaults
log global
option httplog
timeout connect 3s
timeout client 10s
@ -36,9 +37,6 @@ frontend stats
stats realm Haproxy\ Statistics
stats uri /
stats auth admin:test123
# acl is_proxystats hdr(host) -i some.host.com
# default_backend srv_stats
# use_backend srv_stats if is_proxystats
default_backend srv_stats
backend srv_stats

4
tests/fixtures/services-multiple-hosts vendored
View file

@ -1,3 +1,3 @@
db79d3a910f4={"com.docker.compose.config-hash":"5bde40f52451521ad201e70de1291397376a0498a7c955624a609da3b60e7e8e","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:ea39067705590557dd0cd951664a10970ceefcb725a3c1f43690d6d6d4ed5fce","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-multi-containers.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"haproxy","com.docker.compose.version":"2.8.0"}
3e63154954b0={"com.docker.compose.config-hash":"4e0cbdd8372c6779863799e5021ed8178f74b55bd8e070abcdffaf87eb7baa36","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-multi-containers.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"hello.com, www.helloworld.com","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"19901","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}"}
eb294c110eb1={"com.docker.compose.config-hash":"4e0cbdd8372c6779863799e5021ed8178f74b55bd8e070abcdffaf87eb7baa36","com.docker.compose.container-number":"2","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-multi-containers.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"hello.com, www.helloworld.com","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"19901","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}"}
3e63154954b0={"com.docker.compose.config-hash":"4e0cbdd8372c6779863799e5021ed8178f74b55bd8e070abcdffaf87eb7baa36","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-multi-containers.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"hello.com\n, www.helloworld.com\n","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"19901","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}"}
eb294c110eb1={"com.docker.compose.config-hash":"4e0cbdd8372c6779863799e5021ed8178f74b55bd8e070abcdffaf87eb7baa36","com.docker.compose.container-number":"2","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-multi-containers.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"hello.com\n, www.helloworld.com\n","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"19901","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}"}

37
tests/test_parser.py
View file

@ -19,7 +19,10 @@ def test_parser_doesnt_crash():
line_list = load_fixture("no-services")
result = {
"customerrors": False
"customerrors": False,
"stats": {
"port": "false"
}
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -38,6 +41,9 @@ def test_parser_finds_services():
"customerrors": False,
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
},
"stats": {
"port": 0
}
}
@ -65,6 +71,9 @@ def test_parser_finds_services_changed_label():
"lookup_label": "haproxy",
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
},
"stats": {
"port": 0
}
}
@ -91,6 +100,9 @@ def test_parser_finds_services_raw():
"customerrors": False,
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
},
"stats": {
"port": 0
}
}
@ -277,7 +289,10 @@ def test_parser_tcp():
line_list = load_fixture("services-tcp")
result = {
"customerrors": False
"customerrors": False,
"stats": {
"port": 0
}
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -294,7 +309,10 @@ def test_parser_multi_containers():
line_list = load_fixture("services-multi-containers")
result = {
"customerrors": False
"customerrors": False,
"stats": {
"port": 0
}
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -334,7 +352,10 @@ def test_parser_redirect_ssl():
result = {
"customerrors": False,
"ssl_mode": "loose"
"ssl_mode": "loose",
"stats": {
"port": 0
}
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -352,7 +373,10 @@ def test_parser_ssl_strict():
result = {
"customerrors": False,
"ssl_mode": "strict"
"ssl_mode": "strict",
"stats": {
"port": False
}
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -369,7 +393,7 @@ def test_parser_ssl_loose():
result = {
"customerrors": False,
"ssl_mode": "loose"
"ssl_mode": "loose",
}
cfg = easymapping.HaproxyConfigGenerator(result, CERTS_FOLDER)
@ -387,7 +411,6 @@ def test_parser_ssl_letsencrypt():
result = {
"customerrors": True,
"stats": {
"username": "admin",
"password": "password"
},
"letsencrypt": {