badblocks/docker-easy-haproxy
1
0
Fork 0
Code Activity

Merge branch 'issue/41' into issue/43

Browse source
This commit is contained in:
Joao M 2023-07-03 14:19:00 -05:00 • committed by GitHub
commit 812590d86c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
46 changed files with 859 additions and 542 deletions
Download patch file Download diff file Expand all files Collapse all files

13
README.md
View file

@ -27,11 +27,15 @@ EasyHAProxy is part of some projects:
See detailed instructions on how to install below.
## EasyHAProxy Mission
Easy to set up and low configuration to numerous features.
## Features
EasyHAProxy will discover the services based on the Docker Tags of the containers running on a Docker host or Docker Swarm cluster and dynamically set up the `haproxy.cfg`. Below, EasyHAProxy main features:
- Use Letsencrypt with HAProxy.
- Support Automatic Certificate Management Environment (ACME) protocol compatible with Let's encrypt and other issuers.
- Set your custom SSL certificates
- Balance traffic between multiple replicas
- Set SSL with three different levels of validations and according to the most recent definitions.
@ -67,6 +71,13 @@ Or you can install using tools:
[![MicroK8s](docs/easyhaproxy_microk8s.png)](docs/microk8s.md)
[![Dokku](docs/easyhaproxy_dokku.png)](docs/dokku.md)
## Special Topics
If you already set up the EasyHAProxy, is time to go deeper:
- [Custom SSL](docs/ssl.md)
- Automatic Certificate Issuing (e.g. Let's Encrypt)
## See EasyHAProxy in action
Click on the image to see the videos (use HD for better visualization)

6
deploy/docker/docker-compose.yml
View file

@ -5,13 +5,13 @@ services:
image: byjg/easy-haproxy:4.3.1-rc1
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- certs_letsencrypt:/certs/letsencrypt
- certs_certbot:/certs/certbot
- certs_haproxy:/certs/haproxy
environment:
EASYHAPROXY_DISCOVER: docker
EASYHAPROXY_LABEL_PREFIX: easyhaproxy
EASYHAPROXY_LETSENCRYPT_EMAIL: changeme@example.org
EASYHAPROXY_CERTBOT_EMAIL: changeme@example.org
EASYHAPROXY_SSL_MODE: "default"
HAPROXY_CUSTOMERRORS: "true"
HAPROXY_USERNAME: admin
@ -27,7 +27,7 @@ services:
- easyhaproxy
volumes:
certs_letsencrypt:
certs_certbot:
external: true
certs_haproxy:
external: true

2
deploy/docker/install.sh
View file

@ -3,7 +3,7 @@
ASSETS_DIR="$(dirname "${BASH_SOURCE[0]}")"/../../build/assets/certs/haproxy
docker network create easyhaproxy
docker volume create certs_letsencrypt
docker volume create certs_certbot
docker volume create certs_haproxy
docker run -d --rm --name easyhaproxy_install -v certs_haproxy:/certs alpine tail -f /dev/null

37
docs/acme.md Normal file
View file

@ -0,0 +1,37 @@
# SSL - Automatic Certificate Management Environment (ACME)
The Automatic Certificate Management Environment (ACME) protocol
allow automating interactions between certificate authorities and their users' servers,
allowing the automated deployment of public key infrastructure.
Most of the issuers offers Automatic Issuing free of cost.
To enable the ACME protocol we need to enable Certbot in EasyHAProxy by setting up to 4 environment variables:
- EASYHAPROXY_CERTBOT_AUTOCONFIG (optional): Will use pre-sets for your Certificate Authority (CA). See table below.
- EASYHAPROXY_CERTBOT_EMAIL (required): Your email in the certificate authority.
- EASYHAPROXY_CERTBOT_SERVER (optional): The ACME Endpoint of your certificate authority. If you use AUTOCONFIG, it is set automatically. See table below.
- EASYHAPROXY_CERTBOT_EAB_KID (optional): External Account Binding (EAB) Key Identifier (KID) provided by your certificate authority. Some CA require it. See table below.
- EASYHAPROXY_CERTBOT_EAB_HMAC_KEY (optional): External Account Binding (EAB) HMAC Key provided by your certificate authority. Some CA require it. See table below.
- EASYHAPROXY_CERTBOT_RETRY_COUNT (optional): Wait 'n' requests before retrying issue invalid requests. Default 60.
Here are detailed instructions per Certificate Authority (CA). If anyone is missing, please let's know.
| CA | Auto Config | Free? | Account Required? | EAB KID? | EAB HMAC Key? | More Info |
|----------------------|------------------|-------|--------------------|----------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Let's Encrypt | - | Yes | No | No | No | [Link](letsencrypt.md) |
| Let's Encrypt (Test) | letsencrypt_test | Yes | No | No | No | [Link](letsencrypt.md) |
| ZeroSSL | zerossl | Yes | No | No | No | [Link](https://zerossl.com/documentation/acme/) |
| BuyPass | buypass | Yes | No | No | No | [Link](https://community.buypass.com/t/63d4ay/buypass-go-ssl-endpoints-updated-14-05-2020) |
| BuyPass (test) | buypass_test | Yes | No | No | No | [Link](https://community.buypass.com/t/63d4ay/buypass-go-ssl-endpoints-updated-14-05-2020) |
| Google | google | Yes | Yes | Yes | Yes | [Link](https://cloud.google.com/blog/products/identity-security/automate-public-certificate-lifecycle-management-via--acme-client-api) |
| Google Test | google_test | Yes | Yes | Yes | Yes | [Link](https://cloud.google.com/blog/products/identity-security/automate-public-certificate-lifecycle-management-via--acme-client-api) |
| SSLCOM RCA | sslcom_rca | Yes | No. Keys by email. | Yes | Yes | [Link](https://www.ssl.com/blogs/sslcom-supports-acme-protocol-ssl-tls-certificate-automation/) |
| SSLCOM ECC | sslcom_ecc | Yes | No. Keys by email. | Yes | Yes | [Link](https://www.ssl.com/blogs/sslcom-supports-acme-protocol-ssl-tls-certificate-automation/) |
| Digicert | - | No | Yes | Yes | Yes | [Link](https://docs.digicert.com/en/certcentral/certificate-tools/certificate-lifecycle-automation-guides/use-a-third-party-acme-client-for-host-automations.html) |
| Entrust | - | No | Yes | Yes | Yes | [Link](https://www.entrust.com/knowledgebase/ssl/how-to-use-acme-to-install-ssl-tls-certificates-in-entrust-certificate-services-apache) |
| Sectigo | - | No | Yes | Yes | Yes | [Link](https://www.sectigo.com/resource-library/sectigos-acme-automation) |
This configuration is global. After set up ACME properly, is necessary enable for each domain the certificate request.
To do that add the label: `easyhaproxy.<definition>.certbot=true`. See the method of installation you are using to learn how to set up properly.

28
docs/container-labels.md
View file

@ -2,20 +2,20 @@
## Container (Docker or Swarm) labels
| Tag | Description | Default | Example |
|---------------------------------------|-------------------------------------------------------------------------------------------------------|----------------|--------------|
| easyhaproxy.[definition].host | Host(s) HAProxy is listening. More than one host use comma as delimiter | **required** | somehost.com OR host1.com,host2.com |
| easyhaproxy.[definition].mode | (Optional) Is this `http` or `tcp` mode in HAProxy. | http | http or tcp |
| easyhaproxy.[definition].port | (Optional) Port HAProxy will listen for the host. | 80 | 3000 |
| easyhaproxy.[definition].localport | (Optional) Port container is listening. | 80 | 8080 |
| easyhaproxy.[definition].redirect | (Optional) JSON containing key/value pair from host/to URL redirect. | *empty* | {"foo.com":"https://bla.com", "bar.com":"https://bar.org"} |
| easyhaproxy.[definition].sslcert | (Optional) Cert PEM Base64 encoded. Do not use this if `letsencrypt` is enabled. | *empty* | base64 cert + key |
| easyhaproxy.[definition].ssl | (Optional) If `true` you need to provide certificate as a file. See below. Do not use with `sslcert`. | false | true or false |
| easyhaproxy.[definition].ssl-check | (Optional) `ssl`, enable health check via SSL in `mode tcp` | *empty* | ssl |
| easyhaproxy.[definition].letsencrypt | (Optional) Generate certificate with letsencrypt. Do not use with `sslcert` parameter. | false | true OR false |
| easyhaproxy.[definition].redirect_ssl | (Optional) Redirect all requests to https | false | true OR false |
| easyhaproxy.[definition].clone_to_ssl | (Optional) It copies the configuration to HTTPS(443) and disable SSL from the current config. **Do not use* this with `ssl` or `letsencrypt` parameters | false | true OR false |
| easyhaproxy.[definition].balance | (Optional) HAProxy balance algorithm. See [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-balance) | roundrobin | roundrobin, source, uri, url_param, hdr, rdp-cookie, leastconn, first, static-rr, rdp-cookie, hdr_dom, map-based |
| Tag | Description | Default | Example |
|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------------------------------------------------------------|
| easyhaproxy.[definition].host | Host(s) HAProxy is listening. More than one host use comma as delimiter | **required** | somehost.com OR host1.com,host2.com |
| easyhaproxy.[definition].mode | (Optional) Is this `http` or `tcp` mode in HAProxy. | http | http or tcp |
| easyhaproxy.[definition].port | (Optional) Port HAProxy will listen for the host. | 80 | 3000 |
| easyhaproxy.[definition].localport | (Optional) Port container is listening. | 80 | 8080 |
| easyhaproxy.[definition].redirect | (Optional) JSON containing key/value pair from host/to URL redirect. | *empty* | {"foo.com":"https://bla.com", "bar.com":"https://bar.org"} |
| easyhaproxy.[definition].sslcert | (Optional) Cert PEM Base64 encoded. Do not use this if `certbot` is enabled. | *empty* | base64 cert + key |
| easyhaproxy.[definition].ssl | (Optional) If `true` you need to provide certificate as a file. See below. Do not use with `sslcert`. | false | true or false |
| easyhaproxy.[definition].ssl-check | (Optional) `ssl`, enable health check via SSL in `mode tcp` | *empty* | ssl |
| easyhaproxy.[definition].certbot | (Optional) Generate certificate with certbot. Do not use with `sslcert` parameter. More info [here](acme.md). | false | true OR false |
| easyhaproxy.[definition].redirect_ssl | (Optional) Redirect all requests to https | false | true OR false |
| easyhaproxy.[definition].clone_to_ssl | (Optional) It copies the configuration to HTTPS(443) and disable SSL from the current config. **Do not use* this with `ssl` or `certbot` parameters | false | true OR false |
| easyhaproxy.[definition].balance | (Optional) HAProxy balance algorithm. See [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-balance) | roundrobin | roundrobin, source, uri, url_param, hdr, rdp-cookie, leastconn, first, static-rr, rdp-cookie, hdr_dom, map-based |
The `definition` is a string that will group all configurations togethers. Different `definition` will create different configurations.

34
docs/docker-environment.md
View file

@ -1,21 +1,23 @@
# Docker environment variables
| Environment Variable | Description | Default |
|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|
| EASYHAPROXY_DISCOVER | How the services will be discovered to create `haproxy.cfg`: `static`, `docker`, `swarm` or `kubernetes` | **required** |
| EASYHAPROXY_LABEL_PREFIX | (Optional) The key will search for matching resources. | `easyhaproxy` |
| EASYHAPROXY_LETSENCRYPT_EMAIL | (Optional) The email will be used to request the certificate to Letsencrypt | *empty* |
| EASYHAPROXY_LETSENCRYPT_SERVER | (Optional) Can be `staging` or 'schema://domain.tld'. If set, will try to connect to the Letsencrypt test server | *empty* |
| EASYHAPROXY_SSL_MODE | (Optional) `strict` supports only the most recent TLS version; `default` good SSL integration with recent browsers; `loose` supports all old SSL protocols for old browsers (not recommended). | `default` |
| EASYHAPROXY_REFRESH_CONF | (Optional) Check configuration every N seconds. | 10 |
| EASYHAPROXY_LOG_LEVEL | (Optional) The log level for EasyHAproxy messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | DEBUG |
| CERTBOT_LOG_LEVEL | (Optional) The log level for Certbot messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | DEBUG |
| HAPROXY_LOG_LEVEL | (Optional) The log level for HAProxy messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | INFO |
| HAPROXY_USERNAME | (Optional) The HAProxy username to the statistics. | `admin` |
| HAPROXY_PASSWORD | (Optional) The HAProxy password to the statistics. If not set, statistics will be available with no password | *empty* |
| HAPROXY_STATS_PORT | (Optional) The HAProxy port to the statistics. If set to `false`, disable statistics | `1936` |
| HAPROXY_CUSTOMERRORS | (Optional) If HAProxy will use custom HTML errors. true/false. | `false` |
| Environment Variable | Description | Default |
|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|
| EASYHAPROXY_DISCOVER | How the services will be discovered to create `haproxy.cfg`: `static`, `docker`, `swarm` or `kubernetes` | **required** |
| EASYHAPROXY_LABEL_PREFIX | (Optional) The key will search for matching resources. | `easyhaproxy` |
| EASYHAPROXY_CERTBOT_EMAIL | (Optional) The email will be used to request the certificate to Certbot | *empty* |
| EASYHAPROXY_CERTBOT_SERVER | (Optional) Can be `staging` or 'schema://domain.tld', if using other service than Letsencrypt. Might be necessary set EASYHAPROXY_CERTBOT_EAB_KID and EASYHAPROXY_CERTBOT_EAB_HMAC_KEY | *empty* |
| EASYHAPROXY_CERTBOT_EAB_KID | (Optional) eab-kid configuration when required. | *empty* |
| EASYHAPROXY_CERTBOT_EAB_HMAC_KEY | (Optional) eab-kid-hmac-key configuration when required. | *empty* |
| EASYHAPROXY_CERTBOT_RETRY_COUNT | (Optional) Wait 'n' requests before try re-issue invalid calls |
| EASYHAPROXY_SSL_MODE | (Optional) `strict` supports only the most recent TLS version; `default` good SSL integration with recent browsers; `loose` supports all old SSL protocols for old browsers (not recommended). | `default` |
| EASYHAPROXY_REFRESH_CONF | (Optional) Check configuration every N seconds. | 10 |
| EASYHAPROXY_LOG_LEVEL | (Optional) The log level for EasyHAproxy messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | DEBUG |
| CERTBOT_LOG_LEVEL | (Optional) The log level for Certbot messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | DEBUG |
| HAPROXY_LOG_LEVEL | (Optional) The log level for HAProxy messages. Available: TRACE,DEBUG,INFO,WARN,ERROR,FATAL | DEBUG |
| HAPROXY_USERNAME | (Optional) The HAProxy username to the statistics. | `admin` |
| HAPROXY_PASSWORD | (Optional) The HAProxy password to the statistics. If not set, statistics will be available with no password | *empty* |
| HAPROXY_STATS_PORT | (Optional) The HAProxy port to the statistics. If set to `false`, disable statistics | `1936` |
| HAPROXY_CUSTOMERRORS | (Optional) If HAProxy will use custom HTML errors. true/false. | `false` |
----

4
docs/docker.md
View file

@ -57,9 +57,9 @@ Please follow the [docker label configuration](container-labels.md) to see other
You can configure the behavior of the EasyHAProxy by setup specific environment variables. To get a list of the variables, please follow the [docker container environment](docker-environment.md)
## Setup certificates with Letsencrypt
## Setup certificates with ACME (e.g. Letsencrypt)
Follow [this link](letsencrypt.md)
Follow [this link](acme.md)
## Setup your own certificates

2
docs/helm.md
View file

@ -51,7 +51,7 @@ easyhaproxy:
certbot: DEBUG
easyhaproxy: DEBUG
haproxy: DEBUG
letsencrypt:
certbot:
email: ""
service:

10
docs/kubernetes.md
View file

@ -93,23 +93,25 @@ Notes:
**Important**: The annotations are per ingress and applied to all hosts in that ingress configuration.
## Letsencrypt
## Certbot / ACME / Letsencrypt
It is necessary add the annotation `easyhaproxy.letsencrypt` to the ingress configuration:
It is necessary add the annotation `easyhaproxy.certbot` to the ingress configuration:
```yaml
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: easyhaproxy-ingress
easyhaproxy.letsencrypt: 'true'
easyhaproxy.certbot: 'true'
name: example-ingress
namespace: example
spec:
....
```
Make sure your cluster is accessible both through ports 80 and 443.
More info [here](acme.md).
Make sure your cluster is accessible both through ports 80 and 443.
## Custom SSL Certificates

55
docs/letsencrypt.md
View file

@ -2,49 +2,54 @@
EasyHAProxy can issue a letsencrypt certificate. Follow the steps below:
Run the EasyHAProxy:
## Setting up EasyHAProxy
Run the EasyHAProxy container:
```bash
docker run \
-e EASYHAPROXY_LETSENCRYPT_EMAIL=john@doe.com
.... \
... \
-e EASYHAPROXY_CERTBOT_EMAIL=john@doe.com \
-p 80:80 \
-p 443:443 \
-v /path/to/guest/certbot/certs:/certs/certbot \
... \
byjg/easy-haproxy
```
Run your container:
Notes:
- If you don't setup `EASYHAPROXY_CERTBOT_EMAIL` environment variable, EasyHAProxy will fail silently and **will not request** a certificate.
- The ports 80 and 443 needs to accessible through the internet as [Let's Encrypt requirement](https://letsencrypt.org/docs/allow-port-80/)
Be aware of Letsencrypt issue rate limits:
- https://letsencrypt.org/docs/duplicate-certificate-limit/
- https://letsencrypt.org/docs/rate-limits/
In order to avoid several certificate issuing,
**It is required you to persist the container folder `/certs/certbot` outside the container.**
You cannot delete or change it contents.
If you do not persist, or change/delete the contents, Let's encrypt will not work properly.
## Setting up your container to use Let's Encrypt
```bash
docker run \
... \
--label easyhaproxy.express.port=80 \
--label easyhaproxy.express.localport=3000 \
--label easyhaproxy.express.host=example.org \
--label easyhaproxy.express.letsencrypt=true \
.... \
--label easyhaproxy.express.certbot=true \
... \
some/myimage
```
Requirements:
- Your container **must** listen to port 80. Letsencrypt will not issue the certificate if `easyhaproxy.express.port` is in another port, and EasyHAProxy will fail silently.
- Your container **must** listen to port 80. Letsencrypt will not issue the certificate if `easyhaproxy.<definition>.port` is in another port, and EasyHAProxy will fail silently.
- You cannot set port 443 for the container with the Letsencrypt because EasyHAProxy will create this port automatically once the certificate is issued.
- `EASYHAPROXY_LETSENCRYPT_EMAIL` environment variable is required to be set. If you don't set it up, EasyHAProxy **will not request** a certificate.
Be aware of Letsencrypt issue limits - https://letsencrypt.org/docs/duplicate-certificate-limit/ and https://letsencrypt.org/docs/rate-limits/
## Persist your Letsencrypt certificates
It is a good idea to store the letsencrypt certificate in persistent storage because of the limit on how many certificates can be issued for the same domain in a period.
To do this, map the folder `/certs/letsencrypt` to a docker volume.
```bash
docker volume create certs_letsencrypt
docker run \
/* other parameters */
-v certs_letsencrypt:/certs/letsencrypt \
-d byjg/easy-haproxy
```
----
[Open source ByJG](http://opensource.byjg.com)

19
docs/ssl.md
View file

@ -1,12 +1,15 @@
# Setup custom certificates
You can use your certificates with EasyHAProxy.
You can use your own certificates with EasyHAProxy. You just need to let EasyHAProxy know that certificate.
There are two ways to do that.
- [Setup certificate as a label definition in docker container](#setup-certificate-as-a-label-definition-in-docker-container)
- [Map the certificate as a docker volume](#map-the-certificate-as-a-docker-volume)
## Setup certificate as a label definition in docker container
### Create a single PEM from the certificate and key.
1. Create a single PEM from the certificate and key.
```bash
cat example.com.crt example.com.key > single.pem
@ -24,21 +27,21 @@ MIIEojCCA4qgAwIBAgIUegW2BimwuL4RzRZ2WYkHA6U5nkAwDQYJKoZIhvcNAQEL
-----END PRIVATE KEY-----
```
### Convert the `single.pem` to BASE64 in a single line:
2. Convert the `single.pem` to BASE64 in a single line:
```bash
cat single.pem | base64 -w0
```
### Map the BASE64 to your container
3. Define a label in yout container
Add the Base64 string you generated before to the label `easyhaproxy.[definition].sslcert`
## Map the certificate as a docker volume
EasyHAProxy stores the certificates inside the folder `/certs/haproxy`.
EasyHAProxy stores the certificates inside the container folder `/certs/haproxy`.
### Run EasyHAProxy with the volume for the certificates:
1. Run EasyHAProxy with the volume for the certificates:
```bash
docker volume create certs_haproxy
@ -49,7 +52,7 @@ docker run \
-d byjg/easy-haproxy
```
### Create a single PEM from the certificate and the key.
2. Create a single PEM from the certificate and the key.
```bash
cat example.com.crt example.com.key > single.pem
@ -67,7 +70,7 @@ MIIEojCCA4qgAwIBAgIUegW2BimwuL4RzRZ2WYkHA6U5nkAwDQYJKoZIhvcNAQEL
-----END PRIVATE KEY-----
```
### Copy this certificate to EasyHAProxy volume
3. Copy this certificate to EasyHAProxy volume
```bash
docker cp single.pem easyhaproxy:/certs/haproxy

16
docs/static.md
View file

@ -23,8 +23,9 @@ ssl_mode: default
logLevel:
haproxy: INFO
letsencrypt:
email: "acme@example.org"
certbot: {
"email": "acme@example.org"
}
easymapping:
- port: 80
@ -32,7 +33,7 @@ easymapping:
host1.com.br:
containers:
- container:5000
letsencrypt: true
certbot: true
redirect_ssl: true
host2.com.br:
containers:
@ -90,9 +91,10 @@ logLevel:
easyhaproxy: DEBUG # Optional (default: DEBUG). Can be: TRACE,DEBUG,INFO,WARN,ERROR,FATAL
haproxy: INFO # Optional (default: INFO). Can be: TRACE,DEBUG,INFO,WARN,ERROR,FATAL
# Optional. If you enable `letsencrypt` will need to set up this, otherwise the certificate will be issued
letsencrypt:
email": "acme@example.org"
certbot: { # Optional. If you enable `certbot` will need to setu0p this,
# otherwise the certificate will be issued
"email": "acme@example.org"
}
easymapping:
- port: 80 # Listen port
@ -101,7 +103,7 @@ easymapping:
host1.com.br: # Hostname
containers:
- container:5000 # Endpoints of the hostname above (ip, dns, container, etc)
letsencrypt: true # Optional. it will request a letsencrypt certiticate
certbot: true # Optional. it will request a certbot certiticate
redirect_ssl: true # Optional. It will redirect this site to it SSL.
ssl: true # Optional. Inform this port will listen to SSL, instead of HTTP
clone_to_ssl: true # Optional. Default False. You clone these hosts to its equivalent SSL.

2
examples/docker/docker-compose-portainer-app-example.yml
View file

@ -8,7 +8,7 @@ services:
easyhaproxy.http.host: test.xpto.us
easyhaproxy.http.port: 80
easyhaproxy.http.localport: 8080
easyhaproxy.http.letsencrypt: true
easyhaproxy.http.certbot: true
networks:

12
examples/docker/docker-compose-portainer.yml
View file

@ -1,4 +1,4 @@
# docker volume create certs_letsencrypt
# docker volume create certs_certbot
# docker volume create certs_haproxy
# docker volume create portainer_data
# docker network create easyhaproxy
@ -11,13 +11,13 @@ services:
image: byjg/easy-haproxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- certs_letsencrypt:/certs/letsencrypt
- certs_certbot:/certs/certbot
# - certs_haproxy:/certs/haproxy
environment:
EASYHAPROXY_DISCOVER: docker
EASYHAPROXY_LABEL_PREFIX: easyhaproxy
EASYHAPROXY_LETSENCRYPT_EMAIL: changeme@example.org
EASYHAPROXY_CERTBOT_EMAIL: changeme@example.org
EASYHAPROXY_SSL_MODE: "default"
HAPROXY_CUSTOMERRORS: "true"
HAPROXY_USERNAME: admin
@ -32,18 +32,18 @@ services:
portainer:
image: portainer/portainer-ce:latest
volumes:
- portainer_data:/data portainer
- portainer_data:/data
- /var/run/docker.sock:/var/run/docker.sock
labels:
easyhaproxy.http.redirect_ssl: true
easyhaproxy.http.letsencrypt: true
easyhaproxy.http.certbot: true
easyhaproxy.http.host: portainer.xpto.us
easyhaproxy.http.port: 80
easyhaproxy.http.localport: 9000
volumes:
certs_letsencrypt:
certs_certbot:
external: true
certs_haproxy:
external: true

6
helm/easyhaproxy/templates/deployment.yaml
View file

@ -69,7 +69,7 @@ spec:
value: {{ .Values.easyhaproxy.logLevel.haproxy }}
- name: CERTBOT_LOG_LEVEL
value: {{ .Values.easyhaproxy.logLevel.certbot }}
{{- if .Values.easyhaproxy.letsencrypt.email }}
- name: EASYHAPROXY_LETSENCRYPT_EMAIL
value: {{ .Values.easyhaproxy.letsencrypt.email }}
{{- if .Values.easyhaproxy.certbot.email }}
- name: EASYHAPROXY_CERTBOT_EMAIL
value: {{ .Values.easyhaproxy.certbot.email }}
{{ end }}

2
helm/easyhaproxy/values.yaml
View file

@ -69,7 +69,7 @@ easyhaproxy:
certbot: DEBUG
easyhaproxy: DEBUG
haproxy: DEBUG
letsencrypt:
certbot:
email: ""
# Make sure to create this

43
src/easymapping/__init__.py
View file

@ -1,12 +1,13 @@
import base64
import hashlib
from jinja2 import Environment, FileSystemLoader
import json
import os
import re
from jinja2 import Environment, FileSystemLoader
class DockerLabelHandler:
def __init__(self, label):
self.__data = None
self.__label_base = label
def get_lookup_label(self):
@ -18,19 +19,17 @@ class DockerLabelHandler:
return "{}.{}".format(self.__label_base, ".".join(key))
def get(self, label, default_value = ""):
def get(self, label, default_value=""):
if self.has_label(label):
return self.__data[label]
return default_value
def get_bool(self, label, default_value = False):
def get_bool(self, label, default_value=False):
if self.has_label(label):
return self.__data[label].lower() in ["true", "1", "yes"]
return default_value
def get_json(self, label, default_value = {}):
def get_json(self, label, default_value={}):
if self.has_label(label):
return json.loads(self.__data[label])
return default_value
@ -38,7 +37,6 @@ class DockerLabelHandler:
def set_data(self, data):
self.__data = data
def has_label(self, label):
if label in self.__data:
return True
@ -49,16 +47,16 @@ class HaproxyConfigGenerator:
def __init__(self, mapping):
self.mapping = mapping
self.mapping.setdefault("ssl_mode", 'default')
self.mapping.setdefault("letsencrypt", {"email": "", "staging": False})
self.mapping.setdefault("certbot", {"email": "", "server": False, "eab_kid": False, "eab_hmac_key": False})
self.mapping["ssl_mode"] = self.mapping["ssl_mode"].lower()
self.label = DockerLabelHandler(mapping['lookup_label'] if 'lookup_label' in mapping else "easyhaproxy")
self.letsencrypt_hosts = []
self.certbot_hosts = []
self.serving_hosts = []
self.certs = {}
def generate(self, container_metadata={}):
self.mapping.setdefault("easymapping", [])
if container_metadata != {}:
self.mapping["easymapping"] = self.parse(container_metadata)
@ -70,7 +68,6 @@ class HaproxyConfigGenerator:
template = env.get_template('haproxy.cfg.j2')
return template.render(data=self.mapping)
def parse(self, container_metadata):
easymapping = dict()
@ -106,10 +103,10 @@ class HaproxyConfigGenerator:
"80"
)
letsencrypt = self.label.get_bool(
self.label.create([definition, "letsencrypt"]),
certbot = self.label.get_bool(
self.label.create([definition, "certbot"]),
False
) and self.mapping["letsencrypt"]["email"] != ""
) and self.mapping["certbot"]["email"] != ""
clone_to_ssl = self.label.get_bool(
self.label.create([definition, "clone_to_ssl"])
)
@ -139,9 +136,9 @@ class HaproxyConfigGenerator:
self.serving_hosts.append("%s:%s" % (hostname, port))
easymapping[port]["hosts"].setdefault(hostname, {})
easymapping[port]["hosts"][hostname].setdefault("containers", [])
easymapping[port]["hosts"][hostname].setdefault("letsencrypt", False)
easymapping[port]["hosts"][hostname].setdefault("certbot", False)
easymapping[port]["hosts"][hostname]["containers"] += ["{}:{}".format(container, ct_port)]
easymapping[port]["hosts"][hostname]["letsencrypt"] = letsencrypt
easymapping[port]["hosts"][hostname]["certbot"] = certbot
easymapping[port]["hosts"][hostname]["redirect_ssl"] = self.label.get_bool(
self.label.create([definition, "redirect_ssl"])
)
@ -154,7 +151,7 @@ class HaproxyConfigGenerator:
self.label.create([definition, "redirect"])
)
if letsencrypt or clone_to_ssl:
if certbot or clone_to_ssl:
if "443" not in easymapping:
easymapping["443"] = {
"mode": "http",
@ -164,11 +161,11 @@ class HaproxyConfigGenerator:
"redirect": dict(),
}
easymapping["443"]["hosts"][hostname] = dict(easymapping[port]["hosts"][hostname])
easymapping["443"]["hosts"][hostname]["letsencrypt"] = False
easymapping["443"]["hosts"][hostname]["certbot"] = False
easymapping["443"]["hosts"][hostname]["redirect_ssl"] = False
easymapping["443"]["ssl"] = True
self.letsencrypt_hosts.append(hostname) if letsencrypt and hostname not in self.letsencrypt_hosts else self.letsencrypt_hosts
self.certbot_hosts.append(
hostname) if certbot and hostname not in self.certbot_hosts else self.certbot_hosts
# handle SSL
ssl_label = self.label.create([definition, "sslcert"])

276
src/functions/__init__.py
View file

@ -1,17 +1,90 @@
from datetime import datetime
from multiprocessing import Process, Lock
import subprocess
import shlex
import time
import os
import re
import shlex
import subprocess
import time
from datetime import datetime
from multiprocessing import Process
import requests
from OpenSSL import crypto
class ContainerEnv:
@staticmethod
def read():
env_vars = {
"customerrors": True if os.getenv("HAPROXY_CUSTOMERRORS") == "true" else False,
"ssl_mode": os.getenv("EASYHAPROXY_SSL_MODE").lower() if os.getenv("EASYHAPROXY_SSL_MODE") else 'default'
}
if os.getenv("HAPROXY_PASSWORD"):
env_vars["stats"] = {
"username": os.getenv("HAPROXY_USERNAME") if os.getenv("HAPROXY_USERNAME") else "admin",
"password": os.getenv("HAPROXY_PASSWORD"),
"port": os.getenv("HAPROXY_STATS_PORT") if os.getenv("HAPROXY_STATS_PORT") else "1936",
}
env_vars["lookup_label"] = os.getenv("EASYHAPROXY_LABEL_PREFIX") if os.getenv(
"EASYHAPROXY_LABEL_PREFIX") else "easyhaproxy"
env_vars["certbot"] = {
"autoconfig": os.getenv("EASYHAPROXY_CERTBOT_AUTOCONFIG", ""),
"email": os.getenv("EASYHAPROXY_CERTBOT_EMAIL", ""),
"server": os.getenv("EASYHAPROXY_CERTBOT_SERVER", False),
"eab_kid": os.getenv("EASYHAPROXY_CERTBOT_EAB_KID", ""),
"eab_hmac_key": os.getenv("EASYHAPROXY_CERTBOT_EAB_HMAC_KEY", ""),
"retry_count": int(os.getenv("EASYHAPROXY_CERTBOT_RETRY_COUNT", 60)),
}
if env_vars["certbot"]["autoconfig"] != "" and not env_vars["certbot"]["server"] and env_vars["certbot"]["email"] != "":
if env_vars["certbot"]["autoconfig"] == "letsencrypt":
env_vars["certbot"]["server"] = "https://acme-v02.api.letsencrypt.org/directory"
if env_vars["certbot"]["autoconfig"] == "letsencrypt_test":
env_vars["certbot"]["server"] = "https://acme-staging-v02.api.letsencrypt.org/directory"
if env_vars["certbot"]["autoconfig"] == "buypass":
env_vars["certbot"]["server"] = "https://api.buypass.com/acme/directory"
if env_vars["certbot"]["autoconfig"] == "buypass_test":
env_vars["certbot"]["server"] = "https://api.test4.buypass.no/acme/directory"
if env_vars["certbot"]["autoconfig"] == "sslcom_rca":
env_vars["certbot"]["server"] = "https://acme.ssl.com/sslcom-dv-rsa"
if env_vars["certbot"]["autoconfig"] == "sslcom_ecc":
env_vars["certbot"]["server"] = "https://acme.ssl.com/sslcom-dv-ecc"
if env_vars["certbot"]["autoconfig"] == "google":
env_vars["certbot"]["server"] = "https://dv.acme-v02.api.pki.goog/directory"
if env_vars["certbot"]["autoconfig"] == "google_test":
env_vars["certbot"]["server"] = "https://dv.acme-v02.test-api.pki.goog/directory"
if env_vars["certbot"]["autoconfig"] == "zerossl":
url = "https://api.zerossl.com/acme/eab-credentials-email"
headers = {"Content-Type": "application/x-www-form-urlencoded"}
data = "email=" + env_vars["certbot"]["email"]
resp = requests.post(url, headers=headers, data=data).json()
if resp["success"]:
env_vars["certbot"]["server"] = "https://acme.zerossl.com/v2/DV90"
env_vars["certbot"]["eab_kid"] = os.environ['EASYHAPROXY_CERTBOT_EAB_KID'] = resp["eab_kid"]
env_vars["certbot"]["eab_hmac_key"] = os.environ['EASYHAPROXY_CERTBOT_EAB_HMAC_KEY'] = resp["eab_hmac_key"]
else:
os.environ["EASYHAPROXY_CERTBOT_EMAIL"] = ""
Functions.log(Functions.CERTBOT_LOG, Functions.ERROR, "Could not obtain ZeroSSL credentials " + resp["error"]["type"])
os.environ['EASYHAPROXY_CERTBOT_SERVER'] = env_vars["certbot"]["server"]
return env_vars
class Functions:
HAPROXY_LOG="HAPROXY"
EASYHAPROXY_LOG="EASYHAPROXY"
CERTBOT_LOG="CERTBOT"
INIT_LOG="INIT"
HAPROXY_LOG = "HAPROXY"
EASYHAPROXY_LOG = "EASYHAPROXY"
CERTBOT_LOG = "CERTBOT"
INIT_LOG = "INIT"
TRACE = "TRACE"
DEBUG = "DEBUG"
@ -28,7 +101,7 @@ class Functions:
level_importance = {
Functions.TRACE: 0,
Functions.DEBUG: 1,
Functions.INFO: 2,
Functions.INFO: 2,
Functions.WARN: 3,
Functions.ERROR: 4,
Functions.FATAL: 5
@ -57,7 +130,7 @@ class Functions:
if not isinstance(message, (list, tuple)):
message = [message]
for line in message:
log = "[%s] %s [%s]: %s" % (source, datetime.now().strftime("%x %X"), level, line.rstrip())
print(log)
@ -70,10 +143,10 @@ class Functions:
command = shlex.split(command)
try:
process = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
process = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True)
output = []
@ -92,18 +165,20 @@ class Functions:
Functions.log(source, Functions.WARN, process.stderr.readlines())
break
return output
return [return_code, output]
except Exception as e:
Functions.log(source, Functions.ERROR, "%s" % (e))
Functions.log(source, Functions.ERROR, "%s" % e)
return [-99, e]
class Consts:
easyhaproxy_config = "/etc/haproxy/static/config.yml"
haproxy_config = "/etc/haproxy/haproxy.cfg"
custom_config_folder = "/etc/haproxy/conf.d"
certs_letsencrypt = "/certs/letsencrypt"
certs_certbot = "/certs/certbot"
certs_haproxy = "/certs/haproxy"
class DaemonizeHAProxy:
def __init__(self, custom_config_folder = None):
self.process = None
@ -137,29 +212,28 @@ class DaemonizeHAProxy:
command = shlex.split(command)
try:
self.process = subprocess.Popen(command,
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
bufsize=-1,
universal_newlines=True)
self.process = subprocess.Popen(command,
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
bufsize=-1,
universal_newlines=True)
except Exception as e:
Functions.log(source, Functions.ERROR, "%s" % (e))
Functions.log(source, Functions.ERROR, "%s" % e)
def __start(self):
source = Functions.HAPROXY_LOG
try:
with self.process.stdout:
for line in iter(self.process.stdout.readline, b''):
for line in iter(self.process.stdout.readline, b''):
Functions.log(source, Functions.INFO, line)
returncode = self.process.wait()
Functions.log(source, Functions.DEBUG, "Return code %s" % (returncode))
return_code = self.process.wait()
Functions.log(source, Functions.DEBUG, "Return code %s" % return_code)
except Exception as e:
Functions.log(source, Functions.ERROR, "%s" % (e))
Functions.log(source, Functions.ERROR, "%s" % e)
def is_alive(self):
return self.thread.is_alive()
@ -174,7 +248,7 @@ class DaemonizeHAProxy:
def sleep(self):
if self.sleep_secs is None:
try:
try:
self.sleep_secs = int(os.getenv("EASYHAPROXY_REFRESH_CONF", "10"))
except ValueError:
self.sleep_secs = 10
@ -193,16 +267,39 @@ class DaemonizeHAProxy:
class Certbot:
def __init__(self, certs, email, test_server):
self.certs = certs
self.email = email
self.test_server = self.set_test_server(test_server)
def __init__(self, certs):
env = ContainerEnv.read()
def set_test_server(self, test_server):
if test_server.lower() == "staging":
self.certs = certs
self.email = env["certbot"]["email"]
self.acme_server = self.set_acme_server(env["certbot"]["server"])
self.eab_kid = self.set_eab_kid(env["certbot"]["eab_kid"])
self.eab_hmac_key = self.set_eab_hmac_key(env["certbot"]["eab_hmac_key"])
self.freeze_issue = {}
self.retry_count = env["certbot"]["retry_count"]
@staticmethod
def set_acme_server(acme_server):
if not acme_server:
return ""
if acme_server.lower() == "staging":
return "--staging"
elif test_server.lower().startswith("http"):
return "--server " + test_server
elif acme_server.lower().startswith("http"):
return "--server " + acme_server
else:
return ""
@staticmethod
def set_eab_kid(eab_kid):
if eab_kid != "":
return "--eab-kid \"%s\"" % eab_kid
else:
return ""
@staticmethod
def set_eab_hmac_key(eab_hmac_key):
if eab_hmac_key != "":
return "--eab-hmac-key \"%s\"" % eab_hmac_key
else:
return ""
@ -213,23 +310,25 @@ class Certbot:
try:
request_certs = []
renew_certs = []
current_time = time.time()
for host in hosts:
filename = "%s/%s.pem" % (self.certs, host)
host_arg = '-d %s' % (host)
if not os.path.exists(filename):
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "Request new certificate for %s" % (host))
cert_status = self.get_certificate_status(host)
host_arg = '-d %s' % host
if cert_status == "ok" or cert_status == "error":
continue
elif host in self.freeze_issue:
freeze_count = self.freeze_issue.pop(host, 0)
if freeze_count > 0:
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG,
"Waiting freezing period (%d) for %s due previous errors" % (freeze_count, host))
self.freeze_issue[host] = freeze_count-1
elif cert_status == "not_found" or cert_status == "expired":
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "[%s] Request new certificate for %s" % (cert_status, host))
request_certs.append(host_arg)
else:
creation_time = os.path.getctime(filename)
if (current_time - creation_time) // (24 * 3600) > 90:
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "Request expired certificate for %s" % (host))
request_certs.append(host_arg)
if (current_time - creation_time) // (24 * 3600) >= 45:
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "Renew certificate for %s" % (host))
renew_certs.append(host_arg)
elif cert_status == "expiring":
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "[%s] Renew certificate for %s" % (cert_status, host))
renew_certs.append(host_arg)
certbot_certonly = ('/usr/bin/certbot certonly {test_server}'
certbot_certonly = ('/usr/bin/certbot certonly {acme_server}'
' --standalone'
' --preferred-challenges http'
' --http-01-port 2080'
@ -238,39 +337,80 @@ class Certbot:
' --no-eff-email'
' --non-interactive'
' --max-log-backups=0'
' {certs} --email {email}'.format(certs = ' '.join(request_certs),
email = self.email,
test_server = self.test_server)
)
' {eab_kid} {eab_hmac_key}'
' {certs} --email {email}'.format(eab_kid=self.eab_kid,
eab_hmac_key=self.eab_hmac_key,
certs=' '.join(request_certs),
email=self.email,
acme_server=self.acme_server)
)
ret_reload = False
return_code_issue = 0
return_code_renew = 0
if len(request_certs) > 0:
Functions.run_bash(Functions.CERTBOT_LOG, certbot_certonly, return_result=False)
return_code_issue, output = Functions.run_bash(Functions.CERTBOT_LOG, certbot_certonly, return_result=False)
ret_reload = True
if len(renew_certs) > 0:
Functions.run_bash(Functions.CERTBOT_LOG, "/usb/bin/certbot renew", return_result=False)
return_code_renew, output = Functions.run_bash(Functions.CERTBOT_LOG, "/usr/bin/certbot renew", return_result=False)
ret_reload = True
if ret_reload:
self.find_live_certificates()
if return_code_issue != 0:
self.find_missing_certificates(request_certs)
if return_code_renew != 0:
self.find_missing_certificates(renew_certs)
return ret_reload
except Exception as e:
Functions.log(Functions.CERTBOT_LOG, Functions.ERROR, "%s" % (e))
Functions.log(Functions.CERTBOT_LOG, Functions.ERROR, "%s" % e)
return False
def merge_certificate(self, cert, key, filename):
@staticmethod
def merge_certificate(cert, key, filename):
Functions.save(filename, cert + key)
def find_live_certificates(self):
letsencrypt_certs = "/etc/letsencrypt/live/"
if not os.path.exists(letsencrypt_certs):
certbot_certs = "/etc/letsencrypt/live/"
if not os.path.exists(certbot_certs):
return
for item in os.listdir(letsencrypt_certs):
path = os.path.join(letsencrypt_certs, item)
for item in os.listdir(certbot_certs):
path = os.path.join(certbot_certs, item)
if os.path.isdir(path):
cert = Functions.load(os.path.join(path, "cert.pem"))
key = Functions.load(os.path.join(path, "privkey.pem"))
filename = "%s/%s.pem" % (self.certs, item)
self.merge_certificate(cert, key, filename)
def get_certificate_status(self, host):
current_time = time.time()
filename = "%s/%s.pem" % (self.certs, host)
if not os.path.exists(filename):
return "not_found"
try:
with open(filename, 'rb') as file:
certificate_str = file.read()
certificate = crypto.load_certificate(crypto.FILETYPE_PEM, certificate_str)
expiration_after = datetime.strptime(certificate.get_notAfter().decode()[:-1], '%Y%m%d%H%M%S').timestamp()
if current_time >= expiration_after:
return "expired"
elif (expiration_after - current_time) // (24 * 3600) <= 15:
return "expiring"
except Exception as e:
Functions.log(Functions.CERTBOT_LOG, Functions.ERROR, "Certificate %s error %s" % (host, e))
return "error"
return "ok"
def find_missing_certificates(self, hosts):
for host in hosts:
if host.startswith("-d "):
host = host[3:]
cert_status = self.get_certificate_status(host)
if cert_status != "ok":
self.freeze_issue[host] = self.retry_count
Functions.log(Functions.CERTBOT_LOG, Functions.DEBUG, "Freeze issuing ssl for %s due failure. The certificate is %s" % (host, cert_status))

31
src/main.py
View file

@ -1,20 +1,24 @@
import os
from deepdiff import DeepDiff
from functions import Functions, DaemonizeHAProxy, Certbot, Consts
from processor import ProcessorInterface
import os
from deepdiff import DeepDiff
def start():
processor_obj = ProcessorInterface.factory(os.getenv("EASYHAPROXY_DISCOVER"))
if processor_obj is None:
exit(1)
os.makedirs(Consts.certs_letsencrypt, exist_ok=True)
os.makedirs(Consts.certs_certbot, exist_ok=True)
os.makedirs(Consts.certs_haproxy, exist_ok=True)
processor_obj.save_config(Consts.haproxy_config)
processor_obj.save_certs(Consts.certs_haproxy)
letsencrypt_certs_found = processor_obj.get_letsencrypt_hosts()
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG, 'Found hosts: %s' % ", ".join(processor_obj.get_hosts())) # Needs to run after save_config
certbot_certs_found = processor_obj.get_certbot_hosts()
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG,
'Found hosts: %s' % ", ".join(processor_obj.get_hosts())) # Needs to run after save_config
Functions.log(Functions.EASYHAPROXY_LOG, Functions.TRACE, 'Object Found: %s' % (processor_obj.get_parsed_object()))
old_haproxy = None
@ -23,7 +27,7 @@ def start():
haproxy.haproxy("start")
haproxy.sleep()
certbot = Certbot(Consts.certs_letsencrypt, os.getenv("EASYHAPROXY_LETSENCRYPT_EMAIL"), os.getenv("EASYHAPROXY_LETSENCRYPT_SERVER", "").lower())
certbot = Certbot(Consts.certs_certbot)
while True:
if old_haproxy is not None:
@ -32,13 +36,15 @@ def start():
try:
old_parsed = processor_obj.get_parsed_object()
processor_obj.refresh()
if certbot.check_certificates(letsencrypt_certs_found) or DeepDiff(old_parsed, processor_obj.get_parsed_object()) != {} or not haproxy.is_alive() or DeepDiff(current_custom_config_files, haproxy.get_custom_config_files()) != {}:
if certbot.check_certificates(certbot_certs_found) or DeepDiff(old_parsed, processor_obj.get_parsed_object()) != {} or not haproxy.is_alive() or DeepDiff(current_custom_config_files, haproxy.get_custom_config_files()) != {}:
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG, 'New configuration found. Reloading...')
Functions.log(Functions.EASYHAPROXY_LOG, Functions.TRACE, 'Object Found: %s' % (processor_obj.get_parsed_object()))
Functions.log(Functions.EASYHAPROXY_LOG, Functions.TRACE,
'Object Found: %s' % (processor_obj.get_parsed_object()))
processor_obj.save_config(Consts.haproxy_config)
processor_obj.save_certs(Consts.certs_haproxy)
letsencrypt_certs_found = processor_obj.get_letsencrypt_hosts()
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG, 'Found hosts: %s' % ", ".join(processor_obj.get_hosts())) # Needs to after save_config
certbot_certs_found = processor_obj.get_certbot_hosts()
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG,
'Found hosts: %s' % ", ".join(processor_obj.get_hosts())) # Needs to after save_config
old_haproxy = haproxy
haproxy = DaemonizeHAProxy()
current_custom_config_files = haproxy.get_custom_config_files()
@ -46,14 +52,12 @@ def start():
old_haproxy.terminate()
except Exception as e:
Functions.log(Functions.EASYHAPROXY_LOG, Functions.FATAL, "Err: %s" % (e))
Functions.log(Functions.EASYHAPROXY_LOG, Functions.FATAL, "Err: %s" % e)
Functions.log(Functions.EASYHAPROXY_LOG, Functions.DEBUG, 'Heartbeat')
haproxy.sleep()
def main():
Functions.run_bash(Functions.INIT_LOG, '/usr/sbin/haproxy -v')
@ -71,5 +75,6 @@ def main():
start()
if __name__ == '__main__':
main()

154
src/processor/__init__.py
View file

@ -1,50 +1,26 @@
from easymapping import HaproxyConfigGenerator
from functions import Functions, Consts
import yaml
import sys
import os
import json
import base64
import docker
import socket
import docker
import yaml
from kubernetes import client, config
from kubernetes.client.rest import ApiException
class ContainerEnv:
@staticmethod
def read():
env_vars = {
"customerrors": True if os.getenv("HAPROXY_CUSTOMERRORS") == "true" else False,
"ssl_mode": os.getenv("EASYHAPROXY_SSL_MODE").lower() if os.getenv("EASYHAPROXY_SSL_MODE") else 'default'
}
if os.getenv("HAPROXY_PASSWORD"):
env_vars["stats"] = {
"username": os.getenv("HAPROXY_USERNAME") if os.getenv("HAPROXY_USERNAME") else "admin",
"password": os.getenv("HAPROXY_PASSWORD"),
"port": os.getenv("HAPROXY_STATS_PORT") if os.getenv("HAPROXY_STATS_PORT") else "1936",
}
env_vars["lookup_label"] = os.getenv("EASYHAPROXY_LABEL_PREFIX") if os.getenv("EASYHAPROXY_LABEL_PREFIX") else "easyhaproxy"
if (os.getenv("EASYHAPROXY_LETSENCRYPT_EMAIL")):
env_vars["letsencrypt"] = {
"email": os.getenv("EASYHAPROXY_LETSENCRYPT_EMAIL"),
"server": os.getenv("EASYHAPROXY_LETSENCRYPT_SERVER", "false").lower() in ["true", "1", "yes"]
}
env_vars["logLevel"] = {
"easyhaproxy": os.getenv("EASYHAPROXY_LOG_LEVEL") if os.getenv("EASYHAPROXY_LOG_LEVEL") else Functions.DEBUG,
"haproxy": os.getenv("HAPROXY_LOG_LEVEL") if os.getenv("HAPROXY_LOG_LEVEL") else Functions.INFO,
"certbot": os.getenv("CERTBOT_LOG_LEVEL") if os.getenv("CERTBOT_LOG_LEVEL") else Functions.DEBUG,
}
return env_vars
from easymapping import HaproxyConfigGenerator
from functions import Functions, Consts, ContainerEnv
class ProcessorInterface:
static_file = Consts.easyhaproxy_config
def __init__(self, filename = None):
def __init__(self, filename=None):
self.certbot_hosts = None
self.parsed_object = None
self.cfg = None
self.hosts = None
self.cfg = None
self.certbot_hosts = None
self.hosts = None
self.filename = filename
self.label = ContainerEnv.read()['lookup_label']
self.refresh()
@ -60,11 +36,12 @@ class ProcessorInterface:
elif mode == "kubernetes":
return Kubernetes()
else:
Functions.log("EASYHAPROXY", Functions.FATAL, "Expected mode to be 'static', 'docker', 'swarm' or 'kubernetes'. I got '%s'" % (mode))
Functions.log("EASYHAPROXY", Functions.FATAL,
"Expected mode to be 'static', 'docker', 'swarm' or 'kubernetes'. I got '%s'" % mode)
return None
def refresh(self):
self.letsencrypt_hosts = None
self.certbot_hosts = None
self.parsed_object = None
self.cfg = None
self.hosts = None
@ -72,14 +49,14 @@ class ProcessorInterface:
self.parse()
def inspect_network(self):
#Abstract
# Abstract
pass
def parse(self):
self.cfg = HaproxyConfigGenerator(ContainerEnv.read())
def get_letsencrypt_hosts(self):
return self.letsencrypt_hosts
def get_certbot_hosts(self):
return self.certbot_hosts
def get_hosts(self):
return self.hosts
@ -87,7 +64,7 @@ class ProcessorInterface:
def get_parsed_object(self):
return self.parsed_object
def get_certs(self, key = None):
def get_certs(self, key=None):
if key is None:
return self.cfg.certs
else:
@ -95,7 +72,7 @@ class ProcessorInterface:
def get_haproxy_conf(self):
conf = self.cfg.generate(self.parsed_object)
self.letsencrypt_hosts = self.cfg.letsencrypt_hosts
self.certbot_hosts = self.cfg.certbot_hosts
self.hosts = self.cfg.serving_hosts
return conf
@ -108,20 +85,27 @@ class ProcessorInterface:
class Static(ProcessorInterface):
def __init__(self, filename=None):
self.parsed_object = None
self.static_content = None
self.static_content = None
self.cfg = None
super().__init__(filename)
def inspect_network(self):
self.parsed_object = {}
self.static_content = None
def get_parsed_object(self):
return self.static_content["easymapping"] if "easymapping" in self.static_content else []
def get_hosts(self):
hosts = []
for object in self.get_parsed_object():
if "hosts" not in object:
for obj in self.get_parsed_object():
if "hosts" not in obj:
continue
for host in object["hosts"].keys():
hosts.append("%s:%s" % (host, object["port"]))
for host in obj["hosts"].keys():
hosts.append("%s:%s" % (host, obj["port"]))
return hosts
def parse(self):
@ -130,18 +114,21 @@ class Static(ProcessorInterface):
class Docker(ProcessorInterface):
def __init__(self, filename = None):
def __init__(self, filename=None):
self.parsed_object = None
self.client = docker.from_env()
super().__init__()
def inspect_network(self):
try:
ha_proxy_network_name = next(iter(self.client.containers.get(socket.gethostname()).attrs["NetworkSettings"]["Networks"]))
ha_proxy_network_name = next(
iter(self.client.containers.get(socket.gethostname()).attrs["NetworkSettings"]["Networks"]))
except:
# HAProxy is not running in a container, get first container network
if len(self.client.containers.list()) == 0:
return
ha_proxy_network_name = next(iter(self.client.containers.get(self.client.containers.list()[0].name).attrs["NetworkSettings"]["Networks"]))
ha_proxy_network_name = next(iter(
self.client.containers.get(self.client.containers.list()[0].name).attrs["NetworkSettings"]["Networks"]))
ha_proxy_network = self.client.networks.get(ha_proxy_network_name)
@ -157,7 +144,8 @@ class Docker(ProcessorInterface):
class Swarm(ProcessorInterface):
def __init__(self, filename = None):
def __init__(self, filename=None):
self.parsed_object = None
self.client = docker.from_env()
super().__init__()
@ -201,7 +189,8 @@ class Swarm(ProcessorInterface):
class Kubernetes(ProcessorInterface):
def __init__(self, filename = None):
def __init__(self, filename=None):
self.parsed_object = None
config.load_incluster_config()
config.verify_ssl = False
self.api_instance = client.CoreV1Api()
@ -215,7 +204,7 @@ class Kubernetes(ProcessorInterface):
return annotations[key]
def inspect_network(self):
ret = self.v1.list_ingress_for_all_namespaces(watch=False)
self.parsed_object = {}
@ -227,18 +216,14 @@ class Kubernetes(ProcessorInterface):
ssl_hosts = []
letsencrypt = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.letsencrypt")
certbot = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.certbot")
redirect_ssl = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.redirect_ssl")
redirect = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.redirect")
mode = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.mode")
listen_port = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.listen_port")
if listen_port is None:
listen_port = 80
listen_port = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.listen_port", 80)
data = {}
data["creation_timestamp"] = ingress.metadata.creation_timestamp.strftime("%x %X")
data["resource_version"] = ingress.metadata.resource_version
data["namespace"] = ingress.metadata.namespace
data = {"creation_timestamp": ingress.metadata.creation_timestamp.strftime("%x %X"),
"resource_version": ingress.metadata.resource_version, "namespace": ingress.metadata.namespace}
ingress_name = ingress.metadata.namespace
@ -252,35 +237,37 @@ class Kubernetes(ProcessorInterface):
if tls.secret_name not in self.cert_cache or self.cert_cache[tls.secret_name] != secret.data:
self.cert_cache[tls.secret_name] = secret.data
Functions.save(
"{0}/{1}.pem".format(Consts.certs_haproxy, tls.secret_name),
base64.b64decode(secret.data["tls.crt"]).decode('ascii') + "\n" + base64.b64decode(secret.data["tls.key"]).decode('ascii')
"{0}/{1}.pem".format(Consts.certs_haproxy, tls.secret_name),
base64.b64decode(secret.data["tls.crt"]).decode('ascii') + "\n" + base64.b64decode(
secret.data["tls.key"]).decode('ascii')
)
ssl_hosts.extend(tls.hosts)
except Exception as e:
Functions.log("EASYHAPROXY", Functions.WARN, "Ingress %s - Get secret failed: '%s'" % (ingress_name, e))
Functions.log("EASYHAPROXY", Functions.WARN,
"Ingress %s - Get secret failed: '%s'" % (ingress_name, e))
Functions.log("EASYHAPROXY", Functions.TRACE, "Ingress %s - SSL Hosts found '%s'" % (ingress_name, ssl_hosts))
Functions.log("EASYHAPROXY", Functions.TRACE,
"Ingress %s - SSL Hosts found '%s'" % (ingress_name, ssl_hosts))
for rule in ingress.spec.rules:
rule_data = {}
port_number = rule.http.paths[0].backend.service.port.number
definition = "easyhaproxy.%s_%s" % (rule.host.replace(".", "-"), port_number)
rule_data["%s.host" % (definition)] = rule.host
rule_data["%s.port" % (definition)] = listen_port
rule_data["%s.localport" % (definition)] = port_number
rule_data["%s.host" % definition] = rule.host
rule_data["%s.port" % definition] = listen_port
rule_data["%s.localport" % definition] = port_number
if rule.host in ssl_hosts:
rule_data["%s.clone_to_ssl" % (definition)] = 'true'
rule_data["%s.clone_to_ssl" % definition] = 'true'
if redirect_ssl is not None:
rule_data["%s.redirect_ssl" % (definition)] = redirect_ssl
if letsencrypt is not None:
rule_data["%s.letsencrypt" % (definition)] = letsencrypt
rule_data["%s.redirect_ssl" % definition] = redirect_ssl
if certbot is not None:
rule_data["%s.certbot" % definition] = certbot
if redirect is not None:
rule_data["%s.redirect" % (definition)] = redirect
rule_data["%s.redirect" % definition] = redirect
if mode is not None:
rule_data["%s.mode" % (definition)] = mode
rule_data["%s.balance" % (definition)] = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.balance", "roundrobin")
rule_data["%s.mode" % definition] = mode
rule_data["%s.balance" % definition] = self._check_annotation(ingress.metadata.annotations, "easyhaproxy.balance", "roundrobin")
service_name = rule.http.paths[0].backend.service.name
try:
@ -288,13 +275,10 @@ class Kubernetes(ProcessorInterface):
cluster_ip = api_response.spec.cluster_ip
except ApiException as e:
cluster_ip = None
Functions.log("EASYHAPROXY", Functions.WARN, "Ingress %s - Service %s - Failed: '%s'" % (ingress_name, service_name, e))
Functions.log("EASYHAPROXY", Functions.WARN,
"Ingress %s - Service %s - Failed: '%s'" % (ingress_name, service_name, e))
if cluster_ip is not None:
if cluster_ip not in self.parsed_object.keys():
self.parsed_object[cluster_ip] = data
self.parsed_object[cluster_ip].update(rule_data)

3
src/requirements.txt
View file

@ -4,4 +4,5 @@ jinja2
pytest
docker
kubernetes
deepdiff
deepdiff
pyopenssl

10
src/templates/frontend-mode-http.j2
View file

@ -5,16 +5,16 @@
{%- for k in o["hosts"] %}
{% set host = k.replace(".", "_") + "_{0}".format(o["port"]) %}
{% set letsencrypt = o["hosts"][k]["letsencrypt"] %}
{% set certbot = o["hosts"][k]["certbot"] %}
acl is_rule_{{ host }}_1 hdr(host) -i {{ k }}
acl is_rule_{{ host }}_2 hdr(host) -i {{ k }}:{{ o["port"] }}
{% if letsencrypt %}
acl is_letsencrypt_{{ host }} path_beg /.well-known/acme-challenge/
use_backend letsencrypt_backend if is_letsencrypt_{{ host }} is_rule_{{ host }}_1 OR is_letsencrypt_{{ host }} is_rule_{{ host }}_2
{% if certbot %}
acl is_certbot_{{ host }} path_beg /.well-known/acme-challenge/
use_backend certbot_backend if is_certbot_{{ host }} is_rule_{{ host }}_1 OR is_certbot_{{ host }} is_rule_{{ host }}_2
{% endif %}
{% if o["hosts"][k]["redirect_ssl"] %}
http-request redirect scheme https code 301 if {% if letsencrypt %}!is_letsencrypt_{{ host }} {% endif %}is_rule_{{ host }}_1 OR {% if letsencrypt %}!is_letsencrypt_{{ host }} {% endif %}is_rule_{{ host }}_2
http-request redirect scheme https code 301 if {% if certbot %}!is_certbot_{{ host }} {% endif %}is_rule_{{ host }}_1 OR {% if certbot %}!is_certbot_{{ host }} {% endif %}is_rule_{{ host }}_2
{% else %}
use_backend srv_{{ host }} if is_rule_{{ host }}_1 OR is_rule_{{ host }}_2
{% endif %}

2
src/templates/haproxy.cfg.j2
View file

@ -88,7 +88,7 @@ backend srv_{{ host }}
{% endfor %}
{% endfor %}
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

4
src/tests/context.py
View file

@ -1,5 +1,5 @@
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import easymapping
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

6
src/tests/expected/docker.txt
View file

@ -84,8 +84,8 @@ frontend http_in_90
acl is_rule_host2_local_90_1 hdr(host) -i host2.local
acl is_rule_host2_local_90_2 hdr(host) -i host2.local:90
acl is_letsencrypt_host2_local_90 path_beg /.well-known/acme-challenge/
use_backend letsencrypt_backend if is_letsencrypt_host2_local_90 is_rule_host2_local_90_1 OR is_letsencrypt_host2_local_90 is_rule_host2_local_90_2
acl is_certbot_host2_local_90 path_beg /.well-known/acme-challenge/
use_backend certbot_backend if is_certbot_host2_local_90 is_rule_host2_local_90_1 OR is_certbot_host2_local_90 is_rule_host2_local_90_2
use_backend srv_host2_local_90 if is_rule_host2_local_90_1 OR is_rule_host2_local_90_2
backend srv_host2_local_90
@ -96,6 +96,6 @@ backend srv_host2_local_90
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server srv-0 test_processor_docker:9000 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/no-services.txt
View file

@ -23,6 +23,6 @@ defaults
timeout server 10m
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

8
src/tests/expected/services-letsencrypt.txt
View file

@ -49,9 +49,9 @@ frontend http_in_80
acl is_rule_test_example_org_80_1 hdr(host) -i test.example.org
acl is_rule_test_example_org_80_2 hdr(host) -i test.example.org:80
acl is_letsencrypt_test_example_org_80 path_beg /.well-known/acme-challenge/
use_backend letsencrypt_backend if is_letsencrypt_test_example_org_80 is_rule_test_example_org_80_1 OR is_letsencrypt_test_example_org_80 is_rule_test_example_org_80_2
http-request redirect scheme https code 301 if !is_letsencrypt_test_example_org_80 is_rule_test_example_org_80_1 OR !is_letsencrypt_test_example_org_80 is_rule_test_example_org_80_2
acl is_certbot_test_example_org_80 path_beg /.well-known/acme-challenge/
use_backend certbot_backend if is_certbot_test_example_org_80 is_rule_test_example_org_80_1 OR is_certbot_test_example_org_80 is_rule_test_example_org_80_2
http-request redirect scheme https code 301 if !is_certbot_test_example_org_80 is_rule_test_example_org_80_1 OR !is_certbot_test_example_org_80 is_rule_test_example_org_80_2
acl is_rule_test2_example_org_80_1 hdr(host) -i test2.example.org
acl is_rule_test2_example_org_80_2 hdr(host) -i test2.example.org:80
@ -90,6 +90,6 @@ backend srv_test_example_org_443
server srv-0 f5c645a0dfc6:80 check weight 1 verify none
server srv-1 b63438410b6a:80 check weight 1 verify none
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/services-multi-containers.txt
View file

@ -40,6 +40,6 @@ backend srv_www_helloworld_com_19901
server srv-0 test_nginx.2.t5r94mjlced7m3t5orfjbowmm:80 check weight 1
server srv-1 test_nginx.1.p552hqxkdx88narjrp5kouwb2:80 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/services-multiple-hosts.txt
View file

@ -73,6 +73,6 @@ backend srv_hello_com_19901
server srv-0 3e63154954b0:80 check weight 1
server srv-1 eb294c110eb1:80 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/services-redirect-ssl.txt
View file

@ -75,6 +75,6 @@ backend srv_host1_local_443
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server srv-0 5b69bc7fea1b:8080 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/services-tcp.txt
View file

@ -37,6 +37,6 @@ backend srv_agent_quantum_local_31339
tcp-check connect ssl
server srv-0 test_agent:9001 check weight 1 verify none
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

6
src/tests/expected/services.txt
View file

@ -47,8 +47,8 @@ frontend http_in_31337
acl is_rule_node-exporter_quantum_example_org_31337_1 hdr(host) -i node-exporter.quantum.example.org
acl is_rule_node-exporter_quantum_example_org_31337_2 hdr(host) -i node-exporter.quantum.example.org:31337
acl is_letsencrypt_node-exporter_quantum_example_org_31337 path_beg /.well-known/acme-challenge/
use_backend letsencrypt_backend if is_letsencrypt_node-exporter_quantum_example_org_31337 is_rule_node-exporter_quantum_example_org_31337_1 OR is_letsencrypt_node-exporter_quantum_example_org_31337 is_rule_node-exporter_quantum_example_org_31337_2
acl is_certbot_node-exporter_quantum_example_org_31337 path_beg /.well-known/acme-challenge/
use_backend certbot_backend if is_certbot_node-exporter_quantum_example_org_31337 is_rule_node-exporter_quantum_example_org_31337_1 OR is_certbot_node-exporter_quantum_example_org_31337 is_rule_node-exporter_quantum_example_org_31337_2
use_backend srv_node-exporter_quantum_example_org_31337 if is_rule_node-exporter_quantum_example_org_31337_1 OR is_rule_node-exporter_quantum_example_org_31337_2
backend srv_cadvisor_quantum_example_org_31337
@ -119,6 +119,6 @@ backend srv_www_somehost_com_br_80
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server srv-0 some-service:80 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/ssl-loose.txt
View file

@ -33,6 +33,6 @@ backend srv_stats
mode http
server Local 127.0.0.1:1936
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

2
src/tests/expected/ssl-strict.txt
View file

@ -18,6 +18,6 @@ defaults
timeout server 10m
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

6
src/tests/expected/static.txt
View file

@ -50,8 +50,8 @@ frontend http_in_80
acl is_rule_host1_com_br_80_1 hdr(host) -i host1.com.br
acl is_rule_host1_com_br_80_2 hdr(host) -i host1.com.br:80
acl is_letsencrypt_host1_com_br_80 path_beg /.well-known/acme-challenge/
use_backend letsencrypt_backend if is_letsencrypt_host1_com_br_80 is_rule_host1_com_br_80_1 OR is_letsencrypt_host1_com_br_80 is_rule_host1_com_br_80_2
acl is_certbot_host1_com_br_80 path_beg /.well-known/acme-challenge/
use_backend certbot_backend if is_certbot_host1_com_br_80 is_rule_host1_com_br_80_1 OR is_certbot_host1_com_br_80 is_rule_host1_com_br_80_2
use_backend srv_host1_com_br_80 if is_rule_host1_com_br_80_1 OR is_rule_host1_com_br_80_2
acl is_rule_host2_com_br_80_1 hdr(host) -i host2.com.br
@ -105,6 +105,6 @@ backend srv_host3_com_br_8080
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server srv-0 domain:8181 check weight 1
backend letsencrypt_backend
backend certbot_backend
mode http
server certbot 127.0.0.1:2080

7
src/tests/fixtures/run_bash.sh vendored Executable file
View file

@ -0,0 +1,7 @@
#!/bin/bash
# Just return the exit code
echo "Processing run_bash.sh"
if [ -n "$1" ]; then
exit "$1"
fi

2
src/tests/fixtures/services vendored
View file

@ -1,6 +1,6 @@
{"portainer-agent_agent": {"com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"portainer-agent"},
"my-stack_agent": {"easyhaproxy.agent.host":"agent.quantum.example.org","easyhaproxy.agent.localport":"9001","easyhaproxy.agent.mode":"tcp","easyhaproxy.agent.port":"31339","com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"my-stack_cadvisor": {"easyhaproxy.cadvisor.host":"cadvisor.quantum.example.org","easyhaproxy.cadvisor.localport":"8080","easyhaproxy.cadvisor.port":"31337","com.docker.stack.image":"gcr.io/google-containers/cadvisor:v0.34.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"my-stack_node-exporter": {"easyhaproxy.exp.host":"node-exporter.quantum.example.org","easyhaproxy.exp.localport":"9100","easyhaproxy.exp.port":"31337","com.docker.stack.image":"stefanprodan/swarmprom-node-exporter:v0.16.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring","easyhaproxy.exp.letsencrypt":"true"},
"my-stack_node-exporter": {"easyhaproxy.exp.host":"node-exporter.quantum.example.org","easyhaproxy.exp.localport":"9100","easyhaproxy.exp.port":"31337","com.docker.stack.image":"stefanprodan/swarmprom-node-exporter:v0.16.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring","easyhaproxy.exp.certbot":"true"},
"my-stack_reverse-proxy": {"com.docker.stack.image":"quay.io/pngmbh/easy-haproxy:tcp-mode","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"some-service": {"easyhaproxy.http.port":"80","easyhaproxy.http.host":"www.somehost.com.br","easyhaproxy.http.localport":"80","easyhaproxy.http.redirect":"{\"somehost.com.br\":\"https://www.somehost.com.br\",\"somehost.com\":\"https://www.somehost.com.br\",\"www.somehost.com\":\"https://www.somehost.com.br\",\"byjg.ca\":\"https://www.somehost.com.br\",\"www.byjg.ca\":\"https://www.somehost.com.br\"}","easyhaproxy.https.port":"443","easyhaproxy.https.host":"www.somehost.com.br","easyhaproxy.https.localport":"80","easyhaproxy.https.redirect":"{\"somehost.com.br\":\"https://www.somehost.com.br\",\"somehost.com\":\"https://www.somehost.com.br\",\"www.somehost.com\":\"https://www.somehost.com.br\",\"byjg.ca\":\"https://www.somehost.com.br\",\"www.byjg.ca\":\"https://www.somehost.com.br\"}","easyhaproxy.https.sslcert":"U29tZSBQRU0gQ2VydGlmaWNhdGU="}}

2
src/tests/fixtures/services-changed-label vendored
View file

@ -1,6 +1,6 @@
{"portainer-agent_agent": {"com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"portainer-agent"},
"my-stack_agent": {"haproxy.agent.host":"agent.quantum.example.org","haproxy.agent.localport":"9001","haproxy.agent.mode":"tcp","haproxy.agent.port":"31339","com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"my-stack_cadvisor": {"haproxy.cadvisor.host":"cadvisor.quantum.example.org","haproxy.cadvisor.localport":"8080","haproxy.cadvisor.port":"31337","com.docker.stack.image":"gcr.io/google-containers/cadvisor:v0.34.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"my-stack_node-exporter": {"haproxy.exp.host":"node-exporter.quantum.example.org","haproxy.exp.localport":"9100","haproxy.exp.port":"31337","com.docker.stack.image":"stefanprodan/swarmprom-node-exporter:v0.16.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring","haproxy.exp.letsencrypt":"yes"},
"my-stack_node-exporter": {"haproxy.exp.host":"node-exporter.quantum.example.org","haproxy.exp.localport":"9100","haproxy.exp.port":"31337","com.docker.stack.image":"stefanprodan/swarmprom-node-exporter:v0.16.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring","haproxy.exp.certbot":"yes"},
"my-stack_reverse-proxy": {"com.docker.stack.image":"quay.io/pngmbh/easy-haproxy:tcp-mode","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"},
"some-service": {"haproxy.http.port":"80","haproxy.http.host":"www.somehost.com.br","haproxy.http.localport":"80","haproxy.http.redirect":"{\"somehost.com.br\":\"https://www.somehost.com.br\",\"somehost.com\":\"https://www.somehost.com.br\",\"www.somehost.com\":\"https://www.somehost.com.br\",\"byjg.ca\":\"https://www.somehost.com.br\",\"www.byjg.ca\":\"https://www.somehost.com.br\"}","haproxy.https.port":"443","haproxy.https.host":"www.somehost.com.br","haproxy.https.localport":"80","haproxy.https.redirect":"{\"somehost.com.br\":\"https://www.somehost.com.br\",\"somehost.com\":\"https://www.somehost.com.br\",\"www.somehost.com\":\"https://www.somehost.com.br\",\"byjg.ca\":\"https://www.somehost.com.br\",\"www.byjg.ca\":\"https://www.somehost.com.br\"}","haproxy.https.sslcert":"U29tZSBQRU0gQ2VydGlmaWNhdGU="}}

4
src/tests/fixtures/services-letsencrypt vendored
View file

@ -1,4 +1,4 @@
{"f5c645a0dfc6": {"com.docker.compose.config-hash":"b95ebc27d0e61caa418cdfa632e05a656da9bbc3ea0d4603651971015f10a1f0","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"test.example.org","easyhaproxy.http.letsencrypt":"true","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"80","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}","easyhaproxy.http.redirect_ssl":"true"},
{"f5c645a0dfc6": {"com.docker.compose.config-hash":"b95ebc27d0e61caa418cdfa632e05a656da9bbc3ea0d4603651971015f10a1f0","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"test.example.org","easyhaproxy.http.certbot":"true","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"80","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}","easyhaproxy.http.redirect_ssl":"true"},
"bbd4d1854155": {"com.docker.compose.config-hash":"3dc790bf2bea944359c75a40c45655bd868f1d85beb599d1ca797e8ea2c95ee4","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:0fd95b1512c207048ab3fcc74032354f38143fbb8235ac2a47da903c98a58205","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"haproxy","com.docker.compose.version":"2.8.0"},
"b63438410b6a": {"com.docker.compose.config-hash":"b95ebc27d0e61caa418cdfa632e05a656da9bbc3ea0d4603651971015f10a1f0","com.docker.compose.container-number":"2","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"test.example.org","easyhaproxy.http.letsencrypt":"true","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"80","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}","easyhaproxy.http.redirect_ssl":"true"},
"b63438410b6a": {"com.docker.compose.config-hash":"b95ebc27d0e61caa418cdfa632e05a656da9bbc3ea0d4603651971015f10a1f0","com.docker.compose.container-number":"2","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:bea3509d6fdc8d7f9ec95563a5a226dc977ee74fb3e980e0de70e892c2d38dde","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"nginx","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"test.example.org","easyhaproxy.http.certbot":"true","easyhaproxy.http.localport":"80","easyhaproxy.http.port":"80","easyhaproxy.http.redirect":"{\"google.helloworld.com\": \"www.google.com\"}","easyhaproxy.http.redirect_ssl":"true"},
"83d57d592e26": {"com.docker.compose.config-hash":"8c5871144f1e8a3aeca037207c02f011ab2c6e6c311a3773602b63541762dab5","com.docker.compose.container-number":"1","com.docker.compose.depends_on":"","com.docker.compose.image":"sha256:c4232396c715f3d568816c666e6d9b4a68ef6c36f6243b4007c4ee1d8335fd65","com.docker.compose.oneoff":"False","com.docker.compose.project":"docker","com.docker.compose.project.config_files":"/workspace/docker-easy-haproxy/examples/docker/docker-compose-test.yml","com.docker.compose.project.working_dir":"/workspace/docker-easy-haproxy/examples/docker","com.docker.compose.service":"static","com.docker.compose.version":"2.8.0","easyhaproxy.http.host":"test2.example.org","easyhaproxy.http.localport":"8080","easyhaproxy.http.port":"80","io.buildah.version":"1.21.0"}}

2
src/tests/fixtures/static.yml vendored
View file

@ -11,7 +11,7 @@ easymapping:
host1.com.br:
containers:
- container:5000
letsencrypt: true
certbot: true
host2.com.br:
containers:
- other:3000

259
src/tests/test_containerenv.py
View file

@ -1,147 +1,192 @@
import pytest
import os
from functions import Functions
from processor import ContainerEnv
from functions import Functions, ContainerEnv
def test_container_env_empty():
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
# os.environ['CERTBOT_LOG_LEVEL'] = 'warn'
def test_container_env_customerrors():
os.environ['HAPROXY_CUSTOMERRORS'] = 'true'
try:
assert {
"customerrors": True,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": True,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
finally:
os.environ['HAPROXY_CUSTOMERRORS'] = ''
def test_container_env_sslmode():
os.environ['EASYHAPROXY_SSL_MODE'] = 'STRICT'
try:
assert {
"customerrors": False,
"ssl_mode": "strict",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "strict",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
finally:
os.environ['EASYHAPROXY_SSL_MODE'] = ''
def test_container_env_stats():
os.environ['HAPROXY_USERNAME'] = 'abc'
os.environ['HAPROXY_STATS_PORT'] = '2101'
try:
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
finally:
os.environ['HAPROXY_USERNAME'] = ''
os.environ['HAPROXY_STATS_PORT'] = ''
def test_container_env_stats_password():
os.environ['HAPROXY_PASSWORD'] = 'xyz'
try:
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"stats": {
"username": "admin",
"password": "xyz",
"port": "1936"
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"stats": {
"username": "admin",
"password": "xyz",
"port": "1936"
},
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
},
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
finally:
os.environ['HAPROXY_PASSWORD'] = ''
def test_container_env_stats_password():
def test_container_env_stats_password_2():
os.environ['HAPROXY_USERNAME'] = 'abc'
os.environ['HAPROXY_STATS_PORT'] = '2101'
os.environ['HAPROXY_PASSWORD'] = 'xyz'
try:
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"stats": {
"username": "abc",
"password": "xyz",
"port": "2101"
},
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"stats": {
"username": "abc",
"password": "xyz",
"port": "2101"
},
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {"autoconfig": "",
"eab_hmac_key": "",
"eab_kid": "",
"email": "",
"server": False,
"retry_count": 60}
} == ContainerEnv.read()
finally:
os.environ['HAPROXY_USERNAME'] = ''
os.environ['HAPROXY_STATS_PORT'] = ''
os.environ['HAPROXY_PASSWORD'] = ''
def test_container_env_stats_password():
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = 'acme@example.org'
def test_container_env_certbot_email():
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = 'acme@example.org'
try:
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"letsencrypt": {
"email": "acme@example.org",
"server": False
},
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.DEBUG,
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {
"autoconfig": "",
'eab_hmac_key': "",
'eab_kid': "",
"email": "acme@example.org",
"server": False,
"retry_count": 60
}
} == ContainerEnv.read()
finally:
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = ''
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = ''
def test_container_env_letsencrypt():
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = 'acme@example.org'
os.environ['EASYHAPROXY_LETSENCRYPT_SERVER'] = 'true'
def test_container_env_certbot_full():
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = 'acme@example.org'
os.environ['EASYHAPROXY_CERTBOT_SERVER'] = 'schema://url/a'
os.environ['EASYHAPROXY_CERTBOT_EAB_KID'] = 'eab_kid'
os.environ['EASYHAPROXY_CERTBOT_EAB_HMAC_KEY'] = 'eab_hmac_key'
os.environ['EASYHAPROXY_CERTBOT_RETRY_COUNT'] = "10"
try:
assert {
"customerrors": False,
@ -156,6 +201,14 @@ def test_container_env_letsencrypt():
"haproxy": Functions.INFO,
"certbot": Functions.DEBUG,
},
"certbot": {
"autoconfig": "",
"email": "acme@example.org",
"server": "schema://url/a",
'eab_hmac_key': 'eab_hmac_key',
'eab_kid': 'eab_kid',
'retry_count': 10
}
} == ContainerEnv.read()
finally:
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = ''
@ -166,16 +219,22 @@ def test_container_log_level():
os.environ['HAPROXY_LOG_LEVEL'] = Functions.FATAL
try:
assert {
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.ERROR,
"haproxy": Functions.FATAL,
"certbot": Functions.TRACE,
},
} == ContainerEnv.read()
"customerrors": False,
"ssl_mode": "default",
"lookup_label": "easyhaproxy",
"logLevel": {
"easyhaproxy": Functions.ERROR,
"haproxy": Functions.FATAL,
"certbot": Functions.TRACE,
},
"certbot": {
"autoconfig": "",
'eab_hmac_key': "",
'eab_kid': "",
"email": "acme@example.org",
"server": False,
"retry_count": 60
}
} == ContainerEnv.read()
finally:
os.environ['CERTBOT_LOG_LEVEL'] = ''
os.environ['EASYHAPROXY_LOG_LEVEL'] = ''
os.environ['HAPROXY_LOG_LEVEL'] = ''
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = ''

18
src/tests/test_docker.py
View file

@ -1,10 +1,11 @@
import pytest
import os
import time
import docker
import pytest
from functions import Functions
from processor import ProcessorInterface
from processor import Docker
def _get_hydrated_object(parsed_objects, lookup_key):
@ -17,7 +18,6 @@ def _get_hydrated_object(parsed_objects, lookup_key):
def _get_ip_host(parsed_objects, lookup_key):
hydrated_object = {}
for key in parsed_objects:
for keys in parsed_objects[key]:
if lookup_key in keys:
@ -46,7 +46,7 @@ def test_processor_docker():
"easyhaproxy.http2.port": "90",
"easyhaproxy.http2.localport": "9000",
"easyhaproxy.http2.host": "host2.local",
"easyhaproxy.http2.letsencrypt": "true",
"easyhaproxy.http2.certbot": "true",
})
container2 = client.containers.run("byjg/static-httpserver",
name="test2_processor_docker",
@ -62,10 +62,10 @@ def test_processor_docker():
try:
time.sleep(1)
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = 'docker@example.org'
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = 'docker@example.org'
static = ProcessorInterface.factory("docker")
assert static.get_letsencrypt_hosts() is None
assert static.get_certbot_hosts() is None
assert {
'easyhaproxy.http.host': 'host1.local',
@ -74,7 +74,7 @@ def test_processor_docker():
'easyhaproxy.http2.host': 'host2.local',
'easyhaproxy.http2.localport': '9000',
'easyhaproxy.http2.port': '90',
'easyhaproxy.http2.letsencrypt': 'true',
'easyhaproxy.http2.certbot': 'true',
} == _get_hydrated_object(static.get_parsed_object(), "easyhaproxy.http")
assert {
'easyhaproxy.ssl.host': 'hostssl.local',
@ -90,7 +90,7 @@ def test_processor_docker():
assert haproxy_cfg == Functions.load(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./expected/docker.txt")).replace("test_processor_docker", _get_ip_host(
static.get_parsed_object(), "easyhaproxy.http")).replace("test2_processor_docker", _get_ip_host(static.get_parsed_object(), "easyhaproxy.ssl"))
assert static.get_letsencrypt_hosts() == ['host2.local']
assert static.get_certbot_hosts() == ['host2.local']
assert static.get_hosts() == [
'hostssl.local:443',
'host1.local:80',
@ -100,7 +100,7 @@ def test_processor_docker():
'hostssl.local.pem': 'Some PEM Certificate'
}
finally:
os.environ['EASYHAPROXY_LETSENCRYPT_EMAIL'] = ''
os.environ['EASYHAPROXY_CERTBOT_EMAIL'] = ''
container.stop()
container2.stop()

73
src/tests/test_functions.py
View file

@ -1,11 +1,11 @@
import json
import pytest
import os
import re
import random
import re
import string
from functions import Functions
def test_functions_check_local_level():
assert Functions.skip_log('CERTBOT', Functions.INFO) == False
assert Functions.skip_log('HAPOROXY', Functions.INFO) == False
@ -23,6 +23,7 @@ def test_functions_check_local_level():
assert Functions.skip_log('EASYHAPROXY', Functions.INFO) == True
os.environ['EASYHAPROXY_LOG_LEVEL'] = ''
def test_function_load_and_save():
filename = '/tmp/x.txt'
try:
@ -34,6 +35,7 @@ def test_function_load_and_save():
finally:
os.unlink(filename)
def test_functions_check_log_sanity():
print()
Functions.log(Functions.EASYHAPROXY_LOG, Functions.INFO, "Test 1")
@ -51,51 +53,104 @@ def test_functions_check_log_sanity():
assert re.match("\[EASYHAPROXY\] .* \[INFO\]: Test 3", Functions.debug_log[1])
os.environ['EASYHAPROXY_LOG_LEVEL'] = 'warn'
Functions.log(Functions.EASYHAPROXY_LOG, Functions.INFO, "Test 4") # Should not log to debug
Functions.log(Functions.EASYHAPROXY_LOG, Functions.INFO, "Test 4") # Should not log to debug
assert len(Functions.debug_log) == 2
finally:
os.environ['EASYHAPROXY_LOG_LEVEL'] = ''
Functions.debug_log = None
def test_functions_run_bash_log_output():
print()
Functions.debug_log = []
try:
result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 1'", log_output=True, return_result=False)
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 1'", log_output=True,
return_result=False)
assert return_code == 0
assert result == []
assert len(Functions.debug_log) == 1
assert re.match("\[EASYHAPROXY\] .* \[INFO\]: test run 1", Functions.debug_log[0])
finally:
Functions.debug_log = None
def test_functions_run_bash_no_log_output():
print()
Functions.debug_log = []
try:
result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 2'", log_output=False, return_result=False)
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 2'", log_output=False,
return_result=False)
assert return_code == 0
assert result == []
assert len(Functions.debug_log) == 0
finally:
Functions.debug_log = None
def test_functions_run_bash_return():
print()
Functions.debug_log = []
try:
result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 3'", log_output=False, return_result=True)
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 3'", log_output=False,
return_result=True)
assert return_code == 0
assert len(Functions.debug_log) == 0
assert "".join(result) == 'test run 3'
finally:
Functions.debug_log = None
def test_functions_run_bash_log_and_return_output():
print()
Functions.debug_log = []
try:
result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 4'", log_output=True, return_result=True)
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "echo 'test run 4'", log_output=True, return_result=True)
assert return_code == 0
assert "".join(result) == 'test run 4'
assert len(Functions.debug_log) == 1
assert re.match("\[EASYHAPROXY\] .* \[INFO\]: test run 4", Functions.debug_log[0])
finally:
Functions.debug_log = None
Functions.debug_log = None
def test_functions_run_bash_ok():
print()
Functions.debug_log = []
try:
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "%s/fixtures/run_bash.sh" % os.path.dirname(__file__), log_output=True,
return_result=False)
assert return_code == 0
assert result == []
assert len(Functions.debug_log) == 1
assert re.match("\[EASYHAPROXY\] .* \[INFO\]: Processing run_bash.sh", Functions.debug_log[0])
finally:
Functions.debug_log = None
def test_functions_run_bash_fail():
print()
Functions.debug_log = []
try:
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "%s/fixtures/run_bash.sh 15" % os.path.dirname(__file__), log_output=True,
return_result=False)
assert return_code == 15
assert result == []
assert len(Functions.debug_log) == 1
assert re.match("\[EASYHAPROXY\] .* \[INFO\]: Processing run_bash.sh", Functions.debug_log[0])
finally:
Functions.debug_log = None
def test_functions_run_command_not_found():
print()
Functions.debug_log = []
try:
return_code, result = Functions.run_bash(Functions.EASYHAPROXY_LOG, "no_command_here", log_output=True,
return_result=False)
assert return_code == -99
assert str(result) == "[Errno 2] No such file or directory: 'no_command_here'"
assert len(Functions.debug_log) == 1
assert re.match("\[EASYHAPROXY\] .* \[ERROR\]: \[Errno 2\] No such file or directory: 'no_command_here'", Functions.debug_log[0])
finally:
Functions.debug_log = None

11
src/tests/test_labels.py
View file

@ -1,16 +1,17 @@
from .context import easymapping
import json
import pytest
from easymapping import DockerLabelHandler
def test_label_generation():
label = easymapping.DockerLabelHandler("foo")
label = DockerLabelHandler("foo")
assert label.create("bar") == "foo.bar"
assert label.create(["bar", "foobar"]) == "foo.bar.foobar"
def test_label_data():
label = easymapping.DockerLabelHandler("base")
label = DockerLabelHandler("base")
label.set_data(json.loads('{"base.definitions":"h2"}'))
label_name = label.create("definitions")
@ -20,7 +21,7 @@ def test_label_data():
def test_label_complex_key():
label = easymapping.DockerLabelHandler("till")
label = DockerLabelHandler("till")
data = dict()
data["till.definitions"] = "h2"

157
src/tests/test_parser.py
View file

@ -1,12 +1,14 @@
import easymapping
import pytest
import os
import yaml
import json
import os
import yaml
import easymapping
CERTS_FOLDER = "/tmp/certs"
CERT_FILE = "/tmp/certs/haproxy/www.somehost.com.br.pem"
CERTBOT_EMAIL = "some@email.com"
CERTS_FOLDER="/tmp/certs"
CERT_FILE="/tmp/certs/haproxy/www.somehost.com.br.pem"
LETSENCRYPT_EMAIL="some@email.com"
def load_fixture(file):
path = os.path.dirname(os.path.realpath(__file__))
@ -33,15 +35,16 @@ def test_parser_doesnt_crash():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/no-services.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_finds_services():
line_list = load_fixture("services")
result = {
"customerrors": False,
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
"certbot": {
"email": CERTBOT_EMAIL
},
"stats": {
"port": 0
@ -56,9 +59,10 @@ def test_parser_finds_services():
with open(path + "/expected/services.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert {"www.somehost.com.br.pem":"Some PEM Certificate"} == cfg.certs
assert {"www.somehost.com.br.pem": "Some PEM Certificate"} == cfg.certs
assert ['node-exporter.quantum.example.org'] == cfg.certbot_hosts
assert ['node-exporter.quantum.example.org'] == cfg.letsencrypt_hosts
def test_parser_finds_services_changed_label():
line_list = load_fixture("services-changed-label")
@ -66,8 +70,8 @@ def test_parser_finds_services_changed_label():
result = {
"customerrors": False,
"lookup_label": "haproxy",
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
"certbot": {
"email": CERTBOT_EMAIL
},
"stats": {
"port": 0
@ -85,17 +89,18 @@ def test_parser_finds_services_changed_label():
with open(path + "/expected/services.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert {"www.somehost.com.br.pem":"Some PEM Certificate"} == cfg.certs
assert {"www.somehost.com.br.pem": "Some PEM Certificate"} == cfg.certs
assert ['node-exporter.quantum.example.org'] == cfg.certbot_hosts
assert ['node-exporter.quantum.example.org'] == cfg.letsencrypt_hosts
def test_parser_finds_services_raw():
line_list = load_fixture("services")
result = {
"customerrors": False,
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
"certbot": {
"email": CERTBOT_EMAIL
},
"stats": {
"port": 0
@ -118,12 +123,12 @@ def test_parser_finds_services_raw():
"containers": [
"my-stack_agent:9001"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
}
},
"redirect":{
"redirect": {
}
},
{
@ -136,7 +141,7 @@ def test_parser_finds_services_raw():
"containers": [
"my-stack_cadvisor:8080"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
},
"node-exporter.quantum.example.org":{
@ -144,12 +149,12 @@ def test_parser_finds_services_raw():
"containers": [
"my-stack_node-exporter:9100"
],
"letsencrypt": True,
"certbot": True,
"redirect_ssl": False
}
},
"redirect":{
"redirect": {
},
},
{
@ -162,7 +167,7 @@ def test_parser_finds_services_raw():
"containers": [
"my-stack_node-exporter:9100"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
},
"www.somehost.com.br":{
@ -170,16 +175,16 @@ def test_parser_finds_services_raw():
"containers": [
"some-service:80"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
}
},
"redirect":{
"somehost.com.br":"https://www.somehost.com.br",
"somehost.com":"https://www.somehost.com.br",
"www.somehost.com":"https://www.somehost.com.br",
"byjg.ca":"https://www.somehost.com.br",
"www.byjg.ca":"https://www.somehost.com.br"
"redirect": {
"somehost.com.br": "https://www.somehost.com.br",
"somehost.com": "https://www.somehost.com.br",
"www.somehost.com": "https://www.somehost.com.br",
"byjg.ca": "https://www.somehost.com.br",
"www.byjg.ca": "https://www.somehost.com.br"
},
"ssl": True
},
@ -193,16 +198,16 @@ def test_parser_finds_services_raw():
"containers": [
"some-service:80"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
}
},
"redirect":{
"somehost.com.br":"https://www.somehost.com.br",
"somehost.com":"https://www.somehost.com.br",
"www.somehost.com":"https://www.somehost.com.br",
"byjg.ca":"https://www.somehost.com.br",
"www.byjg.ca":"https://www.somehost.com.br"
"redirect": {
"somehost.com.br": "https://www.somehost.com.br",
"somehost.com": "https://www.somehost.com.br",
"www.somehost.com": "https://www.somehost.com.br",
"byjg.ca": "https://www.somehost.com.br",
"www.byjg.ca": "https://www.somehost.com.br"
},
}
]
@ -210,8 +215,7 @@ def test_parser_finds_services_raw():
processed = list(cfg.parse(line_list))
assert parsed_object == processed
assert ['node-exporter.quantum.example.org'] == cfg.letsencrypt_hosts
assert ['node-exporter.quantum.example.org'] == cfg.certbot_hosts
def test_parser_static():
@ -225,7 +229,8 @@ def test_parser_static():
with open(path + "/expected/static.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_static_raw():
path = os.path.dirname(os.path.realpath(__file__))
@ -247,7 +252,7 @@ def test_parser_static_raw():
"containers": [
"container:5000"
],
"letsencrypt": True
"certbot": True
},
"host2.com.br": {
"containers": [
@ -286,7 +291,6 @@ def test_parser_static_raw():
assert expected == parsed
def test_parser_tcp():
line_list = load_fixture("services-tcp")
@ -305,7 +309,8 @@ def test_parser_tcp():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/services-tcp.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_multi_containers():
line_list = load_fixture("services-multi-containers")
@ -324,7 +329,7 @@ def test_parser_multi_containers():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/services-multi-containers.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_multiple_hosts():
@ -346,7 +351,7 @@ def test_parser_multiple_hosts():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/services-multiple-hosts.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_redirect_ssl():
@ -367,7 +372,7 @@ def test_parser_redirect_ssl():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/services-redirect-ssl.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_ssl_strict():
@ -388,7 +393,8 @@ def test_parser_ssl_strict():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/ssl-strict.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_ssl_loose():
line_list = load_fixture("no-services")
@ -405,7 +411,8 @@ def test_parser_ssl_loose():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/ssl-loose.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
def test_parser_ssl_letsencrypt():
line_list = load_fixture("services-letsencrypt")
@ -415,8 +422,8 @@ def test_parser_ssl_letsencrypt():
"stats": {
"password": "password"
},
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
"certbot": {
"email": CERTBOT_EMAIL
}
}
@ -427,7 +434,7 @@ def test_parser_ssl_letsencrypt():
path = os.path.dirname(os.path.realpath(__file__))
with open(path + "/expected/services-letsencrypt.txt", 'r') as expected_file:
assert expected_file.read() == haproxy_config
assert ["test.example.org"] == cfg.letsencrypt_hosts
assert ["test.example.org"] == cfg.certbot_hosts
def test_parser_finds_services_clone_to_ssl_raw():
@ -435,8 +442,8 @@ def test_parser_finds_services_clone_to_ssl_raw():
result = {
"customerrors": False,
"letsencrypt": {
"email": LETSENCRYPT_EMAIL
"certbot": {
"email": CERTBOT_EMAIL
},
"stats": {
"port": 0
@ -457,7 +464,7 @@ def test_parser_finds_services_clone_to_ssl_raw():
"containers":[
"10.152.183.215:8080"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
},
"valida.me":{
@ -465,7 +472,7 @@ def test_parser_finds_services_clone_to_ssl_raw():
"containers":[
"10.152.183.62:8080"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
},
"www.valida.me":{
@ -473,14 +480,14 @@ def test_parser_finds_services_clone_to_ssl_raw():
"containers":[
"10.152.183.62:8080"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
}
},
"mode":"http",
"port":"80",
"redirect":{
"mode": "http",
"port": "80",
"redirect": {
}
},
{
@ -491,14 +498,14 @@ def test_parser_finds_services_clone_to_ssl_raw():
"containers":[
"10.152.183.215:8080"
],
"letsencrypt": False,
"certbot": False,
"redirect_ssl": False
}
},
"mode":"http",
"port":"443",
"redirect":{
"mode": "http",
"port": "443",
"redirect": {
},
"ssl": True
}
@ -506,12 +513,10 @@ def test_parser_finds_services_clone_to_ssl_raw():
processed = list(cfg.parse(line_list))
assert parsed_object == processed
assert [] == cfg.letsencrypt_hosts
assert [] == cfg.certbot_hosts
#test_parser_finds_services_raw()
#test_parser_tcp()
#test_parser_multiple_hosts()
#test_parser_ssl_letsencrypt()
#test_parser_finds_services()
# test_parser_finds_services_raw()
# test_parser_tcp()
# test_parser_multiple_hosts()
# test_parser_ssl_certbot()
# test_parser_finds_services()

55
src/tests/test_static.py
View file

@ -1,8 +1,8 @@
import pytest
import os
from functions import Functions
from processor import ProcessorInterface
from processor import Static
def test_processor_static():
ProcessorInterface.static_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "./fixtures/static.yml")
@ -10,44 +10,44 @@ def test_processor_static():
parsed_object = [
{
"hosts":{
"host1.com.br":{
"containers":[
"container:5000"
"hosts": {
"host1.com.br": {
"containers": [
"container:5000"
],
"letsencrypt": True
"certbot": True
},
"host2.com.br":{
"containers":[
"other:3000"
"host2.com.br": {
"containers": [
"other:3000"
]
}
},
"port":80,
"redirect":{
"www.host1.com.br":"http://host1.com.br"
"port": 80,
"redirect": {
"www.host1.com.br": "http://host1.com.br"
}
},
{
"hosts":{
"host1.com.br":{
"containers":[
"container:80"
"hosts": {
"host1.com.br": {
"containers": [
"container:80"
]
}
},
"port":443,
"port": 443,
"ssl": True
},
{
"hosts":{
"host3.com.br":{
"containers":[
"domain:8181"
"hosts": {
"host3.com.br": {
"containers": [
"domain:8181"
]
}
},
"port":8080
"port": 8080
}
]
hosts = [
@ -57,17 +57,18 @@ def test_processor_static():
'host3.com.br:8080'
]
assert static.get_letsencrypt_hosts() is None
assert static.get_certbot_hosts() is None
assert static.get_parsed_object() == parsed_object
assert static.get_hosts() == hosts
haproxy_cfg = static.get_haproxy_conf()
assert haproxy_cfg == Functions.load(os.path.join(os.path.dirname(os.path.realpath(__file__)), "./expected/static.txt"))
assert haproxy_cfg == Functions.load(
os.path.join(os.path.dirname(os.path.realpath(__file__)), "./expected/static.txt"))
# @todo: Static doesnt populate this fields
assert static.get_letsencrypt_hosts() == []
assert static.get_certbot_hosts() == []
assert static.get_parsed_object() == parsed_object
assert static.get_hosts() == hosts
# test_processor_static()
# test_processor_static()