commit
82391f0939
22 changed files with 460 additions and 126 deletions
|
|
@ -8,8 +8,6 @@ COPY requirements.txt /scripts
|
||||||
RUN pip3 install --upgrade pip \
|
RUN pip3 install --upgrade pip \
|
||||||
&& pip install -r requirements.txt
|
&& pip install -r requirements.txt
|
||||||
|
|
||||||
COPY swarm.* /scripts/
|
|
||||||
COPY static.* /scripts/
|
|
||||||
COPY templates /scripts/templates/
|
COPY templates /scripts/templates/
|
||||||
COPY easymapping /scripts/easymapping/
|
COPY easymapping /scripts/easymapping/
|
||||||
|
|
||||||
|
|
|
||||||
7
Makefile
Normal file
7
Makefile
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
.PHONY: build
|
||||||
|
build:
|
||||||
|
docker build -t byjg/easy-haproxy -t byjg/easy-haproxy:local .
|
||||||
|
|
||||||
|
.PHONY: test
|
||||||
|
test:
|
||||||
|
pytest tests/
|
||||||
61
README.md
61
README.md
|
|
@ -1,24 +1,24 @@
|
||||||
# Easy HAProxy
|
# Easy HAProxy
|
||||||
|
|
||||||
This Docker image will create dynamically the `haproxy.cfg` based on the labels defined in docker containers or from
|
This Docker image will create dynamically the `haproxy.cfg` based on the labels defined in docker containers or from
|
||||||
a simple Yaml instead docker
|
a simple Yaml instead docker
|
||||||
|
|
||||||
# Features
|
# Features
|
||||||
|
|
||||||
- Enable or disable Stats on port 1936 with custom password
|
- Enable or disable Stats on port 1936 with custom password
|
||||||
- Discover and setup haproxy from Docker Tag
|
- Discover and setup haproxy from Docker Tag
|
||||||
- Discover and setup haproxy redirect from Docker Tag
|
- Discover and setup haproxy redirect from Docker Tag
|
||||||
- Setup HAProxy CFG from a Yaml file.
|
- Setup HAProxy CFG from a Yaml file.
|
||||||
|
|
||||||
|
|
||||||
# Basic Usage
|
# Basic Usage
|
||||||
|
|
||||||
The Easy HAProxy will create the `haproxy.cfg` automatically based on the containers or from a YAML provided.
|
The Easy HAProxy will create the `haproxy.cfg` automatically based on the containers or from a YAML provided.
|
||||||
|
|
||||||
The basic command line to run is:
|
The basic command line to run is:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run -d \
|
docker run -d \
|
||||||
--name easy-haproxy-container \
|
--name easy-haproxy-container \
|
||||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||||
-e DISCOVER="swarm|docker|static" \
|
-e DISCOVER="swarm|docker|static" \
|
||||||
|
|
@ -29,7 +29,7 @@ docker run -d \
|
||||||
|
|
||||||
The mapping to `/var/run/docker.sock` is necessary to discover the docker containers and get the labels;
|
The mapping to `/var/run/docker.sock` is necessary to discover the docker containers and get the labels;
|
||||||
|
|
||||||
The environment variables will setup the HAProxy.
|
The environment variables will setup the HAProxy.
|
||||||
|
|
||||||
{:.table}
|
{:.table}
|
||||||
| Environment Variable | Description |
|
| Environment Variable | Description |
|
||||||
|
|
@ -49,9 +49,9 @@ The environment variable `DISCOVER` will define where is located your containers
|
||||||
|
|
||||||
# DISCOVER: docker
|
# DISCOVER: docker
|
||||||
|
|
||||||
This method will use a regular docker installation to discover the containers and configure the HAProxy.
|
This method will use a regular docker installation to discover the containers and configure the HAProxy.
|
||||||
|
|
||||||
The only requirement is that containers and easy-haproxy must be in the same docker network.
|
The only requirement is that containers and easy-haproxy must be in the same docker network.
|
||||||
|
|
||||||
The discover will occur every minute.
|
The discover will occur every minute.
|
||||||
|
|
||||||
|
|
@ -67,12 +67,12 @@ docker run --network easyhaproxy myimage
|
||||||
|
|
||||||
# DISCOVER: swarm
|
# DISCOVER: swarm
|
||||||
|
|
||||||
This method requires a functional Docker Swarm Cluster. The system will search for the labels in all containers on all
|
This method requires a functional Docker Swarm Cluster. The system will search for the labels in all containers on all
|
||||||
swarm nodes.
|
swarm nodes.
|
||||||
|
|
||||||
The discover will occur every minute.
|
The discover will occur every minute.
|
||||||
|
|
||||||
Important: easyhaproxy needs to be in the same network of the containers or otherwise will not access.
|
Important: easyhaproxy needs to be in the same network of the containers or otherwise will not access.
|
||||||
|
|
||||||
## Tags to be attached in the Docker Container
|
## Tags to be attached in the Docker Container
|
||||||
|
|
||||||
|
|
@ -80,12 +80,13 @@ Important: easyhaproxy needs to be in the same network of the containers or othe
|
||||||
| Tag | Description |
|
| Tag | Description |
|
||||||
|---------------------------------------------|---------------------------------------------------------------------------------------------------------|
|
|---------------------------------------------|---------------------------------------------------------------------------------------------------------|
|
||||||
| com.byjg.easyhaproxy.definitions | A Comma delimited list with the definitions. Each name requires the definition of the parameters below. |
|
| com.byjg.easyhaproxy.definitions | A Comma delimited list with the definitions. Each name requires the definition of the parameters below. |
|
||||||
|
| com.byjg.easyhaproxy.mode.[definition] | (Optional) Is this http or tcp mode in HAProxy. (Defaults to http) |
|
||||||
| com.byjg.easyhaproxy.port.[definition] | (Optional) What is the port that the HAProxy will listen to. (Defaults to 80) |
|
| com.byjg.easyhaproxy.port.[definition] | (Optional) What is the port that the HAProxy will listen to. (Defaults to 80) |
|
||||||
| com.byjg.easyhaproxy.localport.[definition] | (Optional) What is the port that the container is listening. (Defaults to 80) |
|
| com.byjg.easyhaproxy.localport.[definition] | (Optional) What is the port that the container is listening. (Defaults to 80) |
|
||||||
| com.byjg.easyhaproxy.host.[definition] | What is the host that the HAProxy will listen to. |
|
| com.byjg.easyhaproxy.host.[definition] | What is the host that the HAProxy will listen to. |
|
||||||
| com.byjg.easyhaproxy.redirect.[definition] | (Optional) Host redirects from connections in the port defined above. |
|
| com.byjg.easyhaproxy.redirect.[definition] | (Optional) Host redirects from connections in the port defined above. |
|
||||||
| com.byjg.easyhaproxy.sslcert.[definition] | (Optional) Cert PEM Base64 encoded. |
|
| com.byjg.easyhaproxy.sslcert.[definition] | (Optional) Cert PEM Base64 encoded. |
|
||||||
|
| com.byjg.easyhaproxy.health-check.[definition] | (Optional) `ssl`, enable health check via SSL in `mode tcp` (Defaults to "empty") |
|
||||||
|
|
||||||
Note: if you are deploying a stack set labels at the `deploy` level:
|
Note: if you are deploying a stack set labels at the `deploy` level:
|
||||||
|
|
||||||
|
|
@ -126,6 +127,22 @@ docker run \
|
||||||
some/myimage
|
some/myimage
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### TLS passthrough
|
||||||
|
|
||||||
|
Used to pass on SSL-termination to a backend:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run \
|
||||||
|
-l com.byjg.easyhaproxy.defintions=tcp-service \
|
||||||
|
-l com.byjg.easyhaproxy.mode.tcp-service=tcp \
|
||||||
|
-l com.byjg.easyhaproxy.health-check.tcp-service=ssl \
|
||||||
|
-l com.byjg.easyhaproxy.port.tcp-service=443
|
||||||
|
.... \
|
||||||
|
some/tcp-service
|
||||||
|
```
|
||||||
|
|
||||||
|
- enable health-check via SSL on the backend with the optional `health-check` label
|
||||||
|
|
||||||
### Redirect Example:
|
### Redirect Example:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -149,12 +166,12 @@ customerrors: true # Optional (default false)
|
||||||
|
|
||||||
easymapping:
|
easymapping:
|
||||||
- port: 80
|
- port: 80
|
||||||
hosts:
|
hosts:
|
||||||
host1.com.br: container:5000
|
host1.com.br: container:5000
|
||||||
host2.com.br: other:3000
|
host2.com.br: other:3000
|
||||||
redirect:
|
redirect:
|
||||||
www.host1.com.br: http://host1.com.br
|
www.host1.com.br: http://host1.com.br
|
||||||
|
|
||||||
- port: 443
|
- port: 443
|
||||||
ssl_cert: BASE64_PEM_CERTIFICATE
|
ssl_cert: BASE64_PEM_CERTIFICATE
|
||||||
hosts:
|
hosts:
|
||||||
|
|
@ -173,10 +190,10 @@ docker run -v /my/config.yml:/etc/haproxy/easyconfig.yml .... byjg/easyhaproxy
|
||||||
|
|
||||||
# Mapping custom .cfg files
|
# Mapping custom .cfg files
|
||||||
|
|
||||||
Map a folder containing valid HAProxy `.cfg` files to `/etc/haproxy/conf.d`. It will be concatenated to your HAProxy CFG.
|
Map a folder containing valid HAProxy `.cfg` files to `/etc/haproxy/conf.d`. It will be concatenated to your HAProxy CFG.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run \
|
docker run \
|
||||||
/* other parameters */
|
/* other parameters */
|
||||||
-v /your/local/conf.d:/etc/haproxy/conf.d \
|
-v /your/local/conf.d:/etc/haproxy/conf.d \
|
||||||
-d byjg/easy-haproxy
|
-d byjg/easy-haproxy
|
||||||
|
|
@ -185,9 +202,9 @@ docker run \
|
||||||
|
|
||||||
# Handling SSL
|
# Handling SSL
|
||||||
|
|
||||||
You can attach a valid SSL certificate to the request.
|
You can attach a valid SSL certificate to the request.
|
||||||
|
|
||||||
1. First Create a single PEM file including CA.
|
1. First Create a single PEM file including CA.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cat example.com.crt example.com.key > single.pem
|
cat example.com.crt example.com.key > single.pem
|
||||||
|
|
@ -215,8 +232,8 @@ cat single.pem | base64 -w0
|
||||||
|
|
||||||
# Setting Custom Errors
|
# Setting Custom Errors
|
||||||
|
|
||||||
If enabled, map the volume : `/etc/haproxy/errors-custom/` to your container and put a file named `ERROR_NUMBER.http`
|
If enabled, map the volume : `/etc/haproxy/errors-custom/` to your container and put a file named `ERROR_NUMBER.http`
|
||||||
where ERROR_NUMBER is the http error code (e.g. 503.http)
|
where ERROR_NUMBER is the http error code (e.g. 503.http)
|
||||||
|
|
||||||
# Build
|
# Build
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,14 +18,18 @@ else
|
||||||
if [[ "$DISCOVER" == "docker" ]]; then
|
if [[ "$DISCOVER" == "docker" ]]; then
|
||||||
CONTAINERS=$(docker ps -q)
|
CONTAINERS=$(docker ps -q)
|
||||||
LABEL_PATH=".Config.Labels"
|
LABEL_PATH=".Config.Labels"
|
||||||
|
|
||||||
|
for container in ${CONTAINERS}; do
|
||||||
|
docker inspect --format "{{ json $LABEL_PATH }}" ${container} | xargs -I % echo ${container}=% >> ${CONTROL_FILE}
|
||||||
|
done
|
||||||
else
|
else
|
||||||
CONTAINERS=$(docker node ps $(docker node ls -q) --format "{{ .Name }}" --filter desired-state=running | cut -d. -f1 | sort | uniq)
|
CONTAINERS=$(docker node ps $(docker node ls -q) --format "{{ .Name }}" --filter desired-state=running | cut -d. -f1 | sort | uniq)
|
||||||
LABEL_PATH=".Spec.Labels"
|
LABEL_PATH=".Spec.Labels"
|
||||||
fi
|
|
||||||
|
|
||||||
for container in ${CONTAINERS}; do
|
for container in ${CONTAINERS}; do
|
||||||
docker inspect --format "{{ json $LABEL_PATH }}" ${container} | xargs -I % echo ${container}=% >> ${CONTROL_FILE}
|
docker service inspect --format "{{ json $LABEL_PATH }}" ${container} | xargs -I % echo ${container}=% >> ${CONTROL_FILE}
|
||||||
done
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
if cmp -s ${CONTROL_FILE} ${CONTROL_FILE}.old ; then
|
if cmp -s ${CONTROL_FILE} ${CONTROL_FILE}.old ; then
|
||||||
RELOAD="false"
|
RELOAD="false"
|
||||||
|
|
|
||||||
24
assets/scripts/swarm.py
Normal file
24
assets/scripts/swarm.py
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
import os
|
||||||
|
from easymapping import HaproxyConfigGenerator
|
||||||
|
|
||||||
|
# path = os.path.dirname(os.path.realpath(__file__))
|
||||||
|
with open("/tmp/.docker_data", 'r') as content_file:
|
||||||
|
lineList = content_file.readlines()
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"customerrors": True if os.getenv("HAPROXY_CUSTOMERRORS") == "true" else False
|
||||||
|
}
|
||||||
|
|
||||||
|
if os.getenv("HAPROXY_PASSWORD"):
|
||||||
|
result["stats"] = {
|
||||||
|
"username": os.getenv("HAPROXY_USERNAME") if os.getenv("HAPROXY_USERNAME") else "admin",
|
||||||
|
"password": os.getenv("HAPROXY_PASSWORD"),
|
||||||
|
"port": os.getenv("HAPROXY_STATS_PORT") if os.getenv("HAPROXY_STATS_PORT") else "1936",
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg = HaproxyConfigGenerator(result)
|
||||||
|
print(cfg.generate(lineList))
|
||||||
|
|
||||||
|
# print(jsonStr)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1,12 +1,144 @@
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
from jinja2 import Environment, FileSystemLoader
|
from jinja2 import Environment, FileSystemLoader
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
|
||||||
|
|
||||||
|
class DockerLabelHandler:
|
||||||
|
def __init__(self, label):
|
||||||
|
self.__label_base = label
|
||||||
|
|
||||||
|
|
||||||
|
def create(self, key):
|
||||||
|
if isinstance(key, str):
|
||||||
|
return "{}.{}".format(self.__label_base, key)
|
||||||
|
|
||||||
|
return "{}.{}".format(self.__label_base, ".".join(key))
|
||||||
|
|
||||||
|
|
||||||
|
def get(self, label, default_value = ""):
|
||||||
|
if self.has_label(label):
|
||||||
|
return self.__data[label]
|
||||||
|
return default_value
|
||||||
|
|
||||||
|
|
||||||
|
def set_data(self, data):
|
||||||
|
self.__data = data
|
||||||
|
|
||||||
|
|
||||||
|
def has_label(self, label):
|
||||||
|
if label in self.__data:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
class HaproxyConfigGenerator:
|
class HaproxyConfigGenerator:
|
||||||
def __init__(self, mapping):
|
def __init__(self, mapping):
|
||||||
self.mapping = mapping
|
self.mapping = mapping
|
||||||
|
self.label = DockerLabelHandler("com.byjg.easyhaproxy")
|
||||||
|
|
||||||
|
|
||||||
|
def generate(self, lineList = []):
|
||||||
|
# static?
|
||||||
|
if len(lineList) > 0:
|
||||||
|
self.mapping["easymapping"] = self.__parse(lineList)
|
||||||
|
|
||||||
|
# still 'None' -> default to [] for jinja2
|
||||||
|
if self.mapping["easymapping"] is None:
|
||||||
|
self.mapping["easymapping"] = []
|
||||||
|
|
||||||
def generate(self):
|
|
||||||
file_loader = FileSystemLoader('templates')
|
file_loader = FileSystemLoader('templates')
|
||||||
env = Environment(loader=file_loader)
|
env = Environment(loader=file_loader)
|
||||||
|
env.trim_blocks = True
|
||||||
|
env.lstrip_blocks = True
|
||||||
|
env.rstrip_blocks = True
|
||||||
template = env.get_template('haproxy.cfg.j2')
|
template = env.get_template('haproxy.cfg.j2')
|
||||||
return template.render(data=self.mapping)
|
return template.render(data=self.mapping)
|
||||||
|
|
||||||
|
|
||||||
|
def __parse(self, lineList):
|
||||||
|
easymapping = dict()
|
||||||
|
|
||||||
|
for line in lineList:
|
||||||
|
line = line.strip()
|
||||||
|
i = line.find("=")
|
||||||
|
container = line[:i]
|
||||||
|
jsonStr = line[i+1:]
|
||||||
|
d = json.loads(jsonStr)
|
||||||
|
|
||||||
|
if self.label.create("definitions") not in d.keys():
|
||||||
|
continue
|
||||||
|
|
||||||
|
self.label.set_data(d)
|
||||||
|
|
||||||
|
definitions = d[self.label.create("definitions")].split(",")
|
||||||
|
for definition in definitions:
|
||||||
|
mode = self.label.get(
|
||||||
|
self.label.create(["mode", definition]),
|
||||||
|
"http"
|
||||||
|
)
|
||||||
|
|
||||||
|
# TODO: we can ignore "host" in TCP, but it would break the template
|
||||||
|
host_label = self.label.create(["host", definition])
|
||||||
|
if not self.label.has_label(host_label):
|
||||||
|
continue
|
||||||
|
|
||||||
|
port = self.label.get(
|
||||||
|
self.label.create(["port", definition]),
|
||||||
|
"80"
|
||||||
|
)
|
||||||
|
|
||||||
|
if self.label.create(["sslcert", definition]) in d:
|
||||||
|
hash = hashlib.md5(
|
||||||
|
d[self.label.create(["sslcert", definition])].encode('utf-8')
|
||||||
|
).hexdigest()
|
||||||
|
else:
|
||||||
|
hash = ""
|
||||||
|
|
||||||
|
key = port+hash
|
||||||
|
|
||||||
|
if key not in easymapping:
|
||||||
|
easymapping[key] = {
|
||||||
|
"mode": mode,
|
||||||
|
"health-check": "",
|
||||||
|
"port": port,
|
||||||
|
"hosts": dict(),
|
||||||
|
"redirect": dict(),
|
||||||
|
}
|
||||||
|
|
||||||
|
# TODO: this could use `EXPOSE` from `Dockerfile`?
|
||||||
|
ct_port = self.label.get(
|
||||||
|
self.label.create(["localport", definition]),
|
||||||
|
"80"
|
||||||
|
)
|
||||||
|
|
||||||
|
easymapping[key]["health-check"] = self.label.get(
|
||||||
|
self.label.create(["health-check", definition]),
|
||||||
|
""
|
||||||
|
)
|
||||||
|
|
||||||
|
easymapping[key]["hosts"][d[host_label]] = "{}:{}".format(container, ct_port)
|
||||||
|
|
||||||
|
# handle SSL
|
||||||
|
ssl_label = self.label.create(["sslcert", definition])
|
||||||
|
if self.label.has_label(ssl_label):
|
||||||
|
filename = "/etc/haproxy/certs/{}.{}.pem".format(
|
||||||
|
d[ssl_label], str(time.time())
|
||||||
|
)
|
||||||
|
easymapping[key]["ssl_cert"] = filename
|
||||||
|
with open(filename, 'wb') as file:
|
||||||
|
file.write(
|
||||||
|
base64.b64decode(d[ssl_label])
|
||||||
|
)
|
||||||
|
|
||||||
|
# handle redirects
|
||||||
|
redirect = self.label.get(
|
||||||
|
self.label.create(["redirect", definition])
|
||||||
|
)
|
||||||
|
if len(redirect) > 0:
|
||||||
|
for r in redirect.split(","):
|
||||||
|
r_parts = r.split("--")
|
||||||
|
easymapping[key]["redirect"][r_parts[0]] = r_parts[1]
|
||||||
|
|
||||||
|
return easymapping.values()
|
||||||
|
|
|
||||||
2
pytest.ini
Normal file
2
pytest.ini
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
[pytest]
|
||||||
|
addopts = -v -p no:warnings
|
||||||
20
setup.py
Normal file
20
setup.py
Normal file
|
|
@ -0,0 +1,20 @@
|
||||||
|
from setuptools import setup, find_packages
|
||||||
|
|
||||||
|
|
||||||
|
with open('README.md') as f:
|
||||||
|
readme = f.read()
|
||||||
|
|
||||||
|
with open('LICENSE') as f:
|
||||||
|
license = f.read()
|
||||||
|
|
||||||
|
setup(
|
||||||
|
name='easymapping',
|
||||||
|
version='0.1.0',
|
||||||
|
description='HAProxy label based routing',
|
||||||
|
long_description=readme,
|
||||||
|
author='',
|
||||||
|
author_email='',
|
||||||
|
url='',
|
||||||
|
license=license,
|
||||||
|
packages=find_packages(exclude=('tests', 'docs'))
|
||||||
|
)
|
||||||
74
swarm.py
74
swarm.py
|
|
@ -1,74 +0,0 @@
|
||||||
import os
|
|
||||||
import json
|
|
||||||
import time
|
|
||||||
import base64
|
|
||||||
import hashlib
|
|
||||||
from easymapping import HaproxyConfigGenerator
|
|
||||||
|
|
||||||
# path = os.path.dirname(os.path.realpath(__file__))
|
|
||||||
with open("/tmp/.docker_data", 'r') as content_file:
|
|
||||||
lineList = content_file.readlines()
|
|
||||||
|
|
||||||
result = {
|
|
||||||
"easymapping": [],
|
|
||||||
"customerrors": True if os.getenv("HAPROXY_CUSTOMERRORS") == "true" else False
|
|
||||||
}
|
|
||||||
easymapping = dict()
|
|
||||||
|
|
||||||
if os.getenv("HAPROXY_PASSWORD"):
|
|
||||||
result["stats"] = {
|
|
||||||
"username": os.getenv("HAPROXY_USERNAME") if os.getenv("HAPROXY_USERNAME") else "admin",
|
|
||||||
"password": os.getenv("HAPROXY_PASSWORD"),
|
|
||||||
"port": os.getenv("HAPROXY_STATS_PORT") if os.getenv("HAPROXY_STATS_PORT") else "1936",
|
|
||||||
}
|
|
||||||
|
|
||||||
for line in lineList:
|
|
||||||
line = line.strip()
|
|
||||||
i = line.find("=")
|
|
||||||
container = line[:i]
|
|
||||||
jsonStr = line[i+1:]
|
|
||||||
d = json.loads(jsonStr)
|
|
||||||
|
|
||||||
if "com.byjg.easyhaproxy.definitions" in d.keys():
|
|
||||||
definitions = d["com.byjg.easyhaproxy.definitions"].split(",")
|
|
||||||
|
|
||||||
for definition in definitions:
|
|
||||||
if "com.byjg.easyhaproxy.host." + definition not in d:
|
|
||||||
continue
|
|
||||||
|
|
||||||
port = d["com.byjg.easyhaproxy.port." + definition] if "com.byjg.easyhaproxy.port." + definition in d else "80"
|
|
||||||
hash = hashlib.md5(d["com.byjg.easyhaproxy.sslcert." + definition].encode('utf-8')).hexdigest() if "com.byjg.easyhaproxy.sslcert." + definition in d else ""
|
|
||||||
|
|
||||||
key = port+hash
|
|
||||||
|
|
||||||
if key not in easymapping:
|
|
||||||
easymapping[key] = {
|
|
||||||
"port": port,
|
|
||||||
"hosts": dict(),
|
|
||||||
"redirect": dict(),
|
|
||||||
# "ssl_cert": ""
|
|
||||||
}
|
|
||||||
|
|
||||||
easymapping[key]["hosts"][d["com.byjg.easyhaproxy.host." + definition]] = container + ":" + (d["com.byjg.easyhaproxy.localport." + definition] if "com.byjg.easyhaproxy.localport." + definition in d else "80")
|
|
||||||
|
|
||||||
if "com.byjg.easyhaproxy.sslcert." + definition in d:
|
|
||||||
filename = '/etc/haproxy/certs/' + d["com.byjg.easyhaproxy.host." + definition] + "." + str(time.time()) + ".pem"
|
|
||||||
easymapping[key]["ssl_cert"] = filename
|
|
||||||
with open(filename, 'wb') as file:
|
|
||||||
file.write(base64.b64decode(d["com.byjg.easyhaproxy.sslcert." + definition]))
|
|
||||||
|
|
||||||
if "com.byjg.easyhaproxy.redirect." + definition in d:
|
|
||||||
redirect = d["com.byjg.easyhaproxy.redirect." + definition] if "com.byjg.easyhaproxy.redirect." + definition in d else ""
|
|
||||||
for r in redirect.split(","):
|
|
||||||
r_parts = r.split("--")
|
|
||||||
easymapping[key]["redirect"][r_parts[0]] = r_parts[1]
|
|
||||||
|
|
||||||
result["easymapping"] = easymapping.values()
|
|
||||||
|
|
||||||
|
|
||||||
cfg = HaproxyConfigGenerator(result)
|
|
||||||
print(cfg.generate())
|
|
||||||
|
|
||||||
# print(jsonStr)
|
|
||||||
|
|
||||||
|
|
||||||
8
templates/bind.j2
Normal file
8
templates/bind.j2
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
{% if "ssl_cert" in o %}
|
||||||
|
bind *:{{ o["port"] }} ssl crt {{ o["ssl_cert"] }}
|
||||||
|
{% elif "h2" in o and o["h2"] %}
|
||||||
|
bind *:{{ o["port"] }} proto h2
|
||||||
|
option http-use-htx
|
||||||
|
{% else %}
|
||||||
|
bind *:{{ o["port"] }}
|
||||||
|
{% endif %}
|
||||||
11
templates/frontend-mode-http.j2
Normal file
11
templates/frontend-mode-http.j2
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
mode http
|
||||||
|
{% for k in o["redirect"] %}
|
||||||
|
redirect prefix {{ o["redirect"][k] }} code 301 if { hdr(host) -i {{ k }} }
|
||||||
|
{% endfor %}
|
||||||
|
{% for k in o["hosts"] %}
|
||||||
|
{% set host = k.replace(".", "_") + "_{0}_{1}".format(o["port"], salt) %}
|
||||||
|
|
||||||
|
acl is_rule_{{ host }}_1 hdr(host) -i {{ k }}
|
||||||
|
acl is_rule_{{ host }}_2 hdr(host) -i {{ k }}:{{ o["port"] }}
|
||||||
|
use_backend srv_{{ host }} if is_rule_{{ host }}_1 OR is_rule_{{ host }}_2
|
||||||
|
{% endfor %}
|
||||||
6
templates/frontend-mode-tcp.j2
Normal file
6
templates/frontend-mode-tcp.j2
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
mode tcp
|
||||||
|
option tcplog
|
||||||
|
log global
|
||||||
|
{% set backend = (o["hosts"]|first) %}
|
||||||
|
default_backend srv_{{ backend.replace(".", "_") + "_{0}_{1}".format(o["port"], salt) }}
|
||||||
|
|
||||||
|
|
@ -1,3 +1,8 @@
|
||||||
|
global
|
||||||
|
log stdout format raw local0 info
|
||||||
|
maxconn 2000
|
||||||
|
tune.ssl.default-dh-param 2048
|
||||||
|
|
||||||
defaults
|
defaults
|
||||||
log global
|
log global
|
||||||
|
|
||||||
|
|
@ -14,11 +19,6 @@ defaults
|
||||||
errorfile 504 /etc/haproxy/errors-custom/504.http
|
errorfile 504 /etc/haproxy/errors-custom/504.http
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
global
|
|
||||||
log /dev/log local0
|
|
||||||
maxconn 2000
|
|
||||||
tune.ssl.default-dh-param 2048
|
|
||||||
|
|
||||||
{% if "stats" in data %}
|
{% if "stats" in data %}
|
||||||
frontend stats
|
frontend stats
|
||||||
bind *:{{ data["stats"]["port"] | default(1936) }}
|
bind *:{{ data["stats"]["port"] | default(1936) }}
|
||||||
|
|
@ -37,32 +37,31 @@ backend srv_stats
|
||||||
mode http
|
mode http
|
||||||
server Local 127.0.0.1:{{ data["stats"]["port"] | default(1936) }}
|
server Local 127.0.0.1:{{ data["stats"]["port"] | default(1936) }}
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
{% for o in data["easymapping"] -%}
|
||||||
{% for o in data["easymapping"] %}
|
{% set mode = o["mode"] or "http" %}
|
||||||
{% set salt = loop.index %}
|
{% set salt = loop.index %}
|
||||||
frontend http_in_{{ o["port"] }}_{{ salt }}
|
|
||||||
bind *:{{ o["port"] }} {{ " ssl crt " + o["ssl_cert"] if "ssl_cert" in o else "" }}
|
|
||||||
mode http
|
|
||||||
|
|
||||||
{% for k in o["redirect"] -%}
|
frontend {{ mode }}_in_{{ o["port"] }}_{{ salt }}
|
||||||
redirect prefix {{ o["redirect"][k] }} code 301 if { hdr(host) -i {{ k }} }
|
{% include "bind.j2" %}
|
||||||
{% endfor -%}
|
{% if mode == "http" %}
|
||||||
|
{% include "frontend-mode-http.j2" %}
|
||||||
|
{% else %}
|
||||||
|
{% include "frontend-mode-tcp.j2" %}
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
{% for k in o["hosts"] %}
|
{% for k in o["hosts"] -%}
|
||||||
{% set host = k.replace(".", "_") + "_{0}_{1}".format(o["port"], salt) %}
|
|
||||||
acl is_rule_{{ host }}_1 hdr(host) -i {{ k }}
|
|
||||||
acl is_rule_{{ host }}_2 hdr(host) -i {{ k }}:{{ o["port"] }}
|
|
||||||
use_backend srv_{{ host }} if is_rule_{{ host }}_1 OR is_rule_{{ host }}_2
|
|
||||||
{% endfor %}
|
|
||||||
|
|
||||||
{% for k in o["hosts"] %}
|
|
||||||
{% set host = k.replace(".", "_") + "_{0}_{1}".format(o["port"], salt) %}
|
{% set host = k.replace(".", "_") + "_{0}_{1}".format(o["port"], salt) %}
|
||||||
backend srv_{{ host }}
|
backend srv_{{ host }}
|
||||||
balance roundrobin
|
balance roundrobin
|
||||||
mode http
|
mode {{ mode }}
|
||||||
|
{% if mode == "http" %}
|
||||||
option forwardfor
|
option forwardfor
|
||||||
http-request set-header X-Forwarded-Port %[dst_port]
|
http-request set-header X-Forwarded-Port %[dst_port]
|
||||||
http-request add-header X-Forwarded-Proto https if { ssl_fc }
|
http-request add-header X-Forwarded-Proto https if { ssl_fc }
|
||||||
server srv {{ o["hosts"][k] }} check weight 1
|
{% elif mode == "tcp" %}
|
||||||
|
option tcp-check
|
||||||
|
tcp-check connect{{ " ssl" if o["health-check"] == "ssl" }}
|
||||||
|
{% endif %}
|
||||||
|
server srv {{ o["hosts"][k] }} check weight 1{{ " verify none" if o["health-check"] == "ssl" }}
|
||||||
{% endfor %}
|
{% endfor %}
|
||||||
{% endfor %}
|
{% endfor %}
|
||||||
|
|
|
||||||
0
tests/__init__.py
Normal file
0
tests/__init__.py
Normal file
5
tests/context.py
Normal file
5
tests/context.py
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
|
||||||
|
|
||||||
|
import easymapping
|
||||||
5
tests/fixtures/no-services
vendored
Normal file
5
tests/fixtures/no-services
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
swarm-prom_caddy={"com.docker.stack.image":"stefanprodan/caddy","com.docker.stack.namespace":"swarm-prom"}
|
||||||
|
swarm-prom_cadvisor={"com.docker.stack.image":"google/cadvisor","com.docker.stack.namespace":"swarm-prom"}
|
||||||
|
swarm-prom_dockerd-exporter={"com.docker.stack.image":"stefanprodan/caddy","com.docker.stack.namespace":"swarm-prom"}
|
||||||
|
swarm-prom_unsee={"com.docker.stack.image":"cloudflare/unsee:v0.8.0","com.docker.stack.namespace":"swarm-prom"}
|
||||||
|
test_proxy={"com.docker.stack.image":"byjg/easy-haproxy","com.docker.stack.namespace":"test"}
|
||||||
5
tests/fixtures/services
vendored
Normal file
5
tests/fixtures/services
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
portainer-agent_agent={"com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"portainer-agent"}
|
||||||
|
my-stack_agent={"com.byjg.easyhaproxy.definitions":"agent","com.byjg.easyhaproxy.host.agent":"agent.quantum.example.org","com.byjg.easyhaproxy.localport.agent":"9001","com.byjg.easyhaproxy.mode.agent":"tcp","com.byjg.easyhaproxy.port.agent":"31339","com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"}
|
||||||
|
my-stack_cadvisor={"com.byjg.easyhaproxy.definitions":"cadvisor","com.byjg.easyhaproxy.host.cadvisor":"cadvisor.quantum.example.org","com.byjg.easyhaproxy.localport.cadvisor":"8080","com.byjg.easyhaproxy.port.cadvisor":"31337","com.docker.stack.image":"gcr.io/google-containers/cadvisor:v0.34.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"}
|
||||||
|
my-stack_node-exporter={"com.byjg.easyhaproxy.definitions":"exp","com.byjg.easyhaproxy.host.exp":"node-exporter.quantum.example.org","com.byjg.easyhaproxy.localport.exp":"9100","com.byjg.easyhaproxy.port.exp":"31337","com.docker.stack.image":"stefanprodan/swarmprom-node-exporter:v0.16.0","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"}
|
||||||
|
my-stack_reverse-proxy={"com.docker.stack.image":"quay.io/pngmbh/easy-haproxy:tcp-mode","com.docker.stack.namespace":"my-stack","com.planetary-quantum":"monitoring"}
|
||||||
2
tests/fixtures/services-tcp
vendored
Normal file
2
tests/fixtures/services-tcp
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
test_agent={"com.byjg.easyhaproxy.definitions":"agent","com.byjg.easyhaproxy.host.agent":"agent.quantum.local","com.byjg.easyhaproxy.localport.agent":"9001","com.byjg.easyhaproxy.mode.agent":"tcp","com.byjg.easyhaproxy.port.agent":"31339","com.docker.stack.image":"portainer/agent:1.5.1","com.docker.stack.namespace":"test", "com.byjg.easyhaproxy.health-check.agent":"ssl"}
|
||||||
|
test_proxy={"com.docker.stack.image":"byjg/easy-haproxy:local","com.docker.stack.namespace":"test"}
|
||||||
23
tests/fixtures/static.yml
vendored
Normal file
23
tests/fixtures/static.yml
vendored
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
stats:
|
||||||
|
username: admin
|
||||||
|
password: test123
|
||||||
|
port: 1936 # Optional (default 1936)
|
||||||
|
|
||||||
|
customerrors: true # Optional (default false)
|
||||||
|
|
||||||
|
easymapping:
|
||||||
|
- port: 80
|
||||||
|
hosts:
|
||||||
|
host1.com.br: container:5000
|
||||||
|
host2.com.br: other:3000
|
||||||
|
redirect:
|
||||||
|
www.host1.com.br: http://host1.com.br
|
||||||
|
|
||||||
|
- port: 443
|
||||||
|
ssl_cert: BASE64_PEM_CERTIFICATE
|
||||||
|
hosts:
|
||||||
|
host1.com.br: container:80
|
||||||
|
|
||||||
|
- port: 8080
|
||||||
|
hosts:
|
||||||
|
host3.com.br: domain:8181
|
||||||
32
tests/test_labels.py
Normal file
32
tests/test_labels.py
Normal file
|
|
@ -0,0 +1,32 @@
|
||||||
|
from .context import easymapping
|
||||||
|
import json
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
def test_label_generation():
|
||||||
|
label = easymapping.DockerLabelHandler("foo")
|
||||||
|
|
||||||
|
assert label.create("bar") == "foo.bar"
|
||||||
|
assert label.create(["bar", "foobar"]) == "foo.bar.foobar"
|
||||||
|
|
||||||
|
|
||||||
|
def test_label_data():
|
||||||
|
label = easymapping.DockerLabelHandler("base")
|
||||||
|
label.set_data(json.loads('{"base.definitions":"h2"}'))
|
||||||
|
|
||||||
|
label_name = label.create("definitions")
|
||||||
|
assert label_name == "base.definitions"
|
||||||
|
assert label.has_label(label_name)
|
||||||
|
assert label.get(label_name) == "h2"
|
||||||
|
|
||||||
|
|
||||||
|
def test_label_complex_key():
|
||||||
|
label = easymapping.DockerLabelHandler("till")
|
||||||
|
|
||||||
|
data = dict()
|
||||||
|
data["till.definitions"] = "h2"
|
||||||
|
data["till.host.h2"] = "fqdn.example.org"
|
||||||
|
data["till.mode.h2"] = "tcp"
|
||||||
|
label.set_data(json.loads(json.dumps(data)))
|
||||||
|
|
||||||
|
assert label.get(label.create(["host", "h2"])) == "fqdn.example.org"
|
||||||
|
assert label.get(label.create(["mode", "h2"])) == "tcp"
|
||||||
108
tests/test_parser.py
Normal file
108
tests/test_parser.py
Normal file
|
|
@ -0,0 +1,108 @@
|
||||||
|
from .context import easymapping
|
||||||
|
import pytest
|
||||||
|
import os
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
|
||||||
|
def load_fixture(file):
|
||||||
|
path = os.path.dirname(os.path.realpath(__file__))
|
||||||
|
with open(path + "/fixtures/" + file, 'r') as content_file:
|
||||||
|
lineList = content_file.readlines()
|
||||||
|
|
||||||
|
return lineList
|
||||||
|
|
||||||
|
|
||||||
|
def test_parser_doesnt_crash():
|
||||||
|
lineList = load_fixture("no-services")
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"customerrors": False
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg = easymapping.HaproxyConfigGenerator(result)
|
||||||
|
haproxy_config = cfg.generate(lineList)
|
||||||
|
assert len(haproxy_config) > 0
|
||||||
|
assert "frontend" not in haproxy_config
|
||||||
|
assert "backend" not in haproxy_config
|
||||||
|
|
||||||
|
|
||||||
|
def test_parser_finds_services():
|
||||||
|
lineList = load_fixture("services")
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"customerrors": False
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg = easymapping.HaproxyConfigGenerator(result)
|
||||||
|
haproxy_config = cfg.generate(lineList)
|
||||||
|
assert len(haproxy_config) > 0
|
||||||
|
assert "mode tcp" in haproxy_config
|
||||||
|
assert "mode http" in haproxy_config
|
||||||
|
|
||||||
|
assert "frontend tcp_in_31339_1" in haproxy_config
|
||||||
|
assert "frontend http_in_31337_2" in haproxy_config
|
||||||
|
|
||||||
|
|
||||||
|
def test_parser_static():
|
||||||
|
path = os.path.dirname(os.path.realpath(__file__))
|
||||||
|
with open(path + "/fixtures/static.yml", 'r') as content_file:
|
||||||
|
parsed = yaml.load(content_file.read(), Loader=yaml.FullLoader)
|
||||||
|
|
||||||
|
cfg = easymapping.HaproxyConfigGenerator(parsed)
|
||||||
|
haproxy_config = cfg.generate()
|
||||||
|
assert len(haproxy_config) > 0
|
||||||
|
|
||||||
|
# assert on auth on stats
|
||||||
|
assert "stats auth admin:test123" in haproxy_config
|
||||||
|
|
||||||
|
# assert that we found redirect
|
||||||
|
assert "redirect prefix http://host1.com.br code 301 if { hdr(host) -i www.host1.com.br }" in haproxy_config
|
||||||
|
|
||||||
|
# assert that we found the services
|
||||||
|
frontend_http_cfg = "frontend http_in_80_1\n"
|
||||||
|
frontend_http_cfg += " bind *:80"
|
||||||
|
assert frontend_http_cfg in haproxy_config
|
||||||
|
|
||||||
|
frontend_https_cfg = "frontend http_in_443_2\n"
|
||||||
|
frontend_https_cfg += " bind *:443"
|
||||||
|
assert frontend_https_cfg in haproxy_config
|
||||||
|
|
||||||
|
# print(haproxy_config)
|
||||||
|
frontend_http8080_cfg = "frontend http_in_8080_3\n"
|
||||||
|
frontend_http8080_cfg += " bind *:8080"
|
||||||
|
assert frontend_http8080_cfg in haproxy_config
|
||||||
|
|
||||||
|
|
||||||
|
# verify ssl config with certificate
|
||||||
|
frontend_ssl_cfg = "frontend http_in_443_2\n"
|
||||||
|
frontend_ssl_cfg += " bind *:443 ssl crt BASE64_PEM_CERTIFICATE"
|
||||||
|
assert frontend_ssl_cfg in haproxy_config
|
||||||
|
|
||||||
|
|
||||||
|
def test_parser_tcp():
|
||||||
|
lineList = load_fixture("services-tcp")
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"customerrors": False
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg = easymapping.HaproxyConfigGenerator(result)
|
||||||
|
haproxy_config = cfg.generate(lineList)
|
||||||
|
# print(haproxy_config)
|
||||||
|
|
||||||
|
frontend_cfg = "frontend tcp_in_31339_1\n"
|
||||||
|
frontend_cfg += " bind *:31339\n"
|
||||||
|
frontend_cfg += " mode tcp\n"
|
||||||
|
frontend_cfg += " option tcplog\n"
|
||||||
|
frontend_cfg += " log global\n"
|
||||||
|
frontend_cfg += " default_backend srv_agent_quantum_local_31339_1\n\n"
|
||||||
|
assert frontend_cfg in haproxy_config
|
||||||
|
|
||||||
|
backend_cfg = "backend srv_agent_quantum_local_31339_1\n"
|
||||||
|
backend_cfg += " balance roundrobin\n"
|
||||||
|
backend_cfg += " mode tcp\n"
|
||||||
|
backend_cfg += " option tcp-check\n"
|
||||||
|
backend_cfg += " tcp-check connect ssl\n"
|
||||||
|
backend_cfg += " server srv test_agent:9001 check weight 1 verify none"
|
||||||
|
|
||||||
|
assert backend_cfg in haproxy_config
|
||||||
Loading…
Add table
Add a link
Reference in a new issue