badblocks/docker-easy-haproxy
1
0
Fork 0
Code Activity

Update logging level for plugin loading and extend static example documentation

Browse source 
- Changed logging level from `info` to `debug` in plugin loading to reduce verbosity during runtime.
- Significantly expanded `examples/static/README.md` with new sections, examples (basic, certbot, deny pages, JWT validator), and detailed usage instructions.
- Improved `examples/docker/docker-compose-changed-label.yml` with clearer redirect syntax using JSON for better readability.
- Fixed handling of empty or invalid JSON in labels within `easymapping` with fallback to default values and error logging.
- Added additional entries to `.gitignore` to exclude dynamic and temporary files like auto-generated configuration files and certificates.
This commit is contained in:
Joao Gilberto Magalhaes 2025-12-04 09:01:12 -05:00
parent 3a4b428e46
commit 99f4ff325b
7 changed files with 384 additions and 403 deletions
Download patch file Download diff file Expand all files Collapse all files

8
.gitignore vendored
View file

@ -6,3 +6,11 @@ __pycache__
.pytest_cache
*.pyc
.env
/examples/static/conf/config.yml
/examples/docker/certs/haproxy/.place_holder_cert.pem
/examples/static/host1.local.pem
/examples/swarm/certs/host1.local.pem
/examples/docker/host2.local.pem
/examples/swarm/certs/host2.local.pem
/examples/docker/jwt_private.pem
/examples/docker/jwt_pubkey.pem

19
examples/docker/README.md
View file

@ -145,6 +145,25 @@ labels:
myproxy.http.port: 80
```
**Usage:**
```bash
docker compose -f docker-compose-changed-label.yml up -d
```
**Test:**
```bash
# Test load balancing (hostname changes between containers)
curl -H "Host: www.helloworld.com" localhost:19901
# Response: f6d8d45b7411
curl -H "Host: www.helloworld.com" localhost:19901
# Response: 59b213cb8592
# Test redirect
curl -I -H "Host: google.helloworld.com" localhost:19901
# Should redirect to: www.google.com/
```
---
### 5. Portainer Integration (`docker-compose-portainer.yml`)

2
examples/docker/docker-compose-changed-label.yml
View file

@ -27,7 +27,7 @@ services:
container:
image: byjg/static-httpserver
labels:
haproxy.http.redirect: host1.local--https://host1.local
haproxy.http.redirect: '{"host1.local": "https://host1.local"}'
haproxy.http.host: host1.local
haproxy.http.port: 80

728
examples/static/README.md
View file

@ -1,49 +1,312 @@
# Static Configuration Example
This directory demonstrates EasyHAProxy using **static configuration** mode instead of dynamic service discovery.
This directory demonstrates EasyHAProxy using **static YAML configuration** instead of dynamic service discovery.
## What is Static Mode?
Static mode uses a YAML configuration file (`config.yml`) to define HAProxy routing rules instead of discovering services automatically from Docker/Kubernetes/Swarm labels.
**Use cases:**
- Non-containerized backends
- Mixed environments (containers + VMs + bare metal)
- Fixed infrastructure where services don't change frequently
- Testing HAProxy configurations
Static mode is useful for:
- Non-containerized backends (VMs, bare metal)
- Fixed infrastructure
- Explicit routing control
---
## Files in This Example
## Prerequisites
- `conf/config.yml` - Static configuration defining hosts and routing
- `docker-compose.yml` - EasyHAProxy container mounting the config file
- `host1.local.pem` - Example SSL certificate
### 1. Generate SSL Certificates
---
## Configuration Structure
### docker-compose.yml
```yaml
services:
haproxy:
image: byjg/easy-haproxy:4.6.0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./conf:/etc/haproxy/static # Mount static config
- ./host1.local.pem:/certs/haproxy/host1.local.pem
environment:
EASYHAPROXY_DISCOVER: static # Use static mode
EASYHAPROXY_SSL_MODE: "loose"
ports:
- "80:80"
- "443:443"
- "1936:1936"
```bash
# From repository root
./examples/generate-keys.sh
```
### conf/config.yml
### 2. Add Host Entry
```bash
echo "127.0.0.1 host1.local www.host1.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 host2.local www.host2.local" | sudo tee -a /etc/hosts
```
---
## Scenario 1: Basic (HTTP → HTTPS Redirect)
**What it does:** Simple HTTP to HTTPS redirect with SSL termination.
### Getting Started
```bash
cd examples/static
# 1. Copy the basic config
cp conf/config-basic.yml conf/config.yml
# 2. Start backend container
docker run -d --name container -p 8080:8080 byjg/static-httpserver
# 3. Start EasyHAProxy
docker compose up -d
```
### Test
```bash
# Test HTTP redirect
curl -I http://host1.local
# Expected: HTTP/1.1 301 Moved Permanently
# Expected: Location: https://host1.local
# Test HTTPS
curl -k https://host1.local
# Expected: Hello from Static HTTP Server!
# Test www redirect
curl -I http://www.host1.local
# Expected: HTTP/1.1 301 Moved Permanently
# Expected: Location: https://host1.local
```
### Stats Interface
Open: http://localhost:1936
- Username: `admin`
- Password: `password`
### Clean Up
```bash
docker compose down
docker stop container && docker rm container
```
---
## Scenario 2: Certbot (Let's Encrypt SSL)
**What it does:** Automatic SSL certificates from Let's Encrypt using ACME HTTP-01 challenge.
### Requirements
- Public IP address
- Domain pointing to your IP
- Ports 80/443 publicly accessible
### Getting Started
```bash
cd examples/static
# 1. Copy the certbot config
cp conf/config-certbot.yml conf/config.yml
# 2. Edit config.yml and change:
# - Replace "example.com" with your real domain
# - Update EASYHAPROXY_CERTBOT_EMAIL in docker-compose.yml
# 3. Start backend container
docker run -d --name container -p 8080:8080 byjg/static-httpserver
# 4. Start EasyHAProxy
docker compose up -d
# 5. Check logs for certificate generation
docker compose logs -f
```
### What to Expect
```
# Certbot will:
# 1. Request certificate from Let's Encrypt
# 2. Complete HTTP-01 challenge
# 3. Save certificate in /certs/certbot/
# 4. Reload HAProxy with new certificate
```
### Test
```bash
# Test HTTPS with real certificate
curl https://your-domain.com
# Expected: No certificate warnings (valid SSL)
# Test HTTP redirect
curl -I http://your-domain.com
# Expected: HTTP/1.1 301 Moved Permanently
```
### Clean Up
```bash
docker compose down
docker stop container && docker rm container
```
**Note:** Certificates are stored in Docker volume `certs_certbot` and persist across restarts.
---
## Scenario 3: Deny Pages (Block Specific Paths)
**What it does:** Blocks access to sensitive paths like `/admin`, `/wp-login.php`, etc.
### Getting Started
```bash
cd examples/static
# 1. Copy the deny-pages config
cp conf/config-deny-pages.yml conf/config.yml
# 2. Start backend container
docker run -d --name container -p 8080:8080 byjg/static-httpserver
# 3. Start EasyHAProxy
docker compose up -d
```
### Test
```bash
# Test normal page (should work)
curl -k https://host1.local/
# Expected: Hello from Static HTTP Server!
# Test blocked path (should fail)
curl -I -k https://host1.local/admin
# Expected: HTTP/1.1 404 Not Found
curl -I -k https://host1.local/wp-login.php
# Expected: HTTP/1.1 404 Not Found
curl -I -k https://host1.local/.env
# Expected: HTTP/1.1 404 Not Found
```
### What's Blocked
The example blocks these paths:
- `/admin`
- `/wp-admin`
- `/wp-login.php`
- `/.env`
- `/config`
### Customize Blocked Paths
Edit `conf/config.yml`:
```yaml
plugin_config:
deny_pages:
paths: /admin,/private,/internal
status_code: 403 # or 404
```
### Clean Up
```bash
docker compose down
docker stop container && docker rm container
```
---
## Scenario 4: JWT Validator (API Authentication)
**What it does:** Validates JWT tokens in Authorization header before allowing access.
### Getting Started
```bash
cd examples/static
# 1. Copy the JWT validator config
cp conf/config-jwt-validator.yml conf/config.yml
# 2. JWT keys were already generated by generate-keys.sh
# Location: examples/docker/jwt_pubkey.pem and jwt_private.pem
# 3. Start backend container
docker run -d --name container -p 8080:8080 byjg/static-httpserver
# 4. Start EasyHAProxy
docker compose up -d
```
### Test Without Token (Should Fail)
```bash
curl -k https://host1.local/
# Expected: Missing Authorization HTTP header
```
### Test With Valid Token
```bash
# 1. Generate a test JWT token using jwt_private.pem
# You can use https://jwt.io or a JWT library
# 2. Example with valid token:
TOKEN="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."
curl -k -H "Authorization: Bearer $TOKEN" https://host1.local/
# Expected: Hello from Static HTTP Server! (if token is valid)
```
### Generate Test Token
```python
# Python example using PyJWT
import jwt
from datetime import datetime, timedelta
with open('examples/docker/jwt_private.pem', 'r') as f:
private_key = f.read()
payload = {
'iss': 'https://auth.example.com/',
'aud': 'https://api.example.com',
'exp': datetime.utcnow() + timedelta(hours=1)
}
token = jwt.encode(payload, private_key, algorithm='RS256')
print(token)
```
### What's Validated
- Authorization header must be present
- Token must be valid JWT format
- Signature must match public key (`jwt_pubkey.pem`)
- Issuer must match: `https://auth.example.com/`
- Audience must match: `https://api.example.com`
- Token must not be expired
### Customize JWT Settings
Edit `conf/config.yml`:
```yaml
plugin_config:
jwt_validator:
algorithm: RS256
issuer: https://your-auth-server.com/
audience: https://your-api.com
pubkey_path: /certs/haproxy/jwt_pubkey.pem
```
### Clean Up
```bash
docker compose down
docker stop container && docker rm container
```
---
## Configuration File Reference
All scenarios use `/etc/haproxy/static/config.yml` mounted from `./conf/config.yml`.
### Basic Structure
```yaml
stats:
@ -54,416 +317,102 @@ stats:
customerrors: true
easymapping:
# HTTP Port 80 - Redirects to HTTPS
- port: 80
redirect:
host1.local: https://host1.local
www.host1.local: https://host1.local
# HTTPS Port 443
- port: 443
ssl: true
hosts:
host1.local:
containers:
- container:8080 # Backend container
- container:8080
```
---
## How It Works
### 1. Port Definitions
Each item in `easymapping` defines a listening port:
### With Plugins
```yaml
easymapping:
- port: 80 # Listen on port 80
redirect: {...} # Optional redirects
- port: 443 # Listen on port 443
ssl: true # Enable SSL
hosts: {...} # Virtual hosts
```
### 2. Redirect Configuration
Redirect specific domains to different URLs:
```yaml
- port: 80
redirect:
host1.local: https://host1.local # HTTP → HTTPS
www.host1.local: https://host1.local # www → non-www + HTTPS
old.domain.com: https://new.domain.com # Domain change
```
### 3. Virtual Hosts
Define hosts and their backend containers:
```yaml
- port: 443
- port: 443
ssl: true
hosts:
host1.local: # Virtual host domain
host1.local:
containers:
- container:8080 # Backend: container_name:port
- another_container:3000 # Multiple backends = load balancing
host2.local:
containers:
- webserver:80
```
**Backend formats:**
- `container_name:port` - Docker container by name
- `ip_address:port` - Direct IP address
- `hostname:port` - Hostname resolution
### 4. SSL Configuration
```yaml
- port: 443
ssl: true # Enable SSL on this port
hosts:
secure.example.com:
containers:
- app:8080
```
SSL certificates must be placed in:
- `/certs/haproxy/<domain>.pem` inside container
- `./certs/<domain>.pem` on host (if volume mounted)
Certificate format: PEM file containing both certificate and private key.
### 5. Stats Interface
```yaml
stats:
username: admin
password: password
port: 1936
```
Access at: `http://localhost:1936`
---
## Running the Example
### Prerequisites: Generate SSL Certificates
Before running the examples, you need to generate the required SSL certificates:
```bash
# From the repository root
./examples/generate-keys.sh
```
This script automatically generates:
- SSL certificates for host1.local and host2.local
- JWT keys for authentication examples
- All other .pem files needed for testing
### 1. Start the Example
```bash
cd examples/static
docker compose up -d
```
### 2. Create Backend Container
The static config references `container:8080`. Create a container with this name:
```bash
docker run -d --name container \
-p 8080:8080 \
byjg/static-httpserver
```
Or add to `docker-compose.yml`:
```yaml
services:
# ... haproxy service ...
container:
image: byjg/static-httpserver
ports:
- "8080:8080"
```
### 3. Test
```bash
# Add to /etc/hosts:
# 127.0.0.1 host1.local www.host1.local
# Test HTTP redirect
curl -I http://host1.local
# Should return: HTTP/1.1 301 Moved Permanently
# Location: https://host1.local
# Test HTTPS
curl -k https://host1.local
# Access stats
open http://localhost:1936
# Username: admin
# Password: password
- container:8080
plugins:
- deny_pages
plugin_config:
deny_pages:
paths: /admin,/private
status_code: 404
```
---
## Advanced Configuration
## Advanced: Multiple Backends
### Load Balancing Multiple Backends
Load balance across multiple containers:
```yaml
hosts:
api.example.com:
containers:
- api_server_1:8080
- api_server_2:8080
- api_server_3:8080
- api1:8080
- api2:8080
- api3:8080
```
Default algorithm: round-robin
---
### Custom Balance Algorithm
## Advanced: External Backends
```yaml
hosts:
api.example.com:
balance: leastconn # Use least connections instead of round-robin
containers:
- api_1:8080
- api_2:8080
```
**Available algorithms:**
- `roundrobin` - Distribute evenly (default)
- `leastconn` - Send to server with fewest connections
- `source` - Same client IP always goes to same server
### External Backends (Non-Docker)
Route to non-Docker backends:
```yaml
hosts:
legacy.example.com:
containers:
- 192.168.1.100:8080 # VM
- 192.168.1.101:8080 # Another VM
- database.local:5432 # Database server
- 192.168.1.100:8080
- 192.168.1.101:8080
```
### Health Checks
```yaml
hosts:
webapp.example.com:
containers:
- server1:8080
- server2:8080
healthcheck:
path: /health
interval: 5s
```
### Multiple Domains, Same Backend
```yaml
hosts:
example.com:
containers:
- webapp:8080
www.example.com:
containers:
- webapp:8080 # Same backend
app.example.com:
containers:
- webapp:8080 # Same backend
```
### Path-Based Routing
While static mode focuses on host-based routing, you can achieve path-based routing using redirects:
```yaml
- port: 80
redirect:
api.example.com/v1: https://api-v1.internal:8080
api.example.com/v2: https://api-v2.internal:8080
```
Or use HAProxy ACLs via custom templates (advanced).
---
## Plugins with Static Configuration
Enable plugins globally or per-host in static mode:
### Global Plugin Configuration
```yaml
plugins:
enabled: [cleanup]
config:
cleanup:
max_idle_time: 600
```
### Per-Host Plugin Configuration (via env vars)
Since static mode doesn't support per-host plugin configuration directly, use environment variables for domain-specific plugins:
```yaml
environment:
EASYHAPROXY_PLUGINS_ENABLED: cloudflare,deny_pages
EASYHAPROXY_PLUGIN_CLOUDFLARE_IP_LIST_PATH: /etc/haproxy/cloudflare_ips.lst
EASYHAPROXY_PLUGIN_DENY_PAGES_PATHS: /admin,/private
```
See [Using Plugins](../../docs/plugins.md) for more details.
---
## Comparison: Static vs. Dynamic Discovery
| Feature | Static Mode | Docker/Swarm/K8s Mode |
|---------------|-----------------------------|----------------------------------------------|
| Configuration | YAML file | Container labels / Ingress annotations |
| Backend types | Any (containers, VMs, IPs) | Containers only |
| Updates | Manual config edit + reload | Automatic discovery |
| Use case | Fixed infrastructure | Dynamic container environments |
| Plugin config | Global via YAML/env | Per-container/ingress via labels/annotations |
---
## Tips
1. **Reloading Configuration:**
```bash
# EasyHAProxy watches config.yml for changes
# Edit conf/config.yml, changes auto-reload
# Or manually restart:
docker compose restart haproxy
```
2. **Validate Configuration:**
```bash
# Check HAProxy config is valid
docker compose exec haproxy haproxy -c -f /etc/haproxy/haproxy.cfg
```
3. **View Generated Config:**
```bash
docker compose exec haproxy cat /etc/haproxy/haproxy.cfg
```
4. **Debugging:**
```bash
# Enable debug mode
docker compose up
# Watch logs in real-time
```
5. **SSL Certificate Management:**
```bash
# Generate self-signed cert
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout host.key -out host.crt \
-subj "/CN=host1.local"
# Combine into PEM
cat host.crt host.key > host1.local.pem
```
---
## Troubleshooting
### Backend Unreachable
### FileNotFoundError: config.yml
**Error:** `503 Service Unavailable`
**Causes:**
- Backend container not running
- Wrong container name in config
- Wrong port number
- Network connectivity issues
**Debug:**
```bash
# Check backend container is running
docker ps | grep container_name
# Make sure config.yml exists
ls conf/config.yml
# Test backend directly
curl http://container_name:port
# Check HAProxy logs
docker compose logs haproxy
# If missing, copy from an example:
cp conf/config-basic.yml conf/config.yml
```
### Configuration Not Reloading
### 503 Service Unavailable
**Solution:**
```bash
# Restart HAProxy
docker compose restart haproxy
# Check file is mounted correctly
docker compose exec haproxy cat /etc/haproxy/static/config.yml
# Check backend is running
docker ps | grep container
curl http://localhost:8080
```
### SSL Certificate Not Found
**Error:** Certificate errors in logs
**Solution:**
```bash
# Verify certificate is mounted
docker compose exec haproxy ls -la /certs/haproxy/
# Verify certificate exists
ls -la host1.local.pem
# Check certificate format (must be PEM with cert + key)
openssl x509 -in host.pem -text -noout
openssl rsa -in host.pem -check
# Regenerate if needed
cd ../.. && ./examples/generate-keys.sh
```
---
### Changes Not Applied
## Migration from Dynamic to Static
If you have Docker labels and want to convert to static config:
**Docker label:**
```yaml
labels:
easyhaproxy.http.host: api.example.com
easyhaproxy.http.port: 80
easyhaproxy.http.localport: 8080
easyhaproxy.http.redirect_ssl: true
```
**Static config equivalent:**
```yaml
easymapping:
- port: 80
redirect:
api.example.com: https://api.example.com
- port: 443
ssl: true
hosts:
api.example.com:
containers:
- container_name:8080
```bash
# Restart to reload config
docker compose restart
```
---
@ -471,6 +420,5 @@ easymapping:
## Further Reading
- [Static Configuration Guide](../../docs/static.md)
- [Environment Variables](../../docs/environment-variable.md)
- [Using Plugins](../../docs/plugins.md)
- [SSL Configuration](../../docs/ssl.md)
- [Environment Variables](../../docs/environment-variable.md)

21
src/easymapping/__init__.py
View file

@ -4,6 +4,7 @@ import os
import re
from jinja2 import Environment, FileSystemLoader
from functions import loggerEasyHaproxy
class DockerLabelHandler:
@ -32,7 +33,16 @@ class DockerLabelHandler:
def get_json(self, label, default_value={}):
if self.has_label(label):
return json.loads(self.__data[label])
value = self.__data[label]
if not value: # Handle empty strings
return default_value
try:
return json.loads(value)
except json.JSONDecodeError as e:
loggerEasyHaproxy.error(
f"Invalid JSON in label '{label}': {value}. Error: {e}. Using default value."
)
return default_value
return default_value
def set_data(self, data):
@ -67,8 +77,7 @@ class HaproxyConfigGenerator:
self.global_plugin_configs = []
except Exception as e:
# If plugin system fails to initialize, log but continue
import logging
logging.warning(f"Failed to initialize plugin system: {e}")
loggerEasyHaproxy.warning(f"Failed to initialize plugin system: {e}")
self.plugin_manager = None
self.global_plugin_configs = []
@ -102,8 +111,7 @@ class HaproxyConfigGenerator:
global_configs = [r.haproxy_config for r in global_results if r.haproxy_config]
self.global_plugin_configs.extend(global_configs)
except Exception as e:
import logging
logging.warning(f"Failed to execute global plugins: {e}")
loggerEasyHaproxy.warning(f"Failed to execute global plugins: {e}")
templates_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.realpath(__file__))), 'templates')
file_loader = FileSystemLoader(templates_dir)
@ -274,8 +282,7 @@ class HaproxyConfigGenerator:
if result.metadata["fcgi_app_definition"] not in self.global_plugin_configs:
self.global_plugin_configs.append(result.metadata["fcgi_app_definition"])
except Exception as e:
import logging
logging.warning(f"Failed to execute domain plugins for {hostname}: {e}")
loggerEasyHaproxy.warning(f"Failed to execute domain plugins for {hostname}: {e}")
easymapping[port]["hosts"][hostname]["plugin_configs"] = []
else:
easymapping[port]["hosts"][hostname]["plugin_configs"] = []

1
src/main.py
View file

@ -1,5 +1,4 @@
import os
import logging
from deepdiff import DeepDiff

2
src/plugins/__init__.py
View file

@ -145,7 +145,7 @@ class PluginManager:
elif plugin.plugin_type == PluginType.DOMAIN:
self.domain_plugins.append(plugin)
self.logger.info(f"Loaded {source} plugin: {plugin.name} ({plugin.plugin_type.value})")
self.logger.debug(f"Loaded {source} plugin: {plugin.name} ({plugin.plugin_type.value})")
except Exception as e:
self._handle_error(f"Failed to load plugin from {filepath}: {str(e)}")