1
0
Fork 0

Introduce plugin initialization phase and consolidate config handling

- Added `initialize_plugins` method to `PluginManager` for requesting and processing file system resources during plugin initialization.
- Introduced `InitializationResult` and `ResourceRequest` structures for fine-grained resource management (e.g., directories, files).
- Enhanced configuration injection by separating `global_configs`, `defaults_configs`, and backend-level `haproxy_config`.
- Updated existing plugins (e.g., JWT Validator, Cloudflare) to use the initialization phase for resource setup.
- Simplified domain plugin execution by consolidating configuration logic into unified loops.
This commit is contained in:
Joao Gilberto Magalhaes 2026-02-13 16:14:05 -05:00
parent 97845b8a52
commit ece012a884
10 changed files with 453 additions and 107 deletions

View file

@ -101,7 +101,13 @@ Execute **once per discovered domain/host**.
├─ Calls plugin.configure(config) for each plugin
└─ Validates configuration (plugin responsibility)
3. EXECUTION PHASE (per discovery cycle)
3. INITIALIZE PHASE
├─ Calls plugin.initialize() for each plugin
├─ Plugins request file system resources (directories, files)
├─ PluginManager processes resource requests
└─ Creates directories and files as needed
4. EXECUTION PHASE (per discovery cycle)
├─ GLOBAL PLUGINS
│ └─ Executes all global plugins once
@ -109,9 +115,11 @@ Execute **once per discovered domain/host**.
└─ For each discovered domain:
└─ Executes all domain plugins
4. RESULT PROCESSING
5. RESULT PROCESSING
├─ Collects PluginResult from each plugin
├─ Injects haproxy_config into generated config
├─ Injects haproxy_config into backend sections
├─ Injects global_configs into global section
├─ Injects defaults_configs into defaults section
├─ Applies modified_easymapping if provided
└─ Logs metadata for debugging
```
@ -164,7 +172,7 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from plugins import PluginInterface, PluginType, PluginContext, PluginResult
from functions import loggerEasyHaproxy
from functions import logger_easyhaproxy
class MyPlugin(PluginInterface):
@ -306,6 +314,18 @@ class PluginInterface(ABC):
"""
pass
def initialize(self) -> InitializationResult:
"""
Initialize plugin resources (new in v2.0)
Optional method to request file system resources.
Default implementation returns empty result (no-op).
Returns:
InitializationResult with resource requests
"""
return InitializationResult()
@abstractmethod
def process(self, context: PluginContext) -> PluginResult:
"""
@ -328,6 +348,7 @@ class PluginInterface(ABC):
**Methods:**
- `configure(config)` - Receives plugin configuration during initialization
- `initialize()` - **[New in v2.0]** Request file system resources (optional)
- `process(context)` - Main execution logic, returns `PluginResult`
### PluginType
@ -390,6 +411,71 @@ def process(self, context: PluginContext) -> PluginResult:
custom_label = context.host_config.get("custom_label", "default")
```
### ResourceRequest
**[New in v2.0]** Request for file system resources during plugin initialization.
```python
@dataclass
class ResourceRequest:
"""Request for file system resources"""
resource_type: str # "directory" or "file"
path: str
content: str | None = None
overwrite: bool = False
```
**Fields:**
- `resource_type` - Type of resource: `"directory"` or `"file"`
- `path` - Absolute path to create
- `content` - File content (only for `resource_type="file"`)
- `overwrite` - Whether to overwrite existing files (default: False)
**Example:**
```python
ResourceRequest(
resource_type="directory",
path="/etc/haproxy/plugin_data"
)
ResourceRequest(
resource_type="file",
path="/etc/haproxy/plugin_config.txt",
content="config data",
overwrite=True
)
```
### InitializationResult
**[New in v2.0]** Plugin initialization result with resource requests.
```python
@dataclass
class InitializationResult:
"""Plugin initialization result with resource requests"""
resources: list[ResourceRequest] = field(default_factory=list)
metadata: dict[str, Any] = field(default_factory=dict)
```
**Fields:**
- `resources` - List of ResourceRequest objects
- `metadata` - Optional metadata about initialization
**Example:**
```python
def initialize(self) -> InitializationResult:
return InitializationResult(
resources=[
ResourceRequest(resource_type="directory", path="/etc/haproxy/jwt_keys")
]
)
```
### PluginResult
Plugin execution result containing configuration and metadata.
@ -401,6 +487,8 @@ class PluginResult:
haproxy_config: str = "" # HAProxy config snippet to inject
modified_easymapping: Optional[list] = None # Modified easymapping structure
metadata: Dict[str, Any] = field(default_factory=dict) # Plugin metadata for logging
global_configs: list[str] = field(default_factory=list) # [New] Global-level configs
defaults_configs: list[str] = field(default_factory=list) # [New] Defaults-level configs
```
**Fields:**
@ -408,11 +496,13 @@ class PluginResult:
- `haproxy_config` - HAProxy configuration snippet (injected into backend/frontend)
- `modified_easymapping` - Modified easymapping structure (optional, advanced use)
- `metadata` - Dictionary with debugging/logging information
- `global_configs` - **[New in v2.0]** List of global-level HAProxy configs (e.g., fcgi-app definitions)
- `defaults_configs` - **[New in v2.0]** List of defaults-level HAProxy configs (e.g., log-format)
**Examples:**
```python
# Simple config injection
# Simple config injection (backend-level)
return PluginResult(
haproxy_config="http-request deny deny_status 403"
)
@ -427,6 +517,22 @@ return PluginResult(
}
)
# With global-level config (new in v2.0)
return PluginResult(
haproxy_config="use-fcgi-app fcgi_example_com",
global_configs=[
"fcgi-app fcgi_example_com\n docroot /var/www/html"
]
)
# With defaults-level config (new in v2.0)
return PluginResult(
haproxy_config="acl from_cloudflare src -f /etc/haproxy/cloudflare_ips.lst",
defaults_configs=[
'log-format "%{+Q}[var(txn.real_ip)]:-/%ci:%cp [%tr] %ft %b/%s"'
]
)
# No operation (plugin disabled or no action needed)
return PluginResult()
```
@ -439,12 +545,13 @@ Manages plugin loading, configuration, and execution.
class PluginManager:
"""Manages plugin loading, configuration, and execution"""
def __init__(self, plugins_dir: str = "/etc/haproxy/plugins", abort_on_error: bool = False):
def __init__(self, plugins_dir: str | None = None, abort_on_error: bool = False):
"""
Initialize the plugin manager
Args:
plugins_dir: Directory containing plugin files
plugins_dir: Directory containing plugin files (defaults to
EASYHAPROXY_PLUGINS_DIR env var or /etc/haproxy/plugins)
abort_on_error: If True, abort on plugin errors; if False, log and continue
"""
@ -454,6 +561,9 @@ class PluginManager:
def configure_plugins(self, plugins_config: dict) -> None:
"""Configure all loaded plugins with their settings"""
def initialize_plugins(self) -> None:
"""[New in v2.0] Initialize all plugins and process resource requests"""
def execute_global_plugins(self, context: PluginContext, enabled_list: Optional[List[str]] = None) -> List[PluginResult]:
"""Execute all global plugins"""
@ -461,6 +571,10 @@ class PluginManager:
"""Execute all domain plugins for a specific domain"""
```
**Environment Variables:**
- `EASYHAPROXY_PLUGINS_DIR` - Override plugin directory (default: `/etc/haproxy/plugins`)
**Note:** You typically don't interact with PluginManager directly when writing plugins. It's used by EasyHAProxy core.
---
@ -626,7 +740,7 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from plugins import PluginInterface, PluginType, PluginContext, PluginResult
from functions import loggerEasyHaproxy
from functions import logger_easyhaproxy
class FastcgiPlugin(PluginInterface):
@ -720,11 +834,10 @@ class FastcgiPlugin(PluginInterface):
fcgi_app_definition = "\n".join(fcgi_app_lines)
# Build metadata - store fcgi_app_definition to be extracted and added to global configs
# Build metadata
metadata = {
"domain": context.domain,
"fcgi_app_name": fcgi_app_name,
"fcgi_app_definition": fcgi_app_definition, # For top-level injection
"document_root": self.document_root,
"index_file": self.index_file,
"path_info": self.path_info,
@ -734,7 +847,8 @@ class FastcgiPlugin(PluginInterface):
return PluginResult(
haproxy_config=backend_config, # use-fcgi-app directive for the backend
modified_easymapping=None,
metadata=metadata
metadata=metadata,
global_configs=[fcgi_app_definition] # For top-level injection (new in v2.0)
)
```
@ -794,8 +908,15 @@ import sys
# Add parent directory to path for imports
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from plugins import PluginInterface, PluginType, PluginContext, PluginResult
from functions import loggerEasyHaproxy
from plugins import (
InitializationResult,
PluginInterface,
PluginType,
PluginContext,
PluginResult,
ResourceRequest
)
from functions import Functions, logger_easyhaproxy
class JwtValidatorPlugin(PluginInterface):
@ -810,6 +931,8 @@ class JwtValidatorPlugin(PluginInterface):
self.pubkey = None # Public key content (alternative to pubkey_path)
self.paths = [] # List of paths that require JWT validation
self.only_paths = False # If true, only specified paths are accessible
# Make JWT_KEYS_DIR configurable via environment variable
self.jwt_keys_dir = os.getenv("EASYHAPROXY_JWT_KEYS_DIR", "/etc/haproxy/jwt_keys")
@property
def name(self) -> str:
@ -874,6 +997,19 @@ class JwtValidatorPlugin(PluginInterface):
if "only_paths" in config:
self.only_paths = str(config["only_paths"]).lower() in ["true", "1", "yes"]
def initialize(self) -> InitializationResult:
"""
Initialize plugin resources - create JWT keys directory (new in v2.0)
Returns:
InitializationResult with directory creation request
"""
return InitializationResult(
resources=[
ResourceRequest(resource_type="directory", path=self.jwt_keys_dir)
]
)
def process(self, context: PluginContext) -> PluginResult:
"""
Generate HAProxy config to validate JWT tokens
@ -893,9 +1029,17 @@ class JwtValidatorPlugin(PluginInterface):
elif self.pubkey:
# Generate path for pubkey based on domain
domain_safe = context.domain.replace(".", "_").replace(":", "_")
pubkey_file = f"/etc/haproxy/jwt_keys/{domain_safe}_pubkey.pem"
pubkey_file = f"{self.jwt_keys_dir}/{domain_safe}_pubkey.pem"
# Write the public key file (defensive - normally created by initialize())
try:
os.makedirs(self.jwt_keys_dir, exist_ok=True)
Functions.save(pubkey_file, self.pubkey)
logger_easyhaproxy.debug(f"Wrote JWT public key to {pubkey_file} for domain {context.domain}")
except (PermissionError, OSError) as e:
logger_easyhaproxy.debug(f"Could not write JWT public key file: {e}")
else:
loggerEasyHaproxy.warning(f"JWT validator plugin for {context.domain}: No pubkey or pubkey_path configured")
logger_easyhaproxy.warning(f"JWT validator plugin for {context.domain}: No pubkey or pubkey_path configured")
return PluginResult()
# Build HAProxy configuration
@ -1021,7 +1165,7 @@ import time
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from plugins import PluginInterface, PluginType, PluginContext, PluginResult
from functions import loggerEasyHaproxy
from functions import logger_easyhaproxy
class CleanupPlugin(PluginInterface):
@ -1057,7 +1201,7 @@ class CleanupPlugin(PluginInterface):
try:
self.max_idle_time = int(config["max_idle_time"])
except ValueError:
loggerEasyHaproxy.warning(f"Invalid max_idle_time value: {config['max_idle_time']}, using default")
logger_easyhaproxy.warning(f"Invalid max_idle_time value: {config['max_idle_time']}, using default")
if "cleanup_temp_files" in config:
self.cleanup_temp_files = str(config["cleanup_temp_files"]).lower() in ["true", "1", "yes"]
@ -1095,15 +1239,15 @@ class CleanupPlugin(PluginInterface):
if file_age > self.max_idle_time:
os.remove(filepath)
cleanup_actions.append(f"Removed old temp file: {filepath}")
loggerEasyHaproxy.debug(f"Cleanup plugin: Removed {filepath}")
logger_easyhaproxy.debug(f"Cleanup plugin: Removed {filepath}")
except Exception as e:
loggerEasyHaproxy.warning(f"Failed to remove temp file {filepath}: {e}")
logger_easyhaproxy.warning(f"Failed to remove temp file {filepath}: {e}")
except Exception as e:
loggerEasyHaproxy.warning(f"Failed to cleanup {temp_dir}: {e}")
logger_easyhaproxy.warning(f"Failed to cleanup {temp_dir}: {e}")
# Log cleanup summary
if cleanup_actions:
loggerEasyHaproxy.info(f"Cleanup plugin: Performed {len(cleanup_actions)} cleanup action(s)")
logger_easyhaproxy.info(f"Cleanup plugin: Performed {len(cleanup_actions)} cleanup action(s)")
return PluginResult(
haproxy_config="", # No HAProxy config needed for cleanup
@ -1117,9 +1261,104 @@ class CleanupPlugin(PluginInterface):
---
## Environment Variables
**New in v2.0:** Plugins can use environment variables for configuration.
### Core Environment Variables
- `EASYHAPROXY_PLUGINS_DIR` - Override plugin directory (default: `/etc/haproxy/plugins`)
- `EASYHAPROXY_PLUGINS_ENABLED` - Comma-separated list of enabled plugins
- `EASYHAPROXY_PLUGINS_ABORT_ON_ERROR` - Abort on plugin errors (default: `false`)
### Plugin-Specific Environment Variables
- `EASYHAPROXY_PLUGIN_<PLUGIN_NAME>_<CONFIG_KEY>` - Configure plugin settings
**Example:**
```bash
EASYHAPROXY_PLUGINS_ENABLED=jwt_validator,cloudflare
EASYHAPROXY_PLUGIN_JWT_VALIDATOR_ALGORITHM=RS256
EASYHAPROXY_PLUGIN_JWT_VALIDATOR_ISSUER=https://auth.example.com/
EASYHAPROXY_JWT_KEYS_DIR=/custom/path/jwt_keys # Plugin-defined env var
```
### Plugin Resource Directories
**New in v2.0:** Plugins can make their resource directories configurable via environment variables.
**Example:**
```python
class MyPlugin(PluginInterface):
def __init__(self):
# Make resource directory configurable
self.data_dir = os.getenv("EASYHAPROXY_MY_PLUGIN_DATA_DIR", "/etc/haproxy/my_plugin_data")
def initialize(self) -> InitializationResult:
return InitializationResult(
resources=[
ResourceRequest(resource_type="directory", path=self.data_dir)
]
)
```
---
## Best Practices
### 1. Error Handling
### 1. Use Plugin Initialization for Resource Setup
**[New in v2.0]** Use the `initialize()` method to request file system resources.
**Do:**
```python
def initialize(self) -> InitializationResult:
return InitializationResult(
resources=[
ResourceRequest(resource_type="directory", path=self.data_dir)
]
)
def process(self, context: PluginContext) -> PluginResult:
# Directory already exists, just use it
filepath = os.path.join(self.data_dir, "data.txt")
with open(filepath, 'w') as f:
f.write("data")
```
**Don't:**
```python
def process(self, context: PluginContext) -> PluginResult:
# Creating directories in process() is inefficient
os.makedirs(self.data_dir, exist_ok=True) # Called on every execution!
filepath = os.path.join(self.data_dir, "data.txt")
```
### 2. Use Typed Result Fields for Config Injection
**[New in v2.0]** Use `global_configs` and `defaults_configs` fields instead of metadata.
**Do:**
```python
return PluginResult(
haproxy_config="use-fcgi-app fcgi_example",
global_configs=["fcgi-app fcgi_example\n docroot /var/www"],
defaults_configs=['log-format "..."']
)
```
**Don't:**
```python
# Deprecated: Don't put config in metadata
return PluginResult(
haproxy_config="use-fcgi-app fcgi_example",
metadata={
"fcgi_app_definition": "fcgi-app fcgi_example\n docroot /var/www" # Wrong!
}
)
```
### 3. Error Handling
Always handle errors gracefully to avoid breaking HAProxy configuration.
@ -1130,7 +1369,7 @@ def configure(self, config: dict) -> None:
try:
self.port = int(config["port"])
except ValueError:
loggerEasyHaproxy.warning(f"Invalid port value: {config['port']}, using default")
logger_easyhaproxy.warning(f"Invalid port value: {config['port']}, using default")
self.port = 8080
```
@ -1140,7 +1379,7 @@ def configure(self, config: dict) -> None:
self.port = int(config["port"]) # Crashes if not an integer!
```
### 2. Configuration Validation
### 4. Configuration Validation
Validate configuration during `configure()` phase, not during `process()`.
@ -1153,7 +1392,7 @@ def configure(self, config: dict) -> None:
# Validate IPs
if not self.allowed_ips:
loggerEasyHaproxy.warning("IP whitelist plugin: No valid IPs configured")
logger_easyhaproxy.warning("IP whitelist plugin: No valid IPs configured")
self.enabled = False
```
@ -1165,7 +1404,7 @@ def process(self, context: PluginContext) -> PluginResult:
raise ValueError("No IPs configured")
```
### 3. Use Metadata for Debugging
### 5. Use Metadata for Debugging
Include useful debugging information in metadata.
@ -1182,7 +1421,7 @@ return PluginResult(
)
```
### 4. Handle Boolean Configuration
### 6. Handle Boolean Configuration
Support multiple boolean formats (true/false, 1/0, yes/no).
@ -1192,7 +1431,7 @@ def configure(self, config: dict) -> None:
self.enabled = str(config["enabled"]).lower() in ["true", "1", "yes"]
```
### 5. Support Multiple Configuration Formats
### 7. Support Multiple Configuration Formats
Support both list and comma-separated string formats for lists.
@ -1209,7 +1448,7 @@ def configure(self, config: dict) -> None:
self.paths = []
```
### 6. Use Descriptive Names
### 8. Use Descriptive Names
Use clear, descriptive names for plugins, configuration keys, and ACLs.
@ -1233,7 +1472,7 @@ def name(self) -> str:
acl p1 path_beg /api # What is p1?
```
### 7. Document Your Plugin
### 9. Document Your Plugin
Include comprehensive docstrings with configuration examples.
@ -1259,7 +1498,7 @@ Example Container Label:
"""
```
### 8. Return Empty Result When Disabled
### 10. Return Empty Result When Disabled
Always check `enabled` flag and return empty result early.
@ -1271,27 +1510,27 @@ def process(self, context: PluginContext) -> PluginResult:
# Plugin logic here...
```
### 9. Use Logger Appropriately
### 11. Use Logger Appropriately
Use appropriate log levels for different messages.
```python
from functions import loggerEasyHaproxy
from functions import logger_easyhaproxy
# For debugging
loggerEasyHaproxy.debug(f"Processing domain: {context.domain}")
logger_easyhaproxy.debug(f"Processing domain: {context.domain}")
# For informational messages
loggerEasyHaproxy.info(f"Loaded plugin configuration: {self.name}")
logger_easyhaproxy.info(f"Loaded plugin configuration: {self.name}")
# For warnings (non-fatal issues)
loggerEasyHaproxy.warning(f"Invalid configuration value, using default")
logger_easyhaproxy.warning(f"Invalid configuration value, using default")
# For errors (fatal issues)
loggerEasyHaproxy.error(f"Failed to load required file: {filepath}")
logger_easyhaproxy.error(f"Failed to load required file: {filepath}")
```
### 10. Make Domain-Safe Identifiers
### 12. Make Domain-Safe Identifiers
Replace special characters when generating HAProxy identifiers.
@ -1611,8 +1850,8 @@ docker-compose exec haproxy cat /etc/haproxy/haproxy.cfg | grep -A 5 "My Plugin"
2. **Add debug statements**
```python
def process(self, context: PluginContext) -> PluginResult:
loggerEasyHaproxy.debug(f"Plugin {self.name} processing domain: {context.domain}")
loggerEasyHaproxy.debug(f"Plugin config: enabled={self.enabled}, setting={self.my_setting}")
logger_easyhaproxy.debug(f"Plugin {self.name} processing domain: {context.domain}")
logger_easyhaproxy.debug(f"Plugin config: enabled={self.enabled}, setting={self.my_setting}")
# ... rest of plugin logic
```
@ -1630,7 +1869,7 @@ docker-compose exec haproxy cat /etc/haproxy/haproxy.cfg | grep -A 5 "My Plugin"
# Risky operation
result = self.do_something_risky()
except Exception as e:
loggerEasyHaproxy.error(f"Plugin {self.name} error: {str(e)}")
logger_easyhaproxy.error(f"Plugin {self.name} error: {str(e)}")
return PluginResult() # Return empty result on error
```