Joao Gilberto Magalhaes 711bb43572 Prepare version 6.0.0

2026-02-18 17:38:46 -05:00

16 KiB

Raw Blame History

sidebar_position
1

Kubernetes

Setup Kubernetes EasyHAProxy

:::info How it works EasyHAProxy for Kubernetes operates by querying all ingress definitions with either the spec.ingressClassName: easyhaproxy field (recommended) or the deprecated annotation kubernetes.io/ingress.class: easyhaproxy-ingress (for backward compatibility). Upon finding a matching ingress class, EasyHAProxy immediately sets up HAProxy and begins serving traffic. :::

For Kubernetes installations, there are three available installation modes:

DaemonSet: This mode exposes ports 80, 443, and 1936.
NodePort: Ports 31080, 31443, and 31936 are exposed.
ClusterIP: In this mode, no ports are exposed externally, and HAProxy is accessible only within the cluster.

To install EasyHAProxy in your Kubernetes cluster, follow these steps:

1) Identify the node where your EasyHAProxy container will run

:::warning Single Node Deployment EasyHAProxy will be limited to a single node. To understand why, see the limitations page. :::

$ kubectl get nodes

NAME      STATUS   ROLES    AGE    VERSION
node-01   Ready    <none>   561d   v1.21.13-3
node-02   Ready    <none>   561d   v1.21.13-3

Add the EasyHAProxy label to the node.

kubectl label nodes node-01 "easyhaproxy/node=master"

2) Install EasyHAProxy with Kubernetes Manifest

kubectl create namespace easyhaproxy

kubectl apply -f \
    https://raw.githubusercontent.com/byjg/docker-easy-haproxy/6.0.0/deploy/kubernetes/easyhaproxy-daemonset.yml

If necessary, you can configure environment variables. To get a list of the variables, please follow the environment variable guide

Running containers

Your container only requires creating an ingress with the spec.ingressClassName: easyhaproxy field pointing to your service.

e.g.

kind: Ingress
metadata:
  name: example-ingress
  namespace: example
spec:
  # Use ingressClassName (recommended)
  ingressClassName: easyhaproxy
  rules:
  - host: example.org
    http:
      paths:
      - backend:
          service:
            name: example-service
            port:
              number: 8080
        pathType: ImplementationSpecific

:::note Backward Compatibility The deprecated annotation kubernetes.io/ingress.class: easyhaproxy-ingress is still supported for backward compatibility, but spec.ingressClassName is the recommended approach for new deployments. :::

Once the container is running, EasyHAProxy will detect automatically and start to redirect all traffic from example.org:80 to your container at port 8080.

You don't need to expose any port in your container.

:::note Important Limitations

The implementation doesn't support all ingress properties or wildcard domains at this time.
The ingress will publish ports 80 and 443, plus 1936 if stats are enabled.
EasyHAProxy will read all spec.rules[].host specifications, however it will parse only the first path spec.rules[].http.paths[0].port.number for each rule, and ignore the other paths. :::

Kubernetes annotations

annotation	Description	Default	Example
kubernetes.io/ingress.class	(deprecated) Activate EasyHAProxy. Use `spec.ingressClassName` instead.	optional	easyhaproxy-ingress
easyhaproxy.redirect_ssl	(optional) Boolean. Force redirect all endpoints to HTTPS.	false	true or false
easyhaproxy.certbot	(optional) Boolean. It will request certbot certificates for the ingresses domains.	false	true or false
easyhaproxy.redirect	(optional) JSON. Key pair with a domain and its destination.	empty	{"domain":"redirect_url"}
easyhaproxy.mode	(optional) Set the HTTP mode for that connection.	http	http or tcp
easyhaproxy.listen_port	(optional) Override the HTTP listen port created for that ingress	80	8081
easyhaproxy.plugins	(optional) Comma-separated list of plugins to enable for this ingress	empty	cloudflare,deny_pages
easyhaproxy.plugin.`{name}`.`{key}`	(optional) Plugin-specific configuration (see Using Plugins)	varies	See examples below

Important: The annotations are per ingress and applied to all hosts in that ingress configuration.

Using Plugins with Kubernetes

Plugins extend HAProxy configuration with additional functionality like JWT validation, IP whitelisting, or Cloudflare IP restoration. For a complete list of available plugins, see the Using Plugins guide.

Enabling Plugins for an Ingress

Add the easyhaproxy.plugins annotation with a comma-separated list of plugin names:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    easyhaproxy.plugins: "cloudflare,deny_pages"
  name: example-ingress
  namespace: example
spec:
  ingressClassName: easyhaproxy
  rules:
  - host: example.org
    http:
      paths:
      - backend:
          service:
            name: example-service
            port:
              number: 8080
        pathType: ImplementationSpecific

Configuring Plugin Options

Use easyhaproxy.plugin.{plugin_name}.{option} annotations to configure individual plugins:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    easyhaproxy.plugins: "deny_pages"
    easyhaproxy.plugin.deny_pages.paths: "/admin,/private,/config"
    easyhaproxy.plugin.deny_pages.status_code: "403"
  name: secure-app-ingress
  namespace: production
spec:
  ingressClassName: easyhaproxy
  rules:
  - host: myapp.example.com
    http:
      paths:
      - backend:
          service:
            name: myapp-service
            port:
              number: 8080
        pathType: ImplementationSpecific

Common Plugin Examples

Protect API with JWT validation:

metadata:
  annotations:
    easyhaproxy.plugins: "jwt_validator"
    easyhaproxy.plugin.jwt_validator.algorithm: "RS256"
    easyhaproxy.plugin.jwt_validator.issuer: "https://auth.example.com/"
    easyhaproxy.plugin.jwt_validator.audience: "https://api.example.com"
    easyhaproxy.plugin.jwt_validator.pubkey_path: "/etc/easyhaproxy/jwt_keys/api_pubkey.pem"
spec:
  ingressClassName: easyhaproxy

Note: For JWT validation, you'll need to mount the public key file into the EasyHAProxy pod. See Using Plugins for details.

Restrict access to specific IPs:

metadata:
  annotations:
    easyhaproxy.plugins: "ip_whitelist"
    easyhaproxy.plugin.ip_whitelist.allowed_ips: "192.168.1.0/24,10.0.0.5"
    easyhaproxy.plugin.ip_whitelist.status_code: "403"
spec:
  ingressClassName: easyhaproxy

Restore Cloudflare visitor IPs:

metadata:
  annotations:
    easyhaproxy.plugins: "cloudflare"
spec:
  ingressClassName: easyhaproxy

Multiple plugins together:

metadata:
  annotations:
    easyhaproxy.plugins: "cloudflare,deny_pages"
    easyhaproxy.plugin.deny_pages.paths: "/wp-admin,/wp-login.php"
    easyhaproxy.plugin.deny_pages.status_code: "404"
spec:
  ingressClassName: easyhaproxy

Global Plugin Configuration

Some plugins (like cleanup) are global and execute once per discovery cycle. Configure these via environment variables or YAML configuration:

Using Helm values.yaml:

easyhaproxy:
  plugins:
    enabled: cleanup
    config:
      cleanup:
        max_idle_time: 600

Using environment variables:

env:
  - name: EASYHAPROXY_PLUGINS_ENABLED
    value: "cleanup"
  - name: EASYHAPROXY_PLUGIN_CLEANUP_MAX_IDLE_TIME
    value: "600"

For more information on plugin types and available plugins, see the Using Plugins guide.

Loading Plugin Configuration from Kubernetes Secrets

EasyHAProxy supports loading sensitive plugin configuration values directly from Kubernetes Secrets using the k8s_secret pattern. This is a generic, plugin-agnostic feature that works with any plugin.

Why Use Kubernetes Secrets?

Security: Keep sensitive data (API keys, passwords, certificates) out of annotations
Best practices: Follows Kubernetes conventions for managing sensitive data
Simplicity: No need to mount volumes or ConfigMaps for secret data
Encryption: Secrets are encrypted at rest in etcd

Annotation Format

# Auto-detect key (tries common variations):
easyhaproxy.plugin.{plugin_name}.k8s_secret.{config_key}: "secret_name"

# Explicit key (no variations):
easyhaproxy.plugin.{plugin_name}.k8s_secret.{config_key}: "secret_name/key_name"

How It Works

You create a Kubernetes Secret with your sensitive data
You reference the secret in your ingress annotation using the k8s_secret pattern
EasyHAProxy reads the secret from the same namespace as the ingress
EasyHAProxy transforms the annotation to inject the secret value
The plugin receives the value as if it was provided directly in the annotation

Example transformation:

# Input annotation:
easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "my-jwt-secret"

# EasyHAProxy reads the secret and transforms to:
easyhaproxy.plugin.jwt_validator.pubkey: "<base64-encoded-content>"

Auto-Detect vs Explicit Key

Auto-Detect Key Format

When you use "secret_name" (without /), EasyHAProxy tries to find the key automatically:

easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "my-jwt-secret"

EasyHAProxy will try these keys in order:

Exact match: pubkey
Common variations based on the config key name

Auto-detect key variations:

Config Key	Tries (in order)
`pubkey`	`pubkey`, `public-key`, `jwt.pub`, `tls.crt`
`password`	`password`, `pass`, `pwd`
`api_key`	`api_key`, `apikey`, `api-key`, `key`

Explicit Key Format

When you use "secret_name/key_name" (with /), EasyHAProxy only tries the exact key name:

easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "my-jwt-secret/rsa-public-key"

EasyHAProxy will only try: rsa-public-key (no variations)

Use explicit key when:

Your secret uses a non-standard key name
You want to be explicit and avoid ambiguity
Multiple keys exist in the secret

Complete Example

---
# 1. Create a secret with your JWT public key
apiVersion: v1
kind: Secret
metadata:
  name: jwt-pubkey-secret
  namespace: production
type: Opaque
stringData:
  pubkey: |
    -----BEGIN PUBLIC KEY-----
    MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA...
    -----END PUBLIC KEY-----

---
# 2. Reference it in your ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: production
  annotations:
    easyhaproxy.plugins: "jwt_validator"
    easyhaproxy.plugin.jwt_validator.algorithm: "RS256"
    easyhaproxy.plugin.jwt_validator.issuer: "https://auth.example.com/"
    easyhaproxy.plugin.jwt_validator.audience: "https://api.example.com"
    # Load pubkey from Kubernetes secret (auto-detect key)
    easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "jwt-pubkey-secret"
spec:
  ingressClassName: easyhaproxy
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

Example with Explicit Key Name

---
apiVersion: v1
kind: Secret
metadata:
  name: app-credentials
  namespace: production
type: Opaque
stringData:
  # Custom key name
  rsa-public-key: |
    -----BEGIN PUBLIC KEY-----
    ...
    -----END PUBLIC KEY-----

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: production
  annotations:
    easyhaproxy.plugins: "jwt_validator"
    # Use explicit key name after the slash
    easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "app-credentials/rsa-public-key"
spec:
  ingressClassName: easyhaproxy
  # ... rest of configuration

Using with Any Plugin

The k8s_secret pattern works with any plugin configuration:

# JWT Validator - load public key
easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "jwt-secret"

# Hypothetical API auth plugin - load API key
easyhaproxy.plugin.api_auth.k8s_secret.api_key: "api-credentials/key"

# Hypothetical basic auth plugin - load password
easyhaproxy.plugin.basic_auth.k8s_secret.password: "auth-secret/pwd"

Priority Order

When multiple configuration methods are used, this is the priority (highest to lowest):

Explicit annotation (e.g., easyhaproxy.plugin.jwt_validator.pubkey: "value")
k8s_secret annotation (e.g., easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey: "secret")

Explicit annotations always take precedence over k8s_secret annotations.

Troubleshooting

Secret not found:

WARNING: Ingress production/api-ingress - Failed to process k8s_secret annotation
'easyhaproxy.plugin.jwt_validator.k8s_secret.pubkey' with value 'jwt-secret': ...

Verify the secret exists: kubectl get secret jwt-secret -n production
Check the secret is in the same namespace as the ingress

Key not found in secret:

WARNING: Ingress production/api-ingress - Secret 'jwt-secret' found but no matching
key (tried: pubkey, public-key, jwt.pub, tls.crt)

List secret keys: kubectl get secret jwt-secret -n production -o jsonpath='{.data}'
Use explicit key format: "jwt-secret/actual-key-name"

Check EasyHAProxy logs:

kubectl logs -n easyhaproxy -l app=easyhaproxy --tail=100

Look for:

INFO: Loaded 'pubkey' from secret 'jwt-secret' (success)
WARNING: Secret 'xyz' found but no matching key (key not found)

Security Considerations

Secrets are read from the same namespace as the ingress (no cross-namespace access)
EasyHAProxy needs RBAC permissions to read secrets (included in default deployment)
Secrets are encrypted at rest in etcd
Secret values are base64-encoded by Kubernetes automatically
Use Kubernetes RBAC to control which service accounts can read which secrets

Certbot / ACME / Letsencrypt

It is necessary to add the annotation easyhaproxy.certbot to the ingress configuration:

kind: Ingress
metadata:
  annotations:
    easyhaproxy.certbot: 'true'
  name: example-ingress
  namespace: example
spec:
  ingressClassName: easyhaproxy
  ....

More info here.

Make sure your cluster is accessible both through ports 80 and 443.

Custom SSL Certificates

Create a secret with your certificate and key and associate them with your ingress.

---
apiVersion: v1
kind: Secret
metadata:
  name: host2-tls
  namespace: default
data:
  tls.crt: base64 of your certificate
  tls.key: base64 of your certificate private key
type: kubernetes.io/tls

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-example
  namespace: default
spec:
  ingressClassName: easyhaproxy
  tls:
  - hosts:
      - host2.local
    secretName: host2-tls
  rules:
    ...

Open source ByJG

16 KiB Raw Blame History