- Implement setuptools-scm for dynamic version management from git tags
- Refactor CI/CD into separate build and deploy jobs with artifact sharing
- Add versioned releases with timestamp-based deployment directories
- Implement health checks and automatic rollback on deployment failure
- Extract deployment logic into reusable shell scripts
- Add Docker layer caching to speed up builds
- Include version info in Django context and build args
This commit updates the `trade_acceptance.html` template partial to include the `cache` templatetag in its `{% load %}` directive, preventing a missing tag error.
This commit comprehensively addresses issues with the local development and debugging environment, ensuring a smoother and more reliable developer experience.
Key changes include:
- **VSCode Debugger:** Corrected launch configuration (`.vscode/launch.json`) to properly run the Django development server with debugging enabled. It now correctly sets `DEBUG=True`, uses `0.0.0.0:8000`, and specifies the correct working directory.
- **Docker Compose:** Exposed the PostgreSQL port (`5432:5432`) in `docker-compose.yml` to allow direct connections from the host, facilitating local development and debugging without needing to run the full application stack.
- **Environment Variables:**
- Updated `.gitignore` to ignore all `.env.*` files, allowing for environment-specific configurations.
- Modified `src/pkmntrade_club/django_project/settings.py` to use `localhost` for `DJANGO_DATABASE_URL` and `REDIS_URL` by default, aligning with the exposed Docker services for easier local development. Default `DISABLE_SIGNUPS` and `DISABLE_CACHE` are now `True` for a more typical local dev setup.
- **Management Commands & Scripts:**
- Adjusted `manage.py` to correctly append the project's root directory to `sys.path`, resolving potential import issues when running management commands.
- Significantly improved `scripts/reset-db_make-migrations_seed-data.sh`:
- Removed reliance on sourcing `.env` directly.
- Ensured the database service (`db`) is started independently before migrations.
- Added explicit steps for running `prebuild.sh`, migrations, and `collectstatic`.
- Switched to using `uv run manage.py loaddata` for seeding, which is more consistent with the project's tooling.
- **Django Settings:** Added `SESSION_ENGINE = "django.contrib.sessions.backends.signed_cookies"` and `SESSION_COOKIE_HTTPONLY = True` for improved session management and security.
These changes collectively fix the previously problematic development setup, enabling straightforward debugging and a more efficient workflow for local development.
This commit significantly improves the local development experience by enabling hot reloading for the Django application. This is achieved by installing the project as an editable package within the Docker services.
Key changes:
- **Hot Reloading:**
- Modified `docker-compose.yml` for `web` and `celery` services to use `uv pip install --editable . --no-deps`.
- Mounted the project root (`./`) to `/code` in `web` and `celery` services to facilitate the editable install.
- **Docker & Build Enhancements:**
- Added `uv` binary to stage-1 in the `Dockerfile` for faster package operations.
- Adjusted file permissions in `Dockerfile` during the app copy.
- Set `DEBUG=true` for the `web` service in `docker-compose.yml` for easier local debugging.
- Changed `restart` policy to `unless-stopped` for `web` and `celery` dev services.
- Added a healthcheck for the `redis` service in the dev `docker-compose.yml`.
- **Code & Script Cleanup:**
- Removed the custom `HealthCheckView` from the `home` app, as health checks are now handled by django-health-checks.
- Updated paths and commands in `scripts/entrypoint.sh`, `scripts/prebuild.sh`, and `scripts/reset-db_make-migrations_seed-data.sh` to align with the new setup and remove obsolete steps (e.g., db cache table creation; we now use redis).
This commit significantly refactors the Docker setup and application
configuration for improved robustness and flexibility.
Key changes include:
- Centralized Environment Variables:
- Default values for essential settings (database, email, cache, etc.)
are now defined in `django_project/settings.py` using `environ.Env`.
This provides sensible defaults and reduces reliance on `.env` files,
especially during Docker image builds.
- `docker-compose.yml` no longer defines environment variables directly
for `web` and `worker` services, deferring to `.env` and settings defaults.
- Dockerfile & Entrypoint Improvements:
- `DJANGO_SETTINGS_MODULE` is now exclusively set as an ENV in `Dockerfile`, instead of setting it in `entrypoint.sh`
- `entrypoint.sh` now conditionally appends `--static-path-mount`
only to the `granian` command, leveraging the upgraded Granian's
(v2.3.0+) ability to serve static files directly. The `STATIC_ROOT` is
dynamically fetched from Django settings.
- Dependency Updates:
- Upgraded `granian` from 2.2.5 to 2.3.1.
- Upgraded `click` from 8.2.0 to 8.2.1.
- `uv.lock` reflects these and other minor transitive dependency updates.
- Configuration Adjustments in `settings.py`:
- Add defaults for all env variables, and set to default local dev settings
- Introduced a `SCHEME` environment variable (defaulting to 'http')
used for `CSRF_TRUSTED_ORIGINS`, `META_SITE_PROTOCOL`,
`ACCOUNT_DEFAULT_HTTP_PROTOCOL`, etc.
- `TIME_ZONE` and various email settings (host, port, user, password, TLS)
are now configurable via environment variables with defaults.
- `CELERY_TIMEZONE` now defaults to the `TIME_ZONE` setting.
- Removed the unused `SCW_SECRET_KEY` variable (previously used for
EMAIL auth).
The deploy.sh script is now re-added to the entrypoint.sh script
to ensure it runs only during first container startup.
A flag file (/flags/.deployed) is now created after a successful deployment.
The deploy.sh script checks for this flag and will not re-run
deployment steps unless FORCE_DEPLOY is set to true. This prevents
unnecessary re-runs of migrations, collectstatic, etc., on subsequent
container starts within the same deployment.
Corrected permissions for `/app/.cursor-server` and created a `/flags`
directory with appropriate permissions in the `Dockerfile`. Added
ENV DJANGO_SETTINGS_MODULE with default value to `Dockerfile`.
This commit significantly improves the gatekeeper system's robustness, monitoring capabilities, and simplifies host header management for backend services.
Key changes include:
**Gatekeeper Health, Management & Resilience:**
- Implemented active health checking for individual gatekeeper containers within the `gatekeeper-manager` service.
- The manager now periodically curls the `/metrics` endpoint of each gatekeeper container.
- Reports health status to a new Gatus `services_gatekeeper` endpoint.
- Automatically attempts to restart the gatekeeper stack if any gatekeeper instance is unhealthy or if the expected number of gatekeepers is not running.
- Refactored the `gatekeeper-manager` shell script for improved state management and signal handling:
- Introduced `STARTED`, `RESTARTING`, `TERMINATING` state flags for more controlled operations.
- Enhanced SIGTERM and SIGHUP handling to gracefully manage gatekeeper lifecycles.
- Added `apk add curl` to ensure `curl` is available in the manager container.
- Renamed the gatekeeper Docker Compose template from `docker-compose_gatekeeper.template.yml` to `gatekeepers.template.yml` and its output to `gatekeepers.yml`.
- Updated `dockergen-gatekeeper` to watch the new template file and notify the correct `gatekeeper-manager` service instance (e.g., `pkmntrade-club-gatekeeper-manager-1`).
- Discover services that should be protected by looking for a `gatekeeper=true` label.
**Host Header Management & `ALLOWED_HOSTS` Simplification:**
- HAProxy configuration (`haproxy.cfg`) now consistently sets the `Host` HTTP header for requests to all backend services (e.g., `pkmntrade.club`, `staging.pkmntrade.club`). This centralizes and standardizes host information.
- Consequently, explicit `ALLOWED_HOSTS` environment variables have been removed from the `web` and `celery` service definitions in `docker-compose_web.yml` and `docker-compose_staging.yml`. Backend Django applications should now rely on the `Host` header set by HAProxy for request validation.
- The `gatekeepers.template.yml` now defines a `TARGET_HOST` environment variable for proxied services (e.g., `web`, `web-staging`). This aligns with the ALLOWED_HOSTS on the target to ensure requests aren't blocked.
**Gatus Monitoring & Configuration Updates:**
- In Gatus configuration (`gatus/config.template.yaml`):
- The "Redis" external service endpoint has been renamed to "Cache" for better clarity and to fit the theme of simple names.
- A new external service endpoint "Gatekeeper" has been added to monitor the overall health reported by the `gatekeeper-manager`.
- Health checks for "Web Worker" endpoints (both main and staging) now include the appropriate `Host` header (e.g., `Host: pkmntrade.club`) to ensure accurate health assessments by Django.
- In `docker-compose_core.yml`, the `curl` commands used by `db-redis-healthcheck` for database and cache health now append `|| true`. This prevents the script from exiting on a curl error (e.g., timeout, connection refused), ensuring that the failure is still reported to Gatus via the `success=false` parameter rather than the script terminating prematurely.
These changes collectively make the gatekeeper system more fault-tolerant, provide better visibility into its status, and streamline the configuration of backend applications by standardizing how they receive host information.
- **Implemented Dynamic Gatekeeper (Anubis) Proxy:**
- Introduced Anubis as a Gatekeeper proxy layer for services (`web`, `web-staging`, `feedback`, `health`).
- Added `docker-gen` setup (`docker-compose_gatekeeper.template.yml`, `gatekeeper-manager`) to dynamically configure Anubis instances based on container labels (`enable_gatekeeper=true`).
- Updated HAProxy to route traffic through the respective Gatekeeper services.
- **Enhanced Service Health Monitoring & Checks:**
- Integrated `django-health-check` into the Django application, providing detailed health endpoints (e.g., `/health/`).
- Replaced the custom health check view with `django-health-check` URLs.
- Added `psutil` for system metrics in health checks.
- Made Gatus configuration dynamic using `docker-gen` (`config.template.yaml`), allowing automatic discovery and monitoring of service instances (e.g., web workers).
- Externalized Gatus SMTP credentials to environment variables.
- Strengthened `docker-compose_core.yml` with a combined `db-redis-healthcheck` service reporting to Gatus.
- Added explicit health checks for `db` and `redis` services in `docker-compose.yml`.
- **Improved Docker & Compose Configuration:**
- Added `depends_on` conditions in `docker-compose.yml` for `web` and `celery` services to wait for the database.
- Updated `ALLOWED_HOSTS` in `docker-compose_staging.yml` and `docker-compose_web.yml` to include internal container names for Gatekeeper communication.
- Set `DEBUG=False` for staging services.
- Removed `.env.production` from `.gitignore` (standardized to `.env`).
- Streamlined `scripts/entrypoint.sh` by removing the call to the no-longer-present `/deploy.sh`.
- **Dependency Updates:**
- Added `django-health-check>=3.18.3` and `psutil>=7.0.0` to `pyproject.toml` and `uv.lock`.
- Updated `settings.py` to include `health_check` apps, configuration, and use `REDIS_URL` consistently.
- **Streamlined deployment script used in GHA:**
- Updated the workflow to copy new server files and create a new `.env` file in the temporary directory before moving them into place.
- Consolidated the stopping and removal of old containers into a single step for better clarity and efficiency.
- Reduce container downtime by rearranging stop/start steps.
- Introduced `django-linear-migrations` version 2.17.0 to manage migrations more effectively and prevent merges.
- Refactored `INSTALLED_APPS` in `settings.py` to allow linear-migrations to track first party apps explicitly.
- Updated `card_badge.html` to adjust width properties for better layout consistency.
- Modified `trade_offer_png.html` to change padding for improved visual appearance.
- Enhanced `bubble_up_trade_offer_updates` signal to delete cached images when related instances change, ensuring up-to-date content.
- Updated `TradeOfferPNGView` to pass the request context when rendering the template, improving compatibility with Django's template rendering.
- Refactored `render_trade_offer_png` to use constants for dimensions and improve readability, while also updating context handling for better integration.
- Dev and prod both now run with the same exact docker configuration, except the .env file copied in.
- Removed the `.env` file and added a new `.env.dev` file for development settings, including database configuration and API keys.
- Introduced a `.envrc` file for automatic venv activation.
- Updated `deploy.sh` to utilize `uv` for running management commands and added a command for building Tailwind CSS.
- Created `docker-compose.yml` for local development with PostgreSQL, ensuring proper service dependencies.
- Deleted unnecessary files such as `docker-compose_db_only.yml` and `requirements.txt` to streamline the project structure.
- Modified Docker commands in `.vscode/tasks.json` to remove the `-d` flag for running services, allowing for better visibility of logs during development.
- Adjusted `entrypoint.sh` to streamline the startup process by running the development server and npm concurrently, enhancing the development workflow.
- Updated `card_badge` and `card_multiselect` template tags to utilize `reverse_lazy` for URL resolution.
- Enhanced caching mechanisms in `card_badge.html` and `card_multiselect.html` to improve performance.
- Introduced a new template `_card_multiselect_options.html` for rendering multiselect options.
- Improved context management in `card_multiselect` to handle selected cards and dynamic placeholders.
- Added error handling for query hashing in `card_multiselect` to ensure robustness.
- Updated `trade_offer_tags` to optimize database queries using `select_related` for related objects.
