Automated Docker Deployment for Quartz Site

Summary: Creating a robust, automated system for deploying a Quartz static site using Docker. The final solution involves a multi-stage Docker build, a shell script to prepare content from an Obsidian vault and manage deployments with Docker Compose, and a cron job for periodic, change-aware updates, successfully overcoming initial challenges with host mounts and build environment issues.

Git Repo: https://git.c0smere.net/wes/digital_garden

Key Topics & Discussion Points

Initial Goal & Challenges

• Objective: Host a Quartz static site within a Docker container.
• Initial Approach (Abandoned): Using Docker host bind mounts for site files.
  • Problems Encountered: Complex file permission issues (NGINX 403 errors after rebuilds on host), and potential conflicts with host security modules like AppArmor/SELinux when files were modified outside the container. This led to the decision to move to a fully containerized build and deployment.

Transition to Multi-Stage Docker Builds

• Rationale: To create a self-contained, portable image with the Quartz site and NGINX, avoiding host filesystem dependencies and permission issues for the served content.
• Dockerfile Structure:
  • Builder Stage (node:lts-slim):
    • Chosen over node:lts-alpine to resolve an /usr/bin/env: unrecognized option: S error encountered during npx quartz build (Alpine’s env doesn’t support the -S flag used by some Node scripts/npx).
    • WORKDIR /app
    • COPY package*.json ./
    • RUN npm ci --omit=dev (or yarn install) for dependencies.
    • COPY . . to copy the entire Quartz project (including the ./content directory prepared by the host script) into the builder.
    • RUN npx quartz build to compile the Quartz site into /app/public.
  • NGINX Stage (nginx:alpine):
    • RUN rm /etc/nginx/conf.d/default.conf to remove the default NGINX site.
    • COPY custom_nginx.conf /etc/nginx/conf.d/default.conf to install the custom NGINX configuration.
    • COPY --from=builder /app/public /usr/share/nginx/html to copy only the built static site from the builder stage.
    • EXPOSE 80
    • CMD ["nginx", "-g", "daemon off;"]
• Custom NGINX Configuration (custom_nginx.conf):
  • Necessary for handling Quartz’s “clean URLs” (links without .html extensions).
  • Key directive: location / { try_files $uri $uri.html $uri/ =404; }
  • Sets root /usr/share/nginx/html; and index index.html;.

Addressing Symlinked content Directory

• Problem Identified: The user’s local Quartz content directory was a symlink to a folder within their Obsidian vault. The COPY . . Docker instruction copies the symlink itself, which becomes broken inside the container as its target is outside the Docker build context. This resulted in npx quartz build finding no content and not generating an index.html in its output.
• Solution: Prepare the content on the host before each Docker build by copying the actual files from the Obsidian vault into a real ./content directory within the Docker build context. This is handled by the update_quartz_docker.sh script.

Automated Rebuild and Deployment Strategy

• Requirement: Avoid rebuilding the Docker image on every single file change from Obsidian Sync, which could be very frequent. Instead, use a periodic check.
• Solution: A cron job executes a master shell script (update_quartz_docker.sh) at a defined interval (e.g., every 10-15 minutes).
• update_quartz_docker.sh Script Functionality:
  • Configuration: Defines paths for the Quartz project directory and the Obsidian content source directory.
  • Change Detection:
    • Maintains a timestamp file (e.g., .last_build_content_timestamp) in the project directory.
    • Before attempting a build, it checks if any files in the OBSIDIAN_CONTENT_SOURCE directory are newer than this timestamp file.
    • If no changes are detected, the script exits early, avoiding unnecessary builds.
  • Content Preparation (if changes detected):
    1. Removes any existing ./content directory or symlink within the Quartz project folder.
    2. Creates a new, empty ./content directory.
    3. Uses rsync -a --delete to copy the actual content from the OBSIDIAN_CONTENT_SOURCE path into the newly created ./content directory. This ensures the Docker build context has the real files.
  • Docker Deployment (using Docker Compose):
    1. docker-compose down --remove-orphans --volumes: Added to ensure a clean state before each deployment, resolving a KeyError: 'ContainerConfig' issue encountered previously.
    2. docker-compose up -d --build --remove-orphans: This command tells Docker Compose to:
       • --build: Rebuild the image using the Dockerfile (which now has access to the prepared ./content).
       • -d: Run in detached mode.
       • up: Create and start containers, recreating them if necessary.
       • --remove-orphans: Cleans up any containers for services no longer in the docker-compose.yml.
  • Timestamp Update: On successful deployment, the script updates the .last_build_content_timestamp file.
  • Logging: The script includes echo statements for logging its actions, typically redirected to a log file by the cron job.
  • Cleanup (Optional): The script can be configured to remove the copied ./content directory after the build and restore the symlink if desired for local development workflows.
• docker-compose.yml File:
  • Defines the quartz-site service.
  • build: .: Instructs Compose to build from the Dockerfile in the current directory.
  • image: my-quartz-site-prod: Names the resulting image.
  • container_name: quartz-prod-container: Assigns a specific name to the container.
  • ports: - "18000:80": Maps port 18000 on the host to port 80 in the container.
  • restart: unless-stopped: Ensures the container restarts automatically unless manually stopped.

Key Files and Configurations

1. Dockerfile:

# Stage 1: Build the Quartz site
FROM node:lts-slim AS builder

# Create a non-root user to own files and run npm commands
USER node
WORKDIR /home/node/app

# Copy the locally cloned Quartz source code from the build context
# This 'quartz' dir is expected to be in the build context root,
# prepared by the update_quartz_docker.sh script.
COPY --chown=node:node quartz/ .

# Install dependencies based on the cloned Quartz source's package.json
RUN npm ci

# Now, copy your specific custom configuration files and the prepared content
# from the root of your deployment repo (the build context's root)
# into the current WORKDIR (/home/node/app), overwriting any placeholders
# from the cloned Quartz source if they share the same names/paths.
COPY --chown=node:node quartz.config.ts ./quartz.config.ts
COPY --chown=node:node quartz.layout.ts ./quartz.layout.ts

# If you have other custom files/dirs at the root of your deployment repo that Quartz needs:
# e.g., custom assets not part of Obsidian content, or custom components to overlay
# COPY --chown=node:node assets/ ./assets/
# COPY --chown=node:node components/ ./components/

# Copy the Obsidian content prepared by the host script into ./content
COPY --chown=node:node content/ ./content/

# Build the Quartz site
RUN npx quartz build
# Output is expected in /home/node/app/public

# Stage 2: Serve the static files with NGINX
FROM nginx:alpine

RUN rm /etc/nginx/conf.d/default.conf
COPY custom_nginx.conf /etc/nginx/conf.d/default.conf

# Adjust source path for public files from the builder stage
COPY --from=builder /home/node/app/public /usr/share/nginx/html

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

2. custom_nginx.conf:

server {
    listen 80;
    server_name localhost; # Or your actual domain
    root /usr/share/nginx/html;
    index index.html;

    location / {
        try_files $uri $uri.html $uri/ =404;
    }
}

3. docker-compose.yml:

version: '3.8'

services:
  quartz-site:
    build: .
    image: my-quartz-site-prod
    container_name: quartz-prod-container
    ports:
      - "18000:80"
    restart: unless-stopped

4. update_quartz_docker.sh:

This one is a bit long to display here: update_quartz_docker.sh

cron support

cron jobs typically run with a very minimal PATH you’ll probably need to use the full path to invoke docker-compose within the shell script in order for cron to successfully call it.

Relevant line in script: DOCKER_COMPOSE_CMD="/usr/local/bin/docker-compose"

Decisions Made

1. Abandoned Host Mounts: Switched from using Docker host bind mounts for Quartz content to a fully containerized build due to persistent permission and security module (AppArmor/SELinux) complexities.
2. Adopted Multi-Stage Docker Builds: Implemented a Dockerfile with a Node.js builder stage for npx quartz build and an NGINX runtime stage.
3. Switched Node Base Image: Changed from node:lts-alpine to node:lts-slim in the builder stage to resolve the /usr/bin/env: unrecognized option: S error.
4. Content Handling for Symlinks: Decided to handle the symlinked content directory by having a host-side script prepare the actual content within the Docker build context before each build. The RUN --mount BuildKit approach was explored but ultimately bypassed in favor of the more portable host script method due to complexities and errors.
5. Automated Deployment Strategy: Implemented a shell script (update_quartz_docker.sh) to manage content preparation, Docker image building, and container deployment.
6. Deployment Tool: Chose docker-compose within the script for its ease of managing the application lifecycle (build, up, down).
7. Change Detection Mechanism: The script incorporates a timestamp-based check to only rebuild and redeploy if actual changes are detected in the Obsidian source content directory. Important to note that this doesn’t capture folder name changes. Ultimately that doesn’t really matter much to me, the folder name change will eventually be captured when something else changes (markdown files, config files, etc.)
8. Periodic Execution: The update_quartz_docker.sh script is intended to be run by a cron job.
9. Docker Compose Error Resolution: Added docker-compose down --remove-orphans --volumes to the deployment script before docker-compose up ... to resolve a KeyError: 'ContainerConfig' and ensure a clean deployment state.