Publish isolated static artifacts on one host #3

Closed
architect wants to merge 1 commits from architect/site-publish-surfaces into main
10 changed files with 1032 additions and 109 deletions
Showing only changes of commit 7aca96e320 - Show all commits
+16
View File
@@ -0,0 +1,16 @@
name: Test
on:
pull_request:
push:
branches: [main]
jobs:
test:
runs-on: fritzlab
steps:
- uses: actions/checkout@v4
- name: Install test dependencies
run: python3 -m pip install --quiet --break-system-packages jinja2 pyyaml
- name: Run tests
run: python3 -m unittest discover -s tests -v
+77 -12
View File
@@ -1,9 +1,10 @@
# action/site-publish
Composite Gitea Action that publishes a **static-content** website to the
fritzlab k8s cluster. Supports `static`, `hugo`, and `mkdocs`. Content goes
to a Garage S3 bucket; Traefik fronts the bucket via an `ExternalName`
Service with cert-manager TLS.
Composite Gitea Action that publishes static content to the fritzlab cluster.
The legacy form supports one `static`, `hugo`, or `mkdocs` output. The
multi-artifact form publishes caller-built outputs to separate Garage buckets
and gives each URL path its own cache, CORS, credential, and middleware
boundary. Traefik fronts the buckets and cert-manager owns TLS.
> **Containerized web apps (Dockerfile-based) are NOT handled here.** Use the
> standard image-producer chain instead:
@@ -24,6 +25,11 @@ as a Garage `globalAlias` on the bucket and adds it to the Ingress + Certificate
on every deploy. Manual edits to manifests in the apps repo are clobbered;
edit `site.yaml` instead.
New sites with more than one security or caching boundary use the
multi-artifact form below. An artifact is storage and release metadata. A
route is edge behavior. Keeping them separate prevents a public path from
inheriting the authenticated catalogue's cache or credentials.
## Usage
Scaffold a new site (handles repo creation + Garage bucket):
@@ -55,6 +61,65 @@ type: static # static | hugo | mkdocs
# # (also requires an Authentik proxy provider + app for the host).
```
### Multiple artifacts on one host
Build each output before invoking the action, then declare the materialized
directories and their routes:
```yaml
domain: baseline.fritzlab.net
artifacts:
catalogue:
source: apps/catalogue/build
bucket: baseline-catalogue
credential: catalogue
cache:
default: private, no-store
dist:
source: dist
bucket: baseline-dist
credential: dist
cache:
default: public, max-age=0, must-revalidate, no-transform
rules:
- match: releases/*
value: public, max-age=31536000, immutable, no-transform
cors_origins: ["*"]
routes:
- name: catalogue
path: /
artifact: catalogue
access: protected
- name: dist
path: /dist
artifact: dist
access: public
```
`source` is a repository-relative directory and cannot be the repository
root. Every artifact needs an explicit, unique bucket and at least one route.
Routes use Kubernetes `Prefix` matching. The route path becomes part of the
published object key: `dist/baseline.css` is served at `/dist/baseline.css`.
`access` is required. Protected routes receive the organization Authentik
middleware automatically and their artifacts must use `private` or `no-store`
cache metadata. Public and protected routes cannot share an artifact. Cache
rules use aws-cli include patterns in order after the default metadata pass;
use immutable caching only for content-addressed or version-pinned paths.
The `default` credential profile uses the existing `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` inputs. A named profile such as `catalogue` reads
`SITE_PUBLISH_CATALOGUE_S3_ACCESS_KEY_ID` and
`SITE_PUBLISH_CATALOGUE_S3_SECRET_ACCESS_KEY` from the caller environment.
Use a different Garage key for every security boundary. The action never logs
credential values and clones the Apps repository without credentials in the
remote URL.
Each artifact Service resolves through Garage's bucket virtual host and sets
Traefik `passHostHeader` to false. The upstream therefore receives the bucket
host while the browser retains the public host. This lets one DNS host address
isolated Garage buckets without shared proxy configuration or CRDs.
`.gitea/workflows/publish.yaml`:
```yaml
@@ -87,9 +152,9 @@ my-site.fritzlab.net 300 IN CNAME traefik.edge.svc.k8s.sjc001.fritzlab.net.
| Input | Required | Default | Description |
|---|---|---|---|
| `token` | yes | | Gitea token for apps repo push |
| `s3-access-key` | yes | | Garage `ci-deploy-key` access key id |
| `s3-secret-key` | yes | | Garage `ci-deploy-key` secret key |
| `s3-endpoint` | no | `http://garage.storage.svc:3900` | Garage S3 endpoint |
| `s3-access-key` | only for the `default` profile | | Garage access key id |
| `s3-secret-key` | only for the `default` profile | | Garage secret key |
| `s3-endpoint` | no | `http://garage-s3.storage.svc:3900` | Garage S3 endpoint |
| `garage-admin-token` | only if site has `aliases` | | Garage admin API token (`admin-token` from `garage-rpc-secret` in `storage` ns) |
| `garage-admin-endpoint` | no | `http://garage.storage.svc:3903` | Garage admin API endpoint |
| `username` | no | `ci-bot` | Gitea username |
@@ -107,12 +172,12 @@ Org secrets in `websites`: `CI_BOT_TOKEN`, `GARAGE_S3_ACCESS_KEY`,
```
push to websites/<repo>
→ CI runs site-publish action
→ reads site.yaml, builds content (static copy / hugo / mkdocs), runs tidy
→ aws s3 sync → Garage bucket named after the repo
→ reads site.yaml and either builds legacy content or snapshots caller-built artifacts
→ aws s3 sync → one Garage bucket per artifact, with isolated credentials and metadata
→ admin API: ensures every alias from site.yaml is a globalAlias on the bucket
→ renders manifests in fritzlab/apps from templates: ExternalName Service →
garage.storage.svc, Traefik Ingress (canonical + aliases), cert-manager
Certificate (canonical + aliases as SANs), kustomization
→ renders manifests in fritzlab/apps from templates: ExternalName Services
Garage bucket virtual hosts, path-scoped Traefik Ingresses,
cert-manager Certificate (canonical + aliases as SANs), kustomization
→ commits + pushes apps repo only if diff is non-empty
→ ArgoCD syncs → site live with TLS
```
+5 -5
View File
@@ -1,15 +1,15 @@
name: Publish Site
description: Build and deploy a static-content site (static, hugo, mkdocs) to Garage S3 with Traefik + cert-manager. Containerized apps should use action/image-build + action/image-push + action/image-deploy.
description: Publish one or more static artifacts to isolated Garage buckets and route them through Traefik + cert-manager. Containerized apps should use action/image-build + action/image-push + action/image-deploy.
inputs:
token:
description: Gitea token (ci-bot) for apps repo push and API operations
required: true
s3-access-key:
description: Garage ci-deploy-key access key id
required: true
description: Garage access key id for the default credential profile
required: false
s3-secret-key:
description: Garage ci-deploy-key secret access key
required: true
description: Garage secret key for the default credential profile
required: false
s3-endpoint:
# Targets garage-s3 (data-only Service) so requests do not round-robin onto
# the gateway pod, whose emptyDir-backed metadata view intermittently
+56 -3
View File
@@ -5,7 +5,7 @@ import subprocess
import tempfile
from pathlib import Path
from utils import EXCLUDE_FILES, env, parse_site_yaml, run
from utils import EXCLUDE_FILES, env, parse_site_yaml, run_args
def build_static(site_dir, cfg):
@@ -34,11 +34,11 @@ def build_static(site_dir, cfg):
elif cfg["type"] == "hugo":
print(f"Building Hugo site from {src}")
run(f"hugo --source {src} --destination {html_dir}")
run_args(["hugo", "--source", str(src), "--destination", str(html_dir)])
elif cfg["type"] == "mkdocs":
print(f"Building MkDocs site from {src}")
run(f"cd {src} && mkdocs build -d {html_dir}")
run_args(["mkdocs", "build", "-d", str(html_dir)], cwd=src)
if cfg.get("tidy", True):
print("Running tidy on HTML files...")
@@ -54,6 +54,56 @@ def build_static(site_dir, cfg):
print(f"Build complete — content at {html_dir}")
def stage_artifacts(site_dir, cfg):
"""Snapshot caller-built outputs for the deploy phase."""
staging_dir = site_dir / ".site-publish"
if staging_dir.is_symlink():
raise SystemExit("ERROR: .site-publish must not be a symbolic link")
if staging_dir.exists():
shutil.rmtree(staging_dir)
staging_dir.mkdir()
root = site_dir.resolve()
for name, artifact in cfg["artifacts"].items():
source = (site_dir / artifact["source"]).resolve()
try:
source.relative_to(root)
except ValueError:
raise SystemExit(f"ERROR: artifact {name}.source escapes the repository")
if not source.is_dir():
raise SystemExit(f"ERROR: artifact {name}.source is not a directory: {artifact['source']}")
files = [entry for entry in source.rglob("*") if entry.is_file()]
if not files:
raise SystemExit(f"ERROR: artifact {name}.source has no files: {artifact['source']}")
for entry in source.rglob("*"):
if entry.is_symlink():
try:
entry.resolve(strict=True).relative_to(root)
except (FileNotFoundError, ValueError):
raise SystemExit(
f"ERROR: artifact {name}.source contains an unsafe symlink: {entry.relative_to(source)}"
)
destination = staging_dir / name
destination.mkdir()
routes = [route for route in cfg["routes"] if route["artifact"] == name]
targets = set()
print(f"Staging artifact {name} from {artifact['source']}")
for route in routes:
route_root = destination / route["path"].lstrip("/")
for entry in files:
relative = entry.relative_to(source)
target = route_root / relative
if target in targets:
raise SystemExit(
f"ERROR: artifact {name} routes map more than once to {target.relative_to(destination)}"
)
targets.add(target)
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(entry, target, follow_symlinks=True)
print(f"Build complete — staged artifacts at {staging_dir}")
def cmd_build():
site_dir = Path(env("SITE_DIR"))
cfg = parse_site_yaml(site_dir)
@@ -62,4 +112,7 @@ def cmd_build():
print("Site disabled — skipping build")
return
if cfg["mode"] == "multi":
stage_artifacts(site_dir, cfg)
else:
build_static(site_dir, cfg)
+197 -40
View File
@@ -2,16 +2,15 @@
import json
import os
import shlex
import shutil
import tempfile
from pathlib import Path
from urllib.error import HTTPError, URLError
from urllib.parse import urlsplit
from urllib.request import Request, urlopen
from utils import (
DEFAULT_S3_ENDPOINT,
GITEA_HOST,
NAMESPACE,
clone_apps,
commit_and_push,
@@ -20,7 +19,7 @@ from utils import (
k8s_name,
parse_site_yaml,
render_templates,
run,
run_args,
)
GARAGE_ADMIN_ENDPOINT = os.environ.get(
@@ -31,21 +30,66 @@ GARAGE_ADMIN_ENDPOINT = os.environ.get(
CACHE_CONTROL = "public, max-age=0, must-revalidate"
def s3_sync(site_name, site_dir, excludes=None):
endpoint = os.environ.get("GARAGE_S3_ENDPOINT", DEFAULT_S3_ENDPOINT)
html_dir = site_dir / "build" / "html"
if not html_dir.exists():
die(f"build/html not found — did the build step run? ({html_dir})")
env("AWS_ACCESS_KEY_ID")
env("AWS_SECRET_ACCESS_KEY")
os.environ.setdefault("AWS_DEFAULT_REGION", "sjc001")
def credential_environment(profile):
if profile == "default":
access_key = env("AWS_ACCESS_KEY_ID")
secret_key = env("AWS_SECRET_ACCESS_KEY")
else:
prefix = f"SITE_PUBLISH_{profile.upper().replace('-', '_')}"
access_key = env(f"{prefix}_S3_ACCESS_KEY_ID")
secret_key = env(f"{prefix}_S3_SECRET_ACCESS_KEY")
child_env = os.environ.copy()
child_env["AWS_ACCESS_KEY_ID"] = access_key
child_env["AWS_SECRET_ACCESS_KEY"] = secret_key
child_env.setdefault("AWS_DEFAULT_REGION", "sjc001")
return child_env
def configure_cors(bucket, origins, endpoint, child_env):
if origins is None:
return
if not origins:
run_args(
["aws", "--endpoint-url", endpoint, "s3api", "delete-bucket-cors",
"--bucket", bucket],
env=child_env,
)
return
config = {
"CORSRules": [{
"AllowedOrigins": origins,
"AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600,
}]
}
with tempfile.NamedTemporaryFile("w", suffix=".json", encoding="utf-8") as handle:
json.dump(config, handle)
handle.flush()
run_args(
["aws", "--endpoint-url", endpoint, "s3api", "put-bucket-cors",
"--bucket", bucket, "--cors-configuration", f"file://{handle.name}"],
env=child_env,
)
def s3_sync(bucket, source, *, credential="default", endpoint=None,
cache=None, cors_origins=None, excludes=None):
endpoint = endpoint or os.environ.get("GARAGE_S3_ENDPOINT", DEFAULT_S3_ENDPOINT)
if not source.exists():
die(f"staged artifact not found — did the build step run? ({source})")
child_env = credential_environment(credential)
cache = cache or {"default": CACHE_CONTROL, "rules": []}
# `excludes` are patterns (site.yaml `excludes:` list) that should never
# be uploaded *and* should never be deleted from the bucket — escape hatch
# for assets managed out-of-band (e.g. large PDFs uploaded via aws-cli).
exclude_flags = " ".join(f"--exclude {shlex.quote(p)}" for p in (excludes or []))
exclude_args = []
for pattern in excludes or []:
exclude_args.extend(["--exclude", pattern])
if excludes:
print(f"Excluding patterns: {excludes}")
print(f"Syncing {html_dir} → s3://{site_name} via {endpoint}")
print(f"Syncing {source} → s3://{bucket} via {endpoint}")
# `sync --delete` handles new/changed/orphaned files. `cp --recursive`
# then re-uploads everything to refresh metadata (cache-control,
# content-type) on objects sync skipped because nothing changed.
@@ -53,22 +97,63 @@ def s3_sync(site_name, site_dir, excludes=None):
# small enough that that's free; correctness wins over throughput.
# AWS CLI guesses Content-Type from file extension on local→S3 uploads,
# so a fresh upload always carries the right MIME type.
run(
f"aws --endpoint-url {endpoint} s3 sync {html_dir}/ s3://{site_name}/ "
f"--delete --only-show-errors "
f"--cache-control '{CACHE_CONTROL}' "
f"{exclude_flags}".rstrip()
run_args(
["aws", "--endpoint-url", endpoint, "s3", "sync", f"{source}/", f"s3://{bucket}/",
"--delete", "--only-show-errors", "--cache-control", cache["default"], *exclude_args],
env=child_env,
)
print("Re-stamping metadata on all objects...")
run(
f"aws --endpoint-url {endpoint} s3 cp {html_dir}/ s3://{site_name}/ "
f"--recursive --only-show-errors "
f"--cache-control '{CACHE_CONTROL}' "
f"{exclude_flags}".rstrip()
run_args(
["aws", "--endpoint-url", endpoint, "s3", "cp", f"{source}/", f"s3://{bucket}/",
"--recursive", "--only-show-errors", "--cache-control", cache["default"], *exclude_args],
env=child_env,
)
for rule in cache["rules"]:
print(f"Applying cache metadata for {rule['match']}")
run_args(
["aws", "--endpoint-url", endpoint, "s3", "cp", f"{source}/", f"s3://{bucket}/",
"--recursive", "--only-show-errors", "--exclude", "*",
"--include", rule["match"], "--cache-control", rule["value"], *exclude_args],
env=child_env,
)
configure_cors(bucket, cors_origins, endpoint, child_env)
def preflight_artifacts(site_dir, cfg):
"""Validate every source, credential, and bucket before the first write."""
for artifact in cfg["artifacts"].values():
source = site_dir / ".site-publish" / artifact["name"]
if not source.is_dir() or not any(entry.is_file() for entry in source.rglob("*")):
die(f"staged artifact is absent or empty: {source}")
child_env = credential_environment(artifact["credential"])
endpoint = artifact["endpoint"] or os.environ.get(
"GARAGE_S3_ENDPOINT", DEFAULT_S3_ENDPOINT
)
run_args(
["aws", "--endpoint-url", endpoint, "s3api", "head-bucket",
"--bucket", artifact["bucket"]],
env=child_env,
)
def routed_cache(cache, routes):
"""Translate source-relative cache patterns to their public object keys."""
rules = []
for route in routes:
prefix = route["path"].lstrip("/")
for rule in cache["rules"]:
pattern = f"{prefix}/{rule['match']}" if prefix else rule["match"]
rules.append({"match": pattern, "value": rule["value"]})
return {"default": cache["default"], "rules": rules}
def garage_admin(method, path, token, body=None):
parsed = urlsplit(GARAGE_ADMIN_ENDPOINT)
if (
parsed.scheme not in {"http", "https"} or not parsed.netloc
or parsed.username or parsed.password or parsed.query or parsed.fragment
):
die("GARAGE_ADMIN_ENDPOINT must be an HTTP URL without credentials")
url = f"{GARAGE_ADMIN_ENDPOINT}{path}"
data = json.dumps(body).encode() if body is not None else None
headers = {"Authorization": f"Bearer {token}"}
@@ -89,17 +174,17 @@ def ensure_bucket_aliases(site_name, aliases, admin_token):
if not aliases:
return
if not admin_token:
print(" (no GARAGE_ADMIN_TOKEN — skipping bucket alias reconcile)")
return
die("GARAGE_ADMIN_TOKEN is required when aliases are configured")
try:
info = garage_admin("GET", f"/v2/GetBucketInfo?globalAlias={site_name}",
admin_token)
except (HTTPError, URLError) as e:
print(f" WARNING: bucket lookup failed: {e}")
return
raise RuntimeError(f"bucket lookup failed for {site_name}: {e}") from e
bucket_id = info.get("id")
if not isinstance(bucket_id, str) or not bucket_id:
die(f"bucket lookup for {site_name} returned no bucket id")
existing = set(info.get("globalAliases") or [])
print(f" Bucket {site_name} ({bucket_id[:12]}…) currently aliases: {sorted(existing)}")
@@ -119,21 +204,92 @@ def ensure_bucket_aliases(site_name, aliases, admin_token):
def render_site_manifests(site_name, action_dir, app_dir, manifests_dir, cfg):
"""Always re-render manifests from current site.yaml. Templates own
domain + aliases, so changes propagate without manual edits."""
manifests_dir.mkdir(parents=True, exist_ok=True)
if manifests_dir.exists():
shutil.rmtree(manifests_dir)
manifests_dir.mkdir(parents=True)
site_k8s = k8s_name(site_name)
if cfg["mode"] == "legacy":
artifacts = [{
"name": "site",
"bucket": site_name,
"service_name": site_k8s,
"external_name": "garage-s3.storage.svc.k8s.sjc001.fritzlab.net",
"virtual_host": False,
}]
routes = [{
"name": "site",
"path": "/",
"artifact": "site",
"service_name": site_k8s,
"ingress_name": site_k8s,
"middlewares": cfg["middlewares"],
"middleware_refs": [f"{name}@file" for name in cfg["middlewares"]],
}]
else:
artifacts = []
artifact_by_name = {}
for name, artifact in cfg["artifacts"].items():
service_name = k8s_name(f"{site_name}-{name}")
view = {
**artifact,
"service_name": service_name,
"external_name": f"{artifact['bucket']}.web.sjc001.fritzlab.net",
"virtual_host": True,
}
artifacts.append(view)
artifact_by_name[name] = view
routes = []
for route in cfg["routes"]:
artifact = artifact_by_name[route["artifact"]]
ingress_name = k8s_name(f"{site_name}-{route['name']}")
middleware_refs = [f"{name}@file" for name in route["middlewares"]]
routes.append({
**route,
"service_name": artifact["service_name"],
"ingress_name": ingress_name,
"middleware_refs": middleware_refs,
})
template_vars = {
"site": site_name,
"site_k8s": k8s_name(site_name),
"site_k8s": site_k8s,
"domain": cfg["domain"],
"aliases": cfg["aliases"],
"namespace": NAMESPACE,
"middlewares": cfg["middlewares"],
"artifacts": artifacts,
"routes": routes,
"modern": cfg["mode"] == "multi",
}
render_templates(action_dir, template_vars, app_dir, manifests_dir)
def deploy_static(site_name, site_dir, action_dir, token, cfg):
s3_sync(site_name, site_dir, excludes=cfg.get("excludes"))
ensure_bucket_aliases(site_name, cfg["aliases"], os.environ.get("GARAGE_ADMIN_TOKEN"))
if cfg["mode"] == "legacy":
s3_sync(
site_name,
site_dir / "build" / "html",
excludes=cfg.get("excludes"),
)
ensure_bucket_aliases(
site_name, cfg["aliases"], os.environ.get("GARAGE_ADMIN_TOKEN")
)
else:
preflight_artifacts(site_dir, cfg)
for artifact in cfg["artifacts"].values():
routes = [
route for route in cfg["routes"]
if route["artifact"] == artifact["name"]
]
s3_sync(
artifact["bucket"],
site_dir / ".site-publish" / artifact["name"],
credential=artifact["credential"],
endpoint=artifact["endpoint"],
cache=routed_cache(artifact["cache"], routes),
cors_origins=artifact["cors_origins"],
excludes=artifact["excludes"],
)
apps_dir = clone_apps(token)
app_dir = apps_dir / "sjc001" / "websites" / site_name
@@ -141,23 +297,24 @@ def deploy_static(site_name, site_dir, action_dir, token, cfg):
render_site_manifests(site_name, action_dir, app_dir, manifests_dir, cfg)
commit_and_push(apps_dir, f"Deploy {site_name}")
try:
commit_and_push(apps_dir, f"Deploy {site_name}", token)
finally:
shutil.rmtree(apps_dir.parent, ignore_errors=True)
def decommission(site_name, token):
"""Remove manifests from apps repo."""
user = env("CI_BOT_USER", "ci-bot")
with tempfile.TemporaryDirectory() as tmp:
apps_dir = Path(tmp)
run(f"git clone --depth 1 https://{user}:{token}@{GITEA_HOST}/fritzlab/apps.git {apps_dir}")
apps_dir = clone_apps(token)
try:
site_path = apps_dir / "sjc001" / "websites" / site_name
if not site_path.exists():
print(f"No manifests for {site_name} — nothing to remove")
return
shutil.rmtree(site_path)
run(f"git -C {apps_dir} config user.name {user}")
run(f"git -C {apps_dir} config user.email {user}@fritzlab.net")
commit_and_push(apps_dir, f"Decommission {site_name}")
commit_and_push(apps_dir, f"Decommission {site_name}", token)
finally:
shutil.rmtree(apps_dir.parent, ignore_errors=True)
print(f"Bucket {site_name} and its objects are NOT purged automatically.")
print(f" garage bucket delete {site_name} --yes")
+3 -2
View File
@@ -24,9 +24,10 @@ def ensure_aws():
subprocess.run(["aws", "--version"], check=True)
def ensure_jinja2():
def ensure_python_modules():
try:
import jinja2
import yaml
except ImportError:
print("Installing jinja2 + pyyaml...")
subprocess.run(
@@ -36,6 +37,6 @@ def ensure_jinja2():
if __name__ == "__main__":
ensure_jinja2()
ensure_python_modules()
ensure_aws()
print("Setup complete")
+368 -31
View File
@@ -1,13 +1,18 @@
"""Shared utilities for the site-publish action."""
import hashlib
import os
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from urllib.parse import urlsplit
import yaml
from jinja2 import Environment, FileSystemLoader
from jinja2 import Environment, FileSystemLoader, StrictUndefined
from yaml import YAMLError
APPS_REPO = "fritzlab/apps"
GITEA_HOST = "code.fritzlab.net"
@@ -16,11 +21,15 @@ DEFAULT_S3_ENDPOINT = "http://garage-s3.storage.svc:3900"
EXCLUDE_FILES = {
".git", ".gitea", ".gitignore", "site.yaml",
"build", "Makefile", "README.md", "CLAUDE.md",
".site-publish", "build", "Makefile", "README.md", "CLAUDE.md",
"Dockerfile", ".dockerignore", "go.mod", "go.sum",
}
VALID_TYPES = {"static", "hugo", "mkdocs"}
NAME_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$")
MIDDLEWARE_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
PROFILE_RE = re.compile(r"^[a-z][a-z0-9-]*$")
AUTH_MIDDLEWARE = "authentik-forwardauth"
DOCKER_DEPRECATION_MSG = """\
type: docker is no longer supported by action/site-publish.
@@ -43,13 +52,18 @@ example.\
def k8s_name(name):
"""Sanitize for DNS-1035 label (dots → dashes)."""
return name.replace(".", "-")
"""Return a stable DNS-1035 label, including for long host names."""
normalized = re.sub(r"[^a-z0-9-]", "-", name.lower()).strip("-")
normalized = re.sub(r"-+", "-", normalized)
if len(normalized) <= 63:
return normalized
digest = hashlib.sha256(normalized.encode()).hexdigest()[:8]
return f"{normalized[:54].rstrip('-')}-{digest}"
def env(key, default=None):
val = os.environ.get(key, default)
if val is None:
if val is None or val == "":
die(f"Missing required env var: {key}")
return val
@@ -59,9 +73,258 @@ def die(msg):
sys.exit(1)
def run(cmd, **kwargs):
print(f" $ {cmd}")
return subprocess.run(cmd, shell=True, check=True, **kwargs)
def run_args(args, *, display=None, **kwargs):
"""Run an argv vector without shell interpolation or credential logging."""
print(f" $ {display or ' '.join(args)}")
return subprocess.run(args, check=True, **kwargs)
def _string_list(cfg, key):
value = cfg.get(key) or []
if not isinstance(value, list) or any(not isinstance(item, str) for item in value):
die(f"{key} must be a list of strings")
return value
def _patterns(cfg, key, field):
values = _string_list(cfg, key)
if any(not value or "\n" in value or "\r" in value for value in values):
die(f"{field} must contain non-empty single-line patterns")
return values
def _cors_origins(raw, artifact_name):
origins = _string_list(raw, "cors_origins")
for origin in origins:
if origin == "*":
continue
parsed = urlsplit(origin)
if (
parsed.scheme != "https" or not parsed.netloc or parsed.path not in {"", "/"}
or parsed.query or parsed.fragment or parsed.username or parsed.password
):
die(f"artifact {artifact_name}.cors_origins must contain * or HTTPS origins")
return origins
def _validate_domain(value, field):
if not isinstance(value, str) or not value or len(value) > 253:
die(f"{field} must be a DNS name")
labels = value.rstrip(".").split(".")
if any(not NAME_RE.fullmatch(label) for label in labels):
die(f"{field} must be a DNS name")
return value.rstrip(".")
def _validate_middlewares(value, field):
if not isinstance(value, list) or any(
not isinstance(item, str) or not MIDDLEWARE_RE.fullmatch(item)
for item in value
):
die(f"{field} must contain file-provider middleware names")
return value
def _known_keys(value, allowed, field):
unknown = sorted(set(value) - set(allowed))
if unknown:
die(f"{field} has unknown fields: {', '.join(unknown)}")
def _cache_header(value, field):
if not isinstance(value, str) or not value or "\n" in value or "\r" in value:
die(f"{field} must be one Cache-Control header value")
return value
def _cache_config(raw, artifact_name):
cache = raw.get("cache") or {}
if not isinstance(cache, dict):
die(f"artifact {artifact_name}.cache must be a mapping")
_known_keys(cache, {"default", "rules"}, f"artifact {artifact_name}.cache")
default = _cache_header(
cache.get("default", "public, max-age=0, must-revalidate"),
f"artifact {artifact_name}.cache.default",
)
rules_raw = cache.get("rules") or []
if not isinstance(rules_raw, list):
die(f"artifact {artifact_name}.cache.rules must be a list")
rules = []
patterns = set()
for index, rule in enumerate(rules_raw):
field = f"artifact {artifact_name}.cache.rules[{index}]"
if not isinstance(rule, dict):
die(f"{field} must be a mapping")
_known_keys(rule, {"match", "value"}, field)
pattern = rule.get("match")
if (
not isinstance(pattern, str) or not pattern or pattern.startswith("/")
or "\n" in pattern or "\r" in pattern or ".." in Path(pattern).parts
):
die(f"{field}.match must be a relative aws-cli include pattern")
if pattern in patterns:
die(f"artifact {artifact_name} repeats cache match {pattern}")
patterns.add(pattern)
rules.append({
"match": pattern,
"value": _cache_header(rule.get("value"), f"{field}.value"),
})
return {"default": default, "rules": rules}
def _cache_directives(value):
return {part.strip().lower().split("=", 1)[0] for part in value.split(",")}
def _modern_config(cfg, domain, aliases):
artifacts_cfg = cfg.get("artifacts")
routes_cfg = cfg.get("routes")
if not isinstance(artifacts_cfg, dict) or not artifacts_cfg:
die("artifacts must be a non-empty mapping")
if not isinstance(routes_cfg, list) or not routes_cfg:
die("routes must be a non-empty list")
artifacts = {}
buckets = set()
for name, raw in artifacts_cfg.items():
if not isinstance(name, str) or not NAME_RE.fullmatch(name):
die(f"artifact name {name!r} must be a DNS label")
if not isinstance(raw, dict):
die(f"artifact {name} must be a mapping")
_known_keys(
raw,
{"source", "bucket", "credential", "endpoint", "cache", "cors_origins", "excludes"},
f"artifact {name}",
)
source = raw.get("source")
bucket = raw.get("bucket")
credential = raw.get("credential", "default")
if not isinstance(source, str) or not source or Path(source).is_absolute():
die(f"artifact {name}.source must be a relative path")
if source in {".", "./"} or ".." in Path(source).parts:
die(f"artifact {name}.source must name a build output inside the repository")
bucket = _validate_domain(bucket, f"artifact {name}.bucket")
if bucket in buckets:
die(f"artifact bucket {bucket} is used more than once")
buckets.add(bucket)
if not isinstance(credential, str) or not PROFILE_RE.fullmatch(credential):
die(f"artifact {name}.credential must be a lowercase profile name")
endpoint = raw.get("endpoint")
if endpoint is not None:
parsed_endpoint = urlsplit(endpoint) if isinstance(endpoint, str) else None
if (
parsed_endpoint is None or parsed_endpoint.scheme not in {"http", "https"}
or not parsed_endpoint.netloc or parsed_endpoint.username or parsed_endpoint.password
or parsed_endpoint.query or parsed_endpoint.fragment
):
die(f"artifact {name}.endpoint must be an HTTP URL without credentials")
cors_origins = _cors_origins(raw, name)
artifacts[name] = {
"name": name,
"source": source,
"bucket": bucket,
"credential": credential,
"cache": _cache_config(raw, name),
"endpoint": endpoint,
"cors_origins": cors_origins,
"excludes": _patterns(raw, "excludes", f"artifact {name}.excludes"),
}
routes = []
route_names = set()
route_paths = set()
referenced = set()
for raw in routes_cfg:
if not isinstance(raw, dict):
die("each route must be a mapping")
_known_keys(
raw,
{"name", "path", "artifact", "access", "middlewares"},
"route",
)
name = raw.get("name")
path = raw.get("path")
artifact = raw.get("artifact")
access = raw.get("access")
if not isinstance(name, str) or not NAME_RE.fullmatch(name):
die("each route.name must be a DNS label")
if name in route_names:
die(f"route name {name} is used more than once")
route_names.add(name)
if not isinstance(path, str) or not re.fullmatch(
r"/(?:[A-Za-z0-9._~-]+(?:/[A-Za-z0-9._~-]+)*)?", path
):
die(f"route {name}.path must be a canonical absolute URL path")
if path != "/" and path.endswith("/"):
die(f"route {name}.path must not end with /")
if "//" in path or ".." in Path(path).parts or "?" in path or "#" in path:
die(f"route {name}.path is not a canonical URL path")
if path in route_paths:
die(f"route path {path} is used more than once")
route_paths.add(path)
if artifact not in artifacts:
die(f"route {name} references unknown artifact {artifact!r}")
referenced.add(artifact)
if access not in {"public", "protected"}:
die(f"route {name}.access must be public or protected")
route_middlewares = _validate_middlewares(
raw.get("middlewares") or [], f"route {name}.middlewares"
)
if access == "protected":
route_middlewares = [AUTH_MIDDLEWARE, *route_middlewares]
if len(set(route_middlewares)) != len(route_middlewares):
die(f"route {name}.middlewares contains a duplicate")
routes.append({
"name": name,
"path": path,
"artifact": artifact,
"access": access,
"middlewares": route_middlewares,
})
unreferenced = sorted(set(artifacts) - referenced)
if unreferenced:
die(f"artifacts without routes: {', '.join(unreferenced)}")
for artifact_name, artifact in artifacts.items():
access_modes = {
route["access"] for route in routes if route["artifact"] == artifact_name
}
if len(access_modes) != 1:
die(f"artifact {artifact_name} cannot cross public and protected routes")
access = next(iter(access_modes))
policies = [artifact["cache"]["default"], *(
rule["value"] for rule in artifact["cache"]["rules"]
)]
for policy in policies:
directives = _cache_directives(policy)
if access == "protected" and not ({"private", "no-store"} & directives):
die(f"protected artifact {artifact_name} cache policy must be private or no-store")
if access == "protected" and ({"public", "s-maxage"} & directives):
die(f"protected artifact {artifact_name} cannot use shared-cache directives")
if access == "public" and "private" in directives:
die(f"public artifact {artifact_name} cannot use private cache metadata")
if access == "protected" and "*" in artifact["cors_origins"]:
die(f"protected artifact {artifact_name} cannot allow wildcard CORS")
profile_access = {}
for artifact_name, artifact in artifacts.items():
access = next(
route["access"] for route in routes if route["artifact"] == artifact_name
)
existing = profile_access.setdefault(artifact["credential"], access)
if existing != access:
die(
f"credential profile {artifact['credential']} cannot cross public and protected artifacts"
)
return {
"mode": "multi",
"domain": domain,
"aliases": aliases,
"artifacts": artifacts,
"routes": routes,
}
def parse_site_yaml(site_dir):
@@ -69,12 +332,50 @@ def parse_site_yaml(site_dir):
if not path.exists():
die("site.yaml not found in repo root")
with open(path) as f:
try:
with open(path, encoding="utf-8") as f:
cfg = yaml.safe_load(f)
except YAMLError as error:
die(f"site.yaml is not valid YAML: {error}")
if not isinstance(cfg, dict):
die("site.yaml must contain a mapping")
if not cfg.get("domain"):
die("domain is required in site.yaml")
domain = _validate_domain(cfg["domain"], "domain")
aliases = [
_validate_domain(value, "aliases entry")
for value in _string_list(cfg, "aliases")
]
if len(set([domain, *aliases])) != 1 + len(aliases):
die("domain and aliases must be unique")
has_artifacts = "artifacts" in cfg
has_routes = "routes" in cfg
if has_artifacts != has_routes:
die("artifacts and routes must be declared together")
if has_artifacts:
_known_keys(
cfg,
{"domain", "aliases", "enabled", "artifacts", "routes"},
"site.yaml",
)
site = _modern_config(cfg, domain, aliases)
site["type"] = "artifacts"
site["enabled"] = cfg.get("enabled", True)
if not isinstance(site["enabled"], bool):
die("enabled must be true or false")
site["tidy"] = False
site["content_dir"] = ""
site["excludes"] = []
site["middlewares"] = []
print("Site config:")
print(f" domain: {site['domain']}")
print(f" artifacts: {', '.join(site['artifacts'])}")
print(f" routes: {', '.join(route['path'] for route in site['routes'])}")
return site
site_type = cfg.get("type", "static")
if site_type == "docker":
@@ -83,20 +384,26 @@ def parse_site_yaml(site_dir):
if site_type not in VALID_TYPES:
die(f"Unknown site type: {site_type} (valid: {', '.join(sorted(VALID_TYPES))})")
excludes = cfg.get("excludes") or []
if not isinstance(excludes, list) or any(not isinstance(p, str) for p in excludes):
die("excludes must be a list of string patterns")
excludes = _string_list(cfg, "excludes")
middlewares = _validate_middlewares(cfg.get("middlewares") or [], "middlewares")
middlewares = cfg.get("middlewares") or []
if not isinstance(middlewares, list) or any(not isinstance(m, str) for m in middlewares):
die("middlewares must be a list of Traefik file-provider middleware names")
content_dir = cfg.get("content_dir", "")
if not isinstance(content_dir, str) or Path(content_dir).is_absolute():
die("content_dir must be a relative path")
if ".." in Path(content_dir).parts:
die("content_dir cannot escape the repository")
enabled = cfg.get("enabled", True)
if not isinstance(enabled, bool):
die("enabled must be true or false")
site = {
"domain": cfg["domain"],
"mode": "legacy",
"domain": domain,
"type": site_type,
"enabled": cfg.get("enabled", True),
"aliases": cfg.get("aliases") or [],
"content_dir": cfg.get("content_dir", ""),
"enabled": enabled,
"aliases": aliases,
"content_dir": content_dir,
"tidy": cfg.get("tidy", True),
"excludes": excludes,
"middlewares": middlewares,
@@ -110,21 +417,46 @@ def parse_site_yaml(site_dir):
def clone_apps(token):
user = env("CI_BOT_USER", "ci-bot")
apps_dir = Path("/tmp/apps-deploy")
if apps_dir.exists():
shutil.rmtree(apps_dir)
run(f"git clone --depth 1 https://{user}:{token}@{GITEA_HOST}/{APPS_REPO}.git {apps_dir}")
run(f"git -C {apps_dir} config user.name {user}")
run(f"git -C {apps_dir} config user.email {user}@fritzlab.net")
clone_root = Path(tempfile.mkdtemp(prefix="apps-deploy-"))
apps_dir = clone_root / "repo"
askpass = clone_root / "askpass"
askpass.write_text(
"#!/bin/sh\ncase \"$1\" in *Username*) printf '%s\\n' \"$GIT_AUTH_USER\" ;; "
"*) printf '%s\\n' \"$GIT_AUTH_TOKEN\" ;; esac\n",
encoding="utf-8",
)
askpass.chmod(0o700)
git_env = git_auth_environment(token, user, askpass)
try:
run_args(
["git", "clone", "--depth", "1", f"https://{GITEA_HOST}/{APPS_REPO}.git", str(apps_dir)],
display=f"git clone --depth 1 https://{GITEA_HOST}/{APPS_REPO}.git {apps_dir}",
env=git_env,
)
except Exception:
shutil.rmtree(clone_root, ignore_errors=True)
raise
run_args(["git", "-C", str(apps_dir), "config", "user.name", user])
run_args(["git", "-C", str(apps_dir), "config", "user.email", f"{user}@fritzlab.net"])
return apps_dir
def git_auth_environment(token, user, askpass):
child_env = os.environ.copy()
child_env["GIT_ASKPASS"] = str(askpass)
child_env["GIT_TERMINAL_PROMPT"] = "0"
child_env["GIT_AUTH_USER"] = user
child_env["GIT_AUTH_TOKEN"] = token
return child_env
def render_templates(action_dir, template_vars, app_dir, manifests_dir):
"""Render Jinja2 templates for a static-content site."""
templates_dir = Path(action_dir) / "templates"
jinja_env = Environment(
loader=FileSystemLoader(str(templates_dir)),
keep_trailing_newline=True,
undefined=StrictUndefined,
)
tmpl_names = ["app.yaml.j2", "certificate.yaml.j2", "ingress.yaml.j2",
@@ -139,16 +471,21 @@ def render_templates(action_dir, template_vars, app_dir, manifests_dir):
print(f" Rendered {tmpl_name} -> {dest}")
def commit_and_push(apps_dir, message):
run(f"git -C {apps_dir} add -A")
def commit_and_push(apps_dir, message, token):
run_args(["git", "-C", str(apps_dir), "add", "-A"])
result = subprocess.run(
f"git -C {apps_dir} diff --cached --quiet",
shell=True, check=False,
["git", "-C", str(apps_dir), "diff", "--cached", "--quiet"],
check=False,
)
if result.returncode == 0:
print("No manifest changes to commit")
return False
run(f"git -C {apps_dir} commit -m '{message}'")
run(f"git -C {apps_dir} push")
run_args(["git", "-C", str(apps_dir), "commit", "-m", message])
git_env = git_auth_environment(
token,
env("CI_BOT_USER", "ci-bot"),
apps_dir.parent / "askpass",
)
run_args(["git", "-C", str(apps_dir), "push"], env=git_env)
print("Manifests pushed — ArgoCD will sync")
return True
+14 -12
View File
@@ -1,41 +1,43 @@
{% for route in routes %}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ site_k8s }}
name: {{ route.ingress_name }}
namespace: {{ namespace }}
{%- if site_type != "docker" %}
annotations:
traefik.ingress.kubernetes.io/router.middlewares: https-redirect@file,retry-upstream@file{% for m in middlewares %},{{ m }}@file{% endfor %}
{%- endif %}
traefik.ingress.kubernetes.io/router.middlewares: https-redirect@file,retry-upstream@file{% for middleware in route.middleware_refs %},{{ middleware }}{% endfor %}
spec:
ingressClassName: traefik
tls:
- hosts:
- {{ domain }}
{%- for alias in aliases %}
{% for alias in aliases %}
- {{ alias }}
{%- endfor %}
{% endfor %}
secretName: {{ site_k8s }}-tls
rules:
- host: {{ domain }}
http:
paths:
- path: /
- path: {{ route.path }}
pathType: Prefix
backend:
service:
name: {{ site_k8s }}
name: {{ route.service_name }}
port:
number: 80
{%- for alias in aliases %}
{% for alias in aliases %}
- host: {{ alias }}
http:
paths:
- path: /
- path: {{ route.path }}
pathType: Prefix
backend:
service:
name: {{ site_k8s }}
name: {{ route.service_name }}
port:
number: 80
{%- endfor %}
{% endfor %}
{% if not loop.last %}---
{% endif %}
{% endfor %}
+9 -2
View File
@@ -1,11 +1,18 @@
{% for artifact in artifacts %}
apiVersion: v1
kind: Service
metadata:
name: {{ site_k8s }}
name: {{ artifact.service_name }}
namespace: {{ namespace }}
{% if artifact.virtual_host %} annotations:
traefik.ingress.kubernetes.io/service.passhostheader: "false"
{% endif %}
spec:
type: ExternalName
externalName: garage.storage.svc.k8s.sjc001.fritzlab.net
externalName: {{ artifact.external_name }}
ports:
- port: 80
targetPort: 80
{% if not loop.last %}---
{% endif %}
{% endfor %}
+285
View File
@@ -0,0 +1,285 @@
import os
import sys
import tempfile
import textwrap
import unittest
from pathlib import Path
from unittest.mock import patch
import yaml
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "scripts"))
from build import stage_artifacts
from deploy import credential_environment, render_site_manifests, routed_cache, s3_sync
from utils import k8s_name, parse_site_yaml
MULTI_SITE = """
domain: baseline.fritzlab.net
artifacts:
catalogue:
source: apps/catalogue/build
bucket: baseline-catalogue
credential: catalogue
cache:
default: private, no-store
dist:
source: dist
bucket: baseline-dist
credential: dist
cache:
default: public, max-age=0, must-revalidate, no-transform
rules:
- match: releases/*
value: public, max-age=31536000, immutable, no-transform
cors_origins: ["*"]
routes:
- name: catalogue
path: /
artifact: catalogue
access: protected
- name: dist
path: /dist
artifact: dist
access: public
"""
class SiteConfigTests(unittest.TestCase):
def write_config(self, directory, body):
Path(directory, "site.yaml").write_text(textwrap.dedent(body), encoding="utf-8")
def test_multi_artifact_config_separates_storage_and_routes(self):
with tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, MULTI_SITE)
config = parse_site_yaml(tmp)
self.assertEqual(config["mode"], "multi")
self.assertEqual(config["artifacts"]["dist"]["credential"], "dist")
self.assertEqual(config["artifacts"]["dist"]["cache"]["rules"][0]["match"], "releases/*")
self.assertEqual(config["routes"][1]["path"], "/dist")
def test_legacy_config_remains_supported(self):
with tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, """
domain: example.fritzlab.net
type: static
content_dir: html
middlewares: [authentik-forwardauth]
""")
config = parse_site_yaml(tmp)
self.assertEqual(config["mode"], "legacy")
self.assertEqual(config["content_dir"], "html")
def test_duplicate_bucket_fails_before_sync_delete_can_run(self):
with tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, """
domain: example.fritzlab.net
artifacts:
first: {source: out/first, bucket: shared-bucket, cache: {default: "private, no-store"}}
second: {source: out/second, bucket: shared-bucket}
routes:
- {name: first, path: /, artifact: first, access: protected}
- {name: second, path: /second, artifact: second, access: public}
""")
with self.assertRaises(SystemExit):
parse_site_yaml(tmp)
def test_unknown_artifact_and_root_strip_fail_closed(self):
cases = (
"{name: root, path: /, artifact: missing, access: public}",
"{name: root, path: relative, artifact: site, access: public}",
)
for routes in cases:
with self.subTest(routes=routes), tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, "\n".join((
"domain: example.fritzlab.net",
"artifacts:",
" site: {source: out/site, bucket: example-site}",
"routes:",
f" - {routes}",
)))
with self.assertRaises(SystemExit):
parse_site_yaml(tmp)
def test_artifact_build_snapshots_each_output(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
self.write_config(root, MULTI_SITE)
(root / "apps/catalogue/build").mkdir(parents=True)
(root / "apps/catalogue/build/index.html").write_text("catalogue", encoding="utf-8")
(root / "dist").mkdir()
(root / "dist/baseline.css").write_text("tokens", encoding="utf-8")
config = parse_site_yaml(root)
stage_artifacts(root, config)
self.assertEqual((root / ".site-publish/catalogue/index.html").read_text(), "catalogue")
self.assertEqual((root / ".site-publish/dist/dist/baseline.css").read_text(), "tokens")
def test_rendered_routes_have_independent_middleware_chains(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
self.write_config(root, MULTI_SITE)
config = parse_site_yaml(root)
app_dir = root / "app"
manifests = app_dir / "manifests"
app_dir.mkdir()
render_site_manifests("baseline", ROOT, app_dir, manifests, config)
ingress = (manifests / "ingress.yaml").read_text(encoding="utf-8")
service = (manifests / "service.yaml").read_text(encoding="utf-8")
self.assertIn("authentik-forwardauth@file", ingress)
self.assertNotIn("kubernetescrd", ingress)
self.assertIn("baseline-dist.web.sjc001.fritzlab.net", service)
self.assertIn("service.passhostheader: \"false\"", service)
for manifest in manifests.glob("*.yaml"):
documents = list(yaml.safe_load_all(manifest.read_text(encoding="utf-8")))
self.assertTrue(documents)
self.assertNotIn(None, documents, manifest.name)
def test_legacy_render_does_not_emit_crd_middleware(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
self.write_config(root, """
domain: example.fritzlab.net
type: static
content_dir: html
""")
config = parse_site_yaml(root)
app_dir = root / "app"
manifests = app_dir / "manifests"
app_dir.mkdir()
render_site_manifests("example.fritzlab.net", ROOT, app_dir, manifests, config)
self.assertFalse((manifests / "middleware.yaml").exists())
self.assertIn("name: example-fritzlab-net", (manifests / "ingress.yaml").read_text())
def test_protected_artifact_rejects_shared_cache_and_wildcard_cors(self):
invalid_configs = (
"""
domain: example.fritzlab.net
artifacts:
site:
source: out/site
bucket: example-site
cache: {default: "public, max-age=0"}
routes:
- {name: site, path: /, artifact: site, access: protected}
""",
"""
domain: example.fritzlab.net
artifacts:
site:
source: out/site
bucket: example-site
cache: {default: "private, no-store"}
cors_origins: ["*"]
routes:
- {name: site, path: /, artifact: site, access: protected}
""",
)
for body in invalid_configs:
with self.subTest(body=body), tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, body)
with self.assertRaises(SystemExit):
parse_site_yaml(tmp)
def test_credential_profile_cannot_cross_access_boundaries(self):
with tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, """
domain: example.fritzlab.net
artifacts:
private:
source: out/private
bucket: example-private
credential: shared
cache: {default: "private, no-store"}
public:
source: out/public
bucket: example-public
credential: shared
routes:
- {name: private, path: /, artifact: private, access: protected}
- {name: public, path: /public, artifact: public, access: public}
""")
with self.assertRaises(SystemExit):
parse_site_yaml(tmp)
def test_unknown_modern_field_fails_closed(self):
with tempfile.TemporaryDirectory() as tmp:
self.write_config(tmp, """
domain: example.fritzlab.net
artifacts:
site:
source: out/site
storage_bucket: misspelled
bucket: example-site
routes:
- {name: site, path: /, artifact: site, access: public}
""")
with self.assertRaises(SystemExit):
parse_site_yaml(tmp)
def test_empty_artifact_fails_before_staging(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
self.write_config(root, MULTI_SITE)
(root / "apps/catalogue/build").mkdir(parents=True)
(root / "dist").mkdir()
config = parse_site_yaml(root)
with self.assertRaises(SystemExit):
stage_artifacts(root, config)
def test_cache_rules_are_applied_after_default_metadata(self):
with tempfile.TemporaryDirectory() as tmp:
source = Path(tmp)
(source / "release.css").write_text("css", encoding="utf-8")
cache = {
"default": "public, max-age=0, must-revalidate",
"rules": [{
"match": "releases/*",
"value": "public, max-age=31536000, immutable",
}],
}
with patch("deploy.credential_environment", return_value={}), \
patch("deploy.configure_cors"), patch("deploy.run_args") as run_args:
s3_sync("example-public", source, cache=cache)
commands = [call.args[0] for call in run_args.call_args_list]
self.assertEqual(len(commands), 3)
self.assertIn("--include", commands[2])
self.assertIn("releases/*", commands[2])
def test_cache_patterns_follow_the_public_route_prefix(self):
cache = {
"default": "public, max-age=0, must-revalidate",
"rules": [{"match": "releases/*", "value": "public, immutable"}],
}
routed = routed_cache(cache, [{"path": "/dist"}])
self.assertEqual(routed["rules"][0]["match"], "dist/releases/*")
def test_named_credentials_are_selected_without_changing_parent_environment(self):
with patch.dict(os.environ, {
"SITE_PUBLISH_DIST_S3_ACCESS_KEY_ID": "access",
"SITE_PUBLISH_DIST_S3_SECRET_ACCESS_KEY": "secret",
}, clear=False):
child = credential_environment("dist")
self.assertEqual(child["AWS_ACCESS_KEY_ID"], "access")
self.assertEqual(child["AWS_SECRET_ACCESS_KEY"], "secret")
def test_long_resource_names_are_stable_dns_labels(self):
first = k8s_name("a" * 90)
second = k8s_name("a" * 89 + "b")
self.assertLessEqual(len(first), 63)
self.assertNotEqual(first, second)
if __name__ == "__main__":
unittest.main()