18 Commits
Author SHA1 Message Date
dfritz 910341391f Merge pull request 'feat(site-publish): scope publication with an artifacts selection' (#7) from artifacts-input into main
Test / contract (push) Successful in 15s
Reviewed-on: #7
2026-09-06 21:31:50 +00:00
Evelyn ChenandClaude Fable 5.1 304095436c chore: refresh the review head for #7
Test / contract (pull_request) Successful in 7s
The three review rounds on b14f6a8 ended on Mission deadlines while the
runner node was saturated; a fresh head starts a clean round.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjQqc4qFmdpAWaYfy2Aypb
2026-09-06 20:44:36 +00:00
Evelyn ChenandClaude Fable 5.1 b14f6a856b fix(site-publish): refuse to record a contract a scoped run did not publish
Test / contract (pull_request) Successful in 7s
Review found the hole in the first commit's claim. render_site_manifests
advances the stored route contract for every route in site.yaml, and
`access` there is a replacement, not a union the way immutable_paths is.
So a catalogue-only publish could write `protected` for the distributions
bucket that nothing published — and validate_route_migrations then
refuses to put that bucket back public. Unpublished intent became an
irreversible fact.

Reproduced from the repo's own fixture: after a whole publish the record
reads public; after a catalogue-only publish with the route flipped it
reads protected, with nothing written to baseline-dist, and reverting
fails with "artifact distributions cannot become public while reusing
protected bucket baseline-dist".

A scoped run now refuses before the first bucket is touched when an
unselected artifact's path, access or artifact name differs from what is
recorded, naming both contracts. It also refuses an unselected artifact
with no published history, which is the same defect at time zero. Publish
the artifact in the same run.

Three tests: the reviewer's flip scenario (and the same change published
in the same run, which proceeds), the no-history case, and one proving
deploy_static reaches the guard before publish_route_immutables,
reconcile_artifact_cors or s3_sync. Disabling the call site alone turns
the last one red.

The README sentence is narrowed to what the code actually guarantees, and
gains the CORS consequence: a scoped run holds no credential for the
other bucket, so a cors_origins change lands with that artifact's next
publish rather than on the merge that edits site.yaml.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjQqc4qFmdpAWaYfy2Aypb
2026-09-06 01:56:43 +00:00
Evelyn ChenandClaude Fable 5.1 3dfee64335 feat(site-publish): scope publication with an artifacts selection
Test / contract (pull_request) Successful in 7s
A repository whose artifacts ship on different cadences has no way to
publish one of them. Baseline needs it: every merge to main must put the
catalogue live in under five minutes, while `dist/` is content-addressed
and may only be written by a tag release. Today the action iterates
cfg["artifacts"] unconditionally, so the only lever is deleting the
distributions artifact from site.yaml — which changes the stored
publication contract and drives the route-retirement path.

The new `artifacts:` input names the subset this run builds and
publishes. Selection scopes the build, the immutable preflight, the CORS
reconcile, the S3 sync, and credential resolution. It deliberately does
not scope manifest rendering or the immutable-path history: those stay
whole, so a scoped run can never retire another artifact's route or
delete its bucket contents. An undeclared name fails before the first
bucket is touched; `enabled: false` refuses a selection because
decommissioning is whole-site.

Default is unchanged: no input publishes every declared artifact.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjQqc4qFmdpAWaYfy2Aypb
2026-09-06 00:53:35 +00:00
Evelyn Chen 9287e4861a Merge pull request 'feat(site-publish): reconcile split-surface CORS' (#6) from feat/split-surface-cors into main
Test / contract (push) Successful in 7s
2026-08-29 23:53:46 +00:00
Evelyn Chen 5f4325706b fix: make CORS reconciliation recoverable
Test / contract (pull_request) Successful in 6s
2026-08-29 23:48:39 +00:00
Evelyn Chen 310ae6a29d feat(site-publish): reconcile split-surface CORS
Test / contract (pull_request) Successful in 7s
Authored-By: @architect <architect@fritzlab.net>
2026-08-29 23:41:41 +00:00
Evelyn Chen 173f0a3a6d Merge pull request '[bug-7acxk8rf0g6b] close split migration boundaries' (#4)
Test / contract (push) Successful in 7s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:25:54 +00:00
Evelyn Chen deccc4e177 test(site-publish): prove absent bucket tombstone
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:19:43 +00:00
Evelyn Chen 9b0a8c4fd4 Merge concurrent immutable history correction
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:17:33 +00:00
Evelyn Chen 898816db16 fix(site-publish): close split migration boundaries
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:15:52 +00:00
Evelyn Chen 892b6e6441 fix(site-publish): retain immutable route history
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:15:29 +00:00
Evelyn Chen 85a0b41380 fix(site-publish): close split migration boundaries
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 23:08:52 +00:00
Evelyn Chen 19fb4e43ab Merge pull request '[bug-7acxk8rf0g6b] feat(site-publish): add split-surface publishing' (#2)
Test / contract (push) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 22:35:46 +00:00
Evelyn Chen 7b824a61ca fix(site-publish): reject split input symlinks
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 22:30:43 +00:00
Evelyn Chen fb2e440bbb fix(site-publish): preflight all immutable routes
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 22:23:51 +00:00
Dave Kowalski 5261b9b99f test(site-publish): prove immutable failure ordering
Test / contract (pull_request) Successful in 7s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 22:17:24 +00:00
Evelyn Chen 5c2630b972 feat(site-publish): add split-surface publishing
Test / contract (pull_request) Successful in 6s
Authored-By: OpenAI (GPT-5) <noreply@openai.com>
2026-08-29 22:04:50 +00:00
8 changed files with 1592 additions and 74 deletions
+75 -2
View File
@@ -52,6 +52,7 @@ artifacts:
- name: distributions
type: static
content_dir: dist
cors_origins: ['*']
publish:
bucket: baseline-dist
credentials:
@@ -92,8 +93,10 @@ routes:
```
The caller supplies each declared credential name as an environment variable
on the action step. Credential values are passed to `aws` only through its
environment and never appear in a logged command or process argument.
on the action step. Names must be matched `<NAME>_S3_ACCESS_KEY` and
`<NAME>_S3_SECRET_KEY` pairs; arbitrary environment variables cannot become
publication credentials. Values pass to `aws` only through its environment and
never appear in a logged command or process argument.
```yaml
- uses: https://code.fritzlab.net/action/site-publish@v1
@@ -120,6 +123,40 @@ revalidation, or `no-store` plus a positive max-age) are rejected. Protected
artifacts require `private` or `no-store` and cannot emit `public`.
Metadata restamping transfers each artifact once even on a no-op publication;
that is the cost of making policy changes effective on unchanged Garage objects.
An immutable cache path is excluded from sync and deletion. Every object key in
that path must contain exactly one full publication SHA-256, calculated over its
cache policy, content type, and bytes. That content address makes concurrent
writes identical even though Garage v2.2.0 has no conditional destination
write. An identical retry converges; a changed object, missing digest metadata,
wrong address, or nested policy under that immutable prefix fails publication.
Every immutable target across every artifact is validated and published before
any route's mutable objects change.
Mutable default and override partitions receive their final cache policy before
the matching prefix-scoped stale deletion, so publication never exposes a
provisional cache policy or a pointer to a missing immutable target.
Generated Ingress annotations and `site-publish-history.yaml` retain every
seen bucket's access, prior route, and cumulative bucket-relative immutable key
prefixes, including while an artifact is absent. Removed or renamed rules stay
recorded. When a move places a retired prefix inside the new sync scope, its
immutable subtrees are excluded; a current-file collision fails publication.
The bucket-keyed history rejects a protected-to-public transition even when the
artifact is renamed; publishing that artifact publicly requires a new bucket.
Legacy single-surface is public for this downgrade check. Removing or renaming
an immutable rule preserves its prior URLs; current mutable content at one of
those paths is rejected instead of replacing it.
Artifact input directories must be pairwise disjoint after filesystem
resolution. Publication stops before build or upload if one contains another or
escapes the repository. Symlinked roots, components, and descendants are also rejected, preventing
protected input from entering a public artifact through dereference. Split
storage endpoints are pinned to Garage, and each website
authority is derived from its bucket; a site cannot expose an arbitrary backend.
`cors_origins` is reconciled as a bucket policy on every split publication. Values are either `*`
or HTTPS origins; browser access is limited to `GET` and `HEAD`. Omitting the field removes stale
CORS from that bucket. Protected artifacts cannot allow wildcard CORS. All immutable objects and
all bucket CORS policies complete before mutable channels change; if any policy write fails, the
policies already attempted are restored to their pre-publication values.
Each split route gets a bucket-specific `<bucket>.web.sjc001.fritzlab.net`
ExternalName Service annotated to disable pass-host-header and a separate Ingress. Route
@@ -197,10 +234,46 @@ my-site.fritzlab.net 300 IN CNAME gateway.sjc001.fritzlab.net.
| `garage-admin-token` | legacy aliases only | | Garage admin API token (`admin-token` from `garage-rpc-secret` in `storage` ns) |
| `garage-admin-endpoint` | no | `http://garage.storage.svc:3903` | Garage admin API endpoint |
| `username` | no | `ci-bot` | Gitea username |
| `artifacts` | no | every declared artifact | Space- or comma-separated subset of `site.yaml` artifacts to build and publish |
Org secrets in `websites`: `CI_BOT_TOKEN`, `GARAGE_S3_ACCESS_KEY`,
`GARAGE_S3_SECRET_KEY`, `GARAGE_ADMIN_TOKEN`.
### Publishing a subset of artifacts
A repository whose branches ship on different cadences names the ones this run
owns:
```yaml
- uses: https://code.fritzlab.net/action/site-publish@<sha>
with:
token: ${{ secrets.CI_BOT_TOKEN }}
artifacts: catalogue
```
Selection scopes the build and the S3 publication only. Ingresses, Services,
Certificates and the immutable-path history are always rendered from the whole
`site.yaml`, so a scoped run cannot retire another artifact's route or delete
its bucket contents. Credentials are resolved for the selected artifacts alone,
so a workflow need not carry secrets for artifacts it does not publish.
What a scoped run may **not** do is move an artifact it is not publishing. The
generated route contract in `site-publish-history.yaml` is written from the
whole `site.yaml`, and `access` there is a replacement rather than a union — so
recording a change nothing published would turn intent into a fact that
`validate_route_migrations` then refuses to undo. A run whose `site.yaml`
changes an unselected artifact's route path, access or artifact name is refused
before the first bucket is touched, naming both contracts: publish that artifact
in the same run. For the same reason a scoped run cannot introduce an artifact
that has no published history yet.
Two more refusals, both before any bucket changes: an undeclared name, and
`enabled: false` with a selection, because decommissioning is whole-site.
One consequence to know: CORS reconciliation is scoped too, since a scoped run
holds no credential for the other bucket. A `cors_origins:` change lands with
that artifact's next publish, not on the merge that edits `site.yaml`.
## Tools
- **`new-site.sh`** — create a new site: Gitea repo, Garage bucket, web hosting enabled.
+9
View File
@@ -28,6 +28,13 @@ inputs:
description: Gitea username for git operations
required: false
default: ci-bot
artifacts:
# Scopes building and publishing only. Routes, Ingresses and the immutable
# -path history are always rendered from the whole site.yaml, so a scoped
# run never retires another artifact's route.
description: Space- or comma-separated subset of site.yaml artifacts to build and publish (default is every declared artifact)
required: false
default: ''
runs:
using: composite
steps:
@@ -44,6 +51,7 @@ runs:
ACTION_DIR: ${{ github.action_path }}
GITHUB_RUN_NUMBER: ${{ github.run_number }}
CI_BOT_USER: ${{ inputs.username }}
SITE_ARTIFACTS: ${{ inputs.artifacts }}
- name: Deploy
shell: bash
@@ -61,3 +69,4 @@ runs:
GARAGE_ADMIN_ENDPOINT: ${{ inputs.garage-admin-endpoint }}
GARAGE_ADMIN_TOKEN: ${{ inputs.garage-admin-token }}
GITHUB_RUN_NUMBER: ${{ github.run_number }}
SITE_ARTIFACTS: ${{ inputs.artifacts }}
+11 -2
View File
@@ -5,7 +5,14 @@ import subprocess
import tempfile
from pathlib import Path
from utils import EXCLUDE_FILES, env, parse_site_yaml, run
from utils import (
EXCLUDE_FILES,
env,
parse_site_yaml,
run,
selected_artifacts,
validate_artifact_inputs,
)
def build_artifact(site_dir, artifact):
@@ -64,5 +71,7 @@ def cmd_build():
print("Site disabled — skipping build")
return
for artifact in cfg["artifacts"]:
validate_artifact_inputs(site_dir, cfg)
for artifact in selected_artifacts(cfg):
build_artifact(site_dir, artifact)
+451 -43
View File
@@ -1,13 +1,20 @@
"""Deploy phase — S3 sync, manifest rendering, alias reconcile."""
import fnmatch
import hashlib
import json
import mimetypes
import os
import re
import shutil
import subprocess
import tempfile
from pathlib import Path
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
import yaml
from utils import (
NAMESPACE,
clone_apps,
@@ -18,6 +25,9 @@ from utils import (
parse_site_yaml,
render_templates,
run,
selected_artifacts,
selected_routes,
validate_artifact_inputs,
)
GARAGE_ADMIN_ENDPOINT = os.environ.get(
@@ -39,17 +49,113 @@ def validate_artifact_output(site_dir, artifact):
def validate_publication_environment(cfg):
"""Resolve every declared credential before the first bucket is changed."""
for artifact in cfg["artifacts"]:
"""Resolve every credential this run needs before the first bucket is changed."""
for artifact in selected_artifacts(cfg):
env(artifact["credentials"]["access_key_env"])
env(artifact["credentials"]["secret_key_env"])
if cfg["compatibility"] and cfg["aliases"] and not os.environ.get("GARAGE_ADMIN_TOKEN"):
die("GARAGE_ADMIN_TOKEN is required when aliases are declared")
def s3_sync(artifact, route, site_dir, credential_env_names=None):
endpoint = artifact["s3_endpoint"]
html_dir = site_dir / artifact["build_dir"]
def _is_immutable(rule):
return "immutable" in {
part.strip().lower().split("=", 1)[0]
for part in rule["cache_control"].split(",")
}
def _aws_capture(args, aws_env):
"""Run a non-streaming AWS request without exposing environment credentials."""
print(f" $ {' '.join(str(part) for part in args)}")
return subprocess.run(args, env=aws_env, text=True, capture_output=True, check=False)
def _immutable_head(endpoint, bucket, key, aws_env):
args = ["aws", "--endpoint-url", endpoint, "s3api", "head-object",
"--bucket", bucket, "--key", key, "--output", "json"]
result = _aws_capture(args, aws_env)
if result.returncode == 0:
return json.loads(result.stdout)
error = f"{result.stdout}\n{result.stderr}"
if any(marker in error for marker in ("404", "Not Found", "NoSuchKey")):
return None
raise RuntimeError(f"head-object failed for s3://{bucket}/{key}: {error.strip()}")
def _immutable_digests(source, cache_control, content_type):
with source.open("rb") as stream:
content_digest = hashlib.file_digest(stream, "sha256").hexdigest()
publication = hashlib.sha256()
publication.update(cache_control.encode())
publication.update(b"\0")
publication.update(content_type.encode())
publication.update(b"\0")
with source.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
publication.update(block)
return content_digest, publication.hexdigest()
def _same_immutable_object(info, content_digest, publication_digest, cache_control, content_type):
metadata = {key.lower(): value for key, value in (info.get("Metadata") or {}).items()}
return (
metadata.get("sha256") == content_digest
and metadata.get("publication-sha256") == publication_digest
and info.get("CacheControl") == cache_control
and info.get("ContentType") == content_type
)
def publish_immutable_file(endpoint, bucket, key, source, cache_control, aws_env):
"""Publish a content-addressed key; identical retries converge."""
content_type = mimetypes.guess_type(source.name)[0] or "application/octet-stream"
content_digest, publication_digest = _immutable_digests(source, cache_control, content_type)
address_digests = re.findall(r"(?<![0-9a-f])([0-9a-f]{64})(?![0-9a-f])", key.lower())
if address_digests != [publication_digest]:
raise RuntimeError(
f"immutable key must contain its one publication SHA-256 {publication_digest}: "
f"s3://{bucket}/{key}"
)
existing = _immutable_head(endpoint, bucket, key, aws_env)
if existing is not None:
if _same_immutable_object(
existing, content_digest, publication_digest, cache_control, content_type,
):
print(f" Immutable object already matches: s3://{bucket}/{key}")
return False
raise RuntimeError(f"immutable object differs or lacks publisher digest: s3://{bucket}/{key}")
args = ["aws", "--endpoint-url", endpoint, "s3api", "put-object",
"--bucket", bucket, "--key", key, "--body", str(source),
"--content-type", content_type, "--cache-control", cache_control,
"--metadata", f"sha256={content_digest},publication-sha256={publication_digest}"]
result = _aws_capture(args, aws_env)
if result.returncode == 0:
return True
error = f"{result.stdout}\n{result.stderr}"
raise RuntimeError(f"put-object failed for s3://{bucket}/{key}: {error.strip()}")
def publish_immutable_rule(artifact, route, rule, html_dir, aws_env):
"""Publish one immutable cache partition without overwrite or deletion."""
rule_root = html_dir / rule["path"]
child_paths = [candidate["path"] for candidate in artifact["cache_rules"]
if candidate["path"].startswith(f"{rule['path'].rstrip('/')}/")]
object_prefix = route["path"].strip("/")
for source in sorted(path for path in rule_root.rglob("*") if path.is_file()):
relative = source.relative_to(html_dir).as_posix()
if any(relative == child or relative.startswith(f"{child}/") for child in child_paths):
continue
if any(fnmatch.fnmatch(relative, pattern) for pattern in artifact["excludes"]):
continue
key = "/".join(part for part in (object_prefix, relative) if part)
publish_immutable_file(
artifact["s3_endpoint"], artifact["bucket"], key, source,
rule["cache_control"], aws_env,
)
def publication_aws_env(artifact, credential_env_names=None):
"""Build the route-scoped AWS environment without leaking other credentials."""
access_key = env(artifact["credentials"]["access_key_env"])
secret_key = env(artifact["credentials"]["secret_key_env"])
aws_env = os.environ.copy()
@@ -63,11 +169,160 @@ def s3_sync(artifact, route, site_dir, credential_env_names=None):
"AWS_SECRET_ACCESS_KEY": secret_key,
"AWS_DEFAULT_REGION": os.environ.get("AWS_DEFAULT_REGION", "sjc001"),
})
return aws_env
def configure_cors(bucket, origins, endpoint, aws_env):
"""Reconcile read-only browser access without exposing publication credentials."""
if origins is None:
return
config = None
if origins:
config = {
"CORSRules": [{
"AllowedOrigins": origins,
"AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600,
}],
}
set_cors_configuration(bucket, config, endpoint, aws_env)
def set_cors_configuration(bucket, config, endpoint, aws_env):
"""Apply an exact bucket CORS configuration, or remove it when absent."""
if config is None:
run([
"aws", "--endpoint-url", endpoint, "s3api", "delete-bucket-cors",
"--bucket", bucket,
], env=aws_env)
return
with tempfile.NamedTemporaryFile("w", suffix=".json", encoding="utf-8") as handle:
json.dump(config, handle)
handle.flush()
run([
"aws", "--endpoint-url", endpoint, "s3api", "put-bucket-cors",
"--bucket", bucket, "--cors-configuration", f"file://{handle.name}",
], env=aws_env)
def get_cors_configuration(bucket, endpoint, aws_env):
"""Read the exact bucket CORS configuration for rollback."""
result = _aws_capture([
"aws", "--endpoint-url", endpoint, "s3api", "get-bucket-cors",
"--bucket", bucket, "--output", "json",
], aws_env)
if result.returncode == 0:
try:
config = json.loads(result.stdout)
except json.JSONDecodeError as error:
raise RuntimeError(f"get-bucket-cors returned invalid JSON for {bucket}") from error
if not isinstance(config, dict) or not isinstance(config.get("CORSRules"), list):
raise RuntimeError(f"get-bucket-cors returned an invalid policy for {bucket}")
return config
error = f"{result.stdout}\n{result.stderr}"
if "NoSuchCORSConfiguration" in error:
return None
raise RuntimeError(f"get-bucket-cors failed for {bucket}: {error.strip()}")
def reconcile_artifact_cors(artifacts, credential_env_names=None):
"""Reconcile all policies, restoring the prior set if any write fails."""
snapshots = []
for artifact in artifacts:
if artifact["cors_origins"] is None:
continue
aws_env = publication_aws_env(artifact, credential_env_names)
snapshots.append((
artifact,
aws_env,
get_cors_configuration(artifact["bucket"], artifact["s3_endpoint"], aws_env),
))
attempted = []
try:
for artifact, aws_env, previous in snapshots:
attempted.append((artifact, aws_env, previous))
configure_cors(
artifact["bucket"], artifact["cors_origins"], artifact["s3_endpoint"], aws_env,
)
except Exception as error:
rollback_errors = []
for artifact, aws_env, previous in reversed(attempted):
try:
set_cors_configuration(
artifact["bucket"], previous, artifact["s3_endpoint"], aws_env,
)
except Exception as rollback_error:
rollback_errors.append(f"{artifact['bucket']}: {rollback_error}")
if rollback_errors:
raise RuntimeError(
f"CORS reconciliation failed ({error}); rollback also failed for "
f"{'; '.join(rollback_errors)}"
) from error
raise
def publish_route_immutables(artifact, route, site_dir, credential_env_names=None):
"""Publish one route's immutable partitions during the global preflight."""
html_dir = site_dir / artifact["build_dir"]
aws_env = publication_aws_env(artifact, credential_env_names)
for rule in artifact["cache_rules"]:
if _is_immutable(rule):
publish_immutable_rule(artifact, route, rule, html_dir, aws_env)
def immutable_key_prefixes(artifact, route):
"""Return immutable partitions as bucket-relative key prefixes."""
route_prefix = route["path"].strip("/")
return [
"/".join(part for part in (route_prefix, rule["path"]) if part)
for rule in artifact["cache_rules"] if _is_immutable(rule)
]
def retained_immutable_paths(artifact, route, previous_contract):
"""Carry all bucket history forward so later route moves cannot delete it."""
previous_paths = previous_contract["immutable_paths"] if previous_contract else []
return sorted(set(previous_paths) | set(immutable_key_prefixes(artifact, route)))
def retired_immutable_filters(artifact, route, html_dir, previous_contract):
"""Protect historical immutable keys that fall inside the current sync scope."""
if not previous_contract:
return []
current_prefix = route["path"].strip("/")
filters = []
current_immutable = set(immutable_key_prefixes(artifact, route))
for immutable_path in previous_contract["immutable_paths"]:
if immutable_path in current_immutable:
continue
if current_prefix:
marker = f"{current_prefix}/"
if not immutable_path.startswith(marker):
continue
retired_path = immutable_path[len(marker):]
else:
retired_path = immutable_path
collision = html_dir / retired_path
if collision.exists() and any(path.is_file() for path in collision.rglob("*")):
raise RuntimeError(
f"current artifact collides with retired immutable partition: {retired_path}"
)
filters.extend(("--exclude", f"{retired_path}/*"))
return filters
def s3_sync(artifact, route, site_dir, credential_env_names=None, previous_contract=None):
endpoint = artifact["s3_endpoint"]
html_dir = site_dir / artifact["build_dir"]
aws_env = publication_aws_env(artifact, credential_env_names)
bucket = artifact["bucket"]
object_prefix = route["path"].strip("/")
bucket_destination = f"s3://{bucket}/"
destination = f"{bucket_destination}{object_prefix + '/' if object_prefix else ''}"
destination = f"s3://{bucket}/{object_prefix + '/' if object_prefix else ''}"
default_cache = next(rule["cache_control"] for rule in artifact["cache_rules"] if not rule["path"])
immutable_paths = [rule["path"] for rule in artifact["cache_rules"] if _is_immutable(rule)]
# `excludes` are patterns (site.yaml `excludes:` list) that should never
# be uploaded *and* should never be deleted from the bucket — escape hatch
# for assets managed out-of-band (e.g. large PDFs uploaded via aws-cli).
@@ -75,41 +330,30 @@ def s3_sync(artifact, route, site_dir, credential_env_names=None):
if artifact["excludes"]:
print(f"Excluding patterns: {artifact['excludes']}")
print(f"Syncing artifact {artifact['name']}{destination} via {endpoint}")
# `sync --delete` handles new/changed/orphaned files. Partitioned
# `cp --recursive` calls then re-upload each file once to refresh metadata
# (cache-control, content-type) on objects sync skipped as unchanged.
# Upload with the final cache policy before cleanup. Sync and deletion are
# scoped to the same current route prefix and cache partition. A route move
# leaves its old bucket partition intact but unreachable after the old
# Ingress disappears, while stale mutable keys on the serving prefix are
# deleted. Immutable subtrees are structurally excluded. `cp --recursive`
# refreshes metadata atomically per object before `sync --delete` removes
# stale keys without ever exposing new bytes under a provisional policy.
# A no-op deploy therefore transfers the artifact bytes once.
# AWS CLI guesses Content-Type from file extension on local→S3 uploads,
# so a fresh upload always carries the right MIME type.
stage = None
sync_source = html_dir
sync_excludes = exclude_args
if object_prefix:
stage = tempfile.TemporaryDirectory()
sync_source = Path(stage.name)
staged_artifact = sync_source / object_prefix
staged_artifact.parent.mkdir(parents=True, exist_ok=True)
staged_artifact.symlink_to(html_dir.resolve(), target_is_directory=True)
sync_excludes = [arg for pattern in artifact["excludes"]
for arg in ("--exclude", f"{object_prefix}/{pattern}")]
try:
# Sync the complete bucket authority so moving a route prefix also
# deletes objects under its old prefix instead of leaving them public.
run(["aws", "--endpoint-url", endpoint, "s3", "sync", f"{sync_source}/", bucket_destination,
"--delete", "--only-show-errors", "--cache-control", default_cache,
*sync_excludes], env=aws_env)
finally:
if stage:
stage.cleanup()
print("Re-stamping metadata on all objects...")
specific_paths = [rule["path"] for rule in artifact["cache_rules"] if rule["path"]]
default_filters = [arg for path in specific_paths for arg in ("--exclude", f"{path}/*")]
retired_filters = retired_immutable_filters(artifact, route, html_dir, previous_contract)
run(["aws", "--endpoint-url", endpoint, "s3", "cp", f"{html_dir}/", destination,
"--recursive", "--only-show-errors", "--cache-control", default_cache,
*default_filters, *exclude_args], env=aws_env)
*default_filters, *retired_filters, *exclude_args], env=aws_env)
run(["aws", "--endpoint-url", endpoint, "s3", "sync", f"{html_dir}/", destination,
"--delete", "--only-show-errors", "--cache-control", default_cache,
*default_filters, *retired_filters, *exclude_args], env=aws_env)
for rule in artifact["cache_rules"]:
if not rule["path"]:
continue
if _is_immutable(rule):
continue
include = f"{rule['path'].rstrip('/')}/*"
child_filters = [arg for path in specific_paths
if path.startswith(f"{rule['path'].rstrip('/')}/")
@@ -119,6 +363,9 @@ def s3_sync(artifact, route, site_dir, credential_env_names=None):
run(["aws", "--endpoint-url", endpoint, "s3", "cp", f"{html_dir}/", destination,
"--recursive", "--only-show-errors", "--cache-control", rule["cache_control"],
"--exclude", "*", "--include", include, *child_filters, *exclude_args], env=aws_env)
run(["aws", "--endpoint-url", endpoint, "s3", "sync", f"{html_dir}/", destination,
"--delete", "--only-show-errors", "--cache-control", rule["cache_control"],
"--exclude", "*", "--include", include, *child_filters, *exclude_args], env=aws_env)
def garage_admin(method, path, token, body=None):
@@ -167,16 +414,32 @@ def ensure_bucket_aliases(site_name, aliases, admin_token):
raise
def render_site_manifests(site_name, action_dir, app_dir, manifests_dir, cfg):
def render_site_manifests(
site_name, action_dir, app_dir, manifests_dir, cfg, previous_contracts=None,
):
"""Always re-render manifests from current site.yaml. Templates own
domain + aliases, so changes propagate without manual edits."""
manifests_dir.mkdir(parents=True, exist_ok=True)
artifact_by_name = {artifact["name"]: artifact for artifact in cfg["artifacts"]}
routes = []
previous_contracts = previous_contracts or {}
next_contracts = {bucket: dict(contract)
for bucket, contract in previous_contracts.items()}
for route in cfg["routes"]:
artifact = artifact_by_name[route["artifact"]]
resource_name = k8s_name(site_name) if cfg["compatibility"] else f"{k8s_name(site_name)}-{route['name']}"
routes.append({**route, "resource_name": resource_name, "artifact_config": artifact})
previous = previous_contracts.get(artifact["bucket"])
immutable_paths = retained_immutable_paths(artifact, route, previous)
next_contracts[artifact["bucket"]] = {
"path": route["path"],
"access": "public" if route["access"] == "legacy" else route["access"],
"artifact": route["artifact"],
"immutable_paths": immutable_paths,
}
routes.append({
**route, "resource_name": resource_name, "artifact_config": artifact,
"immutable_paths_json": json.dumps(immutable_paths, separators=(",", ":")),
})
template_vars = {
"site": site_name,
"site_k8s": k8s_name(site_name),
@@ -187,6 +450,126 @@ def render_site_manifests(site_name, action_dir, app_dir, manifests_dir, cfg):
"routes": routes,
}
render_templates(action_dir, template_vars, app_dir, manifests_dir)
if not cfg["compatibility"] or previous_contracts:
(app_dir / "site-publish-history.yaml").write_text(yaml.safe_dump(
{"version": 1, "buckets": next_contracts}, sort_keys=True,
))
def _validate_route_contract(bucket, contract, path):
expected = {"path", "access", "artifact", "immutable_paths"}
if (not isinstance(bucket, str) or not bucket or not isinstance(contract, dict)
or set(contract) != expected
or contract.get("access") not in {"public", "protected"}
or not isinstance(contract.get("path"), str)
or not contract["path"].startswith("/")
or not isinstance(contract.get("artifact"), str) or not contract["artifact"]
or not isinstance(contract.get("immutable_paths"), list)
or any(not isinstance(item, str) or not item or item.startswith("/")
or any(part in {"", ".", ".."} for part in item.split("/"))
for item in contract["immutable_paths"])):
raise RuntimeError(f"invalid site-publish route history in {path}")
return {
"path": contract["path"], "access": contract["access"],
"artifact": contract["artifact"],
"immutable_paths": sorted(set(contract["immutable_paths"])),
}
def previous_route_contracts(app_dir):
"""Read bucket-keyed route history from generated Ingresses."""
history_path = app_dir / "site-publish-history.yaml"
if history_path.exists():
document = yaml.safe_load(history_path.read_text())
if (not isinstance(document, dict) or set(document) != {"version", "buckets"}
or document["version"] != 1 or not isinstance(document["buckets"], dict)):
raise RuntimeError(f"invalid site-publish route history in {history_path}")
return {
bucket: _validate_route_contract(bucket, contract, history_path)
for bucket, contract in document["buckets"].items()
}
contracts = {}
manifests = app_dir / "manifests"
if not manifests.exists():
return contracts
for path in sorted(manifests.glob("ingress*.yaml")):
document = yaml.safe_load(path.read_text()) or {}
annotations = document.get("metadata", {}).get("annotations", {})
artifact = annotations.get("site-publish.fritzlab.net/artifact")
access = annotations.get("site-publish.fritzlab.net/access")
bucket = annotations.get("site-publish.fritzlab.net/bucket")
immutable_paths_json = annotations.get("site-publish.fritzlab.net/immutable-paths")
route_path = annotations.get("site-publish.fritzlab.net/route-path")
values = (artifact, access, bucket, immutable_paths_json, route_path)
if all(value is None for value in values):
continue
if (not all(isinstance(value, str) for value in values)
or access not in {"public", "protected"} or not route_path.startswith("/")):
raise RuntimeError(f"invalid site-publish route history in {path}")
try:
immutable_paths = json.loads(immutable_paths_json)
except json.JSONDecodeError as exc:
raise RuntimeError(f"invalid site-publish route history in {path}") from exc
if not isinstance(immutable_paths, list) or any(not isinstance(item, str) for item in immutable_paths):
raise RuntimeError(f"invalid site-publish route history in {path}")
if bucket in contracts:
raise RuntimeError(f"duplicate site-publish route history for bucket {bucket}")
contracts[bucket] = _validate_route_contract(bucket, {
"path": route_path, "access": access, "artifact": artifact,
"immutable_paths": immutable_paths,
}, path)
return contracts
def validate_route_migrations(cfg, previous_contracts):
artifacts = {artifact["name"]: artifact for artifact in cfg["artifacts"]}
for route in cfg["routes"]:
artifact = artifacts[route["artifact"]]
previous = previous_contracts.get(artifact["bucket"])
if (previous and previous["access"] == "protected"
and route["access"] in {"public", "legacy"}
):
raise RuntimeError(
f"artifact {route['artifact']} cannot become public while reusing protected "
f"bucket {artifact['bucket']}"
)
def validate_scoped_history(cfg, previous_contracts):
"""A scoped run may not record a route contract it did not publish.
render_site_manifests advances the stored contract for every route in
site.yaml, and `access` is overwritten rather than unioned the way
immutable_paths is. Without this, a catalogue-only publish could write a
protected access for the distributions bucket that nothing published, and
validate_route_migrations would then refuse to put that bucket back —
unpublished intent turned into an irreversible fact.
"""
chosen = set(cfg["selected"])
if len(chosen) == len(cfg["artifacts"]):
return
artifact_by_name = {artifact["name"]: artifact for artifact in cfg["artifacts"]}
for route in cfg["routes"]:
if route["artifact"] in chosen:
continue
artifact = artifact_by_name[route["artifact"]]
previous = previous_contracts.get(artifact["bucket"])
if previous is None:
raise RuntimeError(
f"a scoped run cannot introduce artifact {route['artifact']}; "
f"publish it in the same run"
)
current = {
"path": route["path"],
"access": "public" if route["access"] == "legacy" else route["access"],
"artifact": route["artifact"],
}
recorded = {key: previous[key] for key in current}
if recorded != current:
raise RuntimeError(
f"a scoped run may not change unselected artifact {route['artifact']}'s route "
f"contract ({recorded} -> {current}); publish it in the same run"
)
def deploy_static(site_name, site_dir, action_dir, token, cfg):
@@ -194,21 +577,43 @@ def deploy_static(site_name, site_dir, action_dir, token, cfg):
credential_env_names = {
name for artifact in cfg["artifacts"] for name in artifact["credentials"].values()
}
# Publication is scoped to the selected artifacts; the rendered route
# contract is not. Manifests and immutable-path history stay whole, so a
# partial publish can never retire another artifact's route or bucket.
publishing = selected_routes(cfg)
validate_publication_environment(cfg)
for artifact in cfg["artifacts"]:
for artifact in selected_artifacts(cfg):
validate_artifact_output(site_dir, artifact)
for route in cfg["routes"]:
s3_sync(artifact_by_name[route["artifact"]], route, site_dir, credential_env_names)
if cfg["compatibility"]:
ensure_bucket_aliases(site_name, cfg["aliases"], os.environ.get("GARAGE_ADMIN_TOKEN"))
apps_dir = clone_apps(token)
app_dir = apps_dir / "sjc001" / "websites" / site_name
manifests_dir = app_dir / "manifests"
previous_contracts = previous_route_contracts(app_dir)
validate_route_migrations(cfg, previous_contracts)
validate_scoped_history(cfg, previous_contracts)
# Complete immutable work across the whole publication before any route's
# mutable pointers can change. Partial immutable success is safe; mixing a
# new route with an old route after a later immutable failure is not.
for route in publishing:
publish_route_immutables(
artifact_by_name[route["artifact"]], route, site_dir, credential_env_names,
)
# Reconcile every browser-read policy before publishing mutable content.
# A CORS failure therefore cannot leave a new channel pointing at a release
# whose cross-origin assets browsers cannot consume.
reconcile_artifact_cors(selected_artifacts(cfg), credential_env_names)
for route in publishing:
s3_sync(
artifact_by_name[route["artifact"]], route, site_dir, credential_env_names,
previous_contracts.get(artifact_by_name[route["artifact"]]["bucket"]),
)
if cfg["compatibility"]:
ensure_bucket_aliases(site_name, cfg["aliases"], os.environ.get("GARAGE_ADMIN_TOKEN"))
render_site_manifests(site_name, action_dir, app_dir, manifests_dir, cfg)
render_site_manifests(
site_name, action_dir, app_dir, manifests_dir, cfg, previous_contracts,
)
commit_and_push(apps_dir, f"Deploy {site_name}")
commit_and_push(apps_dir, f"Deploy {site_name}", token)
def decommission(site_name, token, buckets=None):
@@ -219,7 +624,7 @@ def decommission(site_name, token, buckets=None):
print(f"No manifests for {site_name} — nothing to remove")
return
shutil.rmtree(site_path)
commit_and_push(apps_dir, f"Decommission {site_name}")
commit_and_push(apps_dir, f"Decommission {site_name}", token)
for bucket in buckets or [site_name]:
print(f"Bucket {bucket} and its objects are NOT purged automatically.")
print(f" garage bucket delete {bucket} --yes")
@@ -235,8 +640,11 @@ def cmd_deploy():
cfg = parse_site_yaml(site_dir)
if not cfg["enabled"]:
if len(cfg["selected"]) != len(cfg["artifacts"]):
die("a disabled site decommissions whole; drop the artifacts selection")
print("Site disabled — running decommission...")
decommission(site_name, token, [artifact["bucket"] for artifact in cfg["artifacts"]])
return
validate_artifact_inputs(site_dir, cfg)
deploy_static(site_name, site_dir, action_dir, token, cfg)
+165 -13
View File
@@ -1,5 +1,6 @@
"""Shared utilities for the site-publish action."""
import ipaddress
import os
import re
import shutil
@@ -26,6 +27,7 @@ EXCLUDE_FILES = {
VALID_TYPES = {"static", "hugo", "mkdocs"}
NAME_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$")
ENV_RE = re.compile(r"^[A-Z_][A-Z0-9_]*$")
ACCESS_KEY_ENV_RE = re.compile(r"^([A-Z][A-Z0-9_]*)_S3_ACCESS_KEY$")
BUCKET_RE = re.compile(r"^[a-z0-9](?:[a-z0-9.-]{1,61}[a-z0-9])?$")
MIDDLEWARE_RE = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9_-]{0,61}[A-Za-z0-9])?$")
@@ -103,6 +105,55 @@ def _strings(value, label):
return values
def _cors_origins(value, label):
origins = _list(value, label)
if any(not isinstance(item, str) or not item for item in origins):
raise ConfigError(f"{label} must be a list of non-empty strings")
canonical = []
for origin in origins:
if origin == "*":
canonical.append(origin)
continue
parsed = urlparse(origin)
try:
port = parsed.port
except ValueError:
raise ConfigError(f"{label} must contain '*' or canonical HTTPS origins") from None
if parsed.scheme != "https" or not parsed.hostname or parsed.path or parsed.params or (
parsed.query or parsed.fragment or parsed.username or parsed.password
):
raise ConfigError(f"{label} must contain '*' or canonical HTTPS origins")
try:
address = ipaddress.ip_address(parsed.hostname)
except ValueError:
try:
hostname = parsed.hostname.encode("idna").decode("ascii")
except UnicodeError:
raise ConfigError(
f"{label} must contain '*' or canonical HTTPS origins"
) from None
try:
_hostname(hostname, f"{label} hostname")
except ConfigError:
raise ConfigError(
f"{label} must contain '*' or canonical HTTPS origins"
) from None
else:
if getattr(address, "scope_id", None) is not None:
raise ConfigError(
f"{label} must contain '*' or canonical HTTPS origins"
)
hostname = f"[{address.compressed}]" if address.version == 6 else address.compressed
canonical.append(f"https://{hostname}{f':{port}' if port not in (None, 443) else ''}")
if len(canonical) != len(set(canonical)):
raise ConfigError(f"{label} must not contain duplicate canonical origins")
if "*" in canonical and len(canonical) != 1:
raise ConfigError(f"{label} wildcard must be the only origin")
if origins != canonical:
raise ConfigError(f"{label} must contain '*' or canonical HTTPS origins")
return origins
def _hostname(value, label):
if not isinstance(value, str) or len(value) > 253 or value.endswith("."):
raise ConfigError(f"{label} must be a lowercase DNS hostname without a trailing dot")
@@ -214,6 +265,7 @@ def _legacy_config(raw, site_name):
"website_authority": "garage-s3.storage.svc.k8s.sjc001.fritzlab.net",
"credentials": {"access_key_env": "AWS_ACCESS_KEY_ID", "secret_key_env": "AWS_SECRET_ACCESS_KEY"},
"cache_rules": [{"path": "", "cache_control": DEFAULT_CACHE_CONTROL}],
"cors_origins": None,
}
return {
"version": 1,
@@ -222,6 +274,7 @@ def _legacy_config(raw, site_name):
"aliases": _aliases(raw.get("aliases"), raw["domain"]),
"enabled": raw.get("enabled", True),
"artifacts": [artifact],
"selected": ["site"],
"routes": [{
"name": "site", "path": "/", "artifact": "site", "access": "legacy",
"access_middleware": None,
@@ -233,12 +286,16 @@ def _legacy_config(raw, site_name):
def _artifact(item, index):
label = f"artifacts[{index}]"
item = _mapping(item, label)
_known_keys(item, {"name", "type", "content_dir", "tidy", "excludes", "publish", "cache"}, label)
_known_keys(
item,
{"name", "type", "content_dir", "tidy", "excludes", "publish", "cache", "cors_origins"},
label,
)
name = item.get("name")
if not isinstance(name, str) or not NAME_RE.fullmatch(name):
raise ConfigError(f"{label}.name must be a DNS label")
publish = _mapping(item.get("publish"), f"{label}.publish")
_known_keys(publish, {"bucket", "endpoint", "website_authority", "credentials"}, f"{label}.publish")
_known_keys(publish, {"bucket", "credentials"}, f"{label}.publish")
bucket = publish.get("bucket")
if not isinstance(bucket, str) or not BUCKET_RE.fullmatch(bucket):
raise ConfigError(f"{label}.publish.bucket is not a valid bucket name")
@@ -250,6 +307,15 @@ def _artifact(item, index):
if not isinstance(value, str) or not ENV_RE.fullmatch(value):
raise ConfigError(f"{label}.publish.credentials.{key} must name an environment variable")
normalized_credentials[key] = value
access_match = ACCESS_KEY_ENV_RE.fullmatch(normalized_credentials["access_key_env"])
expected_secret = (
f"{access_match.group(1)}_S3_SECRET_KEY" if access_match else None
)
if normalized_credentials["secret_key_env"] != expected_secret:
raise ConfigError(
f"{label}.publish.credentials must be a matched "
"<NAME>_S3_ACCESS_KEY and <NAME>_S3_SECRET_KEY pair"
)
cache = _mapping(item.get("cache"), f"{label}.cache")
_known_keys(cache, {"rules"}, f"{label}.cache")
rules = _list(cache.get("rules"), f"{label}.cache.rules")
@@ -270,10 +336,15 @@ def _artifact(item, index):
if "" not in paths:
raise ConfigError(f"{label}.cache.rules must declare a '/' default")
cache_rules.sort(key=lambda rule: (len(PurePosixPath(rule["path"]).parts), rule["path"]))
authority = _hostname(
publish.get("website_authority", f"{bucket}.{DEFAULT_WEBSITE_SUFFIX}"),
f"{label}.publish.website_authority",
)
immutable_paths = [rule["path"] for rule in cache_rules
if "immutable" in {part.strip().lower().split("=", 1)[0]
for part in rule["cache_control"].split(",")}]
if "" in immutable_paths:
raise ConfigError(f"{label}.cache.rules immutable paths must be narrower than '/'")
for path in immutable_paths:
if any(other.startswith(f"{path}/") for other in paths):
raise ConfigError(f"{label}.cache.rules cannot nest another policy under immutable /{path}")
authority = f"{bucket}.{DEFAULT_WEBSITE_SUFFIX}"
if not isinstance(item.get("tidy", True), bool):
raise ConfigError(f"{label}.tidy must be a boolean")
return {
@@ -284,10 +355,14 @@ def _artifact(item, index):
"excludes": _strings(item.get("excludes") or [], f"{label}.excludes"),
"build_dir": f".site-publish/{name}/html",
"bucket": bucket,
"s3_endpoint": _endpoint(publish.get("endpoint", DEFAULT_S3_ENDPOINT), f"{label}.publish.endpoint"),
"s3_endpoint": DEFAULT_S3_ENDPOINT,
"website_authority": authority,
"credentials": normalized_credentials,
"cache_rules": cache_rules,
"cors_origins": (
_cors_origins(item["cors_origins"], f"{label}.cors_origins")
if "cors_origins" in item else []
),
}
@@ -374,11 +449,42 @@ def _validate_multi(cfg):
raise ConfigError(f"protected route {route['name']} cannot use shared-cache max-age")
if route["access"] == "public" and "private" in directives:
raise ConfigError(f"public route {route['name']} cannot use private cache policy")
if route["access"] == "protected" and "*" in artifact["cors_origins"]:
raise ConfigError(
f"protected route {route['name']} cannot allow wildcard CORS"
)
root = next(route for route in routes if route["path"] == "/")
if any(route["access"] == "protected" for route in routes) and root["access"] == "public":
raise ConfigError("a public '/' catch-all would expose unmatched protected content")
def _artifact_selection(cfg, requested):
"""Resolve the subset of declared artifacts this run publishes."""
declared = [artifact["name"] for artifact in cfg["artifacts"]]
if not requested or not requested.strip():
return declared
names = [part for part in re.split(r"[,\s]+", requested.strip()) if part]
unknown = sorted({name for name in names if name not in declared})
if unknown:
raise ConfigError(
"artifacts selects undeclared artifact(s) " + ", ".join(unknown)
+ "; site.yaml declares " + ", ".join(declared)
)
return [name for name in declared if name in set(names)]
def selected_artifacts(cfg):
"""Artifacts this run builds and publishes, in declaration order."""
chosen = set(cfg["selected"])
return [artifact for artifact in cfg["artifacts"] if artifact["name"] in chosen]
def selected_routes(cfg):
"""Routes whose artifact this run publishes, in route order."""
chosen = set(cfg["selected"])
return [route for route in cfg["routes"] if route["artifact"] in chosen]
def normalize_site_config(raw, site_name):
raw = _mapping(raw, "site.yaml")
domain = _hostname(raw.get("domain"), "domain")
@@ -404,12 +510,49 @@ def normalize_site_config(raw, site_name):
"routes": sorted(routes, key=lambda item: (-len(item["path"]), item["path"], item["name"])),
}
_validate_multi(cfg)
cfg["selected"] = [artifact["name"] for artifact in cfg["artifacts"]]
for route in cfg["routes"]:
if len(f"{k8s_name(site_name)}-{route['name']}") > 63:
raise ConfigError(f"route {route['name']} makes the generated Service name exceed 63 characters")
return cfg
def validate_artifact_inputs(site_dir, cfg):
"""Reject source containment before a public or protected build starts."""
if cfg["compatibility"]:
return
root = Path(site_dir).resolve()
sources = []
for artifact in cfg["artifacts"]:
declared = root
for component in Path(artifact["content_dir"]).parts:
declared /= component
if declared.is_symlink():
raise ConfigError(
f"artifact {artifact['name']} content_dir contains symlink component: "
f"{declared.relative_to(root)}"
)
source = declared.resolve()
if source != root and root not in source.parents:
raise ConfigError(
f"artifact {artifact['name']} content_dir resolves outside the repository"
)
if source.exists():
symlink = next((path for path in source.rglob("*") if path.is_symlink()), None)
if symlink is not None:
raise ConfigError(
f"artifact {artifact['name']} build input contains symlink: "
f"{symlink.relative_to(root)}"
)
sources.append((artifact["name"], source))
for index, (name, source) in enumerate(sources):
for other_name, other_source in sources[index + 1:]:
if source == other_source or source in other_source.parents or other_source in source.parents:
raise ConfigError(
f"artifact build inputs overlap after resolution: {name} and {other_name}"
)
def parse_site_yaml(site_dir, site_name=None):
path = Path(site_dir) / "site.yaml"
if not path.exists():
@@ -420,12 +563,14 @@ def parse_site_yaml(site_dir, site_name=None):
try:
with open(path) as stream:
cfg = normalize_site_config(yaml.safe_load(stream), site_name)
cfg["selected"] = _artifact_selection(cfg, os.environ.get("SITE_ARTIFACTS"))
except (ConfigError, yaml.YAMLError) as exc:
die(str(exc))
print("Site config:")
print(f" domain: {cfg['domain']}")
print(f" contract: {cfg['compatibility'] or 'split-surface-v2'}")
print(f" artifacts: {[item['name'] for item in cfg['artifacts']]}")
print(f" publishing: {cfg['selected']}")
print(f" routes: {[(item['path'], item['access']) for item in cfg['routes']]}")
return cfg
@@ -436,10 +581,7 @@ def clone_apps(token):
apps_dir = Path("/tmp/apps-deploy")
if apps_dir.exists():
shutil.rmtree(apps_dir)
clone_env = os.environ.copy()
clone_env["CI_BOT_TOKEN"] = token
clone_env["GIT_ASKPASS"] = str(Path(__file__).with_name("git-askpass.sh"))
clone_env["GIT_TERMINAL_PROMPT"] = "0"
clone_env = git_auth_env(token)
url = f"https://{user}@{GITEA_HOST}/{APPS_REPO}.git"
run(["git", "clone", "--depth", "1", url, str(apps_dir)],
display=f"git clone --depth 1 https://{user}@{GITEA_HOST}/{APPS_REPO}.git {apps_dir}", env=clone_env)
@@ -477,13 +619,23 @@ def render_templates(action_dir, template_vars, app_dir, manifests_dir):
print(f" Rendered {out_name}.j2 -> {destination}")
def commit_and_push(apps_dir, message):
def git_auth_env(token):
"""Return credential-safe Git authentication shared by clone and push."""
auth_env = os.environ.copy()
auth_env["CI_BOT_TOKEN"] = token
auth_env["GIT_ASKPASS"] = str(Path(__file__).with_name("git-askpass.sh"))
auth_env["GIT_TERMINAL_PROMPT"] = "0"
return auth_env
def commit_and_push(apps_dir, message, token=None):
run(["git", "-C", str(apps_dir), "add", "-A"])
result = subprocess.run(["git", "-C", str(apps_dir), "diff", "--cached", "--quiet"], check=False)
if result.returncode == 0:
print("No manifest changes to commit")
return False
run(["git", "-C", str(apps_dir), "commit", "-m", message])
run(["git", "-C", str(apps_dir), "push"])
push_env = git_auth_env(token) if token else None
run(["git", "-C", str(apps_dir), "push"], env=push_env)
print("Manifests pushed — ArgoCD will sync")
return True
+7
View File
@@ -4,6 +4,13 @@ metadata:
name: {{ route.resource_name }}
namespace: {{ namespace }}
annotations:
{%- if not compatibility %}
site-publish.fritzlab.net/artifact: {{ route.artifact | tojson }}
site-publish.fritzlab.net/access: {{ route.access | tojson }}
site-publish.fritzlab.net/bucket: {{ route.artifact_config.bucket | tojson }}
site-publish.fritzlab.net/immutable-paths: {{ route.immutable_paths_json | tojson }}
site-publish.fritzlab.net/route-path: {{ route.path | tojson }}
{%- endif %}
traefik.ingress.kubernetes.io/router.middlewares: https-redirect@file,retry-upstream@file{% if route.access_middleware %},{{ route.access_middleware }}@file{% endif %}{% for m in route.middlewares %},{{ m }}@file{% endfor %}
spec:
ingressClassName: traefik
+1
View File
@@ -15,6 +15,7 @@ artifacts:
- name: distributions
type: static
content_dir: dist
cors_origins: ['*']
publish:
bucket: baseline-dist
credentials:
+873 -14
View File
File diff suppressed because it is too large Load Diff