Compare commits

...
11 Commits
Author SHA1 Message Date
dfritz 49979f30e4 Merge pull request '[bug-0zkpwxbsdxcn] Own canonical offline private Go module tooling' (#2) from architect/bug-0zkpwxbsdxcn/private-module-tooling into main
Private module tooling / contract (push) Successful in 30s
Reviewed-on: #2
2026-09-09 02:15:54 +00:00
Evelyn Chen c9bd31a6e9 Run tooling race contract with a pinned compiler image
Private module tooling / contract (pull_request) Successful in 32s
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 17:05:40 +00:00
Evelyn Chen a81131583e Enable CGO for the private module race contract
Private module tooling / contract (pull_request) Failing after 20s
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 17:02:49 +00:00
Evelyn Chen 24a2d4be52 Check native fixture writes and explicit temporary cleanup
Private module tooling / contract (pull_request) Failing after 5s
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 15:12:33 +00:00
Evelyn Chen 23729de1bb Exclude build proxy artifacts from native provider module archives
Private module tooling / contract (pull_request) Failing after 4s
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 15:08:18 +00:00
Evelyn Chen 52c9456208 Resolve selected lazy modules by exact native cache identity
Private module tooling / contract (pull_request) Failing after 5s
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 14:55:37 +00:00
Evelyn Chen e75412e6d4 Make native checksum proof independent of a consumer bundle
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 14:53:04 +00:00
Evelyn Chen 320cab62a0 Own canonical offline private Go module build tooling
Authored-By: Codex (GPT-6) <noreply@openai.com>
2026-09-07 14:52:03 +00:00
Dave Kowalski b2540dcac2 Merge pull request '[bug-s28h753sx24n] fix(image-build): document private pull token' (#1) from dev/bug-s28h753sx24n/document-private-pull-token into main 2026-08-26 13:53:50 +00:00
Dave Kowalski 925be5d5e3 docs(image-build): add pull-token trust contract
Authored-By: Codex (GPT-5) <noreply@openai.com>
2026-08-26 13:39:16 +00:00
Dave Kowalski e20e59ad0d docs(image-build): document private pull capability
Authored-By: Codex (GPT-5) <noreply@openai.com>
2026-08-26 13:23:04 +00:00
9 changed files with 767 additions and 7 deletions

No files matched your search

+14
View File
@@ -0,0 +1,14 @@
name: Private module tooling
on:
pull_request:
push:
branches: [main]
jobs:
contract:
runs-on: fritzlab
steps:
- uses: actions/checkout@v4
- name: Canonical export and native offline boundaries
run: |
python3 -m unittest discover -s tests
docker build --network none --file tests/private-modules.Dockerfile tools/private-modules
+1
View File
@@ -0,0 +1 @@
__pycache__/
+75 -3
View File
@@ -3,9 +3,50 @@
Composite Gitea Action that builds a container image with buildx and optionally Composite Gitea Action that builds a container image with buildx and optionally
runs a smoke test. **Does not push** — pair with `action/image-push` to publish. runs a smoke test. **Does not push** — pair with `action/image-push` to publish.
Splitting build from push lets a PR workflow run `image-build` (no secrets, no Splitting build from push lets a PR workflow run `image-build` without push or
side effects) for validation while `main` runs the full build → push → deploy deploy side effects while `main` runs the full build → push → deploy chain. A
chain. PR build that pulls a private base image still needs a registry token limited to
the `read:package` capability; public-base builds need no token. The action logs
in as `ci-bot`, so the token must be issued to that account.
## Private Go module builds
`tools/private-modules/` owns the stdlib-only offline generator and verifier for
native Go file-proxy bundles. Consumers export exact committed source bytes so
their first check needs no Git, network, module dependency, or credential.
Changes belong here; consumers must not edit their generated copies.
From this checkout, export a reviewed full commit ID into a service checkout:
```sh
python3 tools/export-private-modules.py /path/to/service --revision FULL_COMMIT_ID
python3 tools/export-private-modules.py /path/to/service --revision FULL_COMMIT_ID --check
```
The export includes source revision/path/digest metadata and a SHA256 lock.
Service Make and Docker builds run `sha256sum -c tools/private-modules.sha256`
before `GO111MODULE=off GOTOOLCHAIN=local GOFLAGS= go run tools/private-modules.go`.
Reviewers can reproduce `--check` from that pinned source commit. The local hash
check detects accidental drift; Git review establishes the trusted source pin.
Populate selected private versions through the configured authenticated Go
client once, then run the generated command with `-write` to copy unchanged
`.info`, `.mod`, and `.zip` cache artifacts into `third_party/go-proxy`. The
generator itself enforces offline resolution and a local toolchain. It preserves
the complete original archives, including any licenses and notices. Normal Go
downloads still validate their content against the consumer's `go.sum`.
The generated proxy directory has an exact build-only nested `go.mod` marker.
Native Go module packaging excludes that directory from provider releases, so
downstream consumers never recursively bundle another provider's build inputs.
The checker requires the marker, and a native local-Git archive test proves the
exclusion without permitting Git network protocols.
Consumer resolution uses `GONOPROXY=none`, `GONOSUMDB=code.fritzlab.net`,
`GOFLAGS=-mod=readonly`, `GOTOOLCHAIN=local`, and
`GOPROXY=file:///absolute/service/third_party/go-proxy,https://proxy.golang.org`.
Run the checker before and after downloads; its exact go.mod/go.sum fingerprint
rejects stale dependency closures. Public modules stay on the public proxy.
Registry publication remains a separate authenticated operation.
## Usage ## Usage
@@ -21,6 +62,22 @@ The image is built and tagged as `<image>:<github.run_number>` in the runner's
local Docker daemon. Subsequent steps (e.g. `action/image-push`) can reference local Docker daemon. Subsequent steps (e.g. `action/image-push`) can reference
the same tag. the same tag.
### Private bases in PRs
Pass `token` only when the PR head is trusted, and limit `ci-bot` package access
to the private base images that the build requires. Never expose an
organization-wide package reader to a contributor-controlled Dockerfile: it can
pull and disclose any package that the account can read.
```yaml
with:
token: ${{ secrets.PACKAGE_READ_TOKEN }} # caller-chosen secret name
```
The token must be issued to `ci-bot` with `read:package` capability. Tokens from
other accounts fail because the action's registry username is fixed. Omit the
input for public bases.
## Inputs ## Inputs
| Name | Required | Default | Description | | Name | Required | Default | Description |
@@ -31,6 +88,7 @@ the same tag.
| `build-args` | no | — | Multiline `KEY=VALUE` build args. Visible in `docker history` — never put secrets here. | | `build-args` | no | — | Multiline `KEY=VALUE` build args. Visible in `docker history` — never put secrets here. |
| `secrets` | no | — | Multiline `id=VALUE` BuildKit secrets (`--secret`). For tokens the build needs (e.g. a ci-bot token to `go mod download` a private module) that must not leak into layers. Reference with `RUN --mount=type=secret,id=<id>`. | | `secrets` | no | — | Multiline `id=VALUE` BuildKit secrets (`--secret`). For tokens the build needs (e.g. a ci-bot token to `go mod download` a private module) that must not leak into layers. Reference with `RUN --mount=type=secret,id=<id>`. |
| `smoke-test` | no | — | Shell command run after build. `$IMAGE` is set to `<image>:<run_number>`. Non-zero exit fails the action. | | `smoke-test` | no | — | Shell command run after build. `$IMAGE` is set to `<image>:<run_number>`. Non-zero exit fails the action. |
| `token` | no | — | `ci-bot` access token with `read:package` capability. Required to pull a private base image; omit for public bases. |
## Outputs ## Outputs
@@ -38,6 +96,20 @@ the same tag.
|---|---| |---|---|
| `tag` | Numeric tag assigned (= `github.run_number`). | | `tag` | Numeric tag assigned (= `github.run_number`). |
## Private module contract validation
The private-module contract workflow runs its Go race tests in the digest-pinned
standard Go compiler image in `tests/private-modules.Dockerfile`. The Fritzlab
runner doesn't provide GCC. Only `tools/private-modules` enters that build context;
the test runs with networking and module downloads disabled. The public compiler
image must already be cached or obtainable through the existing image pull path.
This check doesn't publish an image or change runner configuration.
```sh
python3 -m unittest discover -s tests
docker build --network none --file tests/private-modules.Dockerfile tools/private-modules
```
## Smoke test patterns ## Smoke test patterns
Override entrypoint for a binary that expects no args: Override entrypoint for a binary that expects no args:
+7 -4
View File
@@ -37,10 +37,13 @@ inputs:
default: '' default: ''
token: token:
description: | description: |
ci-bot token (CI_BOT_TOKEN) for `docker login code.fritzlab.net`. Required ci-bot access token with `read:package` capability for
when the Dockerfile's FROM is a PRIVATE fritzlab image (e.g. FROM `docker login code.fritzlab.net`. The login username is fixed to `ci-bot`,
code.fritzlab.net/fritzlab/base) — the org is `limited`, so buildx can't pull so a token issued to another account will fail. Required when the
it anonymously. Omit for public-base builds (e.g. base itself = FROM debian). Dockerfile's FROM is a PRIVATE fritzlab image (e.g. FROM
code.fritzlab.net/fritzlab/base) — the org is `limited`, so buildx can't
pull it anonymously. Omit for public-base builds (e.g. base itself = FROM
debian).
required: false required: false
default: '' default: ''
outputs: outputs:
+7
View File
@@ -0,0 +1,7 @@
FROM golang:1.27.0-trixie@sha256:df98008ecd2b0ecc9f0a94d1b07e3564a9c92b555369b33d9b5f60d0765b2db7
WORKDIR /contract
# The runner intentionally needn't provide a C compiler for this race check.
# The build context contains only the canonical Go tooling, without Git state.
COPY *.go ./
ENV CGO_ENABLED=1 GO111MODULE=off GOTOOLCHAIN=local GOFLAGS= GOPROXY=off GOSUMDB=off
RUN go test -race -count=1 ./
+91
View File
@@ -0,0 +1,91 @@
import importlib.util
import pathlib
import os
import json
import subprocess
import tempfile
import unittest
import zipfile
ROOT = pathlib.Path(__file__).resolve().parents[1]
SPEC = importlib.util.spec_from_file_location("exporter", ROOT / "tools/export-private-modules.py")
MODULE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MODULE)
class ExportTest(unittest.TestCase):
def test_native_go_archive_excludes_nested_build_artifacts(self):
with tempfile.TemporaryDirectory() as temporary:
root = pathlib.Path(temporary)
source = root / "source"
source.mkdir()
module = "example.invalid/fixture.git"
(source / "go.mod").write_text("module " + module + "\n\ngo 1.27.0\n")
(source / "provider.go").write_text("package fixture\n")
boundary = source / "third_party/go-proxy"
boundary.mkdir(parents=True)
(boundary / "go.mod").write_text("module example.invalid/build-artifacts\n\ngo 1.27.0\n")
(boundary / "provider.zip").write_bytes(b"must not enter the provider module archive")
subprocess.run(["git", "init", "-q", str(source)], check=True)
subprocess.run(["git", "-C", str(source), "add", "."], check=True)
subprocess.run(["git", "-C", str(source), "-c", "user.name=Fixture",
"-c", "user.email=fixture@example.invalid", "commit", "-qm", "fixture"], check=True)
subprocess.run(["git", "-C", str(source), "tag", "v0.1.0"], check=True)
config = root / "gitconfig"
config.write_text('[url "' + source.as_uri() + '"]\n'
'\tinsteadOf = https://example.invalid/fixture\n')
# Let Go select the synthetic VCS URL; Git still permits only the
# rewritten local file protocol, with no network scheme allowed.
env = dict(os.environ, GO111MODULE="on", GOTOOLCHAIN="local", GOFLAGS="", GOWORK="off",
GOPROXY="direct", GONOPROXY="none", GOSUMDB="off", GOVCS="*:git",
GOINSECURE="example.invalid",
GOMODCACHE=str(root / "cache"), GIT_CONFIG_GLOBAL=str(config),
GIT_CONFIG_NOSYSTEM="1", GIT_CONFIG_COUNT="0", GIT_ALLOW_PROTOCOL="file")
probe = subprocess.run(["git", "ls-remote", "https://example.invalid/fixture"], env=env, capture_output=True)
self.assertEqual(probe.returncode, 0, probe.stderr.decode())
# Native Git accepts only the local file protocol: a missed rewrite
# cannot turn this proof into a network or credential lookup.
result = subprocess.run(["go", "mod", "download", "-json", module + "@v0.1.0"],
cwd=root, env=env, capture_output=True)
self.assertEqual(result.returncode, 0, result.stdout.decode() + result.stderr.decode())
archive = json.loads(result.stdout)["Zip"]
with zipfile.ZipFile(archive) as zipped:
names = zipped.namelist()
self.assertIn(module + "@v0.1.0/provider.go", names)
self.assertFalse(any("third_party/go-proxy" in name for name in names))
def test_committed_bytes_and_provenance_are_exact(self):
with tempfile.TemporaryDirectory() as temporary:
base = pathlib.Path(temporary)
source, target = base / "source", base / "target"
source.mkdir()
target.mkdir()
subprocess.run(["git", "init", "-q", str(source)], check=True)
for path in MODULE.FILES:
file = source / path
file.parent.mkdir(parents=True, exist_ok=True)
file.write_text("package main\n")
subprocess.run(["git", "-C", str(source), "add", "."], check=True)
subprocess.run(["git", "-C", str(source), "-c", "user.name=Fixture",
"-c", "user.email=fixture@example.invalid", "commit", "-qm", "fixture"], check=True)
revision = MODULE.committed(source, "rev-parse", "HEAD").decode().strip()
(source / next(iter(MODULE.FILES))).write_text("dirty worktree must be ignored")
MODULE.export(source, target, revision)
MODULE.export(source, target, revision, True)
subprocess.run(["sha256sum", "-c", "tools/private-modules.sha256"],
cwd=target, check=True, stdout=subprocess.DEVNULL)
destination = target / "tools/private-modules.go"
self.assertEqual(destination.read_text(), "package main\n")
destination.write_text("changed")
with self.assertRaises(ValueError):
MODULE.export(source, target, revision, True)
destination.unlink()
destination.symlink_to(source / next(iter(MODULE.FILES)))
with self.assertRaises(ValueError):
MODULE.export(source, target, revision)
with self.assertRaises(ValueError):
MODULE.export(source, target, "HEAD")
if __name__ == "__main__":
unittest.main()
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env python3
"""Export exact committed offline-bootstrap tooling; never download anything."""
import argparse
import hashlib
import json
import pathlib
import re
import subprocess
SOURCE = "https://code.fritzlab.net/action/image-build"
FILES = {
"tools/private-modules/main.go": "tools/private-modules.go",
"tools/private-modules/main_test.go": "tools/private-modules_test.go",
}
def committed(root, *args):
return subprocess.check_output(["git", "-C", str(root), *args])
def export(root, target, revision, check=False):
if not re.fullmatch(r"[0-9a-f]{40}", revision):
raise ValueError("source revision must be a full Git commit ID")
resolved = committed(root, "rev-parse", revision + "^{commit}").decode().strip()
if resolved != revision:
raise ValueError("source revision is not a commit")
result = {}
entries = []
for source, destination in FILES.items():
tree = committed(root, "ls-tree", revision, "--", source).decode()
if not tree.startswith("100644 blob ") or not tree.endswith("\t" + source + "\n"):
raise ValueError("source must be one regular committed file")
data = committed(root, "show", revision + ":" + source)
if not data or len(data) > 128 * 1024:
raise ValueError("source exceeds its bounded export")
result[destination] = data
entries.append({"source": source, "destination": destination,
"sha256": hashlib.sha256(data).hexdigest()})
result["tools/private-modules.source.json"] = (json.dumps({
"version": 1, "repository": SOURCE, "revision": revision, "files": entries,
}, indent=2) + "\n").encode()
result["tools/private-modules.sha256"] = "".join(
hashlib.sha256(data).hexdigest() + " " + path + "\n"
for path, data in sorted(result.items())
).encode()
target = target.resolve(strict=True)
tools_dir = target / "tools"
if tools_dir.is_symlink():
raise ValueError("target tools directory must not be a symlink")
if not check:
tools_dir.mkdir(exist_ok=True)
for path, data in result.items():
destination = target / path
if destination.is_symlink():
raise ValueError("target file must not be a symlink")
if check:
if destination.read_bytes() != data:
raise ValueError("export differs from pinned canonical source: " + path)
else:
destination.write_bytes(data)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("target", type=pathlib.Path)
parser.add_argument("--revision", required=True)
parser.add_argument("--check", action="store_true")
args = parser.parse_args()
export(pathlib.Path(__file__).resolve().parents[1], args.target,
args.revision, args.check)
if __name__ == "__main__":
main()
+301
View File
@@ -0,0 +1,301 @@
// Command private-modules maintains exact private Go proxy artifacts from an
// already authenticated local module cache. It never downloads dependencies.
package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
)
const bundleRoot = "third_party/go-proxy"
const privatePrefix = "code.fritzlab.net/"
const archiveBoundary = "module code.fritzlab.net/action/image-build/private-module-artifacts\n\ngo 1.27.0\n"
type artifact struct {
Path string `json:"path"`
SHA256 string `json:"sha256"`
Size int64 `json:"size_bytes"`
}
type module struct {
Path string `json:"path"`
Version string `json:"version"`
Files []artifact `json:"files"`
}
type manifest struct {
Version int `json:"version"`
GoModSHA256 string `json:"go_mod_sha256"`
GoSumSHA256 string `json:"go_sum_sha256"`
Modules []module `json:"modules"`
}
func digest(b []byte) string { s := sha256.Sum256(b); return hex.EncodeToString(s[:]) }
func inputDigest(name string) (string, error) {
b, e := os.ReadFile(name)
if e != nil {
return "", e
}
return digest(b), nil
}
func nativeGo(args ...string) ([]byte, error) {
c := exec.Command("go", args...)
c.Env = append(os.Environ(), "GOPROXY=off", "GONOPROXY=none", "GOTOOLCHAIN=local", "GOSUMDB=off", "GO111MODULE=on", "GOFLAGS=", "GOWORK=off")
return c.Output()
}
func generate() error {
m := manifest{Version: 1, Modules: []module{}}
var err error
if m.GoModSHA256, err = inputDigest("go.mod"); err != nil {
return err
}
if m.GoSumSHA256, err = inputDigest("go.sum"); err != nil {
return err
}
cache, err := nativeGo("env", "GOMODCACHE")
if err != nil {
return err
}
cacheRoot := filepath.Join(strings.TrimSpace(string(cache)), "cache", "download")
raw, err := nativeGo("list", "-m", "-json", "all")
if err != nil {
return fmt.Errorf("module metadata must already be cached: %w", err)
}
parent := filepath.Dir(bundleRoot)
if err = os.MkdirAll(parent, 0755); err != nil {
return err
}
temp, err := os.MkdirTemp(parent, ".go-proxy-")
if err != nil {
return err
}
defer func() { _ = os.RemoveAll(temp) }()
if err := os.WriteFile(filepath.Join(temp, "go.mod"), []byte(archiveBoundary), 0444); err != nil {
return err
}
decoder := json.NewDecoder(bytes.NewReader(raw))
var total int64
for {
var src struct {
Path, Version, GoMod string
Main bool
Replace json.RawMessage
}
if err = decoder.Decode(&src); errors.Is(err, io.EOF) {
break
} else if err != nil {
return err
}
if src.Main || !strings.HasPrefix(src.Path, privatePrefix) {
continue
}
if len(src.Replace) != 0 || src.Version == "" || len(m.Modules) >= 64 {
return errors.New("private module must have exact unreplaced version")
}
mod := module{Path: src.Path, Version: src.Version, Files: []artifact{}}
stem, err := cacheStem(cacheRoot, src.Path, src.Version, src.GoMod)
if err != nil {
return err
}
for _, ext := range []string{".info", ".mod", ".zip"} {
name := stem + ext
source := filepath.Join(cacheRoot, name)
info, err := os.Lstat(source)
if err != nil || !info.Mode().IsRegular() || info.Size() > 64<<20 {
return errors.New("private module artifact absent or invalid; populate authenticated cache first")
}
total += info.Size()
if total > 256<<20 {
return errors.New("private module bundle too large")
}
b, err := os.ReadFile(source)
if err != nil {
return err
}
target := filepath.Join(temp, name)
if err = os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return err
}
if err = os.WriteFile(target, b, 0444); err != nil {
return err
}
mod.Files = append(mod.Files, artifact{Path: filepath.ToSlash(name), SHA256: digest(b), Size: int64(len(b))})
}
m.Modules = append(m.Modules, mod)
}
sort.Slice(m.Modules, func(i, j int) bool { return m.Modules[i].Path < m.Modules[j].Path })
if len(m.Modules) == 0 {
return errors.New("no private modules")
}
encoded, err := json.MarshalIndent(m, "", " ")
if err != nil {
return err
}
if err = os.WriteFile(filepath.Join(temp, "manifest.json"), append(encoded, '\n'), 0444); err != nil {
return err
}
// This path is exclusively generated. Refuse to replace an unrelated tree.
if _, err = os.Stat(bundleRoot); err == nil {
if _, err = os.Stat(filepath.Join(bundleRoot, "manifest.json")); err != nil {
return errors.New("refuse to replace unrecognized bundle")
}
if err = os.RemoveAll(bundleRoot); err != nil {
return err
}
} else if !os.IsNotExist(err) {
return err
}
return os.Rename(temp, bundleRoot)
}
var modulePath = regexp.MustCompile(`^code[.]fritzlab[.]net/[A-Za-z0-9._~-]+(/[A-Za-z0-9._~-]+)+$`)
var moduleVersion = regexp.MustCompile(`^v[0-9]+[.][0-9]+[.][0-9]+(-[0-9A-Za-z.-]+)?([+]incompatible)?$`)
// Standard Go proxy escaping replaces each ASCII capital with !lowercase.
func moduleStem(path, version string) (string, error) {
if len(path) > 512 || len(version) > 256 || !modulePath.MatchString(path) || !moduleVersion.MatchString(version) {
return "", errors.New("invalid private module identity")
}
for _, part := range strings.Split(path, "/") {
if part == "." || part == ".." {
return "", errors.New("invalid private module path")
}
}
escape := func(value string) string {
var out strings.Builder
for _, r := range value {
if r >= 'A' && r <= 'Z' {
out.WriteByte('!')
r += 'a' - 'A'
}
out.WriteRune(r)
}
return out.String()
}
return escape(path) + "/@v/" + escape(version), nil
}
func cacheStem(cacheRoot, path, version, goMod string) (string, error) {
stem, err := moduleStem(path, version)
if err != nil {
return "", err
}
// Lazy module graphs omit GoMod for selected but unexpanded modules.
// Their canonical cache path is still determined by exact identity.
if goMod != "" {
rel, err := filepath.Rel(cacheRoot, goMod)
if err != nil || filepath.ToSlash(rel) != stem+".mod" {
return "", errors.New("native artifact path differs from module identity")
}
}
return stem, nil
}
func safePath(p string) bool {
return p != "" && filepath.Clean(p) == p && !filepath.IsAbs(p) && p != ".." && !strings.HasPrefix(p, "../") && !strings.ContainsAny(p, "\\\x00")
}
func check() error {
raw, err := os.ReadFile(filepath.Join(bundleRoot, "manifest.json"))
if err != nil || len(raw) > 1<<20 {
return errors.New("private module manifest missing or oversized")
}
var m manifest
d := json.NewDecoder(bytes.NewReader(raw))
d.DisallowUnknownFields()
if err = d.Decode(&m); err != nil || m.Version != 1 || len(m.Modules) == 0 || len(m.Modules) > 64 {
return errors.New("invalid private module manifest")
}
var trailing any
if err = d.Decode(&trailing); !errors.Is(err, io.EOF) {
return errors.New("trailing private module manifest data")
}
mod, err := inputDigest("go.mod")
if err != nil {
return err
}
sum, err := inputDigest("go.sum")
if err != nil {
return err
}
if mod != m.GoModSHA256 || sum != m.GoSumSHA256 {
return errors.New("private module closure stale; regenerate after go.mod/go.sum changes")
}
boundary, err := os.ReadFile(filepath.Join(bundleRoot, "go.mod"))
if err != nil || string(boundary) != archiveBoundary {
return errors.New("private module archive boundary missing or modified")
}
expected := map[string]bool{"manifest.json": true, "go.mod": true}
last := ""
var total int64
for _, m := range m.Modules {
if !strings.HasPrefix(m.Path, privatePrefix) || m.Path <= last || m.Version == "" || len(m.Files) != 3 {
return errors.New("invalid private module closure")
}
last = m.Path
stem, err := moduleStem(m.Path, m.Version)
if err != nil {
return err
}
for i, f := range m.Files {
if !safePath(f.Path) || expected[f.Path] || f.Path != stem+[]string{".info", ".mod", ".zip"}[i] || f.Size < 0 || f.Size > 64<<20 {
return errors.New("invalid module artifact")
}
total += f.Size
if total > 256<<20 {
return errors.New("private module bundle too large")
}
full := filepath.Join(bundleRoot, f.Path)
info, e := os.Lstat(full)
if e != nil || !info.Mode().IsRegular() || info.Size() != f.Size {
return errors.New("private module artifact missing or invalid")
}
b, e := os.ReadFile(full)
if e != nil || digest(b) != f.SHA256 {
return errors.New("private module artifact digest mismatch")
}
expected[f.Path] = true
}
}
err = filepath.WalkDir(bundleRoot, func(path string, entry os.DirEntry, err error) error {
if err != nil {
return err
}
if entry.Type()&os.ModeSymlink != 0 {
return errors.New("symlink in private module bundle")
}
if entry.IsDir() {
return nil
}
rel, e := filepath.Rel(bundleRoot, path)
if e != nil || !expected[filepath.ToSlash(rel)] {
return errors.New("unexpected private module artifact")
}
return nil
})
return err
}
func main() {
write := flag.Bool("write", false, "regenerate from authenticated local module cache (no network)")
flag.Parse()
var err error
if *write {
err = generate()
} else {
err = check()
}
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println("private module artifacts verified")
}
+197
View File
@@ -0,0 +1,197 @@
package main
import (
"archive/zip"
"bytes"
"encoding/json"
"errors"
"os"
"os/exec"
"path/filepath"
"testing"
)
func TestBundleRejectsStaleMissingModifiedAndUnexpectedArtifacts(t *testing.T) {
t.Chdir(t.TempDir())
mustFixture(t, os.WriteFile("go.mod", []byte("module example.invalid/test\n"), 0600))
mustFixture(t, os.WriteFile("go.sum", []byte("recorded sum\n"), 0600))
mod, _ := inputDigest("go.mod")
sum, _ := inputDigest("go.sum")
m := manifest{Version: 1, GoModSHA256: mod, GoSumSHA256: sum, Modules: []module{{Path: "code.fritzlab.net/agenthub/example", Version: "v0.1.0"}}}
for _, ext := range []string{".info", ".mod", ".zip"} {
name := "code.fritzlab.net/agenthub/example/@v/v0.1.0" + ext
b := []byte("canonical artifact " + ext)
mustFixture(t, os.MkdirAll(filepath.Dir(filepath.Join(bundleRoot, name)), 0700))
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, name), b, 0600))
m.Modules[0].Files = append(m.Modules[0].Files, artifact{Path: name, Size: int64(len(b)), SHA256: digest(b)})
}
b, _ := json.Marshal(m)
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "manifest.json"), b, 0600))
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "go.mod"), []byte(archiveBoundary), 0600))
if err := check(); err != nil {
t.Fatal(err)
}
for _, boundary := range []string{"", "module unexpected.invalid/nested\n"} {
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "go.mod"), []byte(boundary), 0600))
if check() == nil {
t.Fatal("missing or changed archive boundary accepted")
}
}
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "go.mod"), []byte(archiveBoundary), 0600))
originalPath, originalVersion := m.Modules[0].Path, m.Modules[0].Version
for _, change := range []string{"path", "version"} {
if change == "path" {
m.Modules[0].Path = "code.fritzlab.net/agenthub/other"
} else {
m.Modules[0].Version = "v0.2.0"
}
changed, _ := json.Marshal(m)
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "manifest.json"), changed, 0600))
if check() == nil {
t.Fatal("artifact accepted under different module identity")
}
m.Modules[0].Path, m.Modules[0].Version = originalPath, originalVersion
}
mustFixture(t, os.WriteFile(filepath.Join(bundleRoot, "manifest.json"), b, 0600))
mustFixture(t, os.WriteFile("go.mod", []byte("changed graph\n"), 0600))
if check() == nil {
t.Fatal("stale dependency closure accepted")
}
mustFixture(t, os.WriteFile("go.mod", []byte("module example.invalid/test\n"), 0600))
path := filepath.Join(bundleRoot, m.Modules[0].Files[2].Path)
original, _ := os.ReadFile(path)
mustFixture(t, os.WriteFile(path, []byte("corrupted zip"), 0600))
if check() == nil {
t.Fatal("modified artifact accepted")
}
mustFixture(t, os.Remove(path))
if check() == nil {
t.Fatal("missing artifact accepted")
}
mustFixture(t, os.WriteFile(path, original, 0600))
extra := filepath.Join(bundleRoot, "extra")
mustFixture(t, os.WriteFile(extra, []byte("unlisted"), 0600))
if check() == nil {
t.Fatal("unlisted artifact accepted")
}
mustFixture(t, os.Remove(extra))
mustFixture(t, os.Symlink("manifest.json", extra))
if check() == nil {
t.Fatal("symlink accepted")
}
}
func TestNativeGoSumRejectsChangedModuleArtifact(t *testing.T) {
root := t.TempDir()
proxy := filepath.Join(root, "proxy")
modulePath, version := "code.fritzlab.net/fixture/module", "v0.1.0"
base := filepath.Join(proxy, modulePath, "@v", version)
if err := os.MkdirAll(filepath.Dir(base), 0700); err != nil {
t.Fatal(err)
}
mod := []byte("module " + modulePath + "\n\ngo 1.27.0\n")
mustFixture(t, os.WriteFile(base+".mod", mod, 0600))
mustFixture(t, os.WriteFile(base+".info", []byte(`{"Version":"v0.1.0","Time":"2026-01-01T00:00:00Z"}`), 0600))
var archive bytes.Buffer
zw := zip.NewWriter(&archive)
entry, err := zw.Create(modulePath + "@" + version + "/go.mod")
if err != nil {
t.Fatal(err)
}
if _, err := entry.Write(mod); err != nil {
t.Fatal(err)
}
if err = zw.Close(); err != nil {
t.Fatal(err)
}
mustFixture(t, os.WriteFile(base+".zip", archive.Bytes(), 0600))
mustFixture(t, os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.invalid/checksum\n\ngo 1.27.0\n\nrequire "+modulePath+" "+version+"\n"), 0600))
run := func(cache string) ([]byte, error) {
cmd := exec.Command("go", "mod", "download", modulePath+"@"+version)
cmd.Dir = root
cmd.Env = append(os.Environ(), "GO111MODULE=on", "GOMODCACHE="+filepath.Join(root, cache), "GOPROXY=file://"+proxy, "GONOPROXY=none", "GONOSUMDB="+privatePrefix, "GOSUMDB=off", "GOTOOLCHAIN=local", "GOFLAGS=", "GOWORK=off")
return cmd.CombinedOutput()
}
if output, err := run("original-cache"); err != nil {
t.Fatalf("native fixture admission: %v: %s", err, output)
}
mustFixture(t, os.WriteFile(base+".mod", append(mod, '\n'), 0600))
output, err := run("fresh-cache")
if err == nil || !bytes.Contains(output, []byte("checksum mismatch")) {
t.Fatalf("native module checksum guard failed: %v: %s", err, output)
}
}
func TestGeneratorMissingPrivateMetadataNeverInvokesGit(t *testing.T) {
root := t.TempDir()
t.Chdir(root)
mustFixture(t, os.WriteFile("go.mod", []byte("module example.invalid/offline\n\ngo 1.27.0\n\nrequire code.fritzlab.net/agenthub/missing v0.0.1\n"), 0600))
mustFixture(t, os.WriteFile("go.sum", nil, 0600))
bin := filepath.Join(root, "bin")
mustFixture(t, os.Mkdir(bin, 0700))
marker := filepath.Join(root, "git-invoked")
mustFixture(t, os.WriteFile(filepath.Join(bin, "git"), []byte("#!/bin/sh\nprintf invoked > \"$PRIVATE_GIT_PROBE\"\nexit 99\n"), 0700))
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
t.Setenv("PRIVATE_GIT_PROBE", marker)
t.Setenv("GOMODCACHE", filepath.Join(root, "empty-cache"))
t.Setenv("GOPRIVATE", "code.fritzlab.net")
t.Setenv("GONOPROXY", "code.fritzlab.net")
t.Setenv("GOTOOLCHAIN", "auto")
err := generate()
if err == nil {
t.Fatal("missing cached module accepted")
}
var failure *exec.ExitError
if !errors.As(err, &failure) || !bytes.Contains(failure.Stderr, []byte("module lookup disabled by GOPROXY=off")) {
t.Fatalf("not a proven offline refusal: %v", err)
}
if _, err = os.Stat(marker); !os.IsNotExist(err) {
t.Fatal("Git invoked during offline generation")
}
if _, err = os.Stat(bundleRoot); !os.IsNotExist(err) {
t.Fatal("failed generation published output")
}
mustFixture(t, os.WriteFile("go.mod", []byte("module example.invalid/offline\n\ngo 1.999.0\n"), 0600))
_, err = nativeGo("list", "-m", "-json", "all")
if !errors.As(err, &failure) || !bytes.Contains(failure.Stderr, []byte("GOTOOLCHAIN=local")) {
t.Fatalf("toolchain download not disabled: %v", err)
}
}
func TestCanonicalModuleArtifactPathBinding(t *testing.T) {
for _, test := range []struct{ path, version, want string }{
{"code.fritzlab.net/agenthub/Provider", "v1.2.3-RC.1", "code.fritzlab.net/agenthub/!provider/@v/v1.2.3-!r!c.1"},
{"code.fritzlab.net/agenthub/provider", "v2.0.0+incompatible", "code.fritzlab.net/agenthub/provider/@v/v2.0.0+incompatible"},
} {
got, err := moduleStem(test.path, test.version)
if err != nil || got != test.want {
t.Fatalf("stem %q %v", got, err)
}
}
for _, path := range []string{"code.fritzlab.net/agenthub/../provider", "code.fritzlab.net/agenthub/provider!", "other.invalid/agenthub/provider"} {
if _, err := moduleStem(path, "v1.0.0"); err == nil {
t.Fatal("invalid path accepted")
}
}
}
func TestLazyGraphModuleCacheIdentity(t *testing.T) {
cache := t.TempDir()
path, version := "code.fritzlab.net/fixture/module", "v0.1.0"
stem, err := cacheStem(cache, path, version, "")
if err != nil || stem != path+"/@v/"+version {
t.Fatalf("selected lazy module rejected: %q %v", stem, err)
}
for _, metadata := range []string{filepath.Join(cache, "foreign/@v/v0.1.0.mod"), filepath.Join(cache, "../escaped.mod"), filepath.Join(cache, path, "@v/v0.2.0.mod")} {
if _, err := cacheStem(cache, path, version, metadata); err == nil {
t.Fatal("contradictory cache identity accepted")
}
}
}
func mustFixture(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatal(err)
}
}