action/check-naming
9
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Validate visible rendered provenance
test / test (pull_request) Successful in 7s
Details

Browse Source 
Authored-By: Codex (GPT-5) <noreply@openai.com>
This commit is contained in:
architect
2026-08-27 17:26:13 +00:00
parent a97440968c
commit c07ea023c0
5 changed files with 125 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -3
View File
@@ -38,6 +38,8 @@ jobs:
pr-title: ${{ github.event.pull_request.title }}
pr-author: ${{ github.event.pull_request.user.login }}
pr-body: ${{ github.event.pull_request.body }}
server-url: ${{ github.server_url }}
token: ${{ github.token }}
base-sha: ${{ github.event.pull_request.base.sha }}
head-sha: ${{ github.event.pull_request.head.sha }}
```
@@ -53,6 +55,8 @@ job — the step still fails, but the job cannot block the PR.
| `pr-title` | yes | PR title — `github.event.pull_request.title` |
| `pr-author` | no | PR author login — `github.event.pull_request.user.login`; `dfritz` is exempt |
| `pr-body` | yes | PR description — `github.event.pull_request.body` |
| `server-url` | yes | Gitea server URL — `github.server_url` |
| `token` | yes | Gitea Actions token — `github.token` |
| `base-sha` | yes | Base commit — `github.event.pull_request.base.sha` |
| `head-sha` | yes | Head commit — `github.event.pull_request.head.sha` |
@@ -73,9 +77,9 @@ The check validates four things for every non-break-glass Agent PR:
`Fixes bug-<id>` automation token and the matching navigable
`https://agenthub.fritzlab.net/bug-<id>` URL. Every PR has a separate
`## Attribution` section containing the canonical `Authored-By`
product/model watermark. Agent-authored PR bodies cannot contain HTML
comment or fenced code delimiters, so required provenance cannot be hidden
from readers.
product/model watermark. The action asks Gitea to render the body and checks
the rendered `<h2>` sections; fenced, commented, scripted, or collapsed
copies do not satisfy the visible provenance contract.
4. **Commit attribution** — every commit in `base-sha..head-sha`, including
commits on `chore/` branches, ends with
action.yaml
+8
View File
@@ -29,6 +29,12 @@ inputs:
pr-body:
description: PR description — github.event.pull_request.body.
required: true
server-url:
description: Gitea server URL — github.server_url.
required: true
token:
description: Gitea Actions token — github.token.
required: true
base-sha:
description: Base commit SHA — github.event.pull_request.base.sha.
required: true
@@ -46,6 +52,8 @@ runs:
PR_TITLE: ${{ inputs.pr-title }}
PR_AUTHOR: ${{ inputs.pr-author }}
PR_BODY: ${{ inputs.pr-body }}
GITEA_SERVER_URL: ${{ inputs.server-url }}
GITEA_TOKEN: ${{ inputs.token }}
BASE_SHA: ${{ inputs.base-sha }}
HEAD_SHA: ${{ inputs.head-sha }}
run: bash "${{ github.action_path }}/check.sh"
check.sh
+24 -15
View File
@@ -10,6 +10,8 @@ AUTHOR="${PR_AUTHOR:-}"
BODY="${PR_BODY:-}"
BASE="${BASE_SHA:-}"
HEAD="${HEAD_SHA:-}"
SERVER_URL="${GITEA_SERVER_URL:-}"
TOKEN="${GITEA_TOKEN:-}"
# Break-glass: dfritz is exempt from all naming checks.
if [ "${AUTHOR}" = "dfritz" ]; then
@@ -21,16 +23,23 @@ FAILED=0
BRANCH_KIND="invalid"
BRANCH_BUG=""
# Gitea hides HTML comments. Reject their delimiters so required provenance is
# always visible, then parse the raw body without a second Markdown renderer.
VISIBLE_BODY="${BODY}"
if printf '%s\n' "${BODY}" | grep -Fq '<!--' ||
printf '%s\n' "${BODY}" | grep -Fq -- '-->'; then
echo "FAIL[check-naming]: PR body must not contain HTML comment delimiters"
# Gitea's renderer is the visibility contract. Validate its output instead of
# maintaining a second Markdown parser in this action.
RENDERED_BODY=""
if [ -z "${SERVER_URL}" ] || [ -z "${TOKEN}" ]; then
echo "FAIL[check-naming]: server-url and token are required to render the PR body"
FAILED=1
elif ! RENDERED_BODY=$(printf '%s' "${BODY}" |
jq -Rs '{Text: ., Mode: "gfm"}' |
curl --fail --silent --show-error \
--header "Authorization: token ${TOKEN}" \
--header "Content-Type: application/json" \
--data-binary @- "${SERVER_URL%/}/api/v1/markdown"); then
echo "FAIL[check-naming]: Gitea could not render the PR body"
FAILED=1
fi
if printf '%s\n' "${BODY}" | grep -Eq '^[[:blank:]]*(```|~~~)'; then
echo "FAIL[check-naming]: PR body must not contain fenced code delimiters"
if printf '%s\n' "${RENDERED_BODY}" | grep -Eiq '<details([[:space:]>])'; then
echo "FAIL[check-naming]: rendered PR body must not contain collapsed details"
FAILED=1
fi
@@ -72,9 +81,9 @@ if [ "${BRANCH_KIND}" = "role-bug" ]; then
FAILED=1
fi
tracking=$(printf '%s\n' "${VISIBLE_BODY}" | awk '
/^## Tracking[[:space:]]*$/ { in_section=1; next }
/^## / && in_section { exit }
tracking=$(printf '%s\n' "${RENDERED_BODY}" | awk '
/<h2[^>]*>Tracking<\/h2>/ { in_section=1; next }
/<h2[^>]*>/ && in_section { exit }
in_section { print }
')
if [ -z "${tracking}" ]; then
@@ -97,12 +106,12 @@ elif [ "${BRANCH_KIND}" = "chore" ] && [ -n "${TITLE_BUG}" ]; then
fi
if [ "${BRANCH_KIND}" != "invalid" ]; then
attribution=$(printf '%s\n' "${VISIBLE_BODY}" | awk '
/^## Attribution[[:space:]]*$/ { in_section=1; next }
/^## / && in_section { exit }
attribution=$(printf '%s\n' "${RENDERED_BODY}" | awk '
/<h2[^>]*>Attribution<\/h2>/ { in_section=1; next }
/<h2[^>]*>/ && in_section { exit }
in_section { print }
')
watermark_re='^[-*]?[[:space:]]*Authored-By: .+ \(.+\) <noreply@[[:alnum:].-]+>$'
watermark_re='Authored-By: .+ \(.+\) <a href="mailto:noreply@[[:alnum:].-]+"[^>]*>noreply@[[:alnum:].-]+</a>'
if ! printf '%s\n' "${attribution}" | grep -qE "${watermark_re}"; then
echo "FAIL[check-naming]: ## Attribution must contain an Authored-By product/model watermark"
FAILED=1
tests/bin/curl
Executable
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# Deterministic Gitea Markdown renderer stub for check.sh contract tests.
set -euo pipefail
if printf '%s\n' "${PR_BODY}" | grep -Fq '<!--'; then
exit 0
fi
printf '%s\n' "${PR_BODY}" | awk '
{
line=$0
lower=tolower(line)
pos=1
while (pos <= 4 && substr(line, pos, 1) == " ") pos++
char=substr(line, pos, 1)
run=0
if (char == "`" || char == "~") while (substr(line, pos + run, 1) == char) run++
rest=substr(line, pos + run)
if (in_fence) {
if (pos <= 4 && char == fence_char && run >= fence_run && rest ~ /^[[:blank:]]*$/) in_fence=0
next
}
if (in_script) {
if (lower ~ /<\/script[[:blank:]]*>/) in_script=0
next
}
if (pos <= 4 && run >= 3 && (char == "~" || index(rest, "`") == 0)) {
in_fence=1
fence_char=char
fence_run=run
next
}
if (lower ~ /^[ ]{0,3}<script([[:blank:]>])/) {
in_script=1
next
}
if (lower ~ /^[ ]{0,3}<details([[:blank:]>])/) {
print "<details>"
next
}
if (line ~ /^## Tracking[[:space:]]*$/) print "<h2>Tracking</h2>"
else if (line ~ /^## Attribution[[:space:]]*$/) print "<h2>Attribution</h2>"
else if (line ~ /Authored-By:.*<noreply@[[:alnum:].-]+>/) {
email=line
sub(/^.*</, "", email)
sub(/>.*/, "", email)
sub(/ <noreply@[[:alnum:].-]+>.*/, "", line)
print line " <a href=\"mailto:" email "\">" email "</a>"
}
else {
gsub(/</, "\\&lt;", line)
gsub(/>/, "\\&gt;", line)
print line
}
}
'
tests/run
+30 -16
View File
@@ -3,6 +3,7 @@
set -euo pipefail
SCRIPT="$(cd "$(dirname "$0")/.." && pwd)/check.sh"
TEST_BIN="$(cd "$(dirname "$0")" && pwd)/bin"
PASS=0
FAIL=0
FIXTURES=$(mktemp -d)
@@ -28,7 +29,8 @@ check() {
body=$(canonical_body "${bug:-bug-test}")
local out rc=0
out=$(cd "${FIXTURES}" && HEAD_BRANCH="$branch" PR_TITLE="$title" PR_AUTHOR="$author" \
PR_BODY="$body" BASE_SHA="$BASE" HEAD_SHA="$GOOD_HEAD" bash "$SCRIPT" 2>&1) || rc=$?
PR_BODY="$body" GITEA_SERVER_URL="https://code.test" GITEA_TOKEN="test-token" PATH="${TEST_BIN}:$PATH" \
BASE_SHA="$BASE" HEAD_SHA="$GOOD_HEAD" bash "$SCRIPT" 2>&1) || rc=$?
local got_lines=0
echo "$out" | grep -qE "^FAIL" && got_lines=1 || true
if [ "$rc" -eq "$want_fail" ] && [ "$got_lines" -eq "$want_fail" ]; then
@@ -46,7 +48,8 @@ check_contract() {
local out rc=0
out=$(cd "${FIXTURES}" && HEAD_BRANCH="architect/bug-x7k2m9/contract" \
PR_TITLE="[bug-x7k2m9] Enforce contract" PR_AUTHOR="architect" \
PR_BODY="$body" BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
PR_BODY="$body" GITEA_SERVER_URL="https://code.test" GITEA_TOKEN="test-token" PATH="${TEST_BIN}:$PATH" \
BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
if [ "$rc" -eq 1 ] && printf '%s\n' "$out" | grep -Fq "$diagnostic"; then
echo "PASS [$desc]"
PASS=$((PASS + 1))
@@ -62,7 +65,8 @@ check_contract_pass() {
local out rc=0
out=$(cd "${FIXTURES}" && HEAD_BRANCH="architect/bug-x7k2m9/contract" \
PR_TITLE="[bug-x7k2m9] Enforce contract" PR_AUTHOR="architect" \
PR_BODY="$body" BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
PR_BODY="$body" GITEA_SERVER_URL="https://code.test" GITEA_TOKEN="test-token" PATH="${TEST_BIN}:$PATH" \
BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
if [ "$rc" -eq 0 ] && printf '%s\n' "$out" | grep -Fq "check-naming: ok"; then
echo "PASS [$desc]"
PASS=$((PASS + 1))
@@ -78,7 +82,8 @@ check_chore_contract() {
local out rc=0
out=$(cd "${FIXTURES}" && HEAD_BRANCH="chore/contract" \
PR_TITLE="Improve delivery contract" PR_AUTHOR="dev" \
PR_BODY="$body" BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
PR_BODY="$body" GITEA_SERVER_URL="https://code.test" GITEA_TOKEN="test-token" PATH="${TEST_BIN}:$PATH" \
BASE_SHA="$base" HEAD_SHA="$head" bash "$SCRIPT" 2>&1) || rc=$?
if [ "$rc" -eq 1 ] && printf '%s\n' "$out" | grep -Fq "$diagnostic"; then
echo "PASS [$desc]"
PASS=$((PASS + 1))
@@ -140,34 +145,43 @@ check_contract "attribution must be in its own section" \
"$BASE" "$GOOD_HEAD" "## Attribution must contain"
check_contract "hidden provenance does not satisfy the visible contract" \
$'<!--\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "comment removal cannot synthesize section headings" \
$'<!-- hidden -->## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n<!-- hidden -->## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "four-space indented backticks do not expose comments" \
$' ````\n<!--\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "unequal backtick runs do not expose comments" \
$'`<!--``\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "invalid backtick fence info does not expose comments" \
$'```html`oops\n<!--\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "invalid fence closer does not expose comments" \
$'```html\n```oops\n```\n<!--\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "fences inside comments do not expose provenance" \
$'<!--\n```html\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "escaped backticks do not hide a comment opener" \
$'\\`<!--\\`\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n-->' \
"$BASE" "$GOOD_HEAD" "PR body must not contain HTML comment delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "fenced provenance does not satisfy the visible contract" \
$'```\n```oops\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>' \
"$BASE" "$GOOD_HEAD" "PR body must not contain fenced code delimiters"
check_contract "fenced code delimiters are rejected" \
$'```text\nvisible example\n```\n\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>' \
"$BASE" "$GOOD_HEAD" "PR body must not contain fenced code delimiters"
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract "indented fence closer does not expose provenance" \
$'```\n ```\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n```' \
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
check_contract_pass "fenced code examples remain available" \
$'```html\n<script>example only</script>\n```\n\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>' \
"$BASE" "$GOOD_HEAD"
check_contract "details cannot collapse provenance" \
$'<details><summary>Release notes</summary>\n\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n</details>' \
"$BASE" "$GOOD_HEAD" "rendered PR body must not contain collapsed details"
check_contract "script cannot suppress provenance" \
$'<script>\n## Tracking\n- Fixes bug-x7k2m9 — https://agenthub.fritzlab.net/bug-x7k2m9\n\n## Attribution\n- Authored-By: Codex (GPT-5) <noreply@openai.com>\n</script>' \
"$BASE" "$GOOD_HEAD" "PR body must contain a non-empty ## Tracking section"
git -C "${FIXTURES}" commit --allow-empty -q -m "unwatermarked change"
BAD_HEAD=$(git -C "${FIXTURES}" rev-parse HEAD)