Initial commit: dns-webhook MutatingAdmissionWebhook
Build dns-webhook Image / build (push) Has been cancelled

Rewrites dnsPolicy+dnsConfig on ClusterFirst pods to distribute
queries across 3 randomly-selected auth-dns nameservers with
edns0/rotate/ndots:5. Includes Gitea CI workflow and README.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Donavan Fritz
2026-04-24 17:14:56 -05:00
commit 01e4b58c91
6 changed files with 553 additions and 0 deletions
+45
View File
@@ -0,0 +1,45 @@
name: Build dns-webhook Image
on:
push:
branches: [main]
jobs:
build:
runs-on: fritzlab
steps:
- name: Check out repo
uses: actions/checkout@v4
- name: Log in to Gitea registry
uses: docker/login-action@v3
with:
registry: code.fritzlab.net
username: ci-bot
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: code.fritzlab.net/dns/webhook
tags: |
type=raw,value=${{ github.run_number }}
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
push: true
provenance: false
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
network: host
- name: Clean up old image tags
run: |
tea login add --name ci --url https://code.fritzlab.net --token '${{ secrets.CI_BOT_TOKEN }}' --no-version-check
tea api '/packages/dns?type=container' | jq -r '.[] | select(.name=="webhook") | select(.version | test("^[0-9]+$")) | .version' \
| sort -n | head -n -3 \
| while read tag; do
echo "deleting webhook:$tag"
tea api -X DELETE "/packages/dns/container/webhook/$tag"
done
+10
View File
@@ -0,0 +1,10 @@
FROM golang:1.26-alpine AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /dns-webhook .
FROM gcr.io/distroless/static:nonroot
COPY --from=builder /dns-webhook /dns-webhook
ENTRYPOINT ["/dns-webhook"]
+93
View File
@@ -0,0 +1,93 @@
# dns-webhook
A Kubernetes [MutatingAdmissionWebhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) that rewrites DNS configuration on every new pod before it starts.
## What it does
When a pod is created with the default `dnsPolicy: ClusterFirst`, this webhook intercepts the request and:
1. **Picks 3 random nameservers** from the pool of 4 production auth-dns pods (ns1ns4). This distributes DNS query load instead of every pod always hitting the same two servers.
2. **Sets search domains** appropriate for the pod's namespace so short service names resolve correctly.
3. **Enables `edns0`** — allows DNS responses larger than 512 bytes (needed for DNSSEC and large TXT records).
4. **Enables `rotate`** — cycles through nameservers on each query for even load distribution.
Pods that opt out (`dnsPolicy: None`, `Default`, or `ClusterFirstWithHostNet`) are passed through unchanged.
## Architecture
```
kubelet / kubectl apply
Kubernetes API server
│ (Pod CREATE request)
MutatingAdmissionWebhook ──► dns-webhook pod (this service)
│ │
│ ◄── JSON Patch ─────────┘
│ replace dnsPolicy → None
│ add dnsConfig { nameservers, searches, options }
Pod stored with rewritten DNS config
```
The webhook runs as a Deployment in `kube-system` and is registered via a `MutatingWebhookConfiguration`. cert-manager issues the TLS certificate; its cainjector populates the `caBundle` field automatically.
## Logs
Key log lines to watch for during debugging:
| Prefix | Meaning |
|--------|---------|
| `dns-webhook starting: cert=... key=...` | Server startup — confirms TLS paths |
| `MUTATE pod=<ns>/<name> uid=... nameservers=[...] op=add\|replace` | Pod was mutated — shows which nameservers were assigned |
| `SKIP pod=<ns>/<name> uid=... policy=<policy>` | Pod was not mutated — shows why (non-ClusterFirst policy) |
| `ERROR ...` | Decode/encode failures — should never appear in normal operation |
```bash
# Stream logs from all webhook replicas
kubectl --context sjc001 logs -n kube-system -l app=dns-webhook -f
# Verify a running pod received the correct DNS config
kubectl --context sjc001 exec -n <namespace> <pod> -- cat /etc/resolv.conf
```
## Deployment
Managed by ArgoCD. Manifests live in the `fritzlab/apps` repo under
`sjc001/kube-system/dns-webhook/manifests/`.
```
apps/sjc001/kube-system/dns-webhook/
├── app.yaml # ArgoCD Application
└── manifests/
├── deployment.yaml # Webhook pods (2 replicas, dnsPolicy: Default)
├── issuer.yaml # cert-manager: selfSigned → CA → leaf cert
├── service.yaml # ClusterIP Service on :443 → pod :8443
├── serviceaccount.yaml
└── webhook.yaml # MutatingWebhookConfiguration
```
The `deployment.yaml` image tag (`code.fritzlab.net/fritzlab/dns-webhook:<run_number>`) must be updated whenever a new image is built. CI in this repo produces the image; update the tag in `apps` to deploy.
## Development
```bash
# Build locally
go build ./...
# Run tests (none yet — the mutation logic is straightforward enough that
# end-to-end verification via a test pod is more useful)
go test ./...
# Build container image
docker build -t dns-webhook:local .
```
## Design notes
- **`dnsPolicy: Default` on the webhook pods themselves**: avoids a circular dependency — if cluster DNS is disrupted, the webhook pods can still start because they use the node's `/etc/resolv.conf` directly.
- **`failurePolicy: Ignore`**: if the webhook is unavailable, pods are admitted without mutation rather than being blocked. Availability of workloads takes priority over DNS load balancing.
- **`imagePullPolicy: IfNotPresent`**: if cluster DNS is down at pod start time, the image pull (which needs DNS to reach the registry) would fail. This policy uses the locally cached image instead.
- **ClusterIP service (not headless)**: webhook calls are short-lived HTTP requests — the keepalive starvation problem that affects long-lived connections doesn't apply here. A stable VIP is the conventional pattern for webhook services.
- **Static nameserver IPs**: the auth-dns pods use `cni.projectcalico.org/ipAddrs` to pin their Calico-allocated IPv6 addresses across restarts, making them safe to hardcode here.
+26
View File
@@ -0,0 +1,26 @@
module code.fritzlab.net/dns/webhook
go 1.23.0
require k8s.io/api v0.32.3
require (
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/net v0.30.0 // indirect
golang.org/x/text v0.19.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
k8s.io/apimachinery v0.32.3 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)
+95
View File
@@ -0,0 +1,95 @@
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.30.0 h1:AcW1SDZMkb8IpzCdQUaIq2sP4sZ4zw+55h6ynffypl4=
golang.org/x/net v0.30.0/go.mod h1:2wGyMJ5iFasEhkwi13ChkO/t1ECNC4X4eBKkVFyYFlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM=
golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/api v0.32.3 h1:Hw7KqxRusq+6QSplE3NYG4MBxZw1BZnq4aP4cJVINls=
k8s.io/api v0.32.3/go.mod h1:2wEDTXADtm/HA7CCMD8D8bK4yuBUptzaRhYcYEEYA3k=
k8s.io/apimachinery v0.32.3 h1:JmDuDarhDmA/Li7j3aPrwhpNBA94Nvk5zLeOge9HH1U=
k8s.io/apimachinery v0.32.3/go.mod h1:GpHVgxoKlTxClKcteaeuF1Ul/lDVb74KpZcxcmLDElE=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 h1:M3sRQVHv7vB20Xc2ybTt7ODCeFj6JSWYFzOFnYeS6Ro=
k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8=
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo=
sigs.k8s.io/structured-merge-diff/v4 v4.4.2 h1:MdmvkGuXi/8io6ixD5wud3vOLwc1rj0aNqRlpuvjmwA=
sigs.k8s.io/structured-merge-diff/v4 v4.4.2/go.mod h1:N8f93tFZh9U6vpxwRArLiikrE5/2tiu1w1AGfACIGE4=
sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=
sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=
+284
View File
@@ -0,0 +1,284 @@
// dns-webhook is a Kubernetes MutatingAdmissionWebhook.
//
// What is a MutatingAdmissionWebhook?
//
// When Kubernetes is about to create a resource (a Pod, Deployment, etc.),
// it first sends the resource's definition to every registered webhook for
// that resource type. The webhook can inspect the definition and return a
// set of JSON Patch operations — instructions that say "change field X to
// value Y before saving". Kubernetes applies those patches, then stores
// the final object.
//
// What does this webhook do?
//
// Every new Pod that uses Kubernetes cluster DNS (dnsPolicy: ClusterFirst)
// gets its DNS configuration rewritten so that:
//
// 1. Three nameserver IPs are chosen at random from the four production
// auth-dns pods (ns1ns4). This spreads DNS query load across the
// pool instead of every pod hitting the same two servers.
//
// 2. The resolv.conf options "edns0" and "rotate" are enabled.
// - edns0 allows DNS responses larger than 512 bytes (needed for
// DNSSEC and large TXT records).
// - rotate makes the resolver cycle through the nameserver list so
// queries are distributed rather than always going to the
// first server that responds.
//
// Pods that already have explicit DNS config (DNSNone), use the node's
// own resolver (DNSDefault), or run with host networking + cluster DNS
// (ClusterFirstWithHostNet) are left untouched.
//
// TLS requirement:
//
// Kubernetes requires admission webhooks to speak HTTPS. This server
// loads a TLS certificate and key from /tls/ (written by cert-manager).
// The paths can be overridden with TLS_CERT / TLS_KEY environment variables.
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"math/rand"
"net/http"
"os"
// admissionv1 contains the AdmissionReview, AdmissionRequest, and
// AdmissionResponse types that Kubernetes sends to / expects from the webhook.
admissionv1 "k8s.io/api/admission/v1"
// corev1 contains Pod, PodSpec, PodDNSConfig, etc.
corev1 "k8s.io/api/core/v1"
)
// nameserverPool lists the IPv6 addresses of the four production auth-dns pods
// (ns1ns4). ns0 is the staging instance and is intentionally excluded.
//
// These are static pod IPs allocated by Calico IPAM. They are stable across
// pod restarts because each Deployment pins its IP with the annotation
// cni.projectcalico.org/ipAddrs.
var nameserverPool = [4]string{
"2602:817:3000:c608::202", // ns1
"2602:817:3008:c607::204", // ns2
"2602:817:3000:c608::203", // ns3
"2602:817:3008:c607::203", // ns4
}
// clusterDomain is the Kubernetes cluster domain configured in kubelet.
// Service FQDNs follow the pattern: <service>.<namespace>.svc.<clusterDomain>
const clusterDomain = "k8s.sjc001.fritzlab.net"
// jsonPatch represents a single RFC 6902 JSON Patch operation.
//
// Kubernetes admission webhooks return patches in this format. The three
// fields mean:
// - Op: the operation — "add", "replace", or "remove"
// - Path: a slash-separated path into the JSON document, e.g. /spec/dnsPolicy
// - Value: the new value to set (omitted for "remove")
type jsonPatch struct {
Op string `json:"op"`
Path string `json:"path"`
Value interface{} `json:"value,omitempty"`
}
// pickThree returns three randomly selected nameserver IPs from nameserverPool.
//
// rand.Perm(4) returns a random permutation of [0, 1, 2, 3], so we take the
// first three indices to get three distinct servers. Using a permutation
// (rather than three independent random picks) guarantees no duplicates.
func pickThree() []string {
idx := rand.Perm(4)
return []string{nameserverPool[idx[0]], nameserverPool[idx[1]], nameserverPool[idx[2]]}
}
// strPtr is a small helper that returns a pointer to a string.
//
// The PodDNSConfigOption type uses *string for option values so that an
// absent value (nil) is distinguishable from an empty string ("").
func strPtr(s string) *string { return &s }
// handleMutate is the HTTP handler for the /mutate endpoint.
//
// Kubernetes calls this endpoint with an AdmissionReview JSON body whenever a
// Pod CREATE request matches the MutatingWebhookConfiguration rules. The
// handler must:
//
// 1. Decode the AdmissionReview to get the Pod definition.
// 2. Decide whether to mutate the Pod.
// 3. Return an AdmissionReview response with Allowed: true and any patches.
func handleMutate(w http.ResponseWriter, r *http.Request) {
// Read the full request body.
body, err := io.ReadAll(r.Body)
if err != nil {
log.Printf("ERROR reading request body: %v", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Unmarshal the outer AdmissionReview envelope.
// AdmissionReview is the top-level wrapper Kubernetes uses for both the
// incoming request and the outgoing response.
var review admissionv1.AdmissionReview
if err := json.Unmarshal(body, &review); err != nil {
log.Printf("ERROR unmarshalling AdmissionReview: %v", err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Extract the Pod from review.Request.Object.Raw.
// Raw is the verbatim JSON of the resource being admitted.
req := review.Request
var pod corev1.Pod
if err := json.Unmarshal(req.Object.Raw, &pod); err != nil {
log.Printf("ERROR unmarshalling Pod (uid=%s): %v", req.UID, err)
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Only mutate pods that use ClusterFirst DNS policy.
//
// DNSPolicy determines how a pod resolves names:
// ClusterFirst — use cluster DNS (CoreDNS); fall back to upstream.
// This is the default for most workloads.
// DNSNone — pod supplies its own dnsConfig; nothing to do.
// DNSDefault — inherit the node's /etc/resolv.conf directly.
// Used by auth-dns pods themselves to avoid a
// circular dependency (DNS pod needs DNS to start).
// ClusterFirstWithHostNet — host-network pod that still wants cluster DNS;
// leave as-is to avoid breaking host-network semantics.
if pod.Spec.DNSPolicy != corev1.DNSClusterFirst {
log.Printf("SKIP pod=%s/%s uid=%s policy=%s (not ClusterFirst)",
req.Namespace, pod.Name, req.UID, pod.Spec.DNSPolicy)
respond(w, review, nil) // no patches, just allow
return
}
// Determine the pod's namespace for the search domain list.
// Kubernetes guarantees this is set for namespaced resources, but fall back
// to "default" just in case.
ns := req.Namespace
if ns == "" {
ns = "default"
}
// Build the DNS configuration we want every pod to have.
//
// Nameservers: three randomly chosen auth-dns pod IPs.
//
// Searches: resolv.conf search domains let short names be resolved without
// a fully-qualified domain name. For a pod in namespace "myapp":
// - myapp.svc.k8s.sjc001.fritzlab.net → finds Services in myapp
// - svc.k8s.sjc001.fritzlab.net → finds Services in any namespace
// - k8s.sjc001.fritzlab.net → catches cluster-level names
// So "kubectl exec mypod -- curl myservice" resolves without a FQDN.
//
// Options:
// ndots:5 — names with fewer than 5 dots are tried with search domains
// before being treated as absolute. This is the Kubernetes
// default and ensures short service names resolve correctly.
// edns0 — enables EDNS extension headers, allowing DNS responses up to
// 65535 bytes (default UDP cap is 512 bytes).
// rotate — cycle through the nameserver list on each query so load is
// distributed rather than always hitting the first entry.
nameservers := pickThree()
dnsConfig := corev1.PodDNSConfig{
Nameservers: nameservers,
Searches: []string{
fmt.Sprintf("%s.svc.%s", ns, clusterDomain),
fmt.Sprintf("svc.%s", clusterDomain),
clusterDomain,
},
Options: []corev1.PodDNSConfigOption{
{Name: "edns0"},
{Name: "rotate"},
{Name: "ndots", Value: strPtr("5")},
},
}
// JSON Patch requires "add" if the field doesn't exist yet in the manifest,
// or "replace" if it does. Sending "replace" for a non-existent field
// (or "add" for an existing one) is a patch error that would reject the pod.
dnsConfigOp := "add"
if pod.Spec.DNSConfig != nil {
dnsConfigOp = "replace"
}
log.Printf("MUTATE pod=%s/%s uid=%s nameservers=%v op=%s",
ns, pod.Name, req.UID, nameservers, dnsConfigOp)
// Build the two patch operations:
// 1. Change dnsPolicy from ClusterFirst → None.
// DNSNone is required when you provide a custom dnsConfig; it tells
// kubelet to use exactly the config we supply and nothing else.
// 2. Set dnsConfig to our constructed value.
patches := []jsonPatch{
{Op: "replace", Path: "/spec/dnsPolicy", Value: corev1.DNSNone},
{Op: dnsConfigOp, Path: "/spec/dnsConfig", Value: dnsConfig},
}
respond(w, review, patches)
}
// respond writes an AdmissionReview response back to the Kubernetes API server.
//
// Every call to the /mutate endpoint must return an AdmissionReview with:
// - UID matching the incoming request (so Kubernetes can correlate them)
// - Allowed: true (we never block pods, just modify them)
// - Patch / PatchType set when we have mutations to apply
func respond(w http.ResponseWriter, review admissionv1.AdmissionReview, patches []jsonPatch) {
resp := &admissionv1.AdmissionResponse{
UID: review.Request.UID,
Allowed: true,
}
if patches != nil {
// Serialize the patch list to JSON and set the patch type.
// JSONPatch is the only patch type supported by admission webhooks.
b, _ := json.Marshal(patches)
pt := admissionv1.PatchTypeJSONPatch
resp.Patch = b
resp.PatchType = &pt
}
// Place the response inside the same AdmissionReview envelope that came in.
// Kubernetes reads review.Response; review.Request is ignored on the way out.
review.Response = resp
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(review); err != nil {
log.Printf("ERROR encoding AdmissionReview response (uid=%s): %v", resp.UID, err)
}
}
func main() {
// TLS cert and key are written by cert-manager into a Kubernetes Secret,
// which is mounted into the pod at /tls/. The environment variables allow
// the paths to be overridden in development or testing.
certFile := getenv("TLS_CERT", "/tls/tls.crt")
keyFile := getenv("TLS_KEY", "/tls/tls.key")
// /mutate receives AdmissionReview requests from the Kubernetes API server.
http.HandleFunc("/mutate", handleMutate)
// /healthz is a simple liveness/readiness probe endpoint.
// kubelet calls this to decide whether the pod is healthy.
http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
log.Printf("dns-webhook starting: cert=%s key=%s", certFile, keyFile)
log.Println("dns-webhook listening on :8443")
// Kubernetes requires admission webhooks to use TLS — plain HTTP is rejected.
log.Fatal(http.ListenAndServeTLS(":8443", certFile, keyFile, nil))
}
// getenv returns the value of the environment variable key, or def if unset.
func getenv(key, def string) string {
if v := os.Getenv(key); v != "" {
return v
}
return def
}