Commit Graph
5 Commits
Author SHA1 Message Date
opsandClaude Sonnet 4.6 c9950348ff flock-agent: GC orphaned allocations; retry birdc on socket-not-ready
flock PR validation / validate (pull_request) Successful in 11s
Three defects enabled the 2026-08-16 Gitea blackhole (bug-wdgjpz3a00gd):

1. Orphaned allocation GC missing: ungraceful eviction (TaintManagerEviction)
   never calls CNI DEL, so the old node keeps advertising the pod's public
   /128 via BGP. Older allocation wins BGP path selection; live pod's node
   yields → blackhole.

   Fix: after the pod informer syncs at startup, sweep all committed
   allocations via orphanedCommitted(). Any allocation whose owner pod is
   absent from the node (or whose UID mismatches, indicating name reuse) is
   torn down, removed from the store, and released from IPAM. A 60 s
   periodic GC goroutine provides the same sweep while the agent runs.

2. renderBird outside-aggregate IP loop lacked pod liveness check: stale
   committed allocations caused BIRD to keep advertising the /128 even in
   steady state between GC ticks.

   Fix: before adding an outside-aggregate primary IP to the BIRD export,
   verify the pod is still in the node-scoped informer cache with a matching
   UID. Orphans are skipped silently; the GC cleans them on the next tick.

3. birdc startup race: the agent's first Render() fires before BIRD has
   bound /run/flock/bird.ctl, so the configure call silently fails with
   "Unable to connect" and the initial routes are never advertised. A
   container-only flock-agent restart (BIRD left running) avoids the race;
   a full pod restart re-hits it.

   Fix: reload() now retries up to 20 × 500 ms on socket-absent and
   "Unable to connect" conditions. Any other birdc failure (syntax error,
   etc.) is not retried.

Fixes bug-wdgjpz3a00gd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-16 06:26:38 +00:00
Donavan FritzandClaude Opus 4.7 9b777ca7d1 bird: per-peer import filter rejects connected subnet
Build flock Image / build (push) Successful in 2m17s
Without a filter, crt001's `network 2602:817:3000:A25::/64` gets
re-advertised to every peer on that subnet. bird installs the BGP /64
with metric 32, beating the kernel-connected route at 256, and all
inter-host VLAN-25 traffic hairpins through the gateway — losing PMTU
9000 and ~30x throughput. Broke Plex 2026-05-04: NFS to nas002 capped
at 7 MB/s, jumbo blackholed.

Add LocalSubnetV6/V4 (CIDR) to NodeBGP. Agent populates by masking the
peer's address to /64 (v6) or /24 (v4) — same fritzlab convention
already in localAddrSameSubnet. Render emits `import where net !=
<subnet>;` per BGP channel when set, falls back to `import all;`
otherwise so existing tests stay green.

Defence in depth: with the matching outbound route-map on crt001
(ROUTE_MAP_CLUSTER_OUT_V{4,6}) the agent now refuses the leak on its
own if the router filter ever drifts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 21:03:59 -05:00
Donavan FritzandClaude Sonnet 4.6 677aec2a42 bird: leading-edge reload + 500ms cooldown (was trailing 500ms debounce)
Build flock Image / build (push) Has been cancelled
A single Ready/NotReady transition no longer pays a 500ms reload wait —
the first call to scheduleReload fires birdc immediately; further calls
within 500ms are coalesced into one tail reload at the cooldown's end.

Burst behavior is the same as before: under heavy churn (deploy rolling
all replicas at once), at most one reload per 500ms.

Steady-state latency from pod Ready transition to crt001 BGP withdraw:
  - probe period (set in pod spec, 1s minimum)
  - ~ms informer + reconcile + birdc + BGP UPDATE
The 500ms hardcoded delay is gone.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 08:26:34 -05:00
Donavan FritzandClaude Sonnet 4.6 3f6dfd3e88 bird: add source address + next hop self (v6 anycast fix)
Build flock Image / build (push) Has been cancelled
Cisco IOS rejects IPv6 BGP advertisements whose next-hop is link-local-
only. BIRD2 was synthesising a link-local next-hop for kernel-learned
routes whose dev had no via gateway (our anycast /128s). Symptom: v4
anycast worked (Cisco doesn't have the same constraint for /32s), v6
anycast didn't make it past crt001.

- pkg/routing/bird/config.go: NodeBGP.LocalV6/LocalV4. Template now
  emits `local <addr> as <asn>` and `next hop self;` in the BGP
  channel for both families, mirroring Calico's `source address` +
  `next hop self` pattern.
- pkg/agent/bird.go: localAddrSameSubnet picks an interface address
  on the peer's /64 or /24 to use as source.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 07:45:51 -05:00
Donavan FritzandClaude Sonnet 4.6 eb1f5e0d8d M2: netlink, IPAM/handler wiring, BIRD sidecar, CNI installer
Build flock Image / build (push) Has been cancelled
Code (Linux build, with no-op stubs for macOS dev):
- pkg/agent/netns_linux.go: ensureVeth → host-side configure (addrgenmode
  none, fe80::1/64, proxy_arp, forwarding) → move peer to pod ns →
  configure pod side (addr, default route via fe80::1, v4 169.254.1.1
  on-link gateway) → host /128 + /32 routes. Idempotent.
- pkg/agent/hostiface.go: deterministic host iface name flock<8hex> from
  FNV-1a-32(containerID).
- pkg/agent/annotations.go: parse flock.fritzlab.net/{ipv6,ipv4,cidr6,
  cidr4,ip-algo,anycast} with design-doc defaults; ParseCNIArgs for the
  K8S_POD_* keys kubelet sets.
- pkg/agent/podinfo.go: shared informer scoped to spec.nodeName==NODE,
  WaitForPod helper for ADD-vs-informer-sync race.
- pkg/agent/handlers.go: PodHandler does
    cache lookup → annotations → IPAM → store(pending) → SetupFunc →
    store(committed) → Result. Idempotent on retry. Del symmetric.
- pkg/routing/bird/config.go: text/template render with stable ordering;
  golden tests for host001 + anycast injection + sort stability.
- pkg/agent/bird.go: writes /etc/flock/bird/bird.conf, debounces 500ms,
  execs `birdc -s /run/flock/bird.ctl configure`. Installs blackhole
  kernel routes for the node summary CIDRs so BIRD's protocol kernel
  imports them.
- pkg/agent/runtime_linux.go: at startup, waits up to 60s for the per-
  node NodeConfig, reconciles committed allocations into IPAM.used,
  garbage-collects pending entries, builds PodHandler, swaps RPC
  handlers in.
- cmd/flock-installer: init-container binary that copies /opt/cni/bin/
  flock and writes 01-flock.conflist (lex-first so kubelet picks it
  over Calico's 10-calico.conflist on flock-labeled nodes).

Deploy:
- Dockerfile: alpine + iproute2 + bird2; multi-binary image.
- deploy/daemonset.yaml: install-cni init container; bird sidecar
  sharing /etc/flock/bird + /run/flock with the agent; ConfigMap-seeded
  bootstrap bird.conf so the sidecar boots before the agent renders.
  Privileged on flock-agent + install-cni; bird sidecar uses
  NET_ADMIN/RAW only.
- RBAC: pods + networkpolicies get/list/watch (the latter is reserved
  for M8 — harmless to grant now).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 22:33:48 -05:00