Commit Graph
9 Commits
Author SHA1 Message Date
opsandClaude Sonnet 4.6 c9950348ff flock-agent: GC orphaned allocations; retry birdc on socket-not-ready
flock PR validation / validate (pull_request) Successful in 11s
Three defects enabled the 2026-08-16 Gitea blackhole (bug-wdgjpz3a00gd):

1. Orphaned allocation GC missing: ungraceful eviction (TaintManagerEviction)
   never calls CNI DEL, so the old node keeps advertising the pod's public
   /128 via BGP. Older allocation wins BGP path selection; live pod's node
   yields → blackhole.

   Fix: after the pod informer syncs at startup, sweep all committed
   allocations via orphanedCommitted(). Any allocation whose owner pod is
   absent from the node (or whose UID mismatches, indicating name reuse) is
   torn down, removed from the store, and released from IPAM. A 60 s
   periodic GC goroutine provides the same sweep while the agent runs.

2. renderBird outside-aggregate IP loop lacked pod liveness check: stale
   committed allocations caused BIRD to keep advertising the /128 even in
   steady state between GC ticks.

   Fix: before adding an outside-aggregate primary IP to the BIRD export,
   verify the pod is still in the node-scoped informer cache with a matching
   UID. Orphans are skipped silently; the GC cleans them on the next tick.

3. birdc startup race: the agent's first Render() fires before BIRD has
   bound /run/flock/bird.ctl, so the configure call silently fails with
   "Unable to connect" and the initial routes are never advertised. A
   container-only flock-agent restart (BIRD left running) avoids the race;
   a full pod restart re-hits it.

   Fix: reload() now retries up to 20 × 500 ms on socket-absent and
   "Unable to connect" conditions. Any other birdc failure (syntax error,
   etc.) is not retried.

Fixes bug-wdgjpz3a00gd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-16 06:26:38 +00:00
Donavan FritzandClaude Opus 4.7 c860e9351b ip-algo: pod annotation > NodeConfig annotation > random
Build flock Image / build (push) Has been cancelled
Add flock.fritzlab.net/ip-algo as a node-wide default via NodeConfig
metadata.annotations. Pod-level annotation still wins. Empty, missing,
or invalid input at either level falls through to the next; invalid
values warn-log via the agent's slog. Both unset → fully random IID
(unchanged baseline).

ParseAnnotations no longer touches ip-algo; ResolveIPAlgo handles the
full precedence chain, called from PodHandler.Add with the cached
NodeConfig's annotations and the agent logger.

Tests: 9 new TestResolveIPAlgo_* cases covering pod-wins, all
fall-through paths, both-absent, nil node map, whitespace, and
duplicate-as-invalid. Fuzz target rebuilt without ip-algo input space
(now exercised by ResolveIPAlgo unit tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:09:09 -05:00
Donavan FritzandClaude Sonnet 4.6 a7dc7bf1f4 anycast: kernel multipath route + L4 hash for multi-pod-per-node
Build flock Image / build (push) Has been cancelled
Move pure resolver logic out of anycast_linux.go into anycast.go so it's
unit-testable on any host. Reshape anycastTarget from a single
{hostIface, via} into a sorted list of nexthops; multiple Ready pods on
the same node binding the same anycast IP now contribute one nexthop
each.

installAnycastRoute uses RTA_MULTIPATH (via netlink.Route.MultiPath)
when the target has more than one nexthop. Single-nexthop targets keep
the simple via-route shape so 1-pod-per-node keeps rendering identically
to today's production form in `ip route show`.

flock-agent writes net.ipv{4,6}.fib_multipath_hash_policy = 1 at
startup so the kernel hashes flows on (saddr, daddr, sport, dport, proto)
rather than just IPs. Best-effort — runs privileged in production, so
it works; falls back to L3 hash on environments where the write fails
(only matters for the multi-pod-per-node case anyway).

resolveAnycastTargets sorts nexthops by canonical(via) for stable
comparison so a quiet reconcile pass doesn't churn the kernel route.

8 new unit tests cover: 1-pod, 2-pods-same-anycast (multi-nexthop),
NotReady drop, no-Ready omits the IP, pending skipped, mixed v6+v4,
family mismatch warns, determinism.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 09:57:32 -05:00
Donavan FritzandClaude Sonnet 4.6 39ede9130b netpol: NetworkPolicy v1 enforcement via nftables
Build flock Image / build (push) Has been cancelled
New pkg/agent/netpol implementing standard networking.k8s.io/v1
NetworkPolicy. Pipeline:

  pods + policies + namespaces  →  Translate  →  Render  →  Apply

Supports ingress + egress, all three peer types (podSelector,
namespaceSelector, ipBlock with except), numeric ports + port ranges,
default-deny semantics derived from PolicyTypes (or inferred from
non-empty Spec.Egress when unset).

Apply path is `nft -f -` shell-out — single transaction, atomic, kernel
guarantees partial-failure rollback. Idempotent dedup via last-applied
script. Reconcile triggers: informer events, 30s self-heal tick, every
CNI ADD/DEL.

Verified against the three live cluster NetPols (calico-apiserver,
remote-proxies/lodge-home-assistant, storage/garage-admin-restrict).
Fuzz target stitches Translate + Render with random selector and peer
inputs; 21 unit tests cover the policy semantics.

Named ports skip with a warn — deferred until kubelet exposes them in a
form that doesn't require shadowing pod state.

Dockerfile: + nftables.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 09:25:58 -05:00
Donavan FritzandClaude Sonnet 4.6 89a3502446 M6: anycast — pod lo + Ready-gated /128/32 + BIRD export
Build flock Image / build (push) Has been cancelled
CNI ADD now adds anycast IPs to the pod's lo interface (NOT eth0 — design
doc rationale: avoid NDP/ARP DAD conflicts when N replicas share an IP).
Allocation persists the anycast list.

AnycastReconciler:
  desired = { ip → flock<8hex> } from
            committed allocations × pod.Status.PodReady=True
  diff against advertised, install/remove host /128 (v6) or /32 (v4)
  re-render bird.conf with the active set

Triggers: 2s tick, AfterCommit (per ADD/DEL), Pod informer Ready
transitions (PodCache.OnReadyChange callback).

The bird template already supported Anycast6/Anycast4 via the export
filter — this turn finally drives those slices from runtime.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-25 07:36:47 -05:00
Donavan FritzandClaude Sonnet 4.6 c7fb159632 agent: maintain NetworkUnavailable=False on owned nodes
Build flock Image / build (push) Has been cancelled
When Calico shuts down on a flock-labeled node, calico-node sets
NetworkUnavailable=True with reason CalicoIsDown. Nothing replaces it,
so kubelet's NodeController applies node.kubernetes.io/network-
unavailable:NoSchedule and new pods can't land.

flock-agent now patches Status.Conditions every 60s with
NetworkUnavailable=False (reason=FlockReady). RBAC: nodes/status patch.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 23:11:47 -05:00
Donavan FritzandClaude Sonnet 4.6 37cc3f6750 runtime: enable BIRD BGP on flock-labeled nodes
Build flock Image / build (push) Has been cancelled
Calico fenced off via Tigera Installation CR (apps@2121892). flock-agent
now renders bird.conf with the per-node BGP peers; bird sidecar reloads
on changes (debounced 500ms). Re-render tick every 15s reacts to
NodeConfig updates.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 23:02:33 -05:00
Donavan FritzandClaude Sonnet 4.6 06110884d4 runtime: skip BIRD render on first cutover
Build flock Image / build (push) Has been cancelled
Calico's calico-node still runs on every node (Tigera-Operator-managed
via ArgoCD with selfHeal). Two birds with the same ASN can't peer to
crt001 from the same source. Use a manual static route on crt001 for
the flock /64 for the first cutover; switch to live BGP after Calico is
fenced off flock-labeled nodes.

The bird sidecar stays running with the bootstrap config (kernel +
device only, no BGP), so flipping live BGP on later is a single-line
change in runtime_linux.go.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 22:41:40 -05:00
Donavan FritzandClaude Sonnet 4.6 eb1f5e0d8d M2: netlink, IPAM/handler wiring, BIRD sidecar, CNI installer
Build flock Image / build (push) Has been cancelled
Code (Linux build, with no-op stubs for macOS dev):
- pkg/agent/netns_linux.go: ensureVeth → host-side configure (addrgenmode
  none, fe80::1/64, proxy_arp, forwarding) → move peer to pod ns →
  configure pod side (addr, default route via fe80::1, v4 169.254.1.1
  on-link gateway) → host /128 + /32 routes. Idempotent.
- pkg/agent/hostiface.go: deterministic host iface name flock<8hex> from
  FNV-1a-32(containerID).
- pkg/agent/annotations.go: parse flock.fritzlab.net/{ipv6,ipv4,cidr6,
  cidr4,ip-algo,anycast} with design-doc defaults; ParseCNIArgs for the
  K8S_POD_* keys kubelet sets.
- pkg/agent/podinfo.go: shared informer scoped to spec.nodeName==NODE,
  WaitForPod helper for ADD-vs-informer-sync race.
- pkg/agent/handlers.go: PodHandler does
    cache lookup → annotations → IPAM → store(pending) → SetupFunc →
    store(committed) → Result. Idempotent on retry. Del symmetric.
- pkg/routing/bird/config.go: text/template render with stable ordering;
  golden tests for host001 + anycast injection + sort stability.
- pkg/agent/bird.go: writes /etc/flock/bird/bird.conf, debounces 500ms,
  execs `birdc -s /run/flock/bird.ctl configure`. Installs blackhole
  kernel routes for the node summary CIDRs so BIRD's protocol kernel
  imports them.
- pkg/agent/runtime_linux.go: at startup, waits up to 60s for the per-
  node NodeConfig, reconciles committed allocations into IPAM.used,
  garbage-collects pending entries, builds PodHandler, swaps RPC
  handlers in.
- cmd/flock-installer: init-container binary that copies /opt/cni/bin/
  flock and writes 01-flock.conflist (lex-first so kubelet picks it
  over Calico's 10-calico.conflist on flock-labeled nodes).

Deploy:
- Dockerfile: alpine + iproute2 + bird2; multi-binary image.
- deploy/daemonset.yaml: install-cni init container; bird sidecar
  sharing /etc/flock/bird + /run/flock with the agent; ConfigMap-seeded
  bootstrap bird.conf so the sidecar boots before the agent renders.
  Privileged on flock-agent + install-cni; bird sidecar uses
  NET_ADMIN/RAW only.
- RBAC: pods + networkpolicies get/list/watch (the latter is reserved
  for M8 — harmless to grant now).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 22:33:48 -05:00