fritzlab-public/flock
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

anycast: put IP on pod eth0, not lo
Build flock Image / build (push) Has been cancelled
Details

Browse Source 
The design doc's lo placement was motivated by avoiding NDP/ARP DAD
conflicts "across nodes advertising the same IP" — but flock pods each
sit on their own /64 veth subnet. DAD on eth0 only sees the host peer,
no cross-node L2.

With the IP on lo, the pod kernel doesn't reply to NDP solicits arriving
on eth0 (Linux default: answer NDP only for addresses on the receiving
interface). The host route `<ip>/128 dev flock<8hex>` causes the host
to do NDP for the destination on the veth; pod ignores; packet drops
silently between forwarding decision and transmit. Symptom: v4 anycast
works (proxy_arp=1 on the host veth handles ARP), v6 anycast doesn't.

Putting on eth0 makes NDP just work.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Donavan Fritz
2026-04-25 07:55:12 -05:00
parent 3f6dfd3e88
commit e1e9544e2e
1 changed files with 13 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pkg/agent/netns_linux.go
+13 -11
View File
@@ -241,16 +241,18 @@ func configurePodSide(req SetupRequest) error {
}
}
// Anycast: assign each IP to pod lo. NOT on eth0 (avoids NDP/ARP
// DAD conflicts when multiple replicas share the same IP).
// Anycast: assign each IP to pod eth0 (NOT lo).
//
// The original design doc proposed lo to avoid NDP/ARP DAD
// conflicts "across nodes advertising the same IP". That concern
// doesn't apply to flock: each pod's veth is its own private /64,
// so DAD on eth0 only sees the veth peer (host) — no cross-node
// L2 contention. Putting the IP on eth0 instead means the pod
// kernel answers NDP solicits arriving on eth0 for that IP, which
// is what the host's /128 host route requires. With anycast on
// lo, NDP from the host side fails and the kernel drops the
// packet between routing decision and transmit.
if len(req.Anycast) > 0 {
lo, err := netlink.LinkByName("lo")
if err != nil {
return fmt.Errorf("lookup pod lo: %w", err)
}
if err := netlink.LinkSetUp(lo); err != nil {
return fmt.Errorf("set up pod lo: %w", err)
}
for _, ip := range req.Anycast {
var mask net.IPMask
if ip.To4() != nil {
@@ -260,8 +262,8 @@ func configurePodSide(req SetupRequest) error {
mask = net.CIDRMask(128, 128)
}
a := &netlink.Addr{IPNet: &net.IPNet{IP: ip, Mask: mask}, Scope: int(netlink.SCOPE_UNIVERSE)}
if err := netlink.AddrAdd(lo, a); err != nil && !errors.Is(err, os.ErrExist) {
return fmt.Errorf("pod lo anycast %s: %w", ip, err)
if err := netlink.AddrAdd(eth0, a); err != nil && !errors.Is(err, os.ErrExist) {
return fmt.Errorf("pod eth0 anycast %s: %w", ip, err)
}
}
}