NodeConfig defaults + code-quality pass + fuzz tests + README
NodeConfig.Spec.Defaults adds per-node IPv6/IPv4 family defaults that pod annotations can override; built-in baseline (v6=true, v4=false) still applies when the field is omitted. bird.Render now validates every operator-supplied value (peer addresses, CIDRs, anycast IPs, source addresses) before templating — fuzz found a peer address containing `}` produced unbalanced braces in bird.conf. Failing input preserved as a regression seed. Fuzz targets added for ParseAnnotations, ParseCNIArgs, HostIfaceName, canonical, IPAM allocate sequences, embed.Embed, and bird.Render. Hardened canonical/ipToU32 against nil and non-IPv4 inputs. README rewritten for outside readers — quickstart, NodeConfig + annotation reference with worked examples, anycast use cases, comparison vs Calico and Cilium, requirements, limitations. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Donavan Fritz
@@ -1,22 +1,335 @@
|
||||
# flock
|
||||
|
||||
Kubernetes CNI for sjc001. Per-pod IPv4 opt-in, IID embedding, Ready-gated anycast via BGP.
|
||||
A small, opinionated Kubernetes CNI built around three ideas:
|
||||
|
||||
Design doc: `k8s-manager/dfritz-cni.md` (in the operator's k8s-manager repo).
|
||||
1. **IPv6-first.** Every pod gets a globally routable IPv6 address. IPv4 is
|
||||
per-pod opt-in for legacy clients.
|
||||
2. **No tunnels, no NAT.** Pod addresses are the real packets on the wire.
|
||||
Each node speaks BGP to its upstream router and advertises its own
|
||||
per-node prefix. The pod network is just the LAN, plus host routes.
|
||||
3. **Anycast as a primitive.** A pod can request an anycast address via
|
||||
an annotation; flock binds it on the pod's loopback and advertises a
|
||||
`/128` (or `/32`) over BGP, but only while the pod is `Ready`. Multiple
|
||||
replicas advertise the same address from different nodes for ECMP load
|
||||
balancing without a separate Service or external LB.
|
||||
|
||||
Status: M1 scaffold. Not functional. See milestones table in the design doc.
|
||||
flock is built for clusters where every node already speaks BGP to one
|
||||
or more upstream routers. It deliberately leaves out features you'd
|
||||
expect from a general-purpose CNI — overlays, IPsec/Wireguard, IPAM
|
||||
coordination across nodes, kube-proxy integration — so the moving parts
|
||||
that remain are easy to reason about.
|
||||
|
||||
## Layout
|
||||
> **Status:** alpha. CRD shape and annotation keys may still change.
|
||||
|
||||
- `cmd/flock` — CNI plugin binary (kubelet-invoked)
|
||||
- `cmd/flock-agent` — DaemonSet binary
|
||||
- `pkg/api/v1alpha1` — `NodeConfig` CRD types
|
||||
- `pkg/cni` — CNI plugin internals + RPC client
|
||||
- `pkg/agent` — agent server, IPAM, state file, anycast, NetworkPolicy
|
||||
- `pkg/embed` — `ip-algo` IID embedding (pure)
|
||||
- `pkg/routing/{bird,ospf}` — routing backends
|
||||
- `deploy/` — CRDs, RBAC, DaemonSet manifests
|
||||
## Table of contents
|
||||
|
||||
- [How it works](#how-it-works)
|
||||
- [Requirements](#requirements)
|
||||
- [Quickstart](#quickstart)
|
||||
- [NodeConfig CRD](#nodeconfig-crd)
|
||||
- [Pod annotations](#pod-annotations)
|
||||
- [Use cases](#use-cases)
|
||||
- [Comparison vs Calico / Cilium](#comparison-vs-calico--cilium)
|
||||
- [Limitations and non-goals](#limitations-and-non-goals)
|
||||
- [Building and testing](#building-and-testing)
|
||||
- [License](#license)
|
||||
|
||||
## How it works
|
||||
|
||||
Each node runs a single `flock-agent` DaemonSet pod with three containers:
|
||||
|
||||
- a privileged init container (`flock-installer`) that drops the CNI
|
||||
plugin binary into `/opt/cni/bin/flock` and writes
|
||||
`/etc/cni/net.d/01-flock.conflist`,
|
||||
- the agent itself, which owns IPAM, programs veth pairs, and tracks
|
||||
pod readiness, and
|
||||
- a [BIRD2](https://bird.network.cz/) sidecar that the agent re-renders
|
||||
and reloads when the per-node config or the active anycast set changes.
|
||||
|
||||
Each node has a `NodeConfig` CR (cluster-scoped, name = node name) that
|
||||
declares its IPv6 and IPv4 prefixes, its local BGP ASN, and its upstream
|
||||
peers. The agent reads the CR via a dynamic informer.
|
||||
|
||||
When kubelet runs the CNI plugin on `ADD`, the plugin opens a unix-socket
|
||||
RPC to the agent. The agent allocates an address from the per-node
|
||||
CIDRs, creates a veth pair, configures the pod side, persists the
|
||||
allocation to `/var/lib/flock/allocations.json`, and returns the result.
|
||||
There is no controller loop and no IPAM coordination across nodes — each
|
||||
node owns a non-overlapping CIDR and allocates locally.
|
||||
|
||||
For anycast, the agent installs `<anycast-ip> via <pod-eth0-ip> dev <veth>`
|
||||
host routes on the node and adds the anycast IP to BIRD's BGP export
|
||||
filter. When a pod loses readiness, the agent withdraws the route from
|
||||
both the kernel and BGP within one reconcile cycle (sub-second).
|
||||
|
||||
### Packet path
|
||||
|
||||
`pod.eth0` (a veth) ↔ host-side veth (with `addrgenmode none`,
|
||||
`fe80::1/64`, proxy-ARP for the v4 default-via) ↔ host kernel ↔ uplink
|
||||
NIC ↔ upstream router. No conntrack, no SNAT, no encapsulation.
|
||||
|
||||
For IPv6 the host side of every veth carries the deterministic link-local
|
||||
gateway `fe80::1`, so every pod can use a fixed default route. For IPv4
|
||||
the host side answers ARP for `169.254.1.1`, providing the same fixed
|
||||
default route in v4.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Linux nodes. flock has not been tested on, and does not target,
|
||||
Windows nodes.
|
||||
- Kubernetes ≥ 1.27.
|
||||
- An upstream router (or pair) that accepts a BGP session from each
|
||||
node. flock has been tested with Cisco IOS-XE, Arista EOS, and FRR
|
||||
acting as the upstream; anything that speaks standard eBGP should work.
|
||||
- Globally routable (or at least datacentre-routable) IPv6 prefix
|
||||
delegated to the cluster, sliced into a per-node /64. IPv4 is
|
||||
optional but supported.
|
||||
- Each node must have a unique local ASN. Private ASNs (`64512–65534`,
|
||||
`4200000000–4294967294`) are typical.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```sh
|
||||
# 1. Install CRD + RBAC + DaemonSet (single bundled manifest):
|
||||
kubectl apply -f deploy/install.yaml
|
||||
|
||||
# 2. Label the node(s) you want flock to manage:
|
||||
kubectl label node <node-name> flock.fritzlab.net/agent=
|
||||
|
||||
# 3. Apply a NodeConfig CR for that node (see "NodeConfig CRD" below):
|
||||
kubectl apply -f my-nodeconfig.yaml
|
||||
|
||||
# 4. Verify the agent is up:
|
||||
kubectl -n kube-system get pod -l app=flock-agent -o wide
|
||||
kubectl -n kube-system exec -it ds/flock-agent -c bird -- \
|
||||
birdc -s /run/flock/bird.ctl show protocols
|
||||
```
|
||||
|
||||
The DaemonSet is gated by the `flock.fritzlab.net/agent` node label, so
|
||||
unlabelled nodes continue to use whatever CNI was installed before. This
|
||||
lets you migrate node-by-node — start with one node, prove it works, then
|
||||
proceed.
|
||||
|
||||
## NodeConfig CRD
|
||||
|
||||
A `NodeConfig` is the only operator-supplied input. One per node, name
|
||||
matches the node name. Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: flock.fritzlab.net/v1alpha1
|
||||
kind: NodeConfig
|
||||
metadata:
|
||||
name: node-a
|
||||
spec:
|
||||
cidr6:
|
||||
- 2001:db8:f001::/64 # Pods on this node get addresses from here.
|
||||
cidr4:
|
||||
- 192.0.2.0/24 # IPv4 pool, used only when a pod opts in.
|
||||
defaults:
|
||||
ipv6: true # Optional. Built-in baseline if omitted.
|
||||
ipv4: false # Optional. Built-in baseline if omitted.
|
||||
bgp:
|
||||
asn: 65101 # This node's local ASN.
|
||||
peers:
|
||||
- address: 2001:db8::1 # Upstream router (IPv6 session).
|
||||
asn: 65000
|
||||
- address: 192.0.2.1 # Same router, IPv4 session.
|
||||
asn: 65000
|
||||
```
|
||||
|
||||
### `spec.defaults`
|
||||
|
||||
`spec.defaults` controls which address families a pod *gets by default*
|
||||
on this node — i.e. when the pod has no explicit `flock.fritzlab.net/ipv6`
|
||||
or `flock.fritzlab.net/ipv4` annotation. Pod annotations always override.
|
||||
If you omit `spec.defaults` (or any individual field inside it) flock
|
||||
falls back to its built-in baseline of **IPv6 on, IPv4 off**.
|
||||
|
||||
| Goal | `spec.defaults` |
|
||||
|---------------------------|----------------------------------------|
|
||||
| IPv6-only (the default) | omit, or `{ ipv6: true, ipv4: false }`|
|
||||
| Dual-stack by default | `{ ipv6: true, ipv4: true }` |
|
||||
| IPv4-only (legacy node) | `{ ipv6: false, ipv4: true }` |
|
||||
|
||||
A NodeConfig that resolves to "neither family" is rejected at allocation
|
||||
time, so misconfiguring both to false will surface as an error on the
|
||||
first `CNI ADD`.
|
||||
|
||||
### `spec.bgp`
|
||||
|
||||
Each `peer` becomes one BGP session. The agent picks a node-local source
|
||||
address on the same subnet as the peer; if there isn't one, BIRD uses
|
||||
its default. Multi-homing (multiple peers per family — or per upstream
|
||||
router pair) is allowed.
|
||||
|
||||
## Pod annotations
|
||||
|
||||
All annotations live under `flock.fritzlab.net/`. Every annotation is
|
||||
optional; leave them off to inherit the per-node defaults.
|
||||
|
||||
| Annotation | Type | Purpose |
|
||||
|-------------------------------------|--------|-----------------------------------------------------------------------------------------------|
|
||||
| `flock.fritzlab.net/ipv6` | bool | Override `spec.defaults.ipv6` for this pod (`true`/`false`). |
|
||||
| `flock.fritzlab.net/ipv4` | bool | Override `spec.defaults.ipv4` for this pod (`true`/`false`). |
|
||||
| `flock.fritzlab.net/cidr6` | CIDRs | Restrict IPv6 allocation to a sub-range of the node's `cidr6`. Comma-separated. |
|
||||
| `flock.fritzlab.net/cidr4` | CIDRs | Restrict IPv4 allocation to a sub-range of the node's `cidr4`. Comma-separated. |
|
||||
| `flock.fritzlab.net/ip-algo` | list | Embed identity into the IPv6 IID. Subset of `namespace,pod,image`, in order, comma-separated. |
|
||||
| `flock.fritzlab.net/anycast` | IPs | Bind these IPs on the pod's `lo`; advertise via BGP while pod is `Ready`. Mixed v6+v4 ok. |
|
||||
|
||||
Bool values must be the literal strings `"true"` or `"false"`
|
||||
(case-insensitive, surrounding whitespace tolerated). Other values —
|
||||
`1`, `0`, `yes`, `no` — are rejected so a typo can't silently flip
|
||||
behaviour.
|
||||
|
||||
### Example pods
|
||||
|
||||
Default IPv6-only — no annotations needed:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: minimal
|
||||
```
|
||||
|
||||
Dual-stack on a node whose default is IPv6-only:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: legacy-client
|
||||
annotations:
|
||||
flock.fritzlab.net/ipv4: "true"
|
||||
```
|
||||
|
||||
Operator-friendly addressing — `fnv(namespace) | fnv(pod) | random`
|
||||
packed into the host bits, so a pod's identity is recognisable from
|
||||
its IP in `kubectl get pods -o wide`:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
annotations:
|
||||
flock.fritzlab.net/ip-algo: "namespace,pod"
|
||||
```
|
||||
|
||||
Anycast service — three replicas, each advertising the same v6+v4
|
||||
anycast pair from the node it lands on. The upstream router does ECMP
|
||||
across the active set:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: dns
|
||||
spec:
|
||||
replicas: 3
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
flock.fritzlab.net/ipv4: "true"
|
||||
flock.fritzlab.net/anycast: "2001:db8:a::53, 192.0.2.53"
|
||||
spec:
|
||||
containers:
|
||||
- name: coredns
|
||||
image: coredns/coredns
|
||||
readinessProbe:
|
||||
httpGet: { path: /ready, port: 8181 }
|
||||
periodSeconds: 1
|
||||
failureThreshold: 1
|
||||
```
|
||||
|
||||
## Use cases
|
||||
|
||||
**Highly-available DNS.** Run N CoreDNS replicas, each annotated with
|
||||
the same `anycast` IP. Point client `/etc/resolv.conf` at the anycast
|
||||
address. Each replica advertises a `/128` from its own node; the
|
||||
upstream router does ECMP. Lose a pod, traffic fails over within a
|
||||
probe cycle.
|
||||
|
||||
**Replacing a kube-proxy `ClusterIP`.** Headless Service plus an anycast
|
||||
IP gives you a single stable address with load-balancing across pods,
|
||||
without the DNAT-pinning that makes long-lived TCP keepalive connections
|
||||
stick to one backend forever. ECMP routes each new flow independently.
|
||||
|
||||
**Per-pod public IPv6.** Because every pod has a globally routable IPv6
|
||||
address and the cluster does no NAT, a pod's `eth0` IP is reachable from
|
||||
the rest of the internet (subject to your firewall). Useful for things
|
||||
like outgoing SMTP, where you want a stable from-address per pod, or for
|
||||
peer-to-peer protocols that don't tolerate NAT.
|
||||
|
||||
**Fast pod identification in `kubectl`.** With
|
||||
`flock.fritzlab.net/ip-algo: namespace,pod` the IPv6 host bits encode
|
||||
the pod's namespace+name, so you can recognise a pod from its IP without
|
||||
a lookup. Reverse-DNS via a wildcard zone makes those IPs human-readable
|
||||
too.
|
||||
|
||||
**Static-IP migration.** Annotation-driven address allocation means you
|
||||
can ask for a specific sub-CIDR (`cidr6: 2001:db8:f001::ab00/120`) for
|
||||
services that previously needed pinned IPs (mail server, ingress
|
||||
controller). When the static-IP requirement goes away, drop the
|
||||
annotation and the pod gets a normal allocation.
|
||||
|
||||
## Comparison vs Calico / Cilium
|
||||
|
||||
| | flock | Calico | Cilium |
|
||||
|--------------------------|-----------------------------|------------------------------|------------------------------|
|
||||
| Default address family | IPv6 | IPv4 | dual |
|
||||
| BGP | yes (BIRD) | yes | optional |
|
||||
| Overlay (VXLAN/IPIP) | never | optional | yes (geneve) or native |
|
||||
| NAT in datapath | never | masquerade by default | masquerade by default |
|
||||
| Anycast pod addressing | first-class | manual | optional, via service mesh |
|
||||
| eBPF datapath | no | optional | yes |
|
||||
| NetworkPolicy | not yet | yes (Felix) | yes (eBPF) |
|
||||
| Cluster size target | small (< 100 nodes) | thousands | thousands |
|
||||
| Operational surface area | low (1 DaemonSet, 1 CRD) | medium | high |
|
||||
| Production-ready | alpha | yes | yes |
|
||||
|
||||
flock is not trying to compete with Calico or Cilium. The right answer
|
||||
for most clusters is one of those two — flock exists for clusters where
|
||||
every node already speaks BGP, the operator wants to think in IPv6-first
|
||||
terms, and per-pod anycast is something they actually want to use rather
|
||||
than work around.
|
||||
|
||||
## Limitations and non-goals
|
||||
|
||||
- No NetworkPolicy enforcement yet (planned).
|
||||
- No NAT, no masquerade, no SNAT-egress. If your pods need to reach a
|
||||
legacy IPv4-only destination, give them an IPv4 address explicitly.
|
||||
- No multi-cluster, no peering across clusters.
|
||||
- Linux-only datapath.
|
||||
- IPAM is per-node — there's no global allocator and no IP mobility.
|
||||
When a pod moves to a different node it gets a new address.
|
||||
- The agent is privileged. It mounts `/var/run/netns`, configures veth
|
||||
pairs, manages kernel routes, and holds `CAP_NET_ADMIN`. This is
|
||||
inherent to being a CNI; reducing privilege further is not a goal.
|
||||
- If BIRD dies but the agent stays up, pods on that node stop being
|
||||
reachable from off-node. The DaemonSet liveness probes catch this.
|
||||
|
||||
## Building and testing
|
||||
|
||||
```sh
|
||||
# Unit tests + fuzz seed corpora (fast, ~1s):
|
||||
go test ./...
|
||||
|
||||
# Targeted fuzz pass:
|
||||
go test -run NEVERMATCH -fuzz=FuzzParseAnnotations -fuzztime=30s ./pkg/agent
|
||||
go test -run NEVERMATCH -fuzz=FuzzRender -fuzztime=30s ./pkg/routing/bird
|
||||
go test -run NEVERMATCH -fuzz=FuzzEmbed -fuzztime=30s ./pkg/embed
|
||||
go test -run NEVERMATCH -fuzz=FuzzIPAM_Allocate -fuzztime=30s ./pkg/agent
|
||||
|
||||
# Build the container image (used by the DaemonSet):
|
||||
docker build -t flock:dev .
|
||||
```
|
||||
|
||||
The fuzz tests are also run as plain unit tests via their seed corpora,
|
||||
so every `go test ./...` exercises the discovered edge cases as
|
||||
regressions.
|
||||
|
||||
`pkg/agent` has Linux-only files (`*_linux.go`) for netlink and netns
|
||||
work; the macOS/Windows build pulls in stubs from `*_stub.go` so tests
|
||||
run cleanly on developer laptops.
|
||||
|
||||
## License
|
||||
|
||||
Apache 2.0.
|
||||
Apache 2.0 — see [LICENSE](LICENSE).
|
||||
|
||||
@@ -20,6 +20,9 @@ spec:
|
||||
openAPIV3Schema:
|
||||
type: object
|
||||
required: [spec]
|
||||
description: |
|
||||
NodeConfig is the per-node operator-supplied configuration for the
|
||||
flock CNI agent. Its name MUST equal the Kubernetes node name.
|
||||
properties:
|
||||
spec:
|
||||
type: object
|
||||
@@ -35,6 +38,25 @@ spec:
|
||||
items:
|
||||
type: string
|
||||
description: IPv4 CIDR owned and aggregate-advertised by this node.
|
||||
defaults:
|
||||
type: object
|
||||
description: |
|
||||
Per-node baseline for which address families a pod receives
|
||||
when its own annotations don't specify. Pod annotations
|
||||
flock.fritzlab.net/ipv6 and flock.fritzlab.net/ipv4 always
|
||||
override these defaults. Built-in fallback (when this block
|
||||
or any field is omitted) is IPv6=true, IPv4=false.
|
||||
properties:
|
||||
ipv6:
|
||||
type: boolean
|
||||
description: |
|
||||
Default IPv6 inclusion for pods on this node. Omit to
|
||||
inherit the built-in baseline (true).
|
||||
ipv4:
|
||||
type: boolean
|
||||
description: |
|
||||
Default IPv4 inclusion for pods on this node. Omit to
|
||||
inherit the built-in baseline (false).
|
||||
bgp:
|
||||
type: object
|
||||
required: [asn, peers]
|
||||
@@ -70,3 +92,9 @@ spec:
|
||||
- name: CIDR4
|
||||
type: string
|
||||
jsonPath: .spec.cidr4
|
||||
- name: DefV6
|
||||
type: boolean
|
||||
jsonPath: .spec.defaults.ipv6
|
||||
- name: DefV4
|
||||
type: boolean
|
||||
jsonPath: .spec.defaults.ipv4
|
||||
|
||||
@@ -20,6 +20,9 @@ spec:
|
||||
openAPIV3Schema:
|
||||
type: object
|
||||
required: [spec]
|
||||
description: |
|
||||
NodeConfig is the per-node operator-supplied configuration for the
|
||||
flock CNI agent. Its name MUST equal the Kubernetes node name.
|
||||
properties:
|
||||
spec:
|
||||
type: object
|
||||
@@ -35,6 +38,25 @@ spec:
|
||||
items:
|
||||
type: string
|
||||
description: IPv4 CIDR owned and aggregate-advertised by this node.
|
||||
defaults:
|
||||
type: object
|
||||
description: |
|
||||
Per-node baseline for which address families a pod receives
|
||||
when its own annotations don't specify. Pod annotations
|
||||
flock.fritzlab.net/ipv6 and flock.fritzlab.net/ipv4 always
|
||||
override these defaults. Built-in fallback (when this block
|
||||
or any field is omitted) is IPv6=true, IPv4=false.
|
||||
properties:
|
||||
ipv6:
|
||||
type: boolean
|
||||
description: |
|
||||
Default IPv6 inclusion for pods on this node. Omit to
|
||||
inherit the built-in baseline (true).
|
||||
ipv4:
|
||||
type: boolean
|
||||
description: |
|
||||
Default IPv4 inclusion for pods on this node. Omit to
|
||||
inherit the built-in baseline (false).
|
||||
bgp:
|
||||
type: object
|
||||
required: [asn, peers]
|
||||
@@ -70,6 +92,12 @@ spec:
|
||||
- name: CIDR4
|
||||
type: string
|
||||
jsonPath: .spec.cidr4
|
||||
- name: DefV6
|
||||
type: boolean
|
||||
jsonPath: .spec.defaults.ipv6
|
||||
- name: DefV4
|
||||
type: boolean
|
||||
jsonPath: .spec.defaults.ipv4
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
||||
+175
-46
@@ -5,77 +5,153 @@ import (
|
||||
"net"
|
||||
"strings"
|
||||
|
||||
flockv1alpha1 "code.fritzlab.net/fritzlab/flock/pkg/api/v1alpha1"
|
||||
"code.fritzlab.net/fritzlab/flock/pkg/embed"
|
||||
)
|
||||
|
||||
// annotationPrefix is the namespace under which all flock pod annotations
|
||||
// live. Anything not starting with this prefix is ignored by the parser.
|
||||
const annotationPrefix = "flock.fritzlab.net/"
|
||||
|
||||
// ParsedAnnotations is the typed view of a Pod's flock annotations.
|
||||
type ParsedAnnotations struct {
|
||||
WantV6 bool
|
||||
WantV4 bool
|
||||
CIDR6 []*net.IPNet
|
||||
CIDR4 []*net.IPNet
|
||||
IPAlgo []embed.Field
|
||||
Anycast []net.IP
|
||||
// Recognised annotation keys (without the prefix).
|
||||
const (
|
||||
annIPv6 = "ipv6"
|
||||
annIPv4 = "ipv4"
|
||||
annCIDR6 = "cidr6"
|
||||
annCIDR4 = "cidr4"
|
||||
annIPAlgo = "ip-algo"
|
||||
annAnycast = "anycast"
|
||||
)
|
||||
|
||||
// FamilyDefaults is the per-call baseline for whether a pod receives an IPv6
|
||||
// and/or IPv4 address. It is the merge of:
|
||||
//
|
||||
// 1. flock's built-in baseline (IPv6=true, IPv4=false), then
|
||||
// 2. any NodeConfig.Spec.Defaults override the operator has applied to
|
||||
// the local node.
|
||||
//
|
||||
// Pod-level `flock.fritzlab.net/ipv{6,4}` annotations override this baseline.
|
||||
//
|
||||
// Use FamilyDefaultsFromNodeConfig to compute a value from a NodeConfig,
|
||||
// or BuiltinFamilyDefaults() if no NodeConfig is in scope.
|
||||
type FamilyDefaults struct {
|
||||
// WantV6 is the default-on value for IPv6 inclusion when the pod has no
|
||||
// explicit ipv6 annotation.
|
||||
WantV6 bool
|
||||
// WantV4 is the default-on value for IPv4 inclusion when the pod has no
|
||||
// explicit ipv4 annotation.
|
||||
WantV4 bool
|
||||
}
|
||||
|
||||
// ParseAnnotations applies the design-doc defaults (ipv6=true, ipv4=false)
|
||||
// and validates the post-merge combination.
|
||||
func ParseAnnotations(in map[string]string) (*ParsedAnnotations, error) {
|
||||
out := &ParsedAnnotations{WantV6: true, WantV4: false}
|
||||
// BuiltinFamilyDefaults returns flock's hard-coded fallback: IPv6 only.
|
||||
// This is the policy applied when no NodeConfig override is in effect.
|
||||
//
|
||||
// We define it as a function rather than a var so callers can't mutate the
|
||||
// shared baseline at runtime.
|
||||
func BuiltinFamilyDefaults() FamilyDefaults {
|
||||
return FamilyDefaults{WantV6: true, WantV4: false}
|
||||
}
|
||||
|
||||
if v, ok := in[annotationPrefix+"ipv6"]; ok {
|
||||
switch strings.ToLower(strings.TrimSpace(v)) {
|
||||
case "true":
|
||||
out.WantV6 = true
|
||||
case "false":
|
||||
out.WantV6 = false
|
||||
default:
|
||||
return nil, fmt.Errorf("annotation ipv6=%q: must be true or false", v)
|
||||
}
|
||||
// FamilyDefaultsFromNodeConfig resolves the effective per-node defaults,
|
||||
// falling back to BuiltinFamilyDefaults for any field the NodeConfig leaves
|
||||
// unset. A nil NodeConfig (or nil Spec.Defaults) returns the built-in
|
||||
// baseline unchanged.
|
||||
func FamilyDefaultsFromNodeConfig(nc *flockv1alpha1.NodeConfig) FamilyDefaults {
|
||||
out := BuiltinFamilyDefaults()
|
||||
if nc == nil || nc.Spec.Defaults == nil {
|
||||
return out
|
||||
}
|
||||
if v, ok := in[annotationPrefix+"ipv4"]; ok {
|
||||
switch strings.ToLower(strings.TrimSpace(v)) {
|
||||
case "true":
|
||||
out.WantV4 = true
|
||||
case "false":
|
||||
out.WantV4 = false
|
||||
default:
|
||||
return nil, fmt.Errorf("annotation ipv4=%q: must be true or false", v)
|
||||
if nc.Spec.Defaults.IPv6 != nil {
|
||||
out.WantV6 = *nc.Spec.Defaults.IPv6
|
||||
}
|
||||
if nc.Spec.Defaults.IPv4 != nil {
|
||||
out.WantV4 = *nc.Spec.Defaults.IPv4
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ParsedAnnotations is the typed view of a pod's flock annotations after the
|
||||
// node-level defaults have been merged in. All slices are non-nil only when
|
||||
// the corresponding annotation was present and parsed cleanly.
|
||||
type ParsedAnnotations struct {
|
||||
// WantV6 is true when the pod should receive an IPv6 address.
|
||||
WantV6 bool
|
||||
// WantV4 is true when the pod should receive an IPv4 address.
|
||||
WantV4 bool
|
||||
// CIDR6 narrows IPv6 allocation to specific operator-approved sub-ranges
|
||||
// of the node's CIDR6 set. nil/empty means "use any node CIDR6".
|
||||
CIDR6 []*net.IPNet
|
||||
// CIDR4 narrows IPv4 allocation. nil/empty means "use any node CIDR4".
|
||||
CIDR4 []*net.IPNet
|
||||
// IPAlgo is the ordered list of identity fields used to build the IID.
|
||||
// nil/empty means "random IID".
|
||||
IPAlgo []embed.Field
|
||||
// Anycast is the set of anycast IPs to bind on the pod's loopback.
|
||||
// nil/empty means "no anycast".
|
||||
Anycast []net.IP
|
||||
}
|
||||
|
||||
// ParseAnnotations applies the supplied per-node defaults and validates the
|
||||
// post-merge combination. It is pure — it does not consult NodeConfig or any
|
||||
// global state — so it is safe to call from tests and fuzz targets.
|
||||
//
|
||||
// Annotation precedence: pod annotation > FamilyDefaults > built-in baseline.
|
||||
// Callers compute FamilyDefaults via FamilyDefaultsFromNodeConfig and pass it
|
||||
// in.
|
||||
//
|
||||
// Errors:
|
||||
// - any unknown ipv6/ipv4 value (must be "true" or "false", case-insensitive)
|
||||
// - any malformed cidr6/cidr4/anycast/ip-algo value
|
||||
// - the post-merge combination resolves to neither IPv6 nor IPv4 (a pod
|
||||
// must have at least one address)
|
||||
func ParseAnnotations(in map[string]string, defaults FamilyDefaults) (*ParsedAnnotations, error) {
|
||||
out := &ParsedAnnotations{WantV6: defaults.WantV6, WantV4: defaults.WantV4}
|
||||
|
||||
if v, ok := in[annotationPrefix+annIPv6]; ok {
|
||||
b, err := parseBoolAnnotation(annIPv6, v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out.WantV6 = b
|
||||
}
|
||||
if v, ok := in[annotationPrefix+annIPv4]; ok {
|
||||
b, err := parseBoolAnnotation(annIPv4, v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out.WantV4 = b
|
||||
}
|
||||
if !out.WantV6 && !out.WantV4 {
|
||||
return nil, fmt.Errorf("ipv6=false requires ipv4=true (pod must have at least one address)")
|
||||
return nil, fmt.Errorf("annotations + defaults resolve to no address family (need at least one of ipv6/ipv4)")
|
||||
}
|
||||
|
||||
if v, ok := in[annotationPrefix+"cidr6"]; ok {
|
||||
nets, err := parseCIDRList(v)
|
||||
if v, ok := in[annotationPrefix+annCIDR6]; ok {
|
||||
nets, err := parseCIDRList(v, familyV6)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("annotation cidr6: %w", err)
|
||||
return nil, fmt.Errorf("annotation %s: %w", annCIDR6, err)
|
||||
}
|
||||
out.CIDR6 = nets
|
||||
}
|
||||
if v, ok := in[annotationPrefix+"cidr4"]; ok {
|
||||
nets, err := parseCIDRList(v)
|
||||
if v, ok := in[annotationPrefix+annCIDR4]; ok {
|
||||
nets, err := parseCIDRList(v, familyV4)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("annotation cidr4: %w", err)
|
||||
return nil, fmt.Errorf("annotation %s: %w", annCIDR4, err)
|
||||
}
|
||||
out.CIDR4 = nets
|
||||
}
|
||||
|
||||
if v, ok := in[annotationPrefix+"ip-algo"]; ok {
|
||||
if v, ok := in[annotationPrefix+annIPAlgo]; ok {
|
||||
fields, err := parseIPAlgo(v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("annotation ip-algo: %w", err)
|
||||
return nil, fmt.Errorf("annotation %s: %w", annIPAlgo, err)
|
||||
}
|
||||
out.IPAlgo = fields
|
||||
}
|
||||
|
||||
if v, ok := in[annotationPrefix+"anycast"]; ok {
|
||||
if v, ok := in[annotationPrefix+annAnycast]; ok {
|
||||
ips, err := parseIPList(v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("annotation anycast: %w", err)
|
||||
return nil, fmt.Errorf("annotation %s: %w", annAnycast, err)
|
||||
}
|
||||
out.Anycast = ips
|
||||
}
|
||||
@@ -83,7 +159,39 @@ func ParseAnnotations(in map[string]string) (*ParsedAnnotations, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func parseCIDRList(s string) ([]*net.IPNet, error) {
|
||||
// parseBoolAnnotation accepts only "true" or "false" (case-insensitive,
|
||||
// surrounding whitespace tolerated). All other values — including "1", "0",
|
||||
// "yes", "no" — are rejected so operator typos are caught loudly rather
|
||||
// than silently producing the "false" default.
|
||||
func parseBoolAnnotation(key, v string) (bool, error) {
|
||||
switch strings.ToLower(strings.TrimSpace(v)) {
|
||||
case "true":
|
||||
return true, nil
|
||||
case "false":
|
||||
return false, nil
|
||||
default:
|
||||
return false, fmt.Errorf("annotation %s=%q: must be \"true\" or \"false\"", key, v)
|
||||
}
|
||||
}
|
||||
|
||||
// addressFamily distinguishes IPv6 vs IPv4 in places where the parser must
|
||||
// validate the family of supplied CIDRs.
|
||||
type addressFamily int
|
||||
|
||||
const (
|
||||
familyAny addressFamily = iota
|
||||
familyV6
|
||||
familyV4
|
||||
)
|
||||
|
||||
// parseCIDRList parses a comma-separated CIDR list. Whitespace around items
|
||||
// is trimmed; empty items are silently dropped. The list must contain at
|
||||
// least one entry post-trim.
|
||||
//
|
||||
// If `want` is familyV6 or familyV4 each entry's family is checked and a
|
||||
// mismatch is reported, so an `flock.fritzlab.net/cidr6` annotation cannot
|
||||
// silently slip a v4 prefix into the v6 allocator.
|
||||
func parseCIDRList(s string, want addressFamily) ([]*net.IPNet, error) {
|
||||
var out []*net.IPNet
|
||||
for _, part := range strings.Split(s, ",") {
|
||||
part = strings.TrimSpace(part)
|
||||
@@ -94,6 +202,17 @@ func parseCIDRList(s string) ([]*net.IPNet, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid CIDR %q: %w", part, err)
|
||||
}
|
||||
isV4 := n.IP.To4() != nil
|
||||
switch want {
|
||||
case familyV6:
|
||||
if isV4 {
|
||||
return nil, fmt.Errorf("CIDR %q is IPv4, expected IPv6", part)
|
||||
}
|
||||
case familyV4:
|
||||
if !isV4 {
|
||||
return nil, fmt.Errorf("CIDR %q is IPv6, expected IPv4", part)
|
||||
}
|
||||
}
|
||||
out = append(out, n)
|
||||
}
|
||||
if len(out) == 0 {
|
||||
@@ -102,6 +221,9 @@ func parseCIDRList(s string) ([]*net.IPNet, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// parseIPList parses a comma-separated literal-IP list. Same trim/empty
|
||||
// semantics as parseCIDRList. Mixed v4 and v6 entries are allowed (anycast
|
||||
// pods can advertise both families together).
|
||||
func parseIPList(s string) ([]net.IP, error) {
|
||||
var out []net.IP
|
||||
for _, part := range strings.Split(s, ",") {
|
||||
@@ -121,6 +243,9 @@ func parseIPList(s string) ([]net.IP, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// parseIPAlgo parses the ip-algo annotation. Each comma-separated token must
|
||||
// match one of: namespace, pod, image. Empty tokens are dropped; unknown
|
||||
// tokens are reported.
|
||||
func parseIPAlgo(s string) ([]embed.Field, error) {
|
||||
var out []embed.Field
|
||||
for _, part := range strings.Split(s, ",") {
|
||||
@@ -128,11 +253,11 @@ func parseIPAlgo(s string) ([]embed.Field, error) {
|
||||
switch part {
|
||||
case "":
|
||||
continue
|
||||
case "namespace":
|
||||
case string(embed.FieldNamespace):
|
||||
out = append(out, embed.FieldNamespace)
|
||||
case "pod":
|
||||
case string(embed.FieldPod):
|
||||
out = append(out, embed.FieldPod)
|
||||
case "image":
|
||||
case string(embed.FieldImage):
|
||||
out = append(out, embed.FieldImage)
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown ip-algo field %q (allowed: namespace, pod, image)", part)
|
||||
@@ -144,8 +269,8 @@ func parseIPAlgo(s string) ([]embed.Field, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// CNIArgs parses the K=V;K=V CNI_ARGS string for the kubelet keys we care
|
||||
// about. Other keys are ignored.
|
||||
// CNIArgs is the typed view of the K=V;K=V CNI_ARGS string passed by kubelet.
|
||||
// We only keep the fields the agent uses; unknown keys are ignored.
|
||||
type CNIArgs struct {
|
||||
PodNamespace string
|
||||
PodName string
|
||||
@@ -153,6 +278,10 @@ type CNIArgs struct {
|
||||
InfraID string
|
||||
}
|
||||
|
||||
// ParseCNIArgs is permissive by design — kubelet versions and runtime
|
||||
// shims pass varying sets of keys. Malformed entries are skipped silently
|
||||
// rather than failing the whole ADD; required-key validation is the
|
||||
// caller's responsibility.
|
||||
func ParseCNIArgs(s string) CNIArgs {
|
||||
var a CNIArgs
|
||||
for _, kv := range strings.Split(s, ";") {
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// FuzzParseAnnotations explores the joint space of {ipv6, ipv4, cidr6, cidr4,
|
||||
// ip-algo, anycast} annotations with random byte strings. Every recognised
|
||||
// key is exercised by deriving a deterministic input map from the fuzzed
|
||||
// bytes; this gives the fuzzer reach into all parser branches at once.
|
||||
//
|
||||
// Properties checked:
|
||||
//
|
||||
// 1. The parser never panics on any input.
|
||||
// 2. On nil-error return, the result satisfies the design-doc invariant
|
||||
// that at least one of WantV6 / WantV4 is true (a pod always has at
|
||||
// least one address).
|
||||
// 3. Anycast IPs and IPAlgo fields are non-nil/empty only when the
|
||||
// annotation was supplied; never spontaneously populated.
|
||||
//
|
||||
// Seed corpus covers known edge cases the spec must handle.
|
||||
func FuzzParseAnnotations(f *testing.F) {
|
||||
// Seeds: each entry is six strings — the literal raw values for the
|
||||
// six parsed keys. Empty string for "key absent".
|
||||
type seed struct {
|
||||
ipv6, ipv4, cidr6, cidr4, ipAlgo, anycast string
|
||||
}
|
||||
seeds := []seed{
|
||||
{},
|
||||
{ipv4: "true"},
|
||||
{ipv6: "false", ipv4: "true"},
|
||||
{ipv6: "TRUE"},
|
||||
{ipv6: " true "},
|
||||
{ipv6: "yes"}, // invalid → expect error
|
||||
{ipv4: "1"}, // invalid
|
||||
{cidr6: ""}, // invalid (empty after split)
|
||||
{cidr6: ","}, // invalid (empty after trim)
|
||||
{cidr6: "2602:817:3000:f001::/64"}, // valid single
|
||||
{cidr6: "2602:817:3000:f001::/64,"}, // trailing comma
|
||||
{cidr6: " 2602:817:3000:f001::/64 "}, // surrounding whitespace
|
||||
{cidr6: "2602:817:3000:f001::/64, 2602:817:3000:f002::/64"},
|
||||
{cidr6: "10.0.0.0/8"}, // family mismatch
|
||||
{cidr4: "172.25.210.0/24"}, // valid
|
||||
{cidr4: "172.25.210.0/24,172.25.211.0/24"}, // multiple
|
||||
{cidr4: "2602:817::/32"}, // family mismatch
|
||||
{ipAlgo: "namespace,pod,image"},
|
||||
{ipAlgo: "namespace, pod , image"}, // whitespace
|
||||
{ipAlgo: "namespace,unknown"}, // invalid
|
||||
{ipAlgo: ""}, // invalid (empty)
|
||||
{ipAlgo: ","}, // invalid
|
||||
{anycast: "2602:817:3000:ac::1"},
|
||||
{anycast: "2602:817:3000:ac::1, 172.25.255.1"},
|
||||
{anycast: "::1"}, // loopback (allowed at parse time)
|
||||
{anycast: "fe80::1"}, // link-local (allowed at parse time)
|
||||
{anycast: "::ffff:10.0.0.1"}, // v4-mapped v6
|
||||
{anycast: "0.0.0.0"}, // unspecified
|
||||
{anycast: "definitely-not-an-ip"}, // invalid
|
||||
{anycast: ""}, // invalid
|
||||
// Embedded NUL bytes
|
||||
{ipv4: "true\x00"},
|
||||
{cidr6: "2602:817:3000:f001::/64\x00"},
|
||||
{anycast: "\x00\x00"},
|
||||
// Unicode
|
||||
{ipv4: "trüe"},
|
||||
{ipAlgo: "námespace"},
|
||||
// Very long
|
||||
{cidr6: longString("2602:817:3000:f001::/64,", 4096)},
|
||||
}
|
||||
for _, s := range seeds {
|
||||
f.Add(s.ipv6, s.ipv4, s.cidr6, s.cidr4, s.ipAlgo, s.anycast)
|
||||
}
|
||||
|
||||
f.Fuzz(func(t *testing.T, ipv6, ipv4, cidr6, cidr4, ipAlgo, anycast string) {
|
||||
in := map[string]string{}
|
||||
// Treat empty as "key absent" so the seed table matches the run-time
|
||||
// shape; Kubernetes annotations cannot have a nil value but they CAN
|
||||
// be missing entirely. Empty-string-with-key is also a real case
|
||||
// (operator typo); add a separate seed below to cover it.
|
||||
if ipv6 != "" {
|
||||
in[annotationPrefix+annIPv6] = ipv6
|
||||
}
|
||||
if ipv4 != "" {
|
||||
in[annotationPrefix+annIPv4] = ipv4
|
||||
}
|
||||
if cidr6 != "" {
|
||||
in[annotationPrefix+annCIDR6] = cidr6
|
||||
}
|
||||
if cidr4 != "" {
|
||||
in[annotationPrefix+annCIDR4] = cidr4
|
||||
}
|
||||
if ipAlgo != "" {
|
||||
in[annotationPrefix+annIPAlgo] = ipAlgo
|
||||
}
|
||||
if anycast != "" {
|
||||
in[annotationPrefix+annAnycast] = anycast
|
||||
}
|
||||
|
||||
got, err := ParseAnnotations(in, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
return // any error is acceptable; we only require no panic
|
||||
}
|
||||
// Property: at least one family must be selected.
|
||||
if !got.WantV6 && !got.WantV4 {
|
||||
t.Fatalf("parser accepted but produced no family: in=%#v", in)
|
||||
}
|
||||
// Property: optional fields populated only when their key was set.
|
||||
if _, hasAlgo := in[annotationPrefix+annIPAlgo]; !hasAlgo && len(got.IPAlgo) != 0 {
|
||||
t.Fatalf("IPAlgo populated without annotation")
|
||||
}
|
||||
if _, hasAny := in[annotationPrefix+annAnycast]; !hasAny && len(got.Anycast) != 0 {
|
||||
t.Fatalf("Anycast populated without annotation")
|
||||
}
|
||||
if _, hasC6 := in[annotationPrefix+annCIDR6]; !hasC6 && len(got.CIDR6) != 0 {
|
||||
t.Fatalf("CIDR6 populated without annotation")
|
||||
}
|
||||
if _, hasC4 := in[annotationPrefix+annCIDR4]; !hasC4 && len(got.CIDR4) != 0 {
|
||||
t.Fatalf("CIDR4 populated without annotation")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// FuzzParseCNIArgs requires the parser to never panic on adversarial inputs.
|
||||
// The parser is permissive by spec — it returns a CNIArgs with whatever it
|
||||
// could extract — so the only invariant is "doesn't crash".
|
||||
func FuzzParseCNIArgs(f *testing.F) {
|
||||
f.Add("")
|
||||
f.Add("=")
|
||||
f.Add(";")
|
||||
f.Add(";=;=;")
|
||||
f.Add("K8S_POD_NAMESPACE=ns;K8S_POD_NAME=p")
|
||||
f.Add("K8S_POD_NAMESPACE=ns;K8S_POD_NAME=p;K8S_POD_UID=abc;K8S_POD_INFRA_CONTAINER_ID=def")
|
||||
f.Add("=value-only")
|
||||
f.Add("key-only=")
|
||||
f.Add("\x00\x00\x00")
|
||||
f.Add("K8S_POD_NAMESPACE=\xff\xfe\xfd")
|
||||
f.Add("K8S_POD_NAME=value;K8S_POD_NAME=other") // duplicate keys: last wins
|
||||
// Long input
|
||||
f.Add(longString("K8S_POD_NAME=x;", 4096))
|
||||
|
||||
f.Fuzz(func(t *testing.T, in string) {
|
||||
_ = ParseCNIArgs(in)
|
||||
})
|
||||
}
|
||||
|
||||
// longString returns s repeated to total >= n bytes, useful for piling up
|
||||
// realistic-looking but oversized inputs.
|
||||
func longString(s string, n int) string {
|
||||
if len(s) == 0 {
|
||||
return ""
|
||||
}
|
||||
var b []byte
|
||||
for len(b) < n {
|
||||
b = append(b, s...)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
@@ -3,11 +3,68 @@ package agent
|
||||
import (
|
||||
"testing"
|
||||
|
||||
flockv1alpha1 "code.fritzlab.net/fritzlab/flock/pkg/api/v1alpha1"
|
||||
"code.fritzlab.net/fritzlab/flock/pkg/embed"
|
||||
)
|
||||
|
||||
func TestParseAnnotations_Defaults(t *testing.T) {
|
||||
a, err := ParseAnnotations(nil)
|
||||
// boolPtr returns a pointer to b — convenient for the *bool pointer fields
|
||||
// in FamilyDefaults where nil means "unset".
|
||||
func boolPtr(b bool) *bool { return &b }
|
||||
|
||||
func TestBuiltinFamilyDefaults(t *testing.T) {
|
||||
d := BuiltinFamilyDefaults()
|
||||
if !d.WantV6 || d.WantV4 {
|
||||
t.Fatalf("built-in defaults wrong: v6=%v v4=%v (want true/false)", d.WantV6, d.WantV4)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFamilyDefaultsFromNodeConfig_NilNodeConfig(t *testing.T) {
|
||||
d := FamilyDefaultsFromNodeConfig(nil)
|
||||
if d != BuiltinFamilyDefaults() {
|
||||
t.Fatalf("nil NodeConfig should yield built-in defaults; got %+v", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFamilyDefaultsFromNodeConfig_NilDefaults(t *testing.T) {
|
||||
nc := &flockv1alpha1.NodeConfig{}
|
||||
d := FamilyDefaultsFromNodeConfig(nc)
|
||||
if d != BuiltinFamilyDefaults() {
|
||||
t.Fatalf("missing Defaults should yield built-in; got %+v", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFamilyDefaultsFromNodeConfig_PartialOverride(t *testing.T) {
|
||||
nc := &flockv1alpha1.NodeConfig{
|
||||
Spec: flockv1alpha1.NodeConfigSpec{
|
||||
Defaults: &flockv1alpha1.FamilyDefaults{
|
||||
IPv4: boolPtr(true),
|
||||
},
|
||||
},
|
||||
}
|
||||
d := FamilyDefaultsFromNodeConfig(nc)
|
||||
// IPv6 was unset → keeps built-in true; IPv4 was set → flipped on.
|
||||
if !d.WantV6 || !d.WantV4 {
|
||||
t.Fatalf("partial override wrong: %+v (want v6=true, v4=true)", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFamilyDefaultsFromNodeConfig_FullOverride(t *testing.T) {
|
||||
nc := &flockv1alpha1.NodeConfig{
|
||||
Spec: flockv1alpha1.NodeConfigSpec{
|
||||
Defaults: &flockv1alpha1.FamilyDefaults{
|
||||
IPv6: boolPtr(false),
|
||||
IPv4: boolPtr(true),
|
||||
},
|
||||
},
|
||||
}
|
||||
d := FamilyDefaultsFromNodeConfig(nc)
|
||||
if d.WantV6 || !d.WantV4 {
|
||||
t.Fatalf("full override wrong: %+v (want v6=false, v4=true)", d)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_BuiltinDefaults(t *testing.T) {
|
||||
a, err := ParseAnnotations(nil, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -16,10 +73,36 @@ func TestParseAnnotations_Defaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_DualStack(t *testing.T) {
|
||||
func TestParseAnnotations_NodeDefaultsApplied(t *testing.T) {
|
||||
// Node config says "IPv4 is on by default for this node".
|
||||
d := FamilyDefaults{WantV6: true, WantV4: true}
|
||||
a, err := ParseAnnotations(nil, d)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !a.WantV6 || !a.WantV4 {
|
||||
t.Fatalf("node defaults not applied: %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_AnnotationOverridesNodeDefault(t *testing.T) {
|
||||
// Node says dual-stack by default; pod opts out of v4 explicitly.
|
||||
d := FamilyDefaults{WantV6: true, WantV4: true}
|
||||
a, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ipv4": "false",
|
||||
}, d)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !a.WantV6 || a.WantV4 {
|
||||
t.Fatalf("annotation override failed: %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_DualStackViaAnnotation(t *testing.T) {
|
||||
a, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ipv4": "true",
|
||||
})
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -31,15 +114,49 @@ func TestParseAnnotations_DualStack(t *testing.T) {
|
||||
func TestParseAnnotations_NoFamily(t *testing.T) {
|
||||
if _, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ipv6": "false",
|
||||
}); err == nil {
|
||||
}, BuiltinFamilyDefaults()); err == nil {
|
||||
t.Fatalf("expected error: ipv6=false ipv4=false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_NoFamily_NodeDefaultsAlsoOff(t *testing.T) {
|
||||
// Pathological NodeConfig that disables both families. Even with no pod
|
||||
// annotation we must reject — otherwise a pod gets an empty allocation.
|
||||
d := FamilyDefaults{WantV6: false, WantV4: false}
|
||||
if _, err := ParseAnnotations(nil, d); err == nil {
|
||||
t.Fatalf("expected error when both defaults are false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_BoolStrictness(t *testing.T) {
|
||||
// Common misuses that should be rejected so typos don't silently flip
|
||||
// behaviour to the implicit-false default.
|
||||
bad := []string{"1", "0", "yes", "no", "TrueFalse", " "}
|
||||
for _, v := range bad {
|
||||
_, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ipv4": v,
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err == nil {
|
||||
t.Errorf("expected error for ipv4=%q", v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_BoolCaseInsensitive(t *testing.T) {
|
||||
for _, v := range []string{"TRUE", "True", " true ", "FALSE", "False"} {
|
||||
_, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ipv4": v,
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Errorf("expected ipv4=%q to parse cleanly: %v", v, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_IPAlgo(t *testing.T) {
|
||||
a, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ip-algo": "namespace,pod,image",
|
||||
})
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -54,10 +171,18 @@ func TestParseAnnotations_IPAlgo(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_IPAlgo_Unknown(t *testing.T) {
|
||||
if _, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "ip-algo": "namespace,foo",
|
||||
}, BuiltinFamilyDefaults()); err == nil {
|
||||
t.Fatalf("expected unknown-field error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_CIDR(t *testing.T) {
|
||||
a, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "cidr6": "2602:817:3000:f001::/64, 2602:817:3000:f002::/64",
|
||||
})
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -66,9 +191,49 @@ func TestParseAnnotations_CIDR(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_CIDR_FamilyMismatch(t *testing.T) {
|
||||
// v4 prefix in a cidr6 annotation must not silently slip through.
|
||||
if _, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "cidr6": "10.0.0.0/8",
|
||||
}, BuiltinFamilyDefaults()); err == nil {
|
||||
t.Fatalf("expected family mismatch error")
|
||||
}
|
||||
if _, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "cidr4": "2602:817::/32",
|
||||
}, BuiltinFamilyDefaults()); err == nil {
|
||||
t.Fatalf("expected family mismatch error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAnnotations_Anycast_Mixed(t *testing.T) {
|
||||
// Anycast accepts both families together — typical for a service that
|
||||
// advertises one v6 and one v4 anycast IP.
|
||||
a, err := ParseAnnotations(map[string]string{
|
||||
annotationPrefix + "anycast": "2602:817:3000:ac::1, 172.25.255.1",
|
||||
}, BuiltinFamilyDefaults())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(a.Anycast) != 2 {
|
||||
t.Fatalf("anycast len=%d", len(a.Anycast))
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCNIArgs(t *testing.T) {
|
||||
args := ParseCNIArgs("IgnoreUnknown=1;K8S_POD_NAMESPACE=mail;K8S_POD_NAME=stalwart-0;K8S_POD_INFRA_CONTAINER_ID=abc123")
|
||||
if args.PodNamespace != "mail" || args.PodName != "stalwart-0" || args.InfraID != "abc123" {
|
||||
t.Fatalf("ParseCNIArgs got %+v", args)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCNIArgs_EmptyAndMalformed(t *testing.T) {
|
||||
// Permissive: malformed entries are skipped, never crash.
|
||||
a := ParseCNIArgs("")
|
||||
if a.PodName != "" {
|
||||
t.Fatalf("empty input should yield empty CNIArgs, got %+v", a)
|
||||
}
|
||||
a = ParseCNIArgs(";;K8S_POD_NAMESPACE=ns;noequalshere;=novalue;K8S_POD_NAME=p")
|
||||
if a.PodNamespace != "ns" || a.PodName != "p" {
|
||||
t.Fatalf("permissive parse failed: %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
// Package agent owns the in-process flock-agent runtime. The agent is a
|
||||
// single Linux DaemonSet pod per node and holds:
|
||||
//
|
||||
// - the durable per-node allocation file at /var/lib/flock/allocations.json
|
||||
// (see Store in state.go),
|
||||
// - an in-memory IPAM seeded from NodeConfig CIDRs and reconciled against
|
||||
// the allocation file at startup (see ipam.go),
|
||||
// - dynamic informers watching the per-node NodeConfig CR (nodeconfig.go)
|
||||
// and the local-node Pod set (podinfo.go),
|
||||
// - an RPC server speaking to the lightweight CNI plugin binary
|
||||
// (cmd/flock and pkg/cni), so kubelet's CNI invocations are answered by
|
||||
// a long-lived process rather than spinning up a fresh binary per ADD,
|
||||
// - the BirdManager that renders bird.conf and triggers `birdc reload`
|
||||
// on changes (bird.go), and
|
||||
// - the AnycastReconciler that programs per-pod /128 and /32 host routes
|
||||
// gated on Pod readiness (anycast_linux.go).
|
||||
//
|
||||
// The package is split between platform-specific files (anycast_linux.go,
|
||||
// netns_linux.go, runtime_linux.go) and stub files used on non-Linux build
|
||||
// hosts so the rest of the package — IPAM, parsing, store, RPC plumbing —
|
||||
// stays unit-testable on macOS and Windows CI.
|
||||
package agent
|
||||
@@ -49,7 +49,8 @@ func (h *PodHandler) Add(ctx context.Context, req flockcni.Request) (*current.Re
|
||||
return nil, fmt.Errorf("lookup pod: %w", err)
|
||||
}
|
||||
|
||||
parsed, err := ParseAnnotations(pod.Annotations)
|
||||
defaults := FamilyDefaultsFromNodeConfig(h.NodeConfig.Load())
|
||||
parsed, err := ParseAnnotations(pod.Annotations, defaults)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse annotations: %w", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestHostIfaceName_Format(t *testing.T) {
|
||||
got := HostIfaceName("0123456789abcdef0123456789abcdef")
|
||||
if !strings.HasPrefix(got, "flock") || len(got) != len("flock")+8 {
|
||||
t.Fatalf("HostIfaceName=%q (want flock + 8 hex)", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHostIfaceName_Determinism(t *testing.T) {
|
||||
a := HostIfaceName("container-xyz")
|
||||
b := HostIfaceName("container-xyz")
|
||||
if a != b {
|
||||
t.Fatalf("not deterministic: %s vs %s", a, b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHostIfaceName_DifferentInputs(t *testing.T) {
|
||||
a := HostIfaceName("a")
|
||||
b := HostIfaceName("b")
|
||||
if a == b {
|
||||
t.Fatalf("collision on trivial inputs")
|
||||
}
|
||||
}
|
||||
|
||||
// FuzzHostIfaceName ensures the host interface name generator never produces
|
||||
// an output longer than IFNAMSIZ-1 (15 chars on Linux) and never panics.
|
||||
// The name format is "flock" + 8 hex chars = 13 chars, always.
|
||||
func FuzzHostIfaceName(f *testing.F) {
|
||||
f.Add("")
|
||||
f.Add("a")
|
||||
f.Add("/var/run/netns/abc")
|
||||
f.Add("0123456789abcdef0123456789abcdef")
|
||||
f.Add(longString("x", 64*1024)) // very long containerID
|
||||
f.Add("\x00\x00\x00")
|
||||
f.Add("ünïcødé/контейнер")
|
||||
|
||||
f.Fuzz(func(t *testing.T, id string) {
|
||||
got := HostIfaceName(id)
|
||||
// Linux IFNAMSIZ is 16 (15 chars + NUL); ours must fit comfortably.
|
||||
if len(got) > 15 {
|
||||
t.Fatalf("HostIfaceName(%q)=%q exceeds 15 chars", id, got)
|
||||
}
|
||||
if !strings.HasPrefix(got, "flock") {
|
||||
t.Fatalf("HostIfaceName(%q)=%q missing prefix", id, got)
|
||||
}
|
||||
// Suffix must be lowercase hex (8 chars).
|
||||
suffix := got[len("flock"):]
|
||||
if len(suffix) != 8 {
|
||||
t.Fatalf("HostIfaceName(%q) suffix len=%d", id, len(suffix))
|
||||
}
|
||||
for _, c := range suffix {
|
||||
if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) {
|
||||
t.Fatalf("HostIfaceName(%q)=%q has non-hex suffix", id, got)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
+43
-20
@@ -62,13 +62,15 @@ func (cryptoRand) PickIndex(n int) int {
|
||||
}
|
||||
|
||||
// AllocRequest describes a pending allocation. Values come from Pod metadata
|
||||
// + annotations at CNI ADD time.
|
||||
// + annotations at CNI ADD time, with per-node FamilyDefaults already merged
|
||||
// in (see ParseAnnotations).
|
||||
type AllocRequest struct {
|
||||
ContainerID string
|
||||
Namespace string
|
||||
Pod string
|
||||
// WantV6 / WantV4 come from the ipv6 / ipv4 annotations (defaults in
|
||||
// design doc: ipv6=true, ipv4=false).
|
||||
// WantV6 / WantV4 are the post-merge address family selection (pod
|
||||
// annotation > NodeConfig.Spec.Defaults > built-in baseline). At least
|
||||
// one MUST be true; Allocate rejects the request otherwise.
|
||||
WantV6 bool
|
||||
WantV4 bool
|
||||
// AnnCIDR6 / AnnCIDR4 come from the cidr6 / cidr4 annotations. Empty
|
||||
@@ -224,34 +226,36 @@ func (i *IPAM) allocV6(cidr *net.IPNet, req AllocRequest) (net.IP, error) {
|
||||
|
||||
// randomV6 picks a random /128 inside cidr. The network prefix bits are
|
||||
// preserved from cidr.IP; the host bits are filled from the random source.
|
||||
//
|
||||
// Implementation: walk the 16 IPv6 bytes once. For each byte we ask whether
|
||||
// it's entirely inside the network mask (skip), entirely inside the host
|
||||
// portion (overwrite with random), or split (combine bits from both).
|
||||
func (i *IPAM) randomV6(cidr *net.IPNet) (net.IP, error) {
|
||||
ones, bits := cidr.Mask.Size()
|
||||
if bits != 128 {
|
||||
return nil, fmt.Errorf("cidr %s is not IPv6", cidr)
|
||||
}
|
||||
out := make(net.IP, 16)
|
||||
out := make(net.IP, net.IPv6len)
|
||||
copy(out, cidr.IP.To16())
|
||||
hostBits := 128 - ones
|
||||
rnd := make([]byte, 16)
|
||||
rnd := make([]byte, net.IPv6len)
|
||||
i.randSrc.FillIID(rnd)
|
||||
// Merge rnd into out where mask bit is 0.
|
||||
for b := 0; b < 16; b++ {
|
||||
// Host bits start at bit index `ones`, byte `b`.
|
||||
for b := 0; b < net.IPv6len; b++ {
|
||||
byteStart := b * 8
|
||||
byteEnd := byteStart + 8
|
||||
if byteEnd <= ones {
|
||||
continue // entirely network
|
||||
}
|
||||
if byteStart >= ones {
|
||||
out[b] = rnd[b] // entirely host
|
||||
switch {
|
||||
case byteEnd <= ones:
|
||||
// Entirely inside the network prefix — leave untouched.
|
||||
continue
|
||||
case byteStart >= ones:
|
||||
// Entirely inside the host portion — fully randomise.
|
||||
out[b] = rnd[b]
|
||||
default:
|
||||
// Split byte: top (ones-byteStart) bits are network, rest host.
|
||||
networkBits := ones - byteStart
|
||||
hostMask := byte(0xFF) >> uint(networkBits)
|
||||
out[b] = (out[b] & ^hostMask) | (rnd[b] & hostMask)
|
||||
}
|
||||
// Split byte: top (ones-byteStart) bits are network, rest is host.
|
||||
networkBits := ones - byteStart
|
||||
hostMask := byte(0xFF) >> uint(networkBits)
|
||||
out[b] = (out[b] & ^hostMask) | (rnd[b] & hostMask)
|
||||
}
|
||||
_ = hostBits
|
||||
return out, nil
|
||||
}
|
||||
|
||||
@@ -360,15 +364,34 @@ func toStringSlice(ns []*net.IPNet) []string {
|
||||
return out
|
||||
}
|
||||
|
||||
// canonical returns the textual form of ip in its native family, so the same
|
||||
// host address is always represented identically regardless of whether it
|
||||
// arrived as a 4-byte slice, a 16-byte v4-in-v6 slice, or a string-parsed
|
||||
// net.IP. Used as the key for the in-use map.
|
||||
//
|
||||
// Returns "" for nil input — callers MUST treat the returned key as opaque
|
||||
// and never use the empty string as a sentinel.
|
||||
func canonical(ip net.IP) string {
|
||||
if ip == nil {
|
||||
return ""
|
||||
}
|
||||
if v4 := ip.To4(); v4 != nil {
|
||||
return v4.String()
|
||||
}
|
||||
return ip.To16().String()
|
||||
if v16 := ip.To16(); v16 != nil {
|
||||
return v16.String()
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ipToU32 reads a 4-byte IPv4 net.IP into a uint32. The caller is expected
|
||||
// to have already validated that ip is an IPv4 address; mis-use returns 0
|
||||
// rather than panicking.
|
||||
func ipToU32(ip net.IP) uint32 {
|
||||
v4 := ip.To4()
|
||||
if v4 == nil {
|
||||
return 0
|
||||
}
|
||||
return uint32(v4[0])<<24 | uint32(v4[1])<<16 | uint32(v4[2])<<8 | uint32(v4[3])
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"net"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// FuzzIPAM_Allocate runs randomly-driven Allocate/Release sequences against
|
||||
// a /120 IPv6 + /28 IPv4 IPAM so the fuzzer can hit address exhaustion.
|
||||
//
|
||||
// Properties checked:
|
||||
//
|
||||
// 1. Allocate never panics regardless of the action stream.
|
||||
// 2. The set of in-use addresses never contains an address that has been
|
||||
// released without a subsequent successful Allocate.
|
||||
// 3. A successful v6 allocation always yields an address inside the
|
||||
// configured /120, and a successful v4 always inside the configured /28.
|
||||
// 4. ipToU32(canonical(allocated v4)) round-trips, and likewise that no
|
||||
// v4 allocation lands on .0 (network) or .15 (broadcast) of the /28.
|
||||
//
|
||||
// The fuzzed bytes are interpreted as an opcode stream:
|
||||
// - bytes[i] & 0x03 selects the action: 0=alloc-v6, 1=alloc-v4,
|
||||
// 2=alloc-dual, 3=release-most-recent.
|
||||
// - bytes[i]>>2 is fed into the deterministic random source so different
|
||||
// fuzzed bytes drive different IID/index choices.
|
||||
func FuzzIPAM_Allocate(f *testing.F) {
|
||||
f.Add([]byte{0, 0, 0, 0})
|
||||
f.Add([]byte{1, 1, 1, 1})
|
||||
f.Add([]byte{2, 2, 2, 2})
|
||||
f.Add([]byte{0, 1, 2, 3})
|
||||
f.Add([]byte(longString("\x00\x01\x02\x03", 256)))
|
||||
|
||||
f.Fuzz(func(t *testing.T, ops []byte) {
|
||||
ipam, err := NewIPAM(
|
||||
[]string{"2001:db8::/120"}, // 256 host slots; 16 bytes of fuzzed nibbles
|
||||
[]string{"10.0.0.0/28"}, // 14 usable hosts (.2..14)
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Deterministic source: replay nibbles cycled from `ops`.
|
||||
fr := &fakeRand{
|
||||
nibbles: append([]byte{}, ops...),
|
||||
iids: [][]byte{
|
||||
// 16 bytes of "host portion" — only the last byte matters
|
||||
// for a /120 prefix.
|
||||
makeIID(ops, 0),
|
||||
makeIID(ops, 1),
|
||||
makeIID(ops, 2),
|
||||
makeIID(ops, 3),
|
||||
},
|
||||
}
|
||||
if len(fr.nibbles) == 0 {
|
||||
fr.nibbles = []byte{0}
|
||||
}
|
||||
ipam.randSrc = fr
|
||||
|
||||
net6 := mustNet(t, "2001:db8::/120")
|
||||
net4 := mustNet(t, "10.0.0.0/28")
|
||||
|
||||
var live []AllocResult
|
||||
seen := map[string]struct{}{}
|
||||
|
||||
for idx, op := range ops {
|
||||
req := AllocRequest{ContainerID: idStr(idx)}
|
||||
switch op & 0x03 {
|
||||
case 0:
|
||||
req.WantV6 = true
|
||||
case 1:
|
||||
req.WantV4 = true
|
||||
case 2:
|
||||
req.WantV6, req.WantV4 = true, true
|
||||
case 3:
|
||||
if len(live) == 0 {
|
||||
continue
|
||||
}
|
||||
rel := live[len(live)-1]
|
||||
live = live[:len(live)-1]
|
||||
ipam.Release(rel.IP6, rel.IP4)
|
||||
delete(seen, canonical(rel.IP6))
|
||||
delete(seen, canonical(rel.IP4))
|
||||
continue
|
||||
}
|
||||
|
||||
res, err := ipam.Allocate(req)
|
||||
if err != nil {
|
||||
continue // exhaustion is acceptable
|
||||
}
|
||||
|
||||
if req.WantV6 {
|
||||
if res.IP6 == nil {
|
||||
t.Fatalf("requested v6 but got nil")
|
||||
}
|
||||
if !net6.Contains(res.IP6) {
|
||||
t.Fatalf("v6 %s outside /120", res.IP6)
|
||||
}
|
||||
if _, dup := seen[canonical(res.IP6)]; dup {
|
||||
t.Fatalf("v6 %s duplicated", res.IP6)
|
||||
}
|
||||
seen[canonical(res.IP6)] = struct{}{}
|
||||
}
|
||||
if req.WantV4 {
|
||||
if res.IP4 == nil {
|
||||
t.Fatalf("requested v4 but got nil")
|
||||
}
|
||||
if !net4.Contains(res.IP4) {
|
||||
t.Fatalf("v4 %s outside /28", res.IP4)
|
||||
}
|
||||
v4 := res.IP4.To4()
|
||||
if v4 == nil {
|
||||
t.Fatalf("v4 result not 4-byte: %s", res.IP4)
|
||||
}
|
||||
// Skip .0 (network) and .15 (broadcast). The allocator
|
||||
// should also skip .1 (gateway) by convention.
|
||||
last := v4[3]
|
||||
if last == 0 || last == 1 || last == 15 {
|
||||
t.Fatalf("v4 %s in reserved range", res.IP4)
|
||||
}
|
||||
if _, dup := seen[canonical(res.IP4)]; dup {
|
||||
t.Fatalf("v4 %s duplicated", res.IP4)
|
||||
}
|
||||
seen[canonical(res.IP4)] = struct{}{}
|
||||
}
|
||||
live = append(live, res)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// FuzzCanonical asserts that canonical never panics and is idempotent.
|
||||
func FuzzCanonical(f *testing.F) {
|
||||
f.Add([]byte{})
|
||||
f.Add([]byte{1, 2, 3, 4})
|
||||
f.Add([]byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0})
|
||||
f.Add([]byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0xff, 10, 0, 0, 1}) // v4-mapped v6
|
||||
f.Add([]byte{0xff})
|
||||
|
||||
f.Fuzz(func(t *testing.T, b []byte) {
|
||||
ip := net.IP(b)
|
||||
s1 := canonical(ip)
|
||||
// Idempotent: re-canonicalising the parsed form yields the same
|
||||
// string for any non-empty result.
|
||||
if s1 != "" {
|
||||
parsed := net.ParseIP(s1)
|
||||
if parsed == nil {
|
||||
t.Fatalf("canonical(%v)=%q is not parseable as IP", b, s1)
|
||||
}
|
||||
if got := canonical(parsed); got != s1 {
|
||||
t.Fatalf("not idempotent: %q -> %q", s1, got)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func makeIID(seed []byte, salt byte) []byte {
|
||||
out := make([]byte, net.IPv6len)
|
||||
for i := range out {
|
||||
if i < len(seed) {
|
||||
out[i] = seed[i] ^ salt
|
||||
} else {
|
||||
out[i] = salt
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func idStr(i int) string {
|
||||
const hex = "0123456789abcdef"
|
||||
return string([]byte{'c', '-', hex[(i>>4)&0xF], hex[i&0xF]})
|
||||
}
|
||||
+3
-3
@@ -1,6 +1,6 @@
|
||||
// Package agent owns the in-process flock-agent runtime: IPAM, netns, state,
|
||||
// anycast, and NetworkPolicy. This file implements the durable per-node
|
||||
// allocation file at /var/lib/flock/allocations.json.
|
||||
// This file implements the durable per-node allocation file at
|
||||
// /var/lib/flock/allocations.json. The package-level doc lives in doc.go.
|
||||
|
||||
package agent
|
||||
|
||||
import (
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
// Package v1alpha1 contains the operator-facing API types for flock.
|
||||
//
|
||||
// Stability: alpha. The shape of these types may change in incompatible ways
|
||||
// between minor releases. CRDs are versioned and the agent reads only its
|
||||
// pinned version.
|
||||
package v1alpha1
|
||||
|
||||
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
@@ -6,27 +11,78 @@ import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
//
|
||||
// The agent reads this on startup and via informer for live updates. There is
|
||||
// no controller and no auto-allocation — purely declarative input.
|
||||
//
|
||||
// A NodeConfig's name MUST equal the Kubernetes node name it configures
|
||||
// (NodeConfigs are cluster-scoped). The agent ignores all NodeConfigs whose
|
||||
// name does not match its own node.
|
||||
type NodeConfigSpec struct {
|
||||
// CIDR6 is the set of IPv6 CIDRs this node owns and advertises as BGP
|
||||
// aggregates. Pod IPv6 addresses are allocated from these.
|
||||
// aggregates. Pod IPv6 addresses are allocated from these. May be empty
|
||||
// only if Defaults disables IPv6 for every pod on this node.
|
||||
CIDR6 []string `json:"cidr6,omitempty"`
|
||||
|
||||
// CIDR4 is the set of IPv4 CIDRs this node owns and advertises as BGP
|
||||
// aggregates. Pod IPv4 addresses are allocated from these.
|
||||
// aggregates. Pod IPv4 addresses are allocated from these. May be empty
|
||||
// when no pod on this node ever opts into IPv4.
|
||||
CIDR4 []string `json:"cidr4,omitempty"`
|
||||
|
||||
// BGP configures the BGP sessions this node establishes upstream.
|
||||
BGP BGPSpec `json:"bgp"`
|
||||
|
||||
// Defaults sets the per-node baseline for which address families a pod
|
||||
// receives when its own annotations don't say. Pod-level
|
||||
// `flock.fritzlab.net/ipv6` and `flock.fritzlab.net/ipv4` annotations
|
||||
// always override these defaults.
|
||||
//
|
||||
// When a field is unset (nil), the agent falls back to its built-in
|
||||
// baseline of IPv6=true, IPv4=false. When the whole Defaults block is
|
||||
// nil, both built-in defaults apply.
|
||||
//
|
||||
// Typical uses:
|
||||
// - dual-stack node: Defaults: { ipv6: true, ipv4: true }
|
||||
// - IPv4-only node: Defaults: { ipv6: false, ipv4: true }
|
||||
// - default (omit Defaults entirely): IPv6-only.
|
||||
//
|
||||
// Validation: at least one of IPv6 or IPv4 must end up true after merging
|
||||
// (annotations + defaults + built-in baseline). The agent rejects pods
|
||||
// that resolve to neither.
|
||||
Defaults *FamilyDefaults `json:"defaults,omitempty"`
|
||||
}
|
||||
|
||||
// FamilyDefaults is the per-node default for which address families a pod
|
||||
// receives when its annotations don't specify. Each field is a pointer so
|
||||
// "unset" is distinguishable from explicit "false".
|
||||
type FamilyDefaults struct {
|
||||
// IPv6 is the default value for the `flock.fritzlab.net/ipv6` annotation.
|
||||
// nil → fall back to the built-in baseline (true).
|
||||
IPv6 *bool `json:"ipv6,omitempty"`
|
||||
|
||||
// IPv4 is the default value for the `flock.fritzlab.net/ipv4` annotation.
|
||||
// nil → fall back to the built-in baseline (false).
|
||||
IPv4 *bool `json:"ipv4,omitempty"`
|
||||
}
|
||||
|
||||
// BGPSpec describes this node's BGP speaker configuration. Each upstream peer
|
||||
// becomes one BGP session in the rendered bird.conf.
|
||||
type BGPSpec struct {
|
||||
ASN uint32 `json:"asn"`
|
||||
// ASN is this node's local autonomous system number. flock uses private
|
||||
// ASNs in the 64512-65534 range by convention but accepts any value.
|
||||
ASN uint32 `json:"asn"`
|
||||
|
||||
// Peers is the set of upstream BGP neighbors. At least one is required
|
||||
// for BGP advertisement to function. Multiple peers of the same family
|
||||
// are allowed (multi-homing).
|
||||
Peers []BGPPeer `json:"peers"`
|
||||
}
|
||||
|
||||
// BGPPeer is a single upstream BGP neighbor.
|
||||
type BGPPeer struct {
|
||||
// Address is the peer's IP. May be IPv4 or IPv6. The agent picks an
|
||||
// appropriate local source address on the same subnet.
|
||||
Address string `json:"address"`
|
||||
ASN uint32 `json:"asn"`
|
||||
|
||||
// ASN is the peer's remote ASN.
|
||||
ASN uint32 `json:"asn"`
|
||||
}
|
||||
|
||||
// NodeConfig is the Schema for the nodeconfigs API. NodeConfigs are cluster-
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
package embed
|
||||
|
||||
import (
|
||||
"net"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// FuzzEmbed verifies that Embed never panics and that any successful return
|
||||
// keeps the output address inside the requested network.
|
||||
func FuzzEmbed(f *testing.F) {
|
||||
type seed struct {
|
||||
prefix string
|
||||
fields string // comma-separated, mapped below to []Field
|
||||
ns, pod string
|
||||
image string
|
||||
fallback string
|
||||
nNibble byte
|
||||
}
|
||||
for _, s := range []seed{
|
||||
{"2602:817:3000:f001::/64", "namespace,pod,image", "mail", "stalwart-0", "", "ctr", 0xe},
|
||||
{"2001:db8::/64", "namespace", "ns", "p", "", "", 0},
|
||||
{"2001:db8::/96", "pod", "", "podname", "", "ctr", 0xf},
|
||||
{"2001:db8::/48", "namespace,pod", "ns", "p", "", "ctr", 0x1},
|
||||
{"2001:db8::/120", "namespace", "n", "p", "", "ctr", 0x0}, // 8 host nibbles
|
||||
{"2001:db8::/124", "namespace", "n", "p", "", "ctr", 0x0}, // 4 host nibbles
|
||||
{"2001:db8::/127", "namespace", "n", "p", "", "ctr", 0x0}, // not nibble-aligned
|
||||
{"2001:db8::/63", "namespace", "n", "p", "", "ctr", 0x0}, // not nibble-aligned
|
||||
{"2001:db8::/64", "namespace,pod,image", "", "", "sha256:abcdef0123456789aabbccddeeff00112233445566778899aabbccddeeff0011", "", 0xa},
|
||||
{"2001:db8::/64", "namespace,pod,image", "", "", "", "ctr", 0xa},
|
||||
{"2001:db8::/64", "namespace", "🦆", "🐧", "", "", 0},
|
||||
{"2001:db8::/64", "namespace", "ns\x00\x00", "p", "", "", 0},
|
||||
} {
|
||||
f.Add(s.prefix, s.fields, s.ns, s.pod, s.image, s.fallback, s.nNibble)
|
||||
}
|
||||
|
||||
f.Fuzz(func(t *testing.T, prefix, fieldsStr, ns, pod, image, fallback string, nNibble byte) {
|
||||
_, network, err := net.ParseCIDR(prefix)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
fields, ok := decodeFields(fieldsStr)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
got, err := Embed(network, fields, Values{
|
||||
Namespace: ns,
|
||||
Pod: pod,
|
||||
Image: image,
|
||||
ImageFallback: fallback,
|
||||
}, nNibble)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if !network.Contains(got) {
|
||||
t.Fatalf("Embed(%s, %v) = %s, outside network", prefix, fields, got)
|
||||
}
|
||||
// Property: low nibble of last byte equals nNibble & 0x0F.
|
||||
if want := nNibble & 0x0F; got[len(got)-1]&0x0F != want {
|
||||
t.Fatalf("low nibble = %x, want %x", got[len(got)-1]&0x0F, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func decodeFields(s string) ([]Field, bool) {
|
||||
if s == "" {
|
||||
return nil, false
|
||||
}
|
||||
var out []Field
|
||||
cur := []byte{}
|
||||
flush := func() bool {
|
||||
if len(cur) == 0 {
|
||||
return true
|
||||
}
|
||||
switch string(cur) {
|
||||
case string(FieldNamespace):
|
||||
out = append(out, FieldNamespace)
|
||||
case string(FieldPod):
|
||||
out = append(out, FieldPod)
|
||||
case string(FieldImage):
|
||||
out = append(out, FieldImage)
|
||||
default:
|
||||
return false
|
||||
}
|
||||
cur = cur[:0]
|
||||
return true
|
||||
}
|
||||
for i := 0; i < len(s); i++ {
|
||||
if s[i] == ',' {
|
||||
if !flush() {
|
||||
return nil, false
|
||||
}
|
||||
continue
|
||||
}
|
||||
cur = append(cur, s[i])
|
||||
}
|
||||
if !flush() {
|
||||
return nil, false
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
return out, true
|
||||
}
|
||||
+129
-6
@@ -9,6 +9,7 @@ import (
|
||||
"fmt"
|
||||
"net"
|
||||
"sort"
|
||||
"strings"
|
||||
"text/template"
|
||||
)
|
||||
|
||||
@@ -118,28 +119,150 @@ protocol bgp upstream4_{{$i}} {
|
||||
{{end}}{{end}}`
|
||||
|
||||
// Render produces the bird.conf text.
|
||||
//
|
||||
// The output is deterministic: the same NodeBGP input always produces the
|
||||
// same string. CIDR lists, anycast lists, and peer lists are sorted before
|
||||
// templating so that the only way the rendered config changes is when
|
||||
// semantically meaningful inputs change. This stability matters because
|
||||
// BirdManager compares Render output against the last-written config to
|
||||
// avoid superfluous birdc reloads.
|
||||
//
|
||||
// Render validates every operator-supplied value that flows into the
|
||||
// templated output (peer addresses, CIDRs, anycast IPs, source addresses)
|
||||
// so a malformed NodeConfig or annotation cannot produce a malformed
|
||||
// bird.conf — even one that BIRD would later reject.
|
||||
func Render(in NodeBGP) (string, error) {
|
||||
if in.RouterID == "" {
|
||||
return "", fmt.Errorf("RouterID is required")
|
||||
return "", fmt.Errorf("bird render: RouterID is required")
|
||||
}
|
||||
if net.ParseIP(in.RouterID) == nil {
|
||||
return "", fmt.Errorf("bird render: RouterID %q is not a valid IP", in.RouterID)
|
||||
}
|
||||
if in.LocalASN == 0 {
|
||||
return "", fmt.Errorf("LocalASN is required")
|
||||
return "", fmt.Errorf("bird render: LocalASN is required")
|
||||
}
|
||||
// Stable order — important so config changes only when something real
|
||||
// changes (avoids needless birdc reloads).
|
||||
if err := validateLocalSource(in.LocalV6, "v6"); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := validateLocalSource(in.LocalV4, "v4"); err != nil {
|
||||
return "", err
|
||||
}
|
||||
for i, p := range in.Peers {
|
||||
if err := validatePeer(p); err != nil {
|
||||
return "", fmt.Errorf("bird render: peer[%d]: %w", i, err)
|
||||
}
|
||||
}
|
||||
if err := validateCIDRs(in.CIDR6, "v6"); err != nil {
|
||||
return "", fmt.Errorf("bird render: cidr6: %w", err)
|
||||
}
|
||||
if err := validateCIDRs(in.CIDR4, "v4"); err != nil {
|
||||
return "", fmt.Errorf("bird render: cidr4: %w", err)
|
||||
}
|
||||
if err := validateAnycastIPs(in.Anycast6, "v6"); err != nil {
|
||||
return "", fmt.Errorf("bird render: anycast6: %w", err)
|
||||
}
|
||||
if err := validateAnycastIPs(in.Anycast4, "v4"); err != nil {
|
||||
return "", fmt.Errorf("bird render: anycast4: %w", err)
|
||||
}
|
||||
|
||||
in = normalize(in)
|
||||
|
||||
t, err := template.New("bird").Parse(tpl)
|
||||
if err != nil {
|
||||
return "", err
|
||||
return "", fmt.Errorf("bird template parse: %w", err)
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := t.Execute(&buf, in); err != nil {
|
||||
return "", err
|
||||
return "", fmt.Errorf("bird template execute: %w", err)
|
||||
}
|
||||
return buf.String(), nil
|
||||
}
|
||||
|
||||
// validatePeer checks that a peer entry has a parseable IP whose family
|
||||
// matches its declared Family field, and a non-zero ASN.
|
||||
func validatePeer(p Peer) error {
|
||||
if p.ASN == 0 {
|
||||
return fmt.Errorf("ASN must be non-zero")
|
||||
}
|
||||
ip := net.ParseIP(p.Address)
|
||||
if ip == nil {
|
||||
return fmt.Errorf("address %q is not a valid IP", p.Address)
|
||||
}
|
||||
isV4 := ip.To4() != nil
|
||||
switch p.Family {
|
||||
case "v6":
|
||||
if isV4 {
|
||||
return fmt.Errorf("address %q is IPv4 but Family is v6", p.Address)
|
||||
}
|
||||
case "v4":
|
||||
if !isV4 {
|
||||
return fmt.Errorf("address %q is IPv6 but Family is v4", p.Address)
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("Family %q must be v6 or v4", p.Family)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// validateCIDRs parses each entry as a CIDR and rejects family mismatches.
|
||||
// fam must be "v6" or "v4".
|
||||
func validateCIDRs(cidrs []string, fam string) error {
|
||||
for _, c := range cidrs {
|
||||
_, n, err := net.ParseCIDR(c)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid CIDR %q: %w", c, err)
|
||||
}
|
||||
isV4 := n.IP.To4() != nil
|
||||
if fam == "v6" && isV4 {
|
||||
return fmt.Errorf("CIDR %q is IPv4, expected IPv6", c)
|
||||
}
|
||||
if fam == "v4" && !isV4 {
|
||||
return fmt.Errorf("CIDR %q is IPv6, expected IPv4", c)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// validateAnycastIPs parses each entry as a literal IP (no prefix) and rejects
|
||||
// family mismatches.
|
||||
func validateAnycastIPs(ips []string, fam string) error {
|
||||
for _, s := range ips {
|
||||
ip := net.ParseIP(s)
|
||||
if ip == nil {
|
||||
return fmt.Errorf("invalid IP %q", s)
|
||||
}
|
||||
isV4 := ip.To4() != nil
|
||||
if fam == "v6" && isV4 {
|
||||
return fmt.Errorf("IP %q is IPv4, expected IPv6", s)
|
||||
}
|
||||
if fam == "v4" && !isV4 {
|
||||
return fmt.Errorf("IP %q is IPv6, expected IPv4", s)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// validateLocalSource validates an optional LocalV6/LocalV4 source address.
|
||||
// Empty is allowed (BIRD picks its own); non-empty must be a parseable IP of
|
||||
// the matching family.
|
||||
func validateLocalSource(s, fam string) error {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
ip := net.ParseIP(s)
|
||||
if ip == nil {
|
||||
return fmt.Errorf("bird render: Local%s %q is not a valid IP", strings.ToUpper(fam), s)
|
||||
}
|
||||
isV4 := ip.To4() != nil
|
||||
if fam == "v6" && isV4 {
|
||||
return fmt.Errorf("bird render: LocalV6 %q is IPv4", s)
|
||||
}
|
||||
if fam == "v4" && !isV4 {
|
||||
return fmt.Errorf("bird render: LocalV4 %q is IPv6", s)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func normalize(in NodeBGP) NodeBGP {
|
||||
cp := in
|
||||
cp.CIDR6 = sortedUnique(in.CIDR6)
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
package bird
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// FuzzRender drives the bird template with a wide range of inputs and
|
||||
// confirms two safety properties:
|
||||
//
|
||||
// 1. Render never panics.
|
||||
// 2. On nil-error return, the output is deterministic (calling Render
|
||||
// twice with the same input yields byte-identical output) and contains
|
||||
// no unbalanced braces (a smoke test for malformed template branches).
|
||||
func FuzzRender(f *testing.F) {
|
||||
type seed struct {
|
||||
routerID string
|
||||
asn uint32
|
||||
peerAddr string
|
||||
peerASN uint32
|
||||
cidr6 string
|
||||
cidr4 string
|
||||
anycast6 string
|
||||
anycast4 string
|
||||
localV6 string
|
||||
localV4 string
|
||||
}
|
||||
seeds := []seed{
|
||||
{routerID: "10.0.0.1", asn: 65101, peerAddr: "2001:db8::1", peerASN: 65000, cidr6: "2001:db8:f001::/64"},
|
||||
{routerID: "172.25.25.101", asn: 65101, peerAddr: "172.25.25.1", peerASN: 65000, cidr4: "172.25.210.0/24"},
|
||||
{routerID: "10.0.0.1", asn: 65101, peerAddr: "2001:db8::1", peerASN: 65000, cidr6: "2001:db8:f001::/64", anycast6: "2001:db8:a::1"},
|
||||
{routerID: "10.0.0.1", asn: 65101, peerAddr: "10.0.0.2", peerASN: 65000, cidr4: "10.0.0.0/24", anycast4: "10.255.0.1"},
|
||||
{routerID: "10.0.0.1", asn: 65101}, // no peer, no cidrs
|
||||
{routerID: "", asn: 65101, peerAddr: "10.0.0.2", peerASN: 1}, // empty routerID → expect error
|
||||
{routerID: "10.0.0.1", asn: 0, peerAddr: "10.0.0.2", peerASN: 1}, // zero ASN → expect error
|
||||
// Backtick-bearing inputs to defend the template against accidental
|
||||
// closure of the raw-string literal.
|
||||
{routerID: "10.0.0.1`", asn: 65101},
|
||||
// Newlines and template-meta in user-supplied addresses
|
||||
{routerID: "10.0.0.1", asn: 65101, peerAddr: "2001:db8::1\n{{kaboom}}", peerASN: 65000, cidr6: "2001:db8:f001::/64"},
|
||||
}
|
||||
for _, s := range seeds {
|
||||
f.Add(s.routerID, s.asn, s.peerAddr, s.peerASN, s.cidr6, s.cidr4, s.anycast6, s.anycast4, s.localV6, s.localV4)
|
||||
}
|
||||
|
||||
f.Fuzz(func(t *testing.T, routerID string, asn uint32, peerAddr string, peerASN uint32, cidr6, cidr4, anycast6, anycast4, localV6, localV4 string) {
|
||||
in := NodeBGP{
|
||||
RouterID: routerID,
|
||||
LocalASN: asn,
|
||||
LocalV6: localV6,
|
||||
LocalV4: localV4,
|
||||
}
|
||||
// Add the peer in whichever family it belongs to, if any. FamilyOf
|
||||
// returns "" for non-IPs; that test exercises the "skip unknown
|
||||
// family" branch in the bird agent code path.
|
||||
if fam := FamilyOf(peerAddr); fam != "" {
|
||||
in.Peers = []Peer{{Family: fam, Address: peerAddr, ASN: peerASN}}
|
||||
}
|
||||
if cidr6 != "" {
|
||||
in.CIDR6 = []string{cidr6}
|
||||
}
|
||||
if cidr4 != "" {
|
||||
in.CIDR4 = []string{cidr4}
|
||||
}
|
||||
if anycast6 != "" {
|
||||
in.Anycast6 = []string{anycast6}
|
||||
}
|
||||
if anycast4 != "" {
|
||||
in.Anycast4 = []string{anycast4}
|
||||
}
|
||||
|
||||
out, err := Render(in)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Determinism.
|
||||
out2, err := Render(in)
|
||||
if err != nil {
|
||||
t.Fatalf("Render became flaky: first ok, second %v", err)
|
||||
}
|
||||
if out != out2 {
|
||||
t.Fatalf("Render not deterministic on identical input")
|
||||
}
|
||||
|
||||
// Smoke test for balanced braces. The template uses `{` and `}`
|
||||
// as BIRD's block delimiters; if our template engine ever
|
||||
// produced an unbalanced output we'd catch it here.
|
||||
if got := strings.Count(out, "{") - strings.Count(out, "}"); got != 0 {
|
||||
t.Fatalf("unbalanced braces: %d", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
go test fuzz v1
|
||||
string("0")
|
||||
uint32(65101)
|
||||
string("0")
|
||||
uint32(1)
|
||||
string("")
|
||||
string("")
|
||||
string("")
|
||||
string("}")
|
||||
string("")
|
||||
string("")
|
||||
Reference in New Issue
Block a user