Skip to content

Use CiliumCIDRGroup to simplify Cilium network policies on AKS

How to use CiliumCIDRGroup on AKS to model external Azure endpoints as reusable, labeled policy targets for cleaner Cilium network policies.

On this page

I’ve been tightening the egress rules on the AKS clusters I run with Cilium. Somewhere along the way, my network policies had filled up with the same handful of IP addresses, copied from file to file. CiliumCIDRGroup turned out to be the fix.

Short version: define each external IP range once as a CiliumCIDRGroup with a name and some labels, then reference it from your policies by name (cidrGroupRef) or by label (cidrGroupSelector).

Why the IPs pile up

Cilium gives every pod an identity based on its labels, and policies select those identities. Anything outside the cluster has no labels, so it all lands in one reserved identity, reserved:world. To allow or deny a specific external address, you have to write out its CIDR.

On AKS, two of those addresses come up again and again: the Instance Metadata Service (IMDS) at 169.254.169.254 and the Azure platform DNS resolver at 168.63.129.16. In my policies, both ended up copied into toCIDRSet blocks from file to file. It works, but it’s hard to review, and changing a range means grepping YAML.

Name the ranges once

The Cilium docs describe it like this:

CiliumCIDRGroup (CCG) is a feature that allows administrators to reference a group of CIDR blocks in a CiliumNetworkPolicy.

It’s a cluster-scoped resource: a list of CIDRs with a name and labels. Here are the two Azure endpoints:

azure-external-endpoints-ccg.yaml
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
  name: azure-imds
  labels:
    cloud: azure
    service: imds
spec:
  externalCIDRs:
    - "169.254.169.254/32"
---
apiVersion: cilium.io/v2
kind: CiliumCIDRGroup
metadata:
  name: azure-internal-dns
  labels:
    cloud: azure
    service: dns
spec:
  externalCIDRs:
    - "168.63.129.16/32"
Terminal window
kubectl apply -f azure-external-endpoints-ccg.yaml

Nothing changes yet. A group does nothing until a policy references it. According to the docs, traffic to those CIDRs also shows up in Hubble annotated with the group’s name and labels.

CIDR groups are for addresses that don’t move. IMDS will always be 169.254.169.254. For anything you reach by hostname, DNS-based policy is the better fit.

Reference a group by name

cidrGroupRef names the group directly. You can read the policy and know exactly which range it targets:

ccnp-by-cidrgroupref.yaml
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: allow-azure-dns-by-name
spec:
  endpointSelector:
    matchLabels:
      io.kubernetes.pod.namespace: kube-system
      k8s-app: kube-dns
  egress:
    - toCIDRSet:
        - cidrGroupRef: azure-internal-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
            - port: "53"
              protocol: TCP
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-imds-by-name
spec:
  endpointSelector: {}
  egressDeny:
    - toCIDRSet:
        - cidrGroupRef: azure-imds
Terminal window
kubectl apply -f ccnp-by-cidrgroupref.yaml

Reference a group by label

cidrGroupSelector works like endpointSelector, but for external ranges. The policy doesn’t need to know any group names:

ccnp-by-cidrgroupselector.yaml
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: allow-azure-dns-by-label
spec:
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: workload-needing-azure-egress
  egress:
    - toCIDRSet:
        - cidrGroupSelector:
            matchLabels:
              cloud: azure
              service: dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
            - port: "53"
              protocol: TCP
---
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: deny-imds-by-label
spec:
  endpointSelector: {}
  egressDeny:
    - toCIDRSet:
        - cidrGroupSelector:
            matchLabels:
              cloud: azure
              service: imds
Terminal window
kubectl apply -f ccnp-by-cidrgroupselector.yaml

Names or labels

Either works. If one team owns both the groups and the policies, names are simpler. If the groups are managed separately from the policies, labels age better: a new group with the right labels is picked up without touching a single policy.

I use labels. With the same policies rolled out to every cluster, I’d rather match on cloud: azure than keep group names in sync everywhere.

The catch, as far as I can tell, is that a typo fails quietly in both styles. A label or name that matches no group should leave the rule with nothing to apply to. I haven’t reproduced that on a cluster. For an allow rule you notice fast, because traffic breaks. For a deny rule like the IMDS one, nothing breaks and pods can still reach IMDS, so it’s worth testing the deny from a pod after changing labels.

References

Published
Updated
Reading
2 min