Calico eBPF by default*

*some conditions may apply

Calico has used eBPF as one of its dataplanes since version 3.13, released more than five years ago. At the time, this was an exciting step forward, introducing a new, innovative data plane that quickly gained traction within the Calico community. Since then, there have been many changes and continued evolution, all thanks to the many adopters of the then-new data plane.

However, there has been one persistent challenge in the installation process since day one: bootstrapping the eBPF data plane required a manual setup step. This extra friction point often frustrated operators and slowed adoption.

With the launch of Calico v3.31, that hurdle to using the eBPF data plane has finally been removed. For many environments (see Limitations section below), you can now install Calico with eBPF enabled right out of the box with** no manual setup required**.

Simply use the provided installation manifest (custom-resources-bpf.yam), which comes preconfigured with the data plane option set to eBPF.

To get started, follow the instructions in Install Calico networking and network policy for on-premises deployments to enjoy a much smoother installation experience.

See Calico eBPF in action with this short demo, highlighting how version 3.31 makes setup easier and increases performance:

The Road to Seamless Installation

The Calico eBPF data plane includes a kube-proxy replacement, enabling faster service implementation than the upstream version of kube-proxy for iptables and better integration with the rest of the data plane for overall superior performance. However, a kube-proxy replacement requires knowing how to reach the Kubernetes API server.

The upstream kube-proxy that comes with Kubernetes installation gets that knowledge during the Kubernetes bootstrapping process. Our replacement is not part of that process; it is installed only after the Kubernetes is up and running. Because that bootstrap information varies widely across environments, it’s not always straightforward to reuse across all environments. Therefore our Install in eBPF mode guide contains a section on how to configure the Kubernetes endpoint ConfigMap to ensure Calico starts up.

Previously, the Kubernetes endpoint used to be specified in the installation manifest which you had to modify. Over the years, Calico gravitated towards operator-based installation, which allows the operator to inspect the system before installing Calico and automatically adjust settings to meet the host environment requirements.

How the Operator Automates Setup

In many Kubernetes environments, by the time you install Calico, kube-proxy is already running and the Kubernetes API server is reachable since the Kubernetes service is implemented. This allows the operator to learn how to program the service prior to installing Calico and before disabling kube-proxy. The operator then passes this knowledge to Calico nodes in the rendered Calico-node Daemonset.

When calico-node instances start, the init container creates the eBPF NAT table and fills the service programming into this table. From this point, eBPF programs can resolve any access to the Kubernetes service to one of its endpoints.

However, setting up the entire eBPF data plane with its many programs attached to all the devices is a complex task. Therefore we opted to bootstrap the system using a connect-time load balancer – a small set of programs attached via cGroups to the connect() system call of all applications. Once these programs are attached, any application in the system can reach the Kubernetes API server without relying on kube-proxy.

Therefore, when you begin the bootstrap process, note that the connect-time load balancer will be turned on by default. However, if your Istio side cars or other cluster mesh setups require connect-time load balancer to be turned off, don’t worry—Calico will automatically reconcile with your settings.

Important: We recommend keeping the connect-time load balancer turned on as it removes per-packet NAT overhead.

• To learn more about the eBPF auto configuration visit Enable the eBPF data plane automatically for self-managed clusters.
• To install eBPF in the automated mode go to the guide: Install Calico networking and network policy for on-premises deployments.

High-Availability (HA) Kubernetes API Server Support

Not only does this change make installing Calico eBPF frictionless in many environments, but it also allows the eBPF kube-proxy replacement to use all Kubernetes service backends in HA Kubernetes API servers setup. It was previously only possible when the KUBERNETES_SERVICE_HOST was represented by a FQDN — for example, in EKS (see How to configure Calico to talk directly to the API server).

Manifest-Based Installations

Although the frictionless method described above only applies to operator based installations of Calico (see limitations below), it is possible to set up the HA Kubernetes API server in a manifest installation as well.

You can manually set the environment variables in the Calico-node Daemon-set that the operator sets automatically:

• KUBERNETES_SERVICE_IPS_PORTS to list all Kubernetes service IPs and ports
• KUBERNETES_APISERVER_ENDPOINTSto list all backend IPs and ports

Both have the format of comma-separated list of ip:port. In a dual-stack environment, just put all the v4 and v6 addresses in the respective variable together. For example:

KUBERNETES_SERVICE_IPS_PORTS=10.1.0.1:443,[deaf:beef::1:0:1]:433 KUBERNETES_APISERVER_ENDPOINTS=192.168.5.5:6443,192.168.5.4:6443,[dead:beef::5:5]:6443

Limitations

The automatic and frictionless installation is only possible when these three conditions are met:

1. Calico is installed using the operator.
2. The environment has kube-proxy running prior to the installation.
3. kube-proxy DaemonSet is not managed by an external entity. (If it is, set kubeProxyManagement to Disabled and bpfKubeProxyIptablesCleanupEnabled to false.)

In managed Kubernetes and some other environments, the Kubernetes API server is typically external to the cluster and custom configuration might lead to unforeseen behaviour. If you use the automated method in such environments, the operator will emit a warning that will guide you how to disable it in case you experience any issues.

Note: We are continually working to expand platform compatibility, and you can expect support for more platforms in future Calico releases.

Join the Conversation

Calico v3.31 makes using the eBPF data plane easier and smoother than ever. What used to be a manual, tricky setup is now mostly automatic, removing a big hurdle for many environments.

Whether you’re using the operator-based installation or a custom setup, you can take advantage of better performance, easier configuration, and a simpler overall experience.

And if you’re looking for more tips or have questions, join our community on the Calico Slack channel to connect with experts and fellow practitioners.