Overview: Exploring Multus Multi-Networking Tech
In traditional setups, we often separate the network cards for different purposes. For example, we might have one for management, another for data, and yet another for hardware control. Sometimes, we even split networks for inbound and outbound data. This is pretty common whether you’re dealing with physical machines or virtual machines.
But in the Kubernetes, it’s different. By default, Pods only have one network interface available for communication with the outside. You can’t simply add another network card like virtual machine environment.
This article will serve as the first piece on the Multus solution, starting with some basic knowledge explanation. Later, we’ll proceed with practical lab operations. Without further ado, here’s what this article will cover:
- Overview
- About Support
- Differences between Thin and Thick
- Types of CNI
- Conclusion
1. Overview
Multus is a Kubernetes plugin developed by Intel. Its primary purpose is to enable Pods to attach multiple network interfaces through Custom Resource Definitions (CRDs). It functions as a meta-plugin to invoke various other CNIs (Container Network Interfaces). Since it’s configured via CRDs, it aligns with Kubernetes’ native configuration approach.
In simple terms, Multus allows Pods to have two or more network interfaces simultaneously.
The diagram above illustrates a Pod with three network interfaces (eth0, net0, net1), where net0 and net1 are connected using different CNIs.
Multus enables multiple network interfaces to be attached to a Pod, achieving the following objectives:
- Isolation of control/data networks
- Support for specific hardware
- Storage-sensitive application services
- Implementation of specific CNI functionalities
- Multi-tenant networking
2. About Support
(1) Multus provides multiple network interfaces for pods running on Kubernetes, supporting most common CNI plugins:
- CNI-developed plugins (Flannel, DHCP, Macvlan)
- Third-party plugins (Calico, Weave, Contiv)
- Other plugins that Kubernetes may not support directly (SR-IOV, DPDK)
(2) Multus utilizes the concept of “delegates” to combine multiple CNI plugins, specifying a primary CNI plugin as the main network and discoverable by Kubernetes.
(3) It can be customized to meet specific CNI requirements (although I haven’t had the chance to do so :))
3. Differences between Thin and Thick
According to the official document, starting from version 4.0, there are new client/server plugin deployments introduced. This new deployment is referred to as the “thick plugin,” while the previous architecture is called the “thin plugin.”
The thick plugin consists of multus-daemon and multus-shim.
- multus-daemon: Deployed on all nodes as a local agent to support additional functionalities such as metrics (which were not available before). Due to this, it requires more resources compared to the previous setup.
In general, it is recommended to use the thick plugin, but if resources are limited, the thin plugin can also be used.
4. Types of CNI
Here’s a quick rundown of the three main types of CNIs.
※ Here’s a breakdown:
(1) Main: Creating additional networks within the cluster
- Bridge: Establishes an additional network based on a bridge. Allows communication between Pods on the same node and with the node itself.
- Host-device: Sets up an additional network based on a host device. Enables Pods to communicate with physical network interfaces on the node system.
- Macvlan: Establishes an additional network based on macvlan. Allows Pods on the node to communicate with other Pods (including the node itself) on other nodes. Each Pod attached to the macvlan-based network gets a unique MAC.
- Ipvlan: Creates an additional network based on ipvlan. Allows Pods on the node to communicate with Pods on other nodes (including the node itself), similar to macvlan. However, each Pod shares the same MAC as the upper-layer physical network interface.
- SR-IOV: Sets up an additional network based on SR-IOV. Allows Pods to attach to virtual function (VF) ports with SR-IOV support on the node.
(2) IP Address Management (IPAM): Responsible for IP Allocation
- DHCP: Requires a DHCP server on the node.
- Host-local: Allocates IPs based on a given subnet range on a single node.
- Static: Specifies IP addresses directly.
- Calico-IPAM: IP allocation specific to Calico CNI.
- Whereabouts: Less commonly used, typically for SR-IOV devices.
(3) Meta: Maintained by the CNI Community
- Flannel: Specifically for the Flannel CNI plugin.
- Tuning: Binary file for adjusting network device parameters via sysctl.
- Portmap: Binary file for configuring port mapping via iptables.
- Bandwidth: Uses Token Bucket Filter (TBF) for traffic shaping.
- Calico: Specifically for the Calico CNI plugin.
5. Conclusion
This article aims to explain some fundamental concepts learned from previous Multus testing, hoping to provide an understanding of the basics before diving into actual deployment and configuration.
Additionally, Multus is a solution that integrates multiple CNIs. When deploying it, it’s advisable to not only confirm Multus’s supported range but also understand if the functionalities of the CNIs you intend to use align with your actual needs, to avoid wasted efforts.
※ Reference:
