2. Agenda
1. KubeVirt
a. Why we need it?
b. Components
c. How it works?
2. Storage
3. Networking
4. Container Native Virtualization
5. Demo
6. References
3. VIRTUAL MACHINES AND CONTAINERS
VIRTUAL MACHINES CONTAINERS
VM virtualizes the hardware Container virtualizes isolates
the process
VM
OS Dependencies
Kernel
Hypervisor
Hardware
App App App App
Hardware
Container Host (Kernel)
Containe
r
App
OS
deps
Containe
r
App
OS
deps
Containe
r
App
OS
deps
Containe
r
App
OS
deps
4. Kubernetes
● Orchestrates many containers on many nodes (hosts)
● Initiated by Google
● Huge community now (contributors and users)
● Declarative approach
● Continuously compares declared state with observed state and
takes action
5. KubeWhat?
KubeVirt (http://kubevirt.io) is a virtual management add-on for Kubernetes.
KubeVirt leverages Kubernetes as a common platform for virtual machines and
containers.
6. Use cases
● Migration path from existing VM infrastructure
● Decomposing VM workloads into containers
● Centralize development workflows
○ One pipeline for both VMs and Containers
○ Allows VMs and Containers to coexist
● Centralized Operations
○ One System for both Container and VM workloads
○ Increased Ops efficiency and lowers Ops costs.
● Provide strict(er) isolation of containers
○ Katacontainers, gVisor, Isolates, Firecracker
7. Capabilities
Compute
● Create/Start/Stop/Reset VM
● VNC and serial console access
● Liveness and readiness probes
● Guest agent status
● Prometheus metrics
● Custom hooks
● VM (anti)affinity
● VM migrations / Node drain
Performance
● CPU pinning
● Huge pages
● IO threads
● Multi-queue
● Pinned IO
Networking
● Pod and L2 networking (via Multus or Genie)
● SR-IOV interface support
● Custom DHCP options
● PXE boot
● Configure MAC address
Storage
● Block PVs
● DataVolume
● ContainerDisk
● Host assisted cloning
● Import and upload flows
8. How it was built
● KubeVirt is built upon K8s
● Leverages K8s native infrastructure
○ Scheduler
○ Networking
■ Ingress
■ Services and routes
■ NetworkPolicies
○ Storage
■ PVs and PVCs
■ storageClass
○ Operators
■ Uses CRDs for many entities including VM and VMI
● Provides virtctl and vmctl(experimental)
● Tooling ecosystem (vagrant, ansible etc)
9. Components
● K8s controllers
○ Manages VM process life cycle
● Virt APIs
○ Defines, Validates, Implements VM, VMI, VMRS, and VMPreset APIs
● Stream APIs
○ WebSocket access to VM’s via console and VNC
● Virt specific pods
○ Virt handler and launcher
● Network and storage integration
○ The glue code integrates VMs with network and storage solutions
● Kubevirt operator
○ Install/update/uninstall Kubevirt components
11. ● PVC/PVs
○ PVCs and PVs are K8s APIs for providing persistent volumes to Pods
○ These APIs abstract away storage details from how storage is consumed
● DataVolume
○ Built on top of PVC, it helps with PVC readiness issue
● StorageClass Provisioners
○ Provisions storage using CSI provider (GlusterFS, Ember, etc...)
○ Offload more complex storage operations to the provisioner
● ContainerDisk
○ Use regular container image exposed as file system
K8s Storage subsystem
12. Data importing
● Containerized Data Importer (CDI)
○ Controller that watches for PVC created with special annotations
○ Attaches PVCs to short lived Pod to inject data into PVC
● CDI Use Cases
○ Leveraged by KubeVirt to import or upload VM disks into PVCs
○ Clone existing disks to new VM
○ Possible other use cases outside of KubeVirt
● Data cloning methods
○ Smart cloning (copy on write, provided by underlying implementation)
○ Host assisted cloning (heavyweight generic approach)
● Consumption methods
○ File based (PVC based)
○ Block device
13. Disk provisioning flow - example
● Post a PVC
○ VM needs a 10gb disk containing a fedora image
○ Post PVC with annotations asking CDI to import a fedora image.
● StorageClass provisions PV
○ GlusterFS StorageClass provisioner sees PVC
○ Dynamically generates PV and underlying Gluster storage volume
● CDI Injects data into PVC
○ CDI sees bound PVC/PV with special annotation
○ Launches Pod to inject fedora image into PVC
● PVC is ready for consumption
○ VM spec is posted referencing new PVC as a volume source
○ Possible to use DataVolume to simplify readiness check
14. VM networking
● Pod network
○ IP Takeover
○ Laverages K8s DNS
○ Services and Routes
● Container networking plugins L2
○ CNI-Multus
○ CNI-Genie
■ Abstraction over other CNI plugins