There are many different tools available to bootstrap and manage Kubernetes cluster within various platform, but they don't all interoperate. Cluster API project attempts to solve this issue, which creates common declarative API, tools, and best practices for deploying, configuring, and managing Kubernetes in multiple platforms. It supports many public cloud provider plugins, and on-premise with vSphere and OpenStack. However, running Cluster API for raw baremetal KVM machines without IaaS vendor remains a challenge. In this talk, we will show you how we use and configure Libvirt to create custom provider for Cluster API. Libvirt is an open-source API and management tool widely used for managing various virtualization platform. This enables us to manage Kubernetes on baremetal KVM/Qemu machines easily, with extensibility to run in Xen and other platform not supported by Cluster API.

1. Creating a custom
Cluster API Provider
Himani Agrawal
@himani_93
Giri Kuncoro
@girikuncoro

2. History

3. 70+ ways to deploy
Kubernetes

4. What is missing?
➔ Declarative, Kubernetes-style API
➔ Compatibility with tooling
➔ Cloud agnostic
➔ Single interface for infra/cluster

5. Cluster API

6. Kubernetes
Cluster
Cluster API
Reference: Deep Dive: Cluster API KubeCon EU 2019

7. Declarative
Config
Kubernetes
Cluster
Cluster API
Reference: Deep Dive: Cluster API KubeCon EU 2019

8. Kubernetes
Cluster
Cluster API
Declarative
Config
Reference: Deep Dive: Cluster API KubeCon EU 2019

9. Cluster API CRDs
Cluster

10. Cluster
apiVersion: "cluster-api.k8s.io/v1alpha1"
kind: Cluster
metadata:
name: fujisan
spec:
providerSpec:
...
clusterNetwork:
services:
cidrBlocks: ["10.96.0.0/12"]
pods:
cidrBlocks: ["192.168.0.0/16"]
serviceDomain: "cluster.local"

11. Cluster API CRDs
Cluster Machine

12. Machine
apiVersion: "cluster-api.k8s.io/v1alpha1"
kind: Machine
metadata:
name: kube-cp
spec:
providerSpec:
vcpu: 2
...
versions:
kubelet: 1.15.0

13. Cluster API CRDs
Cluster Machine Machine Set

14. Cluster API CRDs
Cluster Machine Machine Set
Machine
Deployment

15. Cluster API CRDs
Cluster Machine Machine Set
Machine
Deployment
Machine
Class

16. Cluster API CRDs
Cluster Machine Machine Set
Machine
Deployment
Machine
Class
Pod Replica Set Deployment
Storage
Class

17. How Cluster API works?
Controller Actuator KVM Baremetal
apiVersion:
"cluster-api.k8s.io/v1alpha1"
kind: Machine
metadata:
name: kube-cp
spec:
providerSpec:
vcpu: 2
versions:
kubelet: 1.15.0
Machine CRD
Machine

18. Provider Controller Manager
Cluster Controller Machine Controller

19. Demo
Picture Credits: Ashley McNamara

20. Code
func main() {
mgr, _ := manager.New(cfg, opts)
machineActuator, _ := machine.NewActuator(machine.ActuatorParams{
Client: mgr.GetClient(),
})
apis.AddToScheme(mgr.GetScheme())
clusterapis.AddToScheme(mgr.GetScheme())
capimachine.AddWithActuator(mgr, machineActuator)
mgr.Start(signals.SetupSignalHandler())
}
cmd/manager/main.go

21. Code
func init() {
// AddToManagerFuncs is a list of functions to create controllers
// and add them to a manager.
AddToManagerFuncs = append(AddToManagerFuncs, func(m manager.Manager) error {
return capimachine.AddWithActuator(m, &machine.Actuator{})
})
}
pkg/controller/add_machine_controller.go

22. Code
type LibvirtMachineProviderSpecSpec struct {
// Number of virtual CPU
VCPU int `json:"vcpu"`
// Amount of RAM in GBs
MemoryInGB int `json:"memoryInGB"`
// Image URL to be provisioned
ImageURI string `json:"imageURI"`
// UserData URI of cloud-init image
UserDataURI string `json:"userDataURI"`
}
pkg/apis/libvirt/v1alpha1/libvirtmachineproviderspec_types.go

23. Code
// CreateDomain connects to libvirt daemon and creates a new domain
// as per domainXML.
func CreateDomain(domainXML string) error {
...
}
// DomainExists checks if the domain with the given name exists.
func DomainExists(domainName string) bool {
...
}
// defineDomain returns the XML representation of the domain to be created.
// Output of this func is passed into CreateDomain() to create a new domain.
func defineDomain() string {
...
}
pkg/cloud/libvirt/domain.go

24. func (a *Actuator) Create() error {
return libvirt.CreateDomain()
}
func (a *Actuator) Exists() (bool, error) {
return libvirt.DomainExists()
}
func (a *Actuator) Delete() { ... }
func (a *Actuator) Update() { ... }
Code
pkg/cloud/libvirt/actuators/machine/actuator.go

25. /himani93/cluster-api-provider-libvirt

26. ● Cluster API v0.1.0
● Kubebuilder v1.0.8
● Kustomize v1.0.11
● Kubectl v1.13
● Minikube v1.2.0
Dependency Versions

27. Discussion
● Cluster API is super cool!
● Early work for production version

28. Getting Involved
➔ Github Repo
github.com/kubernetes-sigs/cluster-api
➔ Mailing List: kubernetes-sig-cluster-lifecycle
https://groups.google.com/forum/#!forum/kubernetes-sig-clus
ter-lifecycle
➔ Slack: #cluster-api
https://kubernetes.slack.com/messages/C8TSNPY4T

29. Provider Implementations
● AWS, https://github.com/kubernetes-sigs/cluster-api-provider-aws
● Azure, https://github.com/kubernetes-sigs/cluster-api-provider-azure
● Baidu Cloud, https://github.com/baidu/cluster-api-provider-baiducloud
● Bare Metal, https://github.com/metal3-io/cluster-api-provider-baremetal
● DigitalOcean, https://github.com/kubernetes-sigs/cluster-api-provider-digitalocean
● Exoscale, https://github.com/exoscale/cluster-api-provider-exoscale
● GCP, https://github.com/kubernetes-sigs/cluster-api-provider-gcp
● IBM Cloud, https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud
● OpenStack, https://github.com/kubernetes-sigs/cluster-api-provider-openstack
● Talos, https://github.com/talos-systems/cluster-api-provider-talos
● Tencent Cloud, https://github.com/TencentCloud/cluster-api-provider-tencent
● vSphere, https://github.com/kubernetes-sigs/cluster-api-provider-vsphere

30. API Adoption
Following are the implementations managed by third-parties adopting the standard cluster-api and/or
machine-api being developed here.
● Kubermatic machine-controller, https://github.com/kubermatic/machine-controller/tree/master
● Machine API Operator, https://github.com/openshift/machine-api-operator/tree/master
● Machine-controller-manager, https://github.com/gardener/machine-controller-manager/tree/cluster-api

31. References
● https://kccna18.sched.com/event/Greh
● https://kccnceu19.sched.com/event/MPkR
● https://blogs.vmware.com/cloudnative/2019/03/14/what-and-why-of-cluster-api/
● https://blog.heptio.com/the-kubernetes-cluster-api-de5a1ff870a5
● https://github.com/kubernetes-sigs/cluster-api
● https://cluster-api.sigs.k8s.io/

32. ありがとうございます。