Kube Your Enthusiasm

© Copyright 2018 Pivotal Software, Inc. All rights Reserved.
Paul Czarkowski
pczarkowski@pivotal.io
Twitter: @pczarkowski
Kube Your
Enthusiasm

Cover w/ Image
Topics
■ Platforms
■ Containers
■ Kubernetes
■ Helm
■ Spinnaker
■ Operators
■ Pivotal Container Service
■ Cloud Native Operations

What is a platform ?
https://en.wikipedia.org/wiki/Computing_platform

Different platforms
abstract differently

Hardware
IaaS
CaaS
PaaS
FaaS
HPE, Dell,
IBM, Lenovo
AWS, Microsoft
Azure, GCP,
VMware
PKS, GKE,
OpenShift, AWS
Fargate, Kubernetes
PCF, Azure App
Service, Heroku
AWS Lambda, Azure
Functions, OpenWhisk,
kubeless, PFS

A modern software
platform provides
API driven compute
resources.

API
Users
Storage Compute NetworkDatabase AccessArtifacts
Creative Commons
[1] Jon Trillana
[2] Simon Child
1 2

API
Users
Systems
Admin
Network
Engineer
SecurityDBA QA
Storage
Admin

Traditional
Ticket
Based
Human
Toil
IaaS
Hardware
Platform
PXE boot ?
15
More Control Less Control
Less Efficiency More Efficiency

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Container Runtime
Container Hosts
Infrastructure
Platform
Infrastructure
As
Code
IaaS API
Config
Management
IaaS
Hardware
Platform
PXE boot ?
16

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
App → to the
Platform
Container Runtime
Container Hosts
PaaS
Application Platform
Infrastructure
Platform
Application
Platform
Infrastructure
As
Code
IaaS API
CF APIConfig
Management
IaaS
Hardware
Platform
PXE boot ?
17

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
Infrastructure
Platform
Container
Platform
Application
Platform
Infrastructure
As
Code
IaaS API
CF API
K8s API
Config
Management
Deployment
Manifest
IaaS
Hardware
Platform
PXE boot ?
18

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
Infrastructure
Platform
Container
Platform
Application
Platform
Infrastructure
As
Code
IaaS API
CF API
K8s API
Config
Management
Deployment
Manifest
IaaS
Hardware
Platform
PXE boot ?
19
?????
PaaS
Function
Platform
??? API

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
Infrastructure
Platform
Container
Platform
Application
Platform
Infrastructure
As
Code
IaaS API
CF API
K8s API
Config
Management
Deployment
Manifest
IaaS
Hardware
Platform
PXE boot ?
20
?????
PaaS
Function
Platform
??? API

Traditional
Ticket
Based
Human
Toil
Build App
Artifact
Build App
Container(s)
App → to the
Platform
Container Runtime
Container Hosts
CaaS
Container
Orchestrator
PaaS
VMware
aws/gce/azure
Pivotal
Container Service
Pivotal
App Service
Infrastructure
As
Code
Pivotal Cloud Foundry 2.0
IaaS API
CF API
K8s API
Config
Management
Deployment
Manifest
IaaS
Dell
Or whatever
PXE boot ?
21
?????
PaaS
Pivotal
Function Service
??? API

Build App
Container(s)
CaaS
Container
Orchestrator
Pivotal
Container Service
Pivotal Cloud Foundry 2.0
K8s API
Deployment
Manifest
22

APP
APP
APP
APP
Gitlab Concourse Spinnaker

Saurabh Gupta. "Containers and Pivotal Cloud Foundry" 2016.

FROM maven:3.6-jdk-11-slim as BUILD
COPY . /src
WORKDIR /src
RUN mvn install -DskipTests
FROM openjdk:11.0.1-jre-slim-stretch
EXPOSE 8080
WORKDIR /app
ARG JAR=hello-0.0.1-SNAPSHOT.jar
COPY --from=BUILD /src/target/$JAR /app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

$ docker build -t paulczar/hello .
$ docker push paulczar/hello
$ docker pull paulczar/hello
$ docker run -d -p 8080:8080 paulczar/hello

Hardware
IaaS
CaaS
PaaS
FaaS
Strategic goal: Push as many workloads as technically
feasible to the top of the platform hierarchy
Higher flexibility and
less enforcement of
standards
Lower development
complexity and higher
operational efficiency

Worker
Master
API
Server
Users
Control Plane
Data Plane
etcd
Cloud Ctrl
Manager
Kubelet
kube-proxy
docker
Scheduler
Controller
Manager

Master
Master
Master
API
Server
Users
Control Plane
Data Plane
etcd
Cloud Ctrl
Manager
Worker
Kubelet
kube-proxy
docker
Scheduler
Controller
Manager
Worker
Kubelet
kube-proxy
docker
Worker
Kubelet
kube-proxy
docker
Flannel Flannel Flannel

Unix Philosophy:
Do one thing. Do it well.

Imperative
$ kubectl run hello
--image=paulczar/go-hello-world
$ kubectl scale hello
--replicas=3
$ kubectl create service clusterip
hello --tcp=80:80

Declarative
$ kubectl apply -f hello-world.yaml

Imperative
apiVersion: v1
kind: Pod
metadata:
name: hello
spec:
containers:
- image: paulczar/go-hello-world
imagePullPolicy: Always
name: hello

● Pods
● Services
● Volumes

one or more containers that share
a network and storage

the minimum scalable unit
of your application

MASTER
Node 1 Node 2 Node 3 Node 4
hello
kubelet kubelet kubelet kubelet
Scheduler
Pod
Name: hello
Image: hello1

MASTER
hello
Scheduler
hello
Pod
Name: hello
Image: hello1

MASTER
hello-a
Scheduler
Controller
Manager
Replica Set
Name: hello
Image: hello1
Size: 3
hello-ghello-s
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-b
Image: hello1
Pod
Name: hello-c
Image: hello1

MASTER
hello-a
Scheduler
Controller
Manager
Replica Set
Name: hello
Image: hello1
Size: 3
hello-ghello-s hello-d
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-b
Image: hello1
Pod
Name: hello-d
Image: hello1

MASTER
hello-a
Scheduler
Controller
Manager
Replica Set
Name: hello
Image: hello1
Size: 5
hello-ghello-s hello-d
hello-t hello-z
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-a
Image: hello1
Pod
Name: hello-a
Image: hello1

MASTER
Scheduler
Controller
Manager
Replica Set
Name: hello
Image: hello1
Size: 0

MASTER
hello-A-c
Scheduler
Controller
Manager
Deployment
Name: hello
Image: hello1
Size: 3
hello-A-ghello-A-s
Replica Set
Name: hello-A
Image: hello1
Size: 3

MASTER
hello-A-c
Scheduler
Controller
Manager
Deployment
Name: hello
Image: hello2
Size: 3
hello-A-ghello-A-s
Replica Set
Name: hello-A
Image: hello1
Size: 3
Replica Set
Name: hello-B
Image: hello2
Size: 3
hello-B-g
hello-B-r hello-B-c

MASTER
Scheduler
Controller
Manager
Deployment
Name: hello
Image: hello2
Size: 3
Replica Set
Name: hello-A
Image: hello1
Size: 0
Replica Set
Name: hello-B
Image: hello2
Size: 3
hello-B-g
hello-B-r hello-B-c

MASTER
hello-1
Scheduler
Controller
Manager
StatefulSet
Name: db
Image: hello1
Size: 3
Pod
Name: hello-1
Image: hello1

MASTER
hello-1
Scheduler
Controller
Manager
StatefulSet
Name: hello
Image: hello1
Size: 3
hello-2
Pod
Name: hello-1
Image: hello1
Pod
Name: hello-2
Image: hello1

MASTER
hello-1
Scheduler
Controller
Manager
StatefulSet
Name: hello
Image: hello1
Size: 3
hello-3hello-2
Pod
Name: hello-1
Image: hello1
Pod
Name: hello-2
Image: hello1
Pod
Name: hello-3
Image: hello1

MASTER
db-1
Scheduler
Controller
Manager
StatefulSet
Name: db
Image: cassandra
Size: 3
db-3db-2
Pod
Name: hello-a
Image:
Pod
Name: hello-b
Image:
Pod
Name: db-1
Image: ... vol vol vol
Pod
Name: hello-a
Image:
Pod
Name: hello-b
Image:
PVC
Name: db-1
Image: ...

$ kubectl run hello
--image=paulczar/hello
-- port=8080

● kubectl run created a deployment “deployments.apps/hello”
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/hello 1 1 1 1 1m
● The deployment created a replicaset “replicaset.apps/hello-64f6bf9dd4”
NAME DESIRED CURRENT READY AGE
replicaset.apps/hello-64f6bf9dd4 1 1 1 1m
● Which created a pod “pod/hello-64f6bf9dd4-tq5dq”
NAME READY STATUS RESTARTS AGE
pod/hello-64f6bf9dd4-tq5dq 1/1 Running 0 2s

$ kubectl scale --replicas=3
deployment/hello

$ kubectl scale --replicas=3 deployment/hello
deployment.extensions/hello scaled
$ kubectl get all
pod/hello-64f6bf9dd4-2bndq 1/1 Running 0 15m
pod/hello-64f6bf9dd4-4kq9l 0/1 ContainerCreating 0 2s
pod/hello-64f6bf9dd4-8lkcs 1/1 Running 0 5s

$ kubectl edit deployment hello
...
spec:
containers:
- env:
- name: MESSAGE
value: HELLO I LOVE YOU!!!!
image: paulczar/go-hello
imagePullPolicy: Always
name: hello

$ kubectl get all
pod/hello-5c75b546c7-4lwnn 1/1 Running 0 1m
pod/hello-5c75b546c7-bwxxq 1/1 Running 0 1m
pod/hello-5c75b546c7-sl2pg 1/1 Running 0 1m
replicaset.apps/hello-5c75b546c7 3 3 3 1m

$ kubectl port-forward deployment/hello 8080
Forwarding from 127.0.0.1:8080 -> 8080
$ curl localhost:8080
<html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE
YOU!!!!!</body></html>

$ kubectl expose deployment
hello --type=LoadBalancer
--port 80 --target-port 8080

kubectl expose deployment hello
● creates a service with a ClusterIP that acts as an internal loadbalancer to all
pods in the “hello” deployment
--type=LoadBalancer
● Creates a NodePort
● Configures a LoadBalancer to access the pods via the NodePort
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello LoadBalancer 10.39.248.123 35.184.17.129 80:30468/TCP 5m
$ curl 35.184.17.129
<html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE
YOU!!!!!</body></html>

Service
track Pods based on metadata and provides
connectivity and service discovery (DNS, Env
variables) for them.
Type
ClusterIP (default) exposes service on a
cluster-internal IP.
Container
Container
Pod
app=bacon
Container
Container
Pod
app=bacon
Service
app=bacon 10.3.55.7

Service
Type
NodePort extends ClusterIP to expose services on
each node’s IP via a static port.
Container
Container
Pod
app=bacon
Container
Container
Pod
app=bacon
Service
app=bacon 10.3.55.7
192.168.0.5:4530
K8s Worker K8s Worker
192.168.0.6:4530

Service
Type
LoadBalancer extends NodePort to configure a cloud
provider’s load balancer using the
cloud-controller-manager.
Container
Container
Pod
app=bacon
Container
Container
Pod
app=bacon
Service
app=bacon 10.3.55.7
192.168.0.5:4530
K8s Worker K8s Worker
192.168.0.6:4530
Load Balancer
33.6.5.22:80

Ingress
a controller that manages an external entity to provide
load balancing, SSL termination and name-based
virtual hosting to services based on a set of rules.
Ingress
Service
app=bacon
https://example.com
Service
app=eggs
/bacon /eggs

Container
Container
Pod
Volume
Is [effectively] a Directory, possibly with data in it,
available to all containers in a Pod.
Usually Shares lifecycle of a Pod (Created when Pod
is created, destroyed when Pod is destroyed).
Persistent Volumes outlive Pods.
Can be mounted from local disk, or from a network
storage device such as a EBS volume, iscsi, NFS, etc.

$ kubectl create configmap hello
--from-literal=’message=Hello S1T’

kubectl create configmap hello --from-file=index.html
● creates a configmap called “hello” containing the contents index.html
$ kubectl get configmap hello -o yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: hello
data:
index.html: "<html>n<head>nt<title>Hello to my
friends</title>n</head>n<body>ntHello
to my friendsn</body>n</html>nn"

kubectl create secret generic hello --from-file=index.html
● creates a secret called “hello” containing a base64 hash of contents index.html
$ kubectl get secret hello -o yaml
apiVersion: v1
kind: Secret
metadata:
name: hello
data:
index.html:
PGh0bWw+CjxoZWFkPgoJPHRpdGxlPkhlbGxvIHRvIG15IGZyaWVuZHM8L3RpdGxlPgo8L2hlYWQ+Cjxib2R5
PgoJSGVsbG8gdG8gbXkgZnJpZW5kcwo8L2JvZHk+CjwvaHRtbD4KCg==

Provides key-value pairs to be injected into a pod much like user-data is injected into a Virtual
Machine in the cloud.
Allows you to do last minute configuration of applications running on Kubernetes such as
setting a database host, or a admin password.
ConfigMaps store values as strings, Secrets store them as byte arrays (serialized as base64
encoded strings).
Secrets are [currently] not encrypted by default. This is likely to change.
Can be injected as files in a Volume, or as Environment Variables.
ConfigMaps/Secrets (user-data)

Helm is the best way to
find, share, and use
software built for Kubernetes
@pczarkowski

custom
load balancer
Chart.yaml
Values.yaml
templates/
ci
services
db

Discover & launch great
Kubernetes-ready apps
Search charts
231 charts ready to deploy
Wordpress, Jenkins, Kubeless...
Secure | https://hub.kubeapps.com
@pczarkowski

apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Chart.name }}-cm
data:
db: {{ .Value.db }}
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: {{ .Chart.name }}-app
labels:
app: {{ .Chart.name }}
...
...
spec:
containers:
- image: paulczar/hello
name: hello
volumeMounts:
- name: config
mountPath: /etc/hello
volumes:
- name: config
configMap:
name: {{ .Chart.name }}-cm
apiVersion: v1
kind: Service
metadata:
name: {{ .Chart.name }}-svc
labels:
app: {{ .Chart.name }}-world
spec:
ports:
- port: {{ .Value.port }}
protocol: TCP
targetPort: 8080
selector:
app: {{ .Chart.name }}-world
type: NodePort
@pczarkowski

$ helm install --name staging .
--set db=’user:pass@staging.mysql/dbname’
$ helm install --name production .
--set db=’user:pass@production.mysql/dbname’
@pczarkowski

https://medium.com/netflix-techblog/announcing-ribbon-tying-the-netflix-mid
-tier-services-together-a89346910a62

https://giphy.com/gifs/frustrated-keyboard-g8GfH3i5F0hby
@pczarkowski

https://unsplash.com/photos/WHWYBmtn3_0
@pczarkowski

APP
APP
APP
APP
Gitlab Concourse Spinnaker
@pczarkowski

Cluster Management
● Server Group
● Cluster
● Applications
● Load Balancer
● Firewall
Pipelines
● Pipeline
● Stage
● Deployment Strategies
@pczarkowski

Multi-Cloud Inventory
● Server Group
● Cluster
● Applications
● Load Balancer
● Firewall
Actions and Reactions
● Pipeline
● Stage
@pczarkowski

Cluster Management
● Server Group
● Cluster
● Applications
● Load Balancer
● Firewall
Deployment Management
● Pipeline
● Stage
Deployment Strategies

Spinnaker
Cloud
API
App App App
@pczarkowski

Halyard
https://en.wikipedia.org/wiki/Halyard

Watchers
Watch the Kubernetes
API for changes to
resources and perform
arbitrary actions.

Watchers
Prometheus watches
Services and Pods for
certain annotations ...

Watchers
Spring Cloud Kubernetes
watches Services and Endpoints
to do service discovery on
kubernetes.

It also watches and reads
ConfigMaps to allow for dynamic
configuration of your
applications.

Dynamic Access Control
After a request is
authorized it goes
through Admission
Control

Image Policy Webhook

Admission Webhook

Initializers

Custom Controllers
Kubernetes functionality
is implemented using
controllers.

Custom Controllers
The External DNS
Controller

Custom Controllers
The Cert Manager
Controller

Operators
GCP Cloud Compute
Operator
https://github.com/paulczar/gcp-cloud-compute-operator

Operators
https://github.com/paulczar/gcp-cloud-compute-operator

> kubectl
Storage NetworkingCompute
Kubernetes Dashboard
Dev / Apps IT / Ops
App User
Kubernetes is a Runtime for Containerized Workloads

Dev / Apps
App User
IT / Ops
> kubectl
Load Balancing / Routing
Container Image
Registry
App Monitoring
App Logging
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
Command Line
/ API
Management
GUI
Monitoring GUI
...but Kubernetes alone is not enough for enterprises

Pivotal Container Service (PKS) provides what’s missing
Dev / Apps
App User
IT / Ops
> kubectl
Container Image
Registry
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
App Logging
PKS Control Plane
> pks
Operations Manager
vRealize Operations*
*integration
GCP Service Broker

Dev / Apps
App User
IT / Ops
> kubectl
Container Image
Registry
K8S Updates Log & Monitor Backup & Restore
External
Data Services
Cluster
Provisioning
App Logging
PKS Control Plane
GCP Service Broker
> pks
Operations Manager
vRealize Operations*
*integration
on any Cloud

Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
OpenServiceBrokerAPI
Repair
— CVEs
Repave Rotate
— Credhub

BOSH
Reliable and consistent operational experience for any cloud.
BOSH
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
PKS Control Plane
Use the PKS CLI and API to
create, operate, and scale your
clusters.
VMware GCP Azure Openstack AWS
PKSControlPlane
Built with open-source
Kubernetes
Constant compatibility with the
latest stable release of Google
Kubernetes Engine—no
proprietary extensions.
Harbor
An enterprise-class container registry.
Includes vulnerability scanning, identity
management, and more.
NSX-T
Network management, security, and
load balancing out-of-the-box with
VMware NSX-T. Multi-cloud,
multi-hypervisor.
Enterprise-Grade Kubernetes

What PKS adds to Kubernetes
PKS
value-added
features
Built into
Kubernetes
Multi-container pods
Stateful Sets of pods
Persistent disks
Single tenant ingress
Pod scaling and high availability
Rolling upgrades to pods
Cluster provisioning and scaling
Embedded, hardened Operating System
Monitoring and recovery of cluster VMs and processes
Rolling upgrades to cluster infrastructure
Secure multi-tenant ingress
Secure container registry

PKS Vision
To provide enterprise customers with the ability to
safely and efficiently deliver container services
on their preferred infrastructure so that they can
excel in their market with a cloud native platform

PKS does for your Kubernetes
what
Kubernetes does for your apps

Operational
Efficiency
● Employ 500:1 developer
to operator ratio
● Perform zero-downtime
upgrades
● Runs the same way
on every public/private
cloud
Developer
Productivity
Comprehensive
Security
● Accelerate feedback
loops by improving
delivery velocity
● Focus on applications,
not infrastructure
● Give developers the
tools and frameworks
to build resilient apps
● Adopt a
defense-in-depth
approach
● Continuously update
platforms to limit
threat impact
● Apply the 3 R’s →
repair, repave, rotate
● Run platforms that
stays online under
all circumstances
● Scale up and down, in
and out, through
automation
● Deploy multi-cloud
resilience patterns
High Availability
Platform Team Delivering Real Value

BOSH
Pivotal Container Service
Platform Ops
Pivotal
Ops Manager
PKS tile
upload
and config
Pivotal
Network
Install
Installing PKS
NSX-T

PKSControlPlane
GCP
Service Broker
Harbor
BOSH
Platform Ops
deploy
Install
Pivotal
Ops Manager
Installing PKS
NSX-T

… or
...
Pivotal
NetworkConcourse
pipelinePlatform Ops
Execute
Verify pre-reqs
Provision
infrastructure
Download
binaries
Install
Product
Config
Install PKS

PKS User Interaction
● The PKS Management VM runs the PKS API
together with the Broker, UAA and a MySQL DB.
●
● The PKS API orchestrates the initial kubernetes
cluster deployments and scaling of those clusters.
● A single PKS VM can manage hundreds of
Kubernetes cluster.
● The PKS CLI is a single binary that can be installed
on a Mac, Windows, or Linux to drive the PKS API.
PKS CLI
PKS
Management
VM
PKS API

Creating a new K8s Cluster
Platform User
PKSControlPlane
CLI
API
PKS CREATE CLUSTER
BOSH
deploy
Kubernetes cluster
Create
Harbor
NSX-T
GCP SB
Master
Worker
WorkerWorker
etcd Worker
Master
etcd

Deploying a Kubernetes Cluster via PKS
BOSH
PKS CONTROL PLANE
PKS API
MySQL
PivotalOpsManager
Master / etcd
Worker 1
Worker 2
cluster
UAA BROKER

Availability Zone B
Availability Zone A
Health Management and HA
(1) Kubelet watches and restart containers
Kubelet
Kube-proxy
Pod
Pod
K8s Node
Pod
API Server
Kube Scheduler
K8s Master
Controller
Manager
Bosh agent
Bosh agent
Bosh Health
Manager
Watches and restarts VMs
Availability Zone A
Availability Zone B
4 levels of built-in High Availability
(2) BOSH agent watches and restarts processes
(3) BOSH HM watches and restarts VMs
(4) BOSH distributes deployments across AZs

Multi-Tenancy
PKSControlPlane
Kubernetes cluster
Kubernetes cluster
Harbor
GCP SB
NSX-T
BOSH
Kubernetes cluster
Master
Worker
Worker
etcd
Worker
Master
etcd
Worker
How to isolate and secure access from different tenants?

Deployment Topologies & Multi-Tenancy
Multi-cluster Single cluster
K8s Cluster A
K8s Cluster
BOSH
Namespace A
Namespace B
Namespace C
BOSH
K8s Cluster B
K8s Cluster C
NSX-T
cluster-based namespace-based
PKSControlPlane
PKSControlPlane

Multiple Kubernetes clusters deployed and managed
independently by BOSH
Independent networks with independent policies.
Each cluster has a separate Master and Workers, with
possibly different configs and resources (volumes,
namespaces, policies, affinity rules)
Provides complete isolation for multiple tenants
Single Kubernetes cluster deployed by BOSH
Different tenants use different Kubernetes Namespaces
NSX-T is used to logically isolate each tenant’s network
(Namespace)
Provides logical multi-tenant isolation for managing a
single cluster
Multi-cluster Single cluster
cluster-based namespace-based
Deployment Topologies & Multi-Tenancy

Scaling a Kubernetes Cluster
Platform User
PKSControlPlane
CLI
API
PKS SCALE CLUSTER
BOSH
deploy
Kubernetes cluster
Scale
Harbor
NSX-T
GCP SB
Master
Worker
WorkerWorker
etcd
Worker
Master
etcd
Worker

A new security patch is released for Kubernetes.
Pivotal releases a new CVE for PKS within a few hours.
The Platform Operator can then apply the CVE with no
platform downtime.

BOSH
Platform Ops
Pivotal
Ops Manager
PKS tile
upload
and config
Pivotal
Network
Update
Platform Ops updates PKS
New!!
PKSControlPlane
GCP
Service Broker
Harbor
NSX-T
Kubernetes cluster
Master
Worker
WorkerWorker
etcd
Worker
Master
etcd
Worker

PKSControlPlane
GCP
Service Broker
Harbor
BOSH
Platform Ops
deploy
Update
Pivotal
Ops Manager
Platform Ops updates PKS
Rolling Updates
NSX-T
Kubernetes cluster
Master
Worker
WorkerWorker
etcd
Worker
Master
etcd
Worker

… or
...
Pivotal
NetworkConcourse
pipelinePlatform Ops
Execute
Verify pre-reqs
Verify current
install
Download
updated binaries
Rolling
Updates
Config
Update PKS

BUCKET 1
Independent Software Vendor (ISV) COTS
BUCKET 2
Middleware Vendor
BUCKET 3
.NET Core or .NET (Windows Server)
BUCKET 4
Legacy Java
BUCKET 5
Modern Java
Application
Prioritization
Criteria
Vendor provided software (ISV
or COTS) or no access to
source code
IBM Websphere, Weblogic,
Mulesoft, TIBCO etc
3-5 years old Java (under 7 years old)
Java (Spring / NO Application
Server Specific libraries)
Vendor provides PCF
buildpack, docker images or
kubernetes artifacts
Vendor provides PCF
buildpack, docker images,
kubernetes artifacts
Access to source code Access to source code Access to source code
Vendor availability to support
the migration
Vendor availability to support
the migration
Limited or no Windows
dependencies
Linux or Windows Server Linux Server
Limited or no access to the
code
Example
Example ISV product. Depends on MySQL
DB and stores large files on disk.
Example app that is built on WebSphere.
No dependency on WebSphere libraries.
Example app. 4 services built using .NET
core and uses Microsoft SQL Server.
Example app uses Java EE, fronted by API
gateway ISV product, uses OracleDB.
Example App uses Spring Boot, 6
Microservices, some legacy data sources
but there are behind an API.
Application 1
?
Application 2
?
Application
n…?
First Round: App Portfolio Identification by Bucket

TIME Methodology
TECHNICALQUALITY
BUSINESS VALUEWORSE BETTER
WORSEBETTER
Tolerate Invest
MigrateEliminate
* Gartner’s TIME methodology for Application Portfolio Rationalization
TECHNICAL QUALITY - Technical Debt Level
BUSINESS VALUE - Revenue / Cost Impact
Identify top 10s list

Source: "It's All About Delivering: A Journey From AWS to Cloud Foundry,"
Daniel Basten, Talanx, s1p 2018.

Sources: "Sky is the Limit for Cloud Foundry at AirFrance-KLM," Nathan Wattimena & Fabien Lebrere, AirFrance-KLM, Oct. 2018.; “Why
Change? Small batch thinking,” Coté, Sep. 2018; "Transformation Digitale de la Direction Enterprise France," Philippe Benaben, Gan Zifroni,
Nicolas Gilot, Orange France, July 2018.

https://medium.com/netflix-techblog/how-we-build-code-at-netflix-c5d9bd727f15

You’re a Product Team
The Platform is your product

Infra
Services
App
Platform
Change!!!
Platform
Team
Application
Team
Build common services
for App Teams
Take business
requirements and turn
them into features
IaaS
Virtual Infrastructure
Physical Infrastructure
Abstract infrastructure
complexity with easy
consumption
DBaaSELK
App2App1 App3
Middleware
ML
Creds/CertsMessaging
???
Container Services
Container Hosts | Kubernetes
Infrastructure
Team

Measure
AUTOMATE
Share
CULTURE
LEAN
LEAN

http://engineering.pivotal.io/post/transformation-roi/

Source: "Adopting PCF At An Automobile Manufacturer," Thomas Seibert and
Gregor Zurowski, s1p 2017.

PLATFORM VALUE STREAM AND METRICS
REPLATFORM > MODERNIZE > OPTIMIZE
ESTABLISH, MEASURE AND UPDATE
KEY OBJECTIVES AND RESULTS (OKRs)
SPEED & AGILITY STABILITY
SCALABILITY SAVINGS
$SECURITY
40-60%*
More Projects With
Same Staff
Millions
Annual Savings on
HW, SW and
Support
25-50%*
Fewer Support
Incidents
40%*
Faster Patching
Delivery @ Zero
Downtime
-90%*
Time to Scale
$
$
%
Measure and Share

Sample CIO Dashboard
60 Days
Avg Lead Time
500
Stories per week
10%
Apps on a CD
Pipeline to Prod
15ms
Avg Response Time
YTD
60 Mins
MTTR YTD
20%
% of Systems
Patched YTD
125 Mins
Total Impacted
User Minutes YTD
20
Releases in last
month
Speed Stability & Security

Kube Your Enthusiasm

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Kube Your Enthusiasm

Similar to Kube Your Enthusiasm (20)

More from VMware Tanzu

More from VMware Tanzu (20)

Recently uploaded

Recently uploaded (20)

Kube Your Enthusiasm