SpringOne Platform 2018 Using CredHub for Kubernetes Deployments Eugene Kiselev, Altoros; Peter Blum, Pivotal

1. Using CredHub for Kubernetes
Peter Blum
Pivotal
@_pblum
Eugene Kiselev
Altoros

2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
All Applications Have Secrets!
2

• For Databases
• For Messaging Queues
• For Internal Services
• Everything?

3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes Secrets
3

• Secrets enable developers not to
place secrets in manifests instead use
parameters
• Secrets are base64 encoded
• Secrets stored in etcd along with all
other K8s objects

4. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Configuring Credentials is Hard
Equifax: Website secured by `admin`, `admin`
Leaking Credentials is Easy
Uber: Code found on github which included usernames and
passwords
Detecting Credential Leaks is Hard
Equifax: Hackers accessed files on nearly half the U.S.
population.
G
enerate
Encrypt
Log
Access
Rotate
Story of CredHub

5. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Configuring Credentials is Hard
Equifax: Website secured by `admin`, `admin`
Leaking Credentials is Easy
Uber: Code found on github which included usernames and
passwords
Detecting Credential Leaks is Hard
Equifax: Hackers accessed files on nearly half the U.S.
population.
G
enerate
Encrypt
Audit
Rotate
Story of CredHub

6. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
CredHubCredHub
CLI
BOSH
Config Server
REST
Client
Authentication
Provider
Backing SQL
Database
Encryption
Provider
(HSM)
CredHub Architecture

7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Operations
• Get/Set/Generate/Delete Credential
• Get/Add/Delete Permission
• Interpolate (VCAP_SERVICES)
Authentication
• Mutual TLS
• OAuth2 with UAA
https://credhub-api.cfapps.io
• value - a simple string, used for configuration and other non-generated properties
• password - a simple string, used for generated secrets
• user - username and password pair
• json - a JSON object
• certificate - an object containing a root CA, certificate and private key
• rsa - an object containing an RSA public key and private key
• ssh - an object containing an SSH-formatted public key and private key
CredHub API
Credential types

8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Credhub features and benefits
● Stateless
● Scalable
● Supports HSM
● 100% free
● Granular Access
control
● Several creds types (passwords,
keys, certs)
● Json cred type for customization

9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Credhub deployment types
Credhub NEAR Kubernetes

10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Credhub(s) INSIDE the k8s
Great. Smth else for k8s?
??

11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Helm. Package manager for kubernetes
Helm
Package Manager (pip or npm)
Focus on Managing Applications
Charts
How to install applications
How to upgrade applications
Value
Fast & Simple
Full representation of K8s API
All yaml -- no golang required
Huge existing library of application
charts

12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
postgreSQL
UAA
credhub
deployment
secret
service
deployment secret config-map
service
Credhub chart architecture

13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
K8s Secrets
ETCD Encrypting isn’t easy or the
default
Base64 encoding is NOT
encryption
Secrets exposed when master
VMs are compromised
K8s + CredHub Idea
Why CredHub
Organization wide solution
Auditing
Credential Generation
Storage
Encryption
Native integration into CF and now
K8s!
Pre-existing with Pivotal
Application Service, or OSS Cloud
Foundry.

14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Integration with Kubernetes
Generating Credentials
Goal:
Secured Secret
Implementation:
Custom Resource Definitions
Operators API
API Extension Server
Kubebuilder
Service Catalog
Injecting Credentials
Goal:
Pull Creds at Runtime and Place Into Pod
Implementation:
Mutating Admission Webhook
Custom Kubernetes Controller
Init Containers

15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Kube - API
(Master)
Requested
Container(s)Credhub
Secrets
Volume
CredHub
Webhook
Credhub
Controller
Credhub Init
Container
Requested Pod
Control Plane Runtime Plane
Credhub Service
credhub.pivotal.io
Super Secret
Credential
Springone Namespace
x509 Cert
(ns:springone)
Integration with Kubernetes

16. > Demo!

17. Learn More. Stay Connected.
How to deploy, helper deployment scripts, etc...
PRs Welcomed!
https://github.com/Oskoss/kubernetes-credhub
17
#springone@s1p

18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Ideas for Credhub Operator
How control works:
● Operator listens for k8s create secret command
● Intercepts command
● Determines what namespace you are in
● Stores secret in credhub adding a permission that ONLY allows namespace X
to access secret
1
8


19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Ideas for Credhub Operator
How runtime works:
• Init container looks at your k8s deployment
• If init container finds secrets
• Init container authenticates with credhub
• Init container pulls secrets for you
• Init container puts secrets into the environment for you
• We could even put it onto disk if we wanted to but this would be
something we could do if we have time
1
9


20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Ideas for Credhub Operator
Open questions:
Secrets are scoped to namespaces. Are we okay with allowing everyone in the
namespace access all secrets in credhub? (Not a bad thing, if required can create
new namespace per application)
Do we need to create a new user every time a new namespace is created? No, at
least to start we can have a `credhub_admin` that creates and manages all
secrets but tags the namespace with the secret
How about K8s operator to deploy credhub itself? Eugine is going to do some
research around this to determine if we deploy new credhub per namespace or
have a single one per cluster. (Definitely will deploy credhub in a cluster)
How about deletion? Let’s put this in the icebox for now.
2
0


21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/

22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Credhub bosh deployment
Bosh - the main deployment tool
● Mature
● Secure
● Flexible
● Reliable
● Comfortable (of course if you comfortable with bosh)
Credhub release is living here:
https://github.com/pivotal-cf/credhub-release

23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Helm - client
Tiller - server
Helm architecture
Tiller

24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Helm chart
● k8s deployment objects representation
● Parameters
● And set of dependencies

25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Helm chart deployment
Helm-chart filesvariables
Container image

26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Prerequisites
No need to push subchart, local dir accepted

27. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Credhub chart structure

28. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
uaa.yml as config-map
Sensitive data as secrets
UAA chart
config-map
UAA.yaml
secret

29. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
UAA chart deployment

30. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
UAA chart secrets

31. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
UAA chart config-map

32. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Pivotal Logos on Light Background
Looking for more Pivotal logos, PCF services icons, or OSS logos?
Visit: brandfolder.com/pivotal-assets
3
2


33. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Sample Table
33
2007 2008 2009 2010
Category 1 2.4 6.4 4.1 6.6
Category 2 8.2 4.5 3.2 3.8
Category 3 4.6 3.2 1.9 9.6
Category 4 6.7 3.3 3.4 2.2
Category 5 4.3 5.6 7.1 3.4

34. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Design Assets

35. > Stay Connected.
<Your CTA>
<Related Session>
#springon
e
@s1p

36. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Brand Colors
SpringOne Platform
2018 Primary Colors
3
6
Spring
Brand Color
Pivotal
Brand Color

37. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
STOP! Download Fonts Now
PLEASE DOWNLOAD AND INSTALL PROXIMA NOVA FONTS BEFORE CREATING YOUR PRESENTATION. You can download
the fonts here…
https://brandfolder.com/pivotal Password: keepitsimple
Fonts included in the ZIP file:
Proxima Nova (headline and body text)
http://www.fontspring.com/support/installing/how-do-i-install-fonts-on-my-mac
http://www.fontspring.com/support/installing/how-do-i-install-fonts-on-my-windows-pc
3
7

38. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Body Slide - Dark Background
All body text is Proxima Nova Regular
• Subhead
• Level Two
• Level Three
• Level Four
3
8

39. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Two Columns – Dark Background
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque
3
9
Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Aenean
commodo ligula eget dolor. Aenean
massa. Cum sociis natoque penatibus
et magnis dis parturient montes,
nascetur ridiculus mus. Donec quam
felis, ultricies nec, pellentesque

40. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Code Slide
4
0
// This is Roboto Mono: 42pt or higher please
public class TransferServiceImpl implements TransferService {
public TransferServiceImpl(AccountRepository ar) {
this.accountRepository = ar;
}
…
}

41. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Pivotal Logos on Dark Background
41
Looking for more Pivotal logos, PCF services icons, or OSS logos?
Visit: brandfolder.com/pivotal-assets

42. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Event Logos
4
2
On dark
Horizontal Vertical
On light
On mint

43. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Spring Logo and Project Icons
43
Spring
Framework
Spring
Security
Spring
Data
Spring
Batch
Spring
Integration
Project
Reactor
Spring
AMQP
Spring
Hateoas
Spring
Mobile
Spring
Android
Spring
Social
Spring
Web Services
Spring
Web Flow
Spring
XD
Spring
Boot
Spring
LDAP
Spring Tool
Suite
Spring Cloud
Data Flow
Spring
Kafka
Spring
Cloud