An exploration of the issues in using Kubernetes with suggestions about how to move forward.

1. Progress for big data in Kubernetes
TED DUNNING
Chief Application Architect

2. 2 © 2018 MapR Technologies, Inc. // MapR Confidential
Ted Dunning, PhD
Chief Application Architect, MapR Technologies
Board member, Apache Software Foundation
O’Reilly author
Email tdunning@mapr.com tdunning@apache.org
Twitter @ted_dunning
Contact Information

3. 3 © 2018 MapR Technologies, Inc. // MapR Confidential
kubernetes is coming!

4. 4 © 2018 MapR Technologies, Inc. // MapR Confidential
why?

5. 5 © 2018 MapR Technologies, Inc. // MapR Confidential
Source: Shippable.com http://blog.shippable.com/why-the-adoption-of-kubernetes-will-explode-in-2018
kubernetes = major community support

6. 6 © 2018 MapR Technologies, Inc. // MapR Confidential
every cloud supports kubernetes
https://www.sinax.be/en/aws/
https://www.westconcomstor.com/za/en/vendors/wc-vendors/microsoft-azure-EN-UK.html
https://www.g2crowd.com/products/google-kubernetes-engine-gke/details

7. 7 © 2018 MapR Technologies, Inc. // MapR Confidential
massive customer adoption rate

8. 8 © 2018 MapR Technologies, Inc. // MapR Confidential

9. 9 © 2018 MapR Technologies, Inc. // MapR Confidential
what is kubernetes?

10. 10 © 2018 MapR Technologies, Inc. // MapR Confidential
kubernetes (n.) - greek word for pilot or
helm

11. 11 © 2018 MapR Technologies, Inc. // MapR Confidential
kubernetes started life as a
successor to google’s borg
project...
https://cloud.google.com/security/encryption-in-transit/

12. 12 © 2018 MapR Technologies, Inc. // MapR Confidential
kubernetes is an ecosystem...
Source: Redmonk - http://redmonk.com/sogrady/2017/09/22/cloud-native-license-choices/

13. 13 © 2018 MapR Technologies, Inc. // MapR Confidential
container and resource orchestration engine...

14. 14 © 2018 MapR Technologies, Inc. // MapR Confidential
Source: Shippable.com http://blog.shippable.com/why-the-adoption-of-kubernetes-will-explode-in-2018
kubernetes won the container orchestration war...

15. 15 © 2018 MapR Technologies, Inc. // MapR Confidential
what is kubernetes?

16. 16 © 2018 MapR Technologies, Inc. // MapR Confidential
it runs containers

17. 17 © 2018 MapR Technologies, Inc. // MapR Confidential
what is a container?

18. 18 © 2018 MapR Technologies, Inc. // MapR Confidential
not a vm

19. 19 © 2018 MapR Technologies, Inc. // MapR Confidential
hardware
vm vs container
os
hypervisor
vm
os
libs
app
vm
os
libs
app
hardware
os
container
libs
app
container
libs
app
container
libs
app

20. 20 © 2018 MapR Technologies, Inc. // MapR Confidential
pets vs cattle
https://fwallpapers.com/view/cat-jeans
http://www.clipartpanda.com/clipart_images/free-clip-art-1083418

21. 21 © 2018 MapR Technologies, Inc. // MapR Confidential
pets vs cattle
- long lived
- name them
- care for them
- ephemeral
- brand them with #’s
- well..vets are expensive

22. 22 © 2018 MapR Technologies, Inc. // MapR Confidential

23. 27 © 2018 MapR Technologies, Inc. // MapR Confidential
File File File Container Image
chroot
container = image + isolation
cgroups
● cpu
● memory
● network
● etc.
namespaces
● pids
● mnts
● etc.

24. 28 © 2018 MapR Technologies, Inc. // MapR Confidential
containers are good

25. 29 © 2018 MapR Technologies, Inc. // MapR Confidential
containers are good

26. 30 © 2018 MapR Technologies, Inc. // MapR Confidential
containers have a problem

27. 31 © 2018 MapR Technologies, Inc. // MapR Confidential
you can never get away from
pets unless:
- you handle the problem of
container state
- you need an environment to
support cattle
MapR and kubernetes are the
solution

28. 32 © 2018 MapR Technologies, Inc. // MapR Confidential
solve port mapping hell
monitor running containers
handle dead containers
move containers so utilization improves
autoscale container instances to handle load
Things docker can’t (or won’t) do...

29. 33 © 2018 MapR Technologies, Inc. // MapR Confidential
Magical View of Kubernetes

30. 34 © 2018 MapR Technologies, Inc. // MapR Confidential
App 1
Kubernetes
Magical View of Kubernetes
Kubernetes starts application
containers “somewhere”

31. 35 © 2018 MapR Technologies, Inc. // MapR Confidential
Magical View of Kubernetes
App 1 App 3
Kubernetes
Later containers may be started
elsewhere due to “affinities”

32. 36 © 2018 MapR Technologies, Inc. // MapR Confidential
Magical View of Kubernetes
App 1 App 2 App 3
Kubernetes
Kubernetes provides super fast
naming via DNS so containers
can find each other

33. 37 © 2018 MapR Technologies, Inc. // MapR Confidential
Note that you don’t think about
which machine at all

34. 38 © 2018 MapR Technologies, Inc. // MapR Confidential
You don’t think about which
machine at all
No more names from The Hobbit
Just cattle

35. 39 © 2018 MapR Technologies, Inc. // MapR Confidential
Software engineering can be viewed as freezing bits
Initially, everything is possible, nothing is actual
We freeze the source
Then the binary
Then the package
Then the environment
Ultimately the system
The Impact of Kubernetes

36. 40 © 2018 MapR Technologies, Inc. // MapR Confidential

37. 41 © 2018 MapR Technologies, Inc. // MapR Confidential

38. 42 © 2018 MapR Technologies, Inc. // MapR Confidential

39. 43 © 2018 MapR Technologies, Inc. // MapR Confidential
git cc/ld
java/jar
Build
docker build
resources
config libraries
Package
helm package
Construct

40. 44 © 2018 MapR Technologies, Inc. // MapR Confidential
git cc/ld
java/jar
Build
docker build
resources
config libraries
Package
helm package
Construct
helm install/scale
Load balancer
Deploy

41. 45 © 2018 MapR Technologies, Inc. // MapR Confidential
This is glorious

42. 46 © 2018 MapR Technologies, Inc. // MapR Confidential
but we still have a problem

43. 47 © 2018 MapR Technologies, Inc. // MapR Confidential
state

44. 48 © 2018 MapR Technologies, Inc. // MapR Confidential
Not Done Yet
Load balancer

45. 49 © 2018 MapR Technologies, Inc. // MapR Confidential
Not Done Yet
Load balancer
Here’s the problem

46. 50 © 2018 MapR Technologies, Inc. // MapR Confidential
Not Really Ready at All
State in containers messes things up
Restarts lose the state
Replicating state makes services
complex
Application developers just aren’t
systems developers
State life-cycle doesn’t match app life-
cycle
Load balancer
Here’s the problem

47. 51 © 2018 MapR Technologies, Inc. // MapR Confidential
What is a Service Anyway?
Load balancer
RPC in

48. 52 © 2018 MapR Technologies, Inc. // MapR Confidential
Synchronous RPC-based services only serve one need
In a synchronous service it’s common to do some, defer some
But deferring work is hard in a synchronous world … we have to give up the return call
in some sense
This is the germ of streaming architecture
But … Not Entirely

49. 53 © 2018 MapR Technologies, Inc. // MapR Confidential
Deferred
What is a Service Anyway?
Load balancer
RPC in

50. 54 © 2018 MapR Technologies, Inc. // MapR Confidential
If I can hide details of who and where, I have a service
If I can hide details of deployment, I have a micro-service
If I can hide details of when, I have a streaming micro-service
Isolation is The Defining Characteristic

51. 55 © 2018 MapR Technologies, Inc. // MapR Confidential
Temporal and Geo Isolation
Producer Consumer isn’t even running

52. 56 © 2018 MapR Technologies, Inc. // MapR Confidential
Temporal and Geo Isolation
Producer Consumer

53. 57 © 2018 MapR Technologies, Inc. // MapR Confidential
Producer
Consumer
Temporal and Geo Isolation
Consumer could be an ocean away

54. 58 © 2018 MapR Technologies, Inc. // MapR Confidential
Files are important
• Config files, image files, archival data data
• Legacy applications like machine learning, web
Tables are important
• Critical to have random update for some applications
• Should scale transparently without dedicated cluster
Streams are important
• Should be co-equal form of persistence
We Need Multiple Forms of Persistence

55. 59 © 2018 MapR Technologies, Inc. // MapR Confidential
App 1 App 2 App 3

56. 60 © 2018 MapR Technologies, Inc. // MapR Confidential
App 1 App 2 App 3
stream
LogFile

57. 61 © 2018 MapR Technologies, Inc. // MapR Confidential
App 1 App 2 App 3
stream
LogFile

58. 62 © 2018 MapR Technologies, Inc. // MapR Confidential
App 1 App 2 App 3

59. 63 © 2018 MapR Technologies, Inc. // MapR Confidential
Global namespace across entire Kubernetes cluster
• Between clusters as well if possible
All three forms of primitive persistence
• Files, streams, tables
Inherently scalable
• Performance, cardinality, locality
Uniform access and control
• Path names for all objects, identical permission scheme
What Does This Data Platform Need to Have?

60. 64 © 2018 MapR Technologies, Inc. // MapR Confidential
Global namespace across entire Kubernetes cluster
• Between clusters as well if possible
All three forms of primitive persistence
• Files, streams, tables
Inherently scalable
• Performance, cardinality, locality
Uniform access and control
• Path names for all objects, identical permission scheme
Oh…. got that already. Just need to wire it up to Kubernetes
What Does This Data Platform Need to Have?

61. 65 © 2018 MapR Technologies, Inc. // MapR Confidential

62. 66 © 2018 MapR Technologies, Inc. // MapR Confidential

63. 67 © 2018 MapR Technologies, Inc. // MapR Confidential
kubelet
docker
pod
Normally pods interact
directly with node resources

64. 68 © 2018 MapR Technologies, Inc. // MapR Confidential
kubelet
docker
pod
plugin We can install a volume
plugin (recently introduced)

65. 69 © 2018 MapR Technologies, Inc. // MapR Confidential
mapr-
fs
kubelet
docker
pod
plugin
fuse
This allows uniform access
to files, tables and streams

66. 70 © 2018 MapR Technologies, Inc. // MapR Confidential
Where does that take us?

67. 71 © 2018 MapR Technologies, Inc. // MapR Confidential
Installation of plugin is K8S level operation
• No per-node attention required
Use of plugin is overlay operation
• No change needed for an container
• Any Helm chart can use the plugin for conventional file access
Can share storage/compute or isolate or scale independently
Consequences

68. 72 © 2018 MapR Technologies, Inc. // MapR Confidential
State is no longer a dirty word for Kubernetes
HPC can run on K8S
Boring things can run on K8S without storage appliances
Previously crazy ideas can now be valuable
Complexity is largely not visible
More Consequences

69. 73 © 2018 MapR Technologies, Inc. // MapR Confidential
Container orchestration is awesome

70. 74 © 2018 MapR Technologies, Inc. // MapR Confidential
Container orchestration is awesome
Data orchestration is, too

71. 75 © 2018 MapR Technologies, Inc. // MapR Confidential
Cloud as-is: No unified data access or security concepts
Public Cloud
Application
✓
API
API Connector
Single cloud vendor strategy:
• Vendor lock in
• No failover in case of global outage
• Limited Edge capabilities
AWS Services:
• Kinesis & Elastic MapReduce
• Redshift & DynamoDB
• S3 & Glacier

72. 76 © 2018 MapR Technologies, Inc. // MapR Confidential
Cloud as-is: No unified data access or security concepts
Public Cloud
Application
✓
API
API Connector
AWS Services:
• Kinesis & Elastic MapReduce
• Redshift & DynamoDB
• S3 & Glacier
API
Azure Services:
• HD Insight
• SQL Server & CosmosDB
• Blob & DataLakeStore
Multi cloud strategy:
• Complex data movement between clouds
• On any other cloud:
• Different API‘s: application breaks
• Different Security concept

73. 77 © 2018 MapR Technologies, Inc. // MapR Confidential
Cloud as-is: No unified data access or security concepts
Edge Private Cloud
On Premise
Public Cloud Public Cloud Public Cloud
API
Application
✓
API API API API
API Connector
Multi cloud strategy:
• Complex data movement between clouds
• On any other cloud:
• Different API‘s, application breaks
• Different Security concept

74. 78 © 2018 MapR Technologies, Inc. // MapR Confidential
Open APIs
How a “Media Company” is Unifying Compute Environments
Application
• Unified Security Model
• Data access decoupled from physical
storage location. Globally.
• No lock-in to proprietary APIs
• Full openness
• Data made portable
API Connector
✓
GLOBAL DATA MANAGEMENT
Edge Private Cloud
On Premise
Public Cloud Public Cloud Public Cloud
Uniform computing
environment everywhere

75. 79 © 2018 MapR Technologies, Inc. // MapR Confidential
Open APIs
How “Manufacturing Company” is Orchestrating Data
Application
• Unified Security Model
• Data access decoupled from physical
storage location. Globally.
• No lock-in to proprietary APIs
• Full openness
• Data made portable
API Connector
✓
GLOBAL DATA MANAGEMENT
Edge Private Cloud
On Premise
Public Cloud Public Cloud Public Cloud
Platform level
data replication

76. 80 © 2018 MapR Technologies, Inc. // MapR Confidential
Tier 1 Bank #1 Creating a Global Filesystem
/mapr/edge1
/mapr/edge2
/mapr/edge3
/mapr/newyork
/mapr/amsterdam
/mapr/azure /mapr/gcp
/mapr/aws-eu-west
NFS POSIX HDFSREST
HOT
WARM
COLD
/mapr
Kafka JSON HBASE SQL S3
Application
✓
Global access
to local data

77. 81 © 2018 MapR Technologies, Inc. // MapR Confidential
Tier 1 Bank #2: Creating a Universal Application Platform
Application
GLOBAL DATA MANAGEMENT
Edge Private Cloud
On Premise
Public Cloud Public Cloud Public Cloud
PodPod Pod Image Classification using
Tensorflow in a Docker container
Classic ETL
Scheduling & Scaling
MapR Kubernetes Volume Driver
Single pane of glass to
control jobs anywhere

78. 82 © 2018 MapR Technologies, Inc. // MapR Confidential
Additional Resources
O’Reilly report by Ted Dunning & Ellen Friedman © September 2017
Read free courtesy of MapR:
https://mapr.com/ebook/machine-learning-logistics/
O’Reilly book by Ted Dunning & Ellen Friedman
© March 2016
Read free courtesy of MapR:
https://mapr.com/streaming-architecture-using-
apache-kafka-mapr-streams/
Ted Dunning & Ellen Friedman
Model Management in theReal World
MachineLearning
Logistics

79. 83 © 2018 MapR Technologies, Inc. // MapR Confidential
Additional Resources
O’Reilly book by Ted Dunning & Ellen Friedman
© June 2014
Read free courtesy of MapR:
https://mapr.com/practical-machine-learning-new-look-
anomaly-detection/
O’Reilly book by Ellen Friedman & Ted Dunning
© February 2014
Read free courtesy of MapR:
https://mapr.com/practical-machine-learning/

80. 84 © 2018 MapR Technologies, Inc. // MapR Confidential
AI & Analytics in Production:
How to Make It Work
Free book signing with authors Ted Dunning & Ellen Friedman
MapR stand #145:
Tues 1:00 pm – 1:45 pm
Wed 1:00 pm – 1:45 pm
Download free pdf via MapR:
https://mapr.com/ebook/ai-and-analytics-in-production/