Ian huston getting started with cloud foundry

Getting Started with
Cloud Foundry
•  Ian Huston, Data ScientistIan Huston, Data Scientist
ODSC London 2016

2
© Copyright 2015 Pivotal. All rights reserved.
Who am I?
Ÿ  Ian Huston
Ÿ  @ianhuston
Ÿ  www.ianhuston.net
Ÿ  More resources: 
https://github.com/ihuston/
python-cf-examples
Ÿ  Data Scientist
Ÿ  Previously a theoretical
physicist using Python for
numerical simulations & HPC

3
Aims for this talk
Goals:
Ÿ  What is Cloud Foundry and how can it help?
Ÿ  Get a taste of CF as a developer/data scientist
Anti-goals:
Ÿ  Understand CF from an operations viewpoint
Ÿ  Install & run a full Cloud Foundry installation

4
Talk Resources
For the full 2 hour tutorial clone this repo:
https://github.com/ihuston/python-cf-examples

What is Cloud Foundry?
http://cloudfoundry.org

Open Source
Multi-Cloud Platform

Simple App Deployment,
Scaling & Availability

Cloud Applications Haiku

Here is my source code
Run it on the cloud for me
I do not care how.
-  Onsi Fakhouri
@onsijoe

How can data scientists use CF?

Data Services
Easy control of incoming data

11
Multi-cloud
Ÿ  Same applications running across diﬀerent cloud providers:
Private
Public
Hosted

12
Cloud Foundry Foundation
PLATINUMGOLDSILVER

13
cf push
Pushing your ﬁrst app

14
Challenge 1: Pushing your first application
Ÿ  Choose PWS API endpoint: $ cf api https://api.run.pivotal.io
Ÿ  Login: $ cf login
Ÿ  Code directory: 01-simple-python-app
Ÿ  Deploy with
$ cf push
Ÿ  Check it worked at http://myapp-RANDOM-WORDS.cfapps.io
Ÿ  Turn off your app when finished with cf stop myapp (but not yet!)

15
Simple Flask App Demo
Ÿ  Simple one page “Hello World” web app
Ÿ  Video: https://www.youtube.com/watch?v=QOfD6tnoAB8
Ÿ  Demonstrates:
–  Installation of requirements
–  Scaling properties
Ÿ  Need to Provide:
–  App files
–  Dependencies listed in requirements.txt file
–  Optional manifest.yml file with configuration for deployment

C
F

R
O"
U"
T"
E"
R
2. Set up domain
Cloud
Controller
Instance
1. Upload code
4. Copy app into
containerised
instances
3. Install Python
&
Dependencies
5. Start app
and accept
connections
Send request to URL
WHAT JUST
HAPPENED?
Source
Code
Instance
$ cf push
Browser
5. Load balance
between
instances

17
What just happened?
1.  Application code is uploaded to CF
2.  Domain URL is set up ready for routing
3.  Cloud controller builds application in container:
–  Python interpreter selected
–  Dependencies installed with pip
4.  Container is replicated to provide instances
5.  App starts and Router load balances requests
Ÿ  See what’s happening using logs: $ cf logs myapp --recent

18
Python on Cloud Foundry
Ÿ  First class language (with Go, Java, Ruby, Node.js, PHP)
Ÿ  Automatic app type detection
–  Looks for requirements.txt or setup.py
Ÿ  Buildpack takes care of
–  Detecting that a Python app is being pushed
–  Installing Python interpreter
–  Installing packages in requirements.txt using pip
–  Starting web app as requested (e.g. python myapp.py)

19
Scaling memory and instances
Ÿ  We can scale our application as needed in terms of memory
and number of instances:
$ cf scale myapp –i 5
$ cf scale myapp –m 256M
Ÿ  Check app in browser to see diﬀerent instances being used.
Ÿ  Scale back down with $ cf scale myapp –i 1

20
How does this work?
Ÿ  Containerised application is cached and ready to be
deployed.
Ÿ  Scaling number of instances replicates container and load
balances requests across all instances.
Ÿ  Scaling memory requires restarting app.
Ÿ  Auto-scaling is also possible.

21
Buildpacks
Pushing data science apps

22
What are buildpacks?
Ÿ  Idea and format from Heroku
Ÿ  Responsible for doing whatever is necessary to get your
app running.
Ÿ  Buildpacks take care of
–  Detecting which type of application is being pushed
–  Installing the appropriate run-time
–  Installing required dependencies or other artifacts
–  Starting the application as requested

23
Containers vs Buildpacks
runtime layer
OS image
application layer
Container (e.g. Docker)
system brings ﬁxed
host OS Kernel
* Devs may bring a custom
buildpack
runtime layer*
OS image
application layer
Buildpack
App container
System Provides
Dev Provides
system brings ﬁxed
host OS Kernel

24
Community Buildpacks
https://github.com/cloudfoundry-community/
cf-docs-contrib/wiki/Buildpacks
Ÿ  Lots of languages:  

Clojure, Haskell, .NET, Erlang, Elixir, etc.
Ÿ  RShiny app buildpack:  

https://github.com/alexkago/cf-buildpack-r
Ÿ  Can also use some Heroku buildpacks without modiﬁcation.

25
Using conda for package management
Ÿ  http://conda.pydata.org
Ÿ  Beneﬁts:
–  Uses precompiled binary packages
–  No ﬁddling with Fortran or C compilers and library paths
–  Known good combinations of main package versions
–  Really simple environment management (better than virtualenv)
–  Easy to run Python 2 and 3 side-by-side
Go try it out if you haven’t already!

26
Challenge 2: Pushing a PyData app
Ÿ  Code directory: 02-pydata-spyre-app
Ÿ  Spyre – Adam Hajari https://github.com/adamhajari/spyre
Ÿ  This app uses the Pydata buildpack to install Matplotlib,
NumPy and more.
Ÿ  Spyre provides a simple way to build interactive web based
visualisations similar to Rshiny.

27
Next Steps
See https://github.com/ihuston/python-cf-examples for
continuation of tutorial:
Ÿ  How to bind data sources like Redis & PostgreSQL
Ÿ  Build a simple machine learning API system
Full tutorial as a video:

https://www.youtube.com/watch?v=lp2c05yDJko

28
Resources
Ÿ  Docs: http://docs.cloudfoundry.org
Ÿ  CF Summit videos: http://cfsummit.com
Ÿ  Join the CF community: http://cloudfoundry.org
Ÿ  CF meetups: http://cloud-foundry.meetup.com

29
@ianhuston

Ian huston getting started with cloud foundry

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Ian huston getting started with cloud foundry

Similar to Ian huston getting started with cloud foundry (20)

More from Jessica Willis

More from Jessica Willis (7)

Recently uploaded

Recently uploaded (20)

Ian huston getting started with cloud foundry