Google is making the power of its datacenter, network, and technology innovations available to the world through its Cloud services. This presentation will provide an overview of the Google Cloud Platform and a deeper dive on Google Compute Engine. Google recently made an open source contribution to SaltStack and now you can now use Salt Cloud to manage your Compute Engine resources (IaaS virtual machine services). Come find out more about Google's Cloud Platform and how you can leverage Google scale with SaltStack.

1. SaltStack and
Google Compute Engine
Eric Johnson
Technical Program Manager

2. Agenda: January 30th, 2014
Whirlwind Tour of Google Cloud Platform
Google Compute Engine
Using SaltStack and Compute Engine (demo)
1
2
3

3. Whirlwind Tour of Google Cloud Platform
Google Compute Engine
Using SaltStack and Compute Engine (demo)
1
2
3

4. For the past 15 years, Google
has been building out the
world’s fastest, most powerful,
highest quality cloud
infrastructure on the planet.
Images by Connie Zhou
Why Google Cloud Platform?

5. A Network that Spans the Globe

6. Innovating Software & Driving Technology Forward
SpannerDremelMapReduce
Big Table Colossus
2012 20132002 2004 2006 2008 2010
GFS
Compute
Engine

7. Building Products that Scale
Google Maps Gmail Google Drive YouTube

8. “[Google's] ability to build, organize, and
operate a huge network of servers and fiber-
optic cables with an efficiency and speed that
rocks physics on its heels.
This is what makes Google Google: its physical
network, its thousands of fiber miles, and those
many thousands of servers that, in aggregate,
add up to the mother of all clouds.”
- Wired
Images by Connie Zhou

9. Storage
Cloud Storage Cloud SQL
Cloud
Datastore
Compute
Compute
Engine
App Engine
App Services
BigQuery
Cloud
Endpoints
Google Cloud Platform

10. Compute
Compute
Engine
App Engine
Storage
Cloud Storage Cloud SQL
Cloud
Datastore
App Services
BigQuery
Cloud
Endpoints
Power Computation

11. Storage
Cloud Storage Cloud SQL
Cloud
Datastore
Storing What You Want, However You Want
Compute
Compute
Engine
App Engine
App Services
BigQuery
Cloud
Endpoints

12. App Services
BigQuery
Cloud
Endpoints
Building Robust, Intelligent Systems
Storage
Cloud Storage Cloud SQL
Cloud
Datastore
Compute
Compute
Engine
App Engine

13. Whirlwind Tour of Google Cloud Platform
Google Compute Engine
Using SaltStack and Compute Engine (demo)
1
2
3

14. • IaaS: VMs, Network, Storage
• Google DNA (speed, scale, reliable, secure)
• Fast Provisioning, Consistent Performance
• Enterprise Ready
• 24x7 Support
• 99.95% monthly SLA
• ISO 27001, SSAE-16 SOC 1,2,3
• Accessible Through
• Web @ https://cloud.google.com/console
• gcutil command-line utility
• REST API
• Partners (Commercial and FOSS)
Google Compute Engine

15. Launching 100 virtual machines

16. Block Storage
Persistent Disks

17. Persistent Disk
10 TB
• Data and root partitions
• Billed only by capacity (GB/month)
• Performance caps scale linearly with size
• Volume striping is automatic
• Differential snapshots
• Create new PDs based on snapshots

18. PD Use-cases
Root
Stateful root
volume
User managed data
volume
Root
RW
Data
RO
Data
Instant distribution of
static content

19. Global Snapshot and Restore
GCS
vol1 vol2 vol3
vol1.
t2
vol2.
t2
vol3.
t3
t1 t1 t1
t1 t1 t1
t1 t1 t1
t1 t1
t1 t1
t2 t2
t2
t2
t3 t3 t3
● Point in time snapshot to
Google Cloud Storage
(GCS)
● Differential snapshots
● GCS global replication!
● Restore from snapshot
anywhere in the world
us-central1-a
europe-west1-a

20. Virtual Machines

21. standard
• For workloads with balanced CPU and
memory
highmem
• For workloads with higher memory
requirements
highcpu
• For workloads with higher CPU requirements
Shared Core Instances
• For inexpensive prototyping and staging
workloads (g1-small, f1-micro)
Machine Types
Machine Type Virtual Cores
Memory
(GB)
n1-standard-1 1 3.75
n1-standard-2 2 7.50
n1-standard-4 4 15.00
n1-standard-8 8 30.00
n1-standard-16 16 60.00
n1-highmem-2 2 13.00
n1-highmem-4 4 26.00
n1-highmem-8 8 52.00
n1-highmem-16 16 104.00
n1-highcpu-2 2 1.80
n1-highcpu-4 4 3.60
n1-highcpu-8 8 7.20
n1-highcpu-16 16 14.40

22. Images
+ Custom Kernel Support
(debian backports works great for docker)

23. Sub-hour Billing
1 minute granularity
10 minute minimum

24. Enhanced Reliability
us-central1-a
scheduled maintenance event
Live Migration
• No downtime during
scheduled datacenter
maintenance events
Automatic Restart
• Instances automatically
restarted if subjected to
system events such as
hardware failure

25. Networking

26. Images by Connie Zhou
Networking
• Google's Network
• Projects are isolated private networks
• TCP, UDP, ICMP only
• Multiple private network groups and firewalls
• Tags and address ranges
• Addresses
• public: static or ephemeral
• private: ephemeral with DNS
• Routes, gateways, VPNs, and IP Forwarding

27. Region: us-central1
Target Pool (tp-a)
Load-balancer
• Region based
• Session affinity
• Hashed on src ip:port and dst ip:port
• Consists of:
• Target pool: collection of instances
• Forwarding rules: protocol:port, pool
• HTTP health check: optional us-central1-b
us-central1-a
www0
www1
www2
www3
Forwarding Rules
tcp:443 ➔ tp-a
tcp:80 ➔ tp-a
Internet
http://googlecloudplatform.blogspot.com/2013/11/compute-engine-load-balancing-hits-1-million-requests-per-second.html

28. ...and

29. Noteworthy
• Metadata
• Startup scripts
• SSH Keys
• Tags (instance/network)
• OAuth2 and Scopes
• Access other Google Cloud Platform services
• Ecosystem is growing
• Partners: RightScale, Scalr, New Relic, MongoLab, MapR, and many more...
• Open Source: Salt, Chef, Puppet, Ansible, Vagrant, Docker, CoreOS, fog, libcloud

30. Whirlwind Tour of Google Cloud Platform
Google Compute Engine
Using SaltStack and Compute Engine (demo)
1
2
3

31. Start to finish
1. Use the Developers Console to create the salt master (metadata: saltdemo=yup)
salt
This is a Compute
Engine instance
named "salt"

32. Start to finish
1. Use the Developers Console to create the salt-master (metadata: saltdemo=yup)
2. Use salt-cloud to create 4 instances (2 per zone), install apache
us-central1-a
minion1
minion3
us-central1-b
minion2
minion4
salt
# salt-cloud -P -m /etc/salt/demo.map
# salt 'minion*' state.highstate
GooglePublicAPI

33. Start to finish
1. Use the Developers Console to create the salt-master (metadata: saltdemo=yup)
2. Use salt-cloud to create 4 instances (2 per zone)
3. Use salt-cloud networking functions Region: us-central1
Target Pool (lb-tp)
us-central1-a
minion1
minion3
us-central1-b
minion2
minion4
Forwarding Rules
tcp:80 ➔ lb-tp
PublicLBIP:a.b.c.d
salt
# salt-cloud -f create_fwrule gce ...
# salt-cloud -f create_lb gce ...

34. Start to finish
1. Use the Developers Console to create the salt-master (metadata: saltdemo=yup)
2. Use salt-cloud to create 4 instances (2 per zone)
3. Use salt-cloud networking functions
4. Generate some HTTP requests to LB IP
Region: us-central1
Target Pool (lb-tp)
us-central1-a
minion1
minion3
us-central1-b
minion2
minion4
Forwarding Rules
tcp:80 ➔ lb-tp
PublicLBIP:a.b.c.d
Internet
dst port 80
salt

35. The salt master via custom "startup-script" metadata
startup-script1 #!/bin/bash
2
3 SALT=$(curl -s http://metadata/computeMetadata/v1beta1/instance/attributes/saltdemo)
4
5 if [ "$SALT" = "yup" ]; then
6 # update and install dependencies (libcloud 0.14.0 + bugfix)
7 apt-get install vim tmux git python-pip -y
8 pip install git+https://github.com/apache/libcloud@58f3722ba54dc79cc9ef40271f6a3d39d6e8841d
9
10 # salt master
11 curl -L http://bootstrap.saltstack.org | sudo sh -s -- -M -N git v2014.1.0rc3
12
13 # dev gce provider
14 git clone https://github.com/erjohnso/salt
15 cp salt/salt/cloud/clouds/gce.py /usr/lib/python2.7/dist-packages/salt/cloud/clouds/gce.py
16
17 # set up root
18 mkdir /root/.gsutil
19 echo 1391212799 > /root/.gsutil/.last_software_update_check
20 gsutil cp gs://5ej/saltconf/root-ssh/goo* /root/.ssh
21 chmod 600 /root/.ssh/google_compute_engine
22
23 # copy salt config/srv files
24 gsutil cp -Rp gs://5ej/saltconf/etc /
25 gsutil cp -Rp gs://5ej/saltconf/srv /
26 fi

36. Compute Engine provider configuration
1 zone1a:
2 minion:
3 master: salt
4 image: debian-7
5 size: n1-standard-1
6 location: us-central1-a
7 make_master: False
8 deploy: True
9 delete_boot_pd: True
10 tags: '["minion", "salt", "z1a"]'
11 provider: gce-config
12
13 zone1b:
14 minion:
15 master: salt
16 image: debian-7
17 size: n1-standard-1
18 location: us-central1-b
19 make_master: False
20 deploy: True
21 delete_boot_pd: True
22 tags: '["minion", "salt", "z1b"]'
23 provider: gce-config
/etc/salt/cloud.profiles
/etc/salt/cloud1 providers:
2 gce-config:
3 project: 'google.com:erjohnso'
4 service_account_email_address: '...@developer.gserviceaccount.com'
5 service_account_private_key: '/etc/salt/pkey.pem'
6 provider: gce
/etc/salt/demo.map1 zone1a:
2 - minion1
3 - minion3
4 zone1b:
5 - minion2
6 - minion4

37. Available now and more coming!
• Standard salt-cloud compatibility
• Create, destroy instances
• --list-sizes, --list-images, --list-locations
• --show_instance myinstance
Available
Now!!
Coming
Soon!!
• Plus... Persistent Disks and Snapshots
• create_disk, delete_disk, show_disk, attach_disk, detach_disk
• create_snapshot, delete_snapshot, show_snapshot
• Plus... Networks and Firewall Rules
• create_network, delete_network, show_network
• create_fwrule, delete_fwrule, show_fwrule
• Plus... Load-balancer and HTTP Health Checks
• create_lb, delete_lb, show_lb, lb_attach, lb_detach

38. cloud.google.com
Google Compute Engine: https://cloud.google.com/products/compute-engine
Salt + Compute Engine: http://salt.readthedocs.org/en/latest/topics/cloud/gce.html