Introduction to Chef, from Velocity 2010.

1. Infrastructure Automation
Speakers:
with Chef
Adam Jacob CTO
Christopher Brown VP Engineering
Joshua Timberman Sr. Solution Engineer
Copyright © 2010 Opscode, Inc - All Rights Reserved 1

2. • Adam Jacob
• Christopher Brown
• Joshua Timberman
http://www.flickr.com/photos/anotherphotograph/2100904507/sizes/o/

3. http://www.flickr.com/photos/timyates/2854357446/sizes/l/

4. • Developers?
http://www.flickr.com/photos/timyates/2854357446/sizes/l/

5. • Developers?
• Systems Administrators?
http://www.flickr.com/photos/timyates/2854357446/sizes/l/

6. • Developers?
• Systems Administrators?
• “Business” People?
http://www.flickr.com/photos/timyates/2854357446/sizes/l/

7. • Developers?
• Systems Administrators?
• “Business” People?
• Executives?
http://www.flickr.com/photos/timyates/2854357446/sizes/l/

8. http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg

9. At a High Level...
http://www.flickr.com/photos/asten/2159525309/sizes/l/

10. At a High Level...
• A library for configuration management
http://www.flickr.com/photos/asten/2159525309/sizes/l/

11. At a High Level...
• A library for configuration management
• A configuration management system
http://www.flickr.com/photos/asten/2159525309/sizes/l/

12. At a High Level...
• A library for configuration management
• A configuration management system
• A systems integration platform
http://www.flickr.com/photos/asten/2159525309/sizes/l/

13. At a High Level...
• A library for configuration management
• A configuration management system
• A systems integration platform
• An API for your entire Infrastructure
http://www.flickr.com/photos/asten/2159525309/sizes/l/

14. http://www.flickr.com/photos/gi/518613153/sizes/o/

15. Principles
• Idempotent
http://www.flickr.com/photos/gi/518613153/sizes/o/

16. Principles
• Idempotent
• Reasonability
http://www.flickr.com/photos/gi/518613153/sizes/o/

17. Principles
• Idempotent
• Reasonability
• Sane defaults, easily changed
http://www.flickr.com/photos/gi/518613153/sizes/o/

18. Principles
• Idempotent
• Reasonability
• Sane defaults, easily changed
• Hackability
http://www.flickr.com/photos/gi/518613153/sizes/o/

19. Principles
• Idempotent
• Reasonability
• Sane defaults, easily changed
• Hackability
• TMTOWTDI
http://www.flickr.com/photos/gi/518613153/sizes/o/

20. http://www.flickr.com/photos/gi/518613153/sizes/o/

21. The world moves
pretty fast
• Primitives
http://www.flickr.com/photos/gi/518613153/sizes/o/

22. The world moves
pretty fast
• Primitives
• Enable you to solve your
problems
http://www.flickr.com/photos/gi/518613153/sizes/o/

23. The world moves
pretty fast
• Primitives
• Enable you to solve your
problems
• Your best skill is knowing
your systems
http://www.flickr.com/photos/gi/518613153/sizes/o/

24. http://www.flickr.com/photos/gi/518613153/sizes/o/

25. So you better be ready
• You need a 3rd generation
language
http://www.flickr.com/photos/gi/518613153/sizes/o/

26. So you better be ready
• You need a 3rd generation
language
• You’re already a
programmer
http://www.flickr.com/photos/gi/518613153/sizes/o/

27. So you better be ready
• You need a 3rd generation
language
• You’re already a
programmer
• You’ve just been lied to
http://www.flickr.com/photos/gi/518613153/sizes/o/

28. Chef enables
Infrastructure as Code
Manage configuration as idempotent Resources.
Put them together in Recipes.
Track it like source code.
Configure your servers.

29. Infrastructure as Code
http://www.flickr.com/photos/wonderlane/2306082998/
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

30. Infrastructure as Code is...
http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/
Copyright © 2010 Opscode, Inc - All Rights Reserved 11

31. Infrastructure as Code is...
A technical
domain revolving
around building
and managing
infrastructure
programmatically
http://www.flickr.com/photos/kwerfeldein/2634561264/sizes/o/
Copyright © 2010 Opscode, Inc - All Rights Reserved 11

32. Enable the reconstruction of
the business from nothing
but a source code
repository, an application
data backup, and bare metal
resources.
Copyright © 2010 Opscode, Inc - All Rights Reserved 12

33. Theory of Constraints
http://www.flickr.com/photos/mabar/313860092/ Copyright © 2010 Opscode, Inc - All Rights Reserved 13

34. Your Prime Constraint Should Be
http://www.flickr.com/photos/visualage/2126833132/sizes/o/
Copyright © 2010 Opscode, Inc - All Rights Reserved 14

35. Your Prime Constraint Should Be
The time it
takes to
restore your
application
data
http://www.flickr.com/photos/visualage/2126833132/sizes/o/
Copyright © 2010 Opscode, Inc - All Rights Reserved 14

36. Philosophy
Sidebar

37. Wax Philosophical
• We are craftsmen and artists
• You need great brushes and tools
• But nobody remembers Gaudi’s hammer
http://www.flickr.com/photos/vgm8383/2686128924/sizes/l/

38. • Can only be
measured
by the final
solution
http://www.flickr.com/photos/aloshbennett/619307160/sizes/l/

39. A Tour of Chef
Copyright © 2010 Opscode, Inc - All Rights Reserved 18

40. Chef Client runs on your
systems
Copyright © 2010 Opscode, Inc - All Rights Reserved 19

41. They talk to Chef
Servers
Copyright © 2010 Opscode, Inc - All Rights Reserved 20

42. They can also stand
alone - Chef Solo
Copyright © 2010 Opscode, Inc - All Rights Reserved 21

43. We call each system you
configure a Node
Copyright © 2010 Opscode, Inc - All Rights Reserved 22

44. Nodes have Attributes
Copyright © 2010 Opscode, Inc - All Rights Reserved 23

45. Attributes are
Searchable
$ knife search node ‘platform:mac_os_x’
search(:node, ‘platform:mac_os_x’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 24

46. Nodes have a Run List
What Roles or Recipes to apply
in Order
Copyright © 2010 Opscode, Inc - All Rights Reserved 25

47. Nodes have a Run List
Copyright © 2010 Opscode, Inc - All Rights Reserved 26

48. Nodes have Roles
Copyright © 2010 Opscode, Inc - All Rights Reserved 27

49. Roles
Copyright © 2010 Opscode, Inc - All Rights Reserved 28

50. Roles
Describe what a node should be
Copyright © 2010 Opscode, Inc - All Rights Reserved 28

51. Roles
Describe what a node should be
Copyright © 2010 Opscode, Inc - All Rights Reserved 28

52. Copyright © 2010 Opscode, Inc - All Rights Reserved 29

53. Roles are Searchable
$ knife search role ‘max_children:50’
search(:role, ‘max_children:50’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 30

54. Roles have a Run List
What Roles or Recipes to apply
in Order
Copyright © 2010 Opscode, Inc - All Rights Reserved 31

55. Run Lists on Roles
expand
Copyright © 2010 Opscode, Inc - All Rights Reserved 32

56. Chef manages
Resources on Nodes
Copyright © 2010 Opscode, Inc - All Rights Reserved 33

57. A Resource...
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

58. A Resource...
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

59. A Resource...
• Is of a given type.
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

60. A Resource...
• Is of a given type.
• Has a name.
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

61. A Resource...
• Is of a given type.
• Has a name.
• Has attributes.
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

62. A Resource...
• Is of a given type.
• Has a name.
• Has attributes.
• Takes actions to
bring the resource to
a declared state.
Is a declarative description of the state you desire a part
of your node to be in.
http://www.flickr.com/photos/xiaming/382205902/sizes/l/

63. Copyright © 2010 Opscode, Inc - All Rights Reserved 35

64. Resources take action
through Providers
Copyright © 2010 Opscode, Inc - All Rights Reserved 36

65. A Provider...
Knows how to actually perform the actions specified by a resource.
http://www.flickr.com/photos/affableslinky/562950216/

66. A Provider...
Knows how to actually perform the actions specified by a resource.
Apt,Yum, Rubygems,
Multiple providers
per resource type.
Portage, Macports,
FreeBSD Ports, etc.
http://www.flickr.com/photos/affableslinky/562950216/

67. A Provider...
Knows how to actually perform the actions specified by a resource.
Apt,Yum, Rubygems,
Multiple providers
per resource type.
Portage, Macports,
FreeBSD Ports, etc.
Can be overridden
with the provider
attribute on a
resource.
http://www.flickr.com/photos/affableslinky/562950216/

68. Copyright © 2010 Opscode, Inc - All Rights Reserved 38

69. Resources
Platform
Provider
http://www.flickr.com/photos/acurbelo/2628837104/sizes/o/

70. Recipes are lists of
Resources
Copyright © 2010 Opscode, Inc - All Rights Reserved 40

71. A Recipe...
Applies resources in the order they are specified
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

72. A Recipe...
Applies resources in the order they are specified
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

73. A Recipe...
Applies resources in the order they are specified
1
• Evaluates resources
in the order they
appear.
2
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

74. A Recipe...
Applies resources in the order they are specified
• Evaluates resources
in the order they
appear. 1
2
• Adds each resource
object to the
Resource Collection
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

75. A Recipe...
Applies resources in the order they are specified
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

76. A Recipe...
Applies resources in the order they are specified
• Can include other
recipes.
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

77. A Recipe...
Applies resources in the order they are specified
• Can include other
recipes.
• Is just Ruby.
http://www.flickr.com/photos/roadsidepictures/2478953342/sizes/o/

78. Order Matters
Copyright © 2010 Opscode, Inc - All Rights Reserved 43

79. Cookbooks are
packages for Recipes
Copyright © 2010 Opscode, Inc - All Rights Reserved 44

80. Cookbooks
Copyright © 2010 Opscode, Inc - All Rights Reserved 45

81. Cookbooks
Distributable
Copyright © 2010 Opscode, Inc - All Rights Reserved 45

82. Cookbooks
Distributable
Infrastructure as Code
Copyright © 2010 Opscode, Inc - All Rights Reserved 45

83. Cookbooks
Distributable
Infrastructure as Code
Version control repository
Copyright © 2010 Opscode, Inc - All Rights Reserved 45

84. Common Cookbook Components
Copyright © 2010 Opscode, Inc - All Rights Reserved 46

85. Common Cookbook Components
Recipes
Copyright © 2010 Opscode, Inc - All Rights Reserved 46

86. Common Cookbook Components
Recipes
Assets (files/templates)
Copyright © 2010 Opscode, Inc - All Rights Reserved 46

87. Common Cookbook Components
Recipes
Assets (files/templates)
Attributes
Copyright © 2010 Opscode, Inc - All Rights Reserved 46

88. Common Cookbook Components
Recipes
Assets (files/templates)
Attributes
Metadata
Copyright © 2010 Opscode, Inc - All Rights Reserved 46

89. Cookbook directory
Copyright © 2010 Opscode, Inc - All Rights Reserved 47

90. Cookbook assets
Copyright © 2010 Opscode, Inc - All Rights Reserved 48

91. Cookbook assets
Files
Copyright © 2010 Opscode, Inc - All Rights Reserved 48

92. Cookbook assets
Files
‣ Static assets
‣ Downloaded via remote_file
‣ File specificity
Copyright © 2010 Opscode, Inc - All Rights Reserved 48

93. Cookbook assets
Copyright © 2010 Opscode, Inc - All Rights Reserved 49

94. Cookbook assets
Templates
Copyright © 2010 Opscode, Inc - All Rights Reserved 49

95. Cookbook assets
Templates
‣ Dynamic assets
‣ ERB (erubis)
‣ File specificity
Copyright © 2010 Opscode, Inc - All Rights Reserved 49

96. Cookbooks
Copyright © 2010 Opscode, Inc - All Rights Reserved 50

97. Cookbooks
Attributes
Copyright © 2010 Opscode, Inc - All Rights Reserved 50

98. Cookbooks
Attributes
‣ Node attributes
Copyright © 2010 Opscode, Inc - All Rights Reserved 50

99. Cookbooks
Copyright © 2010 Opscode, Inc - All Rights Reserved 51

100. Cookbooks
Metadata
Copyright © 2010 Opscode, Inc - All Rights Reserved 51

101. Cookbooks
Metadata
cookbooks/god/metadata.rb
Copyright © 2010 Opscode, Inc - All Rights Reserved 51

102. Cookbooks
Copyright © 2010 Opscode, Inc - All Rights Reserved 52

103. Cookbooks
Other Components
Copyright © 2010 Opscode, Inc - All Rights Reserved 52

104. Cookbooks
Other Components
‣ definitions
‣ libraries
‣ resources
‣ providers
Copyright © 2010 Opscode, Inc - All Rights Reserved 52

105. Cookbooks are
shareable!
Copyright © 2010 Opscode, Inc - All Rights Reserved 53

106. Data bags store
arbitrary data
Copyright © 2010 Opscode, Inc - All Rights Reserved 54

107. A user
$ knife data bag show users adam
Copyright © 2010 Opscode, Inc - All Rights Reserved 55

108. Data Bags are
Searchable
$ knife search users ‘shell:/usr/bin/zsh’
search(:users, ‘/usr/bin/zsh’)
Copyright © 2010 Opscode, Inc - All Rights Reserved 56

109. Data bags make recipes
awesome-r (that’s
totally a word)
Copyright © 2010 Opscode, Inc - All Rights Reserved 57

110. Data bags make recipes
awesome-r (that’s
totally a word)
Copyright © 2010 Opscode, Inc - All Rights Reserved 57

111. Working with Chef
Copyright © 2010 Opscode, Inc - All Rights Reserved 58

112. Create your Chef
Repostiory
$ git clone http://github.com/opscode/chef-
repo.git
Copyright © 2010 Opscode, Inc - All Rights Reserved 59

113. Configure knife
Everything you can do in Chef, you can do from the
command line with Knife.
Per-directory configurations!
Copyright © 2010 Opscode, Inc - All Rights Reserved 60

114. Download some
cookbooks
$ knife cookbook site vendor rails -d
INFO: Downloading rails from the cookbooks site at version 0.8.0
...
INFO: Cookbook rails version 0.8.0 successfully vendored!
INFO: Downloading apache2 from the cookbooks site at version 0.12.0
...
INFO: Cookbook apache2 version 0.12.0 successfully vendored!
INFO: Downloading ruby from the cookbooks site at version 0.7.0
...
INFO: Cookbook ruby version 0.7.0 successfully vendored!
Uses the “vendor branch” pattern, so you can make
changes and track the upstream
Copyright © 2010 Opscode, Inc - All Rights Reserved 61

115. Check out the Rails
recipe...
$ vim cookbooks/rails/recipes/default.rb
Copyright © 2010 Opscode, Inc - All Rights Reserved 62

116. Upload them!
$ knife cookbook upload -a
These run as root, kids.
Let’s not blindly trust the upstream too much, ok?
Copyright © 2010 Opscode, Inc - All Rights Reserved 63

117. Build a role
$ knife role create rails
Copyright © 2010 Opscode, Inc - All Rights Reserved 64

118. Add your Cloudy
credentials to knife
$ vi ~/.chef/knife.rb
Copyright © 2010 Opscode, Inc - All Rights Reserved 65

119. Launch a new Rails
Server
$ knife rackspace server create 'role[rails]'
$ knife ec2 server create ‘role[rails]’
Or just install Chef on
your existing servers
Copyright © 2010 Opscode, Inc - All Rights Reserved 66

120. Chef runs on your new
server
$ sudo chef-client
INFO: Starting Chef Run
...
INFO: Chef Run complete in 11.852033 seconds
Automatically.
Copyright © 2010 Opscode, Inc - All Rights Reserved 67

121. Marvel at Rails being
installed
$ knife ssh role:rails ‘gem list | grep rails’
Knife SSH!
“Search for all nodes with role rails”
Run the following command
Executes in parallel
Supports screen, tmux, and macterm
Copyright © 2010 Opscode, Inc - All Rights Reserved 68

122. Decide you want Rails 3
instead
$ vim cookbooks/rails/default
The vendor pattern means that you can just change it, and
safely merge any upstream changes later on.
Copyright © 2010 Opscode, Inc - All Rights Reserved 69

123. Upload your changed
cookbook
$ knife cookbook upload rails
Only uploads the changed file.
Copyright © 2010 Opscode, Inc - All Rights Reserved 70

124. Run chef and verify
$ knife ssh role:rails ‘sudo chef-client && gem list |
grep rails’
cook-prod-i78f2e213.opscode.us rails (3.0.0-
beta4)
Run chef on all the Rails systems
Then verify the changes.
Copyright © 2010 Opscode, Inc - All Rights Reserved 71

125. Watch now as Mr.
Timberman rocks it live.
Copyright © 2010 Opscode, Inc - All Rights Reserved 72

126. Questions?
www.opscode.com
‣ Twitter: @opscode, #opschef
‣ irc.freenode.net #chef
‣ lists.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 73