This document discusses setting up a manageable Puppet infrastructure. It outlines common pitfalls like unmaintainable codebases and collaboration difficulties. Quick wins to address pitfalls include moving data to Hiera, implementing code reviews, and refactoring constantly. The document recommends designing around server roles, storing all things data in Hiera, and facilitating deployment and workflow through environments and R10k. It provides examples of setting up roles, using different Hiera data types, and Hiera-related functions to retrieve and generate resources from data.

1. Manageable Puppet
infrastructure
~April 2014 edition~
PuppetCampBerlin
Ger Apeldoorn - http://puppetspecialist.nl
1 / 44

2. Freelance PuppetConsultant
TrainerforPuppetLabs Benelux
Who's this?
2 / 44

3. Scope
Also... why this talk?
3 / 44

4. Common
pitfalls
4 / 44

5. Pitfalls
Cause & effect
Pitfalls
Lots of Workarounds
Unmaintainable codebase
Collaboration difficulties
5 / 44

6. Pitfalls
Cause & effect
Quick Wins
Fix your codebase!
Quick wins:
Move data to Hiera
Implement Code Review
Use Puppet-lint in a git-hook
REFACTOR CONSTANTLY
6 / 44

7. A Manageable Design
April 2014 edition
7 / 44

8. Requirements
Whadda we need
8 / 44

9. Our environment should be:
Easyto Use
Easyto Comprehend
Easyto Update
and...Safe
9 / 44

10. This stuff
isn't exactly
easy
10 / 44

11. But we cán make it safe and
manageable
11 / 44

12. Requirements
Easyto:
Use
Comprehend
Update
Safe
Safe
Useenvironments to test everything
Createahugetesting environment
UseGit to promoteyour code
12 / 44

13. Requirements
Easyto:
Use
Comprehend
Update
Safe
Manageable
Manageable
Keepaconsistent modulestructure
Using roles for abstraction
Facilitatecollaboration
13 / 44

14. Domains
Server Roles
All things data
Deployment &Workflow
14 / 44

15. Overview
Software Components
15 / 44

16. Software Components
Puppet Enterpriseor TheForeman
Hieraandhiera-eyaml (HierarchicalDatalookup)
Gerrit (Codereview system)
Git (what else?)
GitFlow, adaptedversionforGerrit
R10K (Environment deployment tool)
16 / 44

17. Domain#1:
Server Roles
17 / 44

18. Alayer ofabstraction
18 / 44

19. How to do it?
Createroles module
root@puppet# puppet module generate gerapeldoorn-role
Createabase-roleto cover generic settings
# modules/role/manifests/base.pp:
class role::base {
include users
include ssh
include motd
...
19 / 44

20. How to do it? -Cont'd-
Put all requiredresources intheclasses
# modules/role/manifests/app.pp:
class role::app {
include apache
include tomcat
apache::virtualhost { 'default':
...
Includeroleinnodedefinition
# site.pp:
node 'app01.autiplan.com' {
include role::base
include role::app
}
20 / 44

21. Domain#2:
All things Data
21 / 44

22. Hiera
Hierarchical data lookup tool
22 / 44

23. ConfiguredHierarchy:
#/etc/puppet/hiera.yaml:
:hierarchy:
- "%{::clientcert}"
- "%{::environment}"
- common
Node app01.autiplan.com:
environment: testing
Hieradata
# hiera/app01.autiplan.com.yaml
---
examplekey: value for
app01.autiplan.com
# hiera/testing.yaml
---
examplekey: value for nodes in
testing environment
# hiera/common.yaml
---
examplekey: value for all nodes
It's all about Hierarchy
What will bein$test?
$test = hiera('examplekey')
23 / 44

24. Types of Hieradata
Regular values
# hiera/app01.autiplan.com.yaml
---
examplekey: value
24 / 44

25. Types of Hieradata
Arrays
# hiera/app01.autiplan.com.yaml
---
array: [ item1, item2, item3 ]
otherarray:
- item1
- item2
- item3
Note: Never use tabs in Hiera files!
25 / 44

26. Types of Hieradata
Hashes
# hiera/app01.autiplan.com.yaml
---
hash:
key1: value
key2: value
26 / 44

27. Types of Hieradata
Combinations
# hiera/app01.autiplan.com.yaml
---
hash:
key1: value
key2: value
key3:
- arrayvalue1
- arrayvalue2
key4:
subhashkey1: value
subhashkey2: value
27 / 44

28. Hiera-relatedfunctions
...and what to use them for
28 / 44

29. Whatdoes itdo?
Retrieves the first-found value in the
hierarchy. (top-down)
Whatto use itfor?
Basic variable-lookup.
Very easy to create exceptions!
Howto use it?
$smarthost = hiera('smarthost')
ExampleHieradata
# hiera/mail.autiplan.com.yaml
---
smarthost: smtp.myprovider.nl
# hiera/testing.yaml
---
smarthost: testsmtp.autiplan.com
# hiera/common.yaml
---
smarthost: mail.autiplan.com
hiera('key' [,default_value])
29 / 44

30. Whatdoes itdo?
Retrieves an array or hash value
in the hierarchy, concatinates all
found results
Whatto use itfor?
Combining data from all
hierarchy levels.
Howto use it?
$users = hiera_array('users')
ExampleHieradata
# hiera/app01.autiplan.com.yaml
---
users: [ 'user1', 'user2' ]
# hiera/testing.yaml
---
users: [ 'testuser' ]
# hiera/common.yaml
---
users: [ 'user3', 'user4' ]
hiera_array('key' [,default_value]) (andhiera_hash)
30 / 44

31. Whatdoes itdo?
Includes all classes listed in the
array that is loaded from Hiera.
Takes elements from ALL
hierarchy levels.
Whatto use itfor?
Lightweight ENC.
Put all classes / roles in Hiera.
Howto use it?
node default {
hiera_include('roles')
}
ExampleHieradata
# hiera/web01.autiplan.com.yaml
---
roles:
- role::web
# hiera/common.yaml
---
roles:
- role::base
hiera_include('classes')
31 / 44

32. Whatdoes itdo?
Generates resources from a
HASH.
Whatto use itfor?
Generate any resource based on
data from Hiera.
Can also be used with
hiera_hash to create resources
from all levels!
Howto use it?
create_resources ('apache::vhost', hiera('vhosts', {}))
ExampleHieradata
# hiera/web01.autiplan.com.yaml
---
vhosts:
autiplan.com:
alias: www.autiplan.com
autiplan.dk:
alias: www.autiplan.dk
docroot: /var/www/html/autiplan.dk
autiplan.nl:
alias: www.autiplan.nl
cdn.autiplan.com:
port: 81
docroot: /var/www/html/cdn
create_resources('type',HASH[,default_values])
32 / 44

33. Databindings
Auto-loading of Hiera data for parameterized classes.
33 / 44

34. Whatdoes itdo?
Automatically loads class
parameters from Hiera.
Whatto use itfor?
Specify all class parameters in
Hiera.
Use all hierarchical benefits for
class parameters.
Simplify the use of
parameterized classes.
Howto use it?
include mysql::server
ExampleHieradata
# hiera/web01.autiplan.com.yaml
---
mysql::server::root_password: m0ars3cr3t
# hiera/common.yaml
---
mysql::server::root_password: t0ps3cr3t
mysql::server::package_name: mysql-server
mysql::server::restart: true
Data bindings
34 / 44

35. Putting it all together
Anything node-specific should be in Hiera!
35 / 44

36. APuppet Run:What calls what?
36 / 44

37. Domain#3:
Deployment & Workflow
37 / 44

38. Environments
Keeping the environmentalists happy
38 / 44

39. Environments
What is anenvironment?
Seperate modulepaths/site.pp.
Common environments: development, testing, production.
Nodes request a specific environment.
Why?
Essential to prevent mistakes.
NEVER edit code in production!
The workflow helps us to 'promote' our code to production.
39 / 44

40. Demo!
40 / 44

41. R10koverview
41 / 44

42. Final remarks
Keeppublic modules as-is,wherever possible
Create wrapper classes in company-module.
Create fork if needed, submit pull request for fixes.
Add forked module (gitrepo) to Puppetfile.
Thinkahead
Always try to anticipate future applications.
If it feels overly complicated, yer doin it wrong.
Refactor!
42 / 44

43. Questions?
43 / 44

44. Freelance PuppetConsultant
TrainerforPuppetLabs Benelux
Thank you!
A howto of setting up this environment (and the workflow!) is available on my
blog: http://puppetspecialist.nl/mpi
44 / 44