Presentation on Techniques and Approaches to Modules Reusability at PuppetCamp 2010 San Francisco

1. RE-USEYOUR MODULES!
Techniques and Approaches to Modules Reusability
Puppet Camp 2010 San Francisco
Alessandro Franceschi
al@lab42.it

2. I have a dream
a small one

3. CrossVendor

4. Reusable

5. Plug & Play

6. Puppet
modules

7. CrossVendor
Reusable
Plug & Play

8. At the beginning...

9. class foo {
package { foo:
ensure => installed
}
service { foo:
ensure => running,
enable => true,
hasstatus => true,
require => Package[foo],
subscribe => File[foo.conf],
}
file { foo.conf:
ensure => present,
path => “/etc/foo.conf”,
mode => 644,
owner => root,
group => root,
source => “puppet:///foo/foo.conf”,
}
}
At the beginning

10. Facing Diversity

11. class foo {
package { foo:
ensure => installed,
name => $operatingsystem ? {
debian => “food”,
default => “foo”,
},
}
service { foo:
ensure => running,
name => $operatingsystem ? {
debian => “food”,
default => “foo”,
},
enable => true,
hasstatus => true,
require => Package[foo],
subscribe => File[foo.conf],
}
file { foo.conf:
ensure => present,
path => $operatingsystem ? {
debian => “/etc/foo/foo.conf”,
default => “/etc/foo.conf”,
},
mode => 644,
Facing Diversity

12. class foo {
package { foo:
ensure => installed
name => $operatingsystem ? {
debian => “food”,
default => “foo”,
}
}
service { foo:
ensure => running,
name => $operatingsystem ? {
debian => “food”,
default => “foo”,
},
enable => true,
hasstatus => $operatingsystem ? {
debian => false,
default => true,
},
pattern => “food”,
require => Package[foo],
subscribe => File[foo.conf],
}
file { foo.conf:
ensure => present,
path => $operatingsystem ? {
debian => “/etc/foo/foo.conf”,
default => “/etc/foo.conf”,
},
mode => 644,
Facing Diversity

13. package { foo:
ensure => installed
name => $operatingsystem ? {
debian => “food”,
ubuntu => “food”,
default => “foo”,
}
}
service { foo:
ensure => running,
name => $operatingsystem ? {
debian => “food”,
ubuntu => “food”,
default => “foo”,
},
enable => true,
hasstatus => $operatingsystem ? {
debian => false,
ubuntu => false,
default => true,
},
pattern => “food”,
require => Package[foo],
subscribe => File[foo.conf],
}
file { foo.conf:
ensure => present,
path => $operatingsystem ? {
debian => “/etc/foo/foo.conf”,
default => “/etc/foo.conf”,
Facing Diversity

14. Centralized settings: foo::params
Qualified variables
OS dedicated classes
Facing Diversity

15. Centralized settings
class foo::params {
$packagename = $operatingsystem ? {
debian => “food”,
ubuntu => “food”
default => "foo",
}
$servicename = $operatingsystem ? {
debian => “food”,
ubuntu => “food”,
default => “foo”,
},
$processname = $operatingsystem ? {
default => "food",
}
$hasstatus = $operatingsystem ? {
debian => false,
ubuntu => false,
default => true,
}
$configfile = $operatingsystem ? {
freebsd => "/usr/local/etc/foo.conf",
ubuntu => “/etc/foo/foo.conf",
Facing Diversity

16. Qualified variables
class foo {
require foo::params
package { "foo":
name => "${foo::params::packagename}",
ensure => present,
}
service { "foo":
name => "${foo::params::servicename}",
ensure => running,
enable => true,
hasstatus => "${foo::params::hasstatus}",
pattern => "${foo::params::processname}",
require => Package["foo"],
subscribe => File["foo.conf"],
}
file { "foo.conf":
path => "${foo::params::configfile}",
mode => "${foo::params::configfile_mode}",
owner => "${foo::params::configfile_owner}",
group => "${foo::params::configfile_group}",
ensure => present,
require => Package["foo"],
Facing Diversity

17. OS dedicated classes
class foo {
[ ... ]
# Include OS specific subclasses, if necessary
# Note that they needn’t to inherit foo
case $operatingsystem {
debian: { include foo::debian }
ubuntu: { include foo::debian }
default: { }
}
}
Facing Diversity

18. Dealing with Users

19. # foo class needs users’ variables
# IE: $foo_server
class foo {
[ ... ]
file { foo.conf:
ensure => present,
path => $operatingsystem ? {
debian => “/etc/foo/foo.conf”,
default => “/etc/foo.conf”,
},
mode => 644,
content => template(“foo/foo.conf.erb”),
}
}
# foo/templates/foo.conf.erb is something like:
# File Managed by Puppet
server = <%= foo_server %>
# Plug & Play ?
Dealing with Users

20. Filtering user variables
Setting default values
Dealing with Users

21. Setting default values
class foo::params {
# Full hostname of foo server
$server = $foo_server ? {
'' => "foo.example42.com",
default => "${foo_server}",
}
# Foo DB management
$db_host = $foo_db_host ? {
'' => "localhost",
default => "${foo_db_host}",
}
$db_user = $foo_db_user ? {
'' => "root",
default => "${foo_db_user}",
}
$db_password = $foo_db_password ? {
'' => "",
default => "${foo_db_host}",
}
Dealing with Users

22. Setting default values
class foo {
include foo::params
[ ... ]
# ${foo_server} is user variable unfiltered
# ${foo::params::server} is user variable filtered
# You may want/need to reassign ${foo_server} value:
${foo_server} = ${foo::params::server}
}
In templates this fails:
server = <%= foo::params::server %>
This works (of course):
server = <%= foo_server %>
This works (even if variable is not set):
server = <%= scope.lookupvar('foo::params::server') %>
Dealing with Users

23. Adapt and Customize

24. debian => “food”,
ubuntu => “food”,
default => “foo”,
},
enable => true,
hasstatus => $operatingsystem ? {
debian => false,
ubuntu => false,
default => true,
},
require => Package[foo],
subscribe => File[foo.conf],
}
file { foo.conf:
ensure => present,
path => “/etc/foo.conf”,
mode => 644,
owner => root,
group => root,
source => [ “puppet:///foo/foo.conf-$hostname”,
“puppet:///foo/foo.conf-$my_role”,
“puppet:///foo/foo.conf” ],
}
}
Adapt and customize

25. }
file { foo.conf:
ensure => present,
path => “/etc/foo.conf”,
mode => 644,
owner => root,
group => root,
source => [ “puppet:///foo/foo.conf-$hostname”,
“puppet:///foo/foo.conf-$my_role”,
“puppet:///foo/foo.conf” ],
}
case $my_role {
mail: {
file { foo.conf-mail:
ensure => present,
path => “/etc/foo.conf.d/mail”,
mode => 644,
owner => root,
group => root,
source => [ “puppet:///foo/foo.conf-mail”,
}
}
default: { }
}
Adapt and customize

26. Isolation of diversity
Project classes || Project module
Adapt and Customize

27. Isolation of diversity
# Class foo::example42
#
# You can use your custom project class to modify
# the standard behavior of foo module
#
# You don't need to use class inheritance if you
# don't override or redefine resources in foo class
#
# You can add custom resources and are free to
# decide how to provide the contents of your files:
# - Via static sourced files ( source => ) according
# to the naming convention you need
# - Via custom templates ( content => )
# - Via some kind of infile line modification tools
# such as Augeas
#
class foo::example42 inherits foo {
File["foo.conf"] {
source => [ "puppet:///foo/foo.conf-$hostname",
"puppet:///foo/foo.conf-$role",
"puppet:///foo/foo.conf" ],
}
}
Adapt and Customize

28. Project Classes ||
Project Modules
class foo {
[ ... ]
# Include project specific class if $my_project is set
# The extra project class is by default looked in foo module
# If $my_project_onmodule == yes it's looked in your project
# module
if $my_project {
case $my_project_onmodule {
yes,true: { include "${my_project}::foo" }
default: { include "foo::${my_project}" }
}
}
}
Adapt and Customize

29. Control and monitor

30. service { foo:
ensure => running,
name => $operatingsystem ? {
debian => “food”,
ubuntu => “food”,
default => “foo”,
},
enable => true,
hasstatus => $operatingsystem ? {
debian => false,
ubuntu => false,
default => true,
},
pattern => “food”,
require => Package[foo],
subscribe => File[foo.conf],
}
# Monitoring stuff: munin and nagios
munin::plugin { "foo_processes":
ensure => present,
}
nagios::service { "foo_${foo_port_real}":
check_command => "tcp_port!${foo_port_real}"
}
Control and Monitor

31. Monitoring abstraction
Monitor meta-module
Control and Monitor

32. Monitoring abstraction
class foo {
[ ... ]
# Include monitor classe if $monitor == yes
# Define the monitoring tools to use
# with the variables $monitor_tools (can be an array)
if $monitor == "yes" { include foo::monitor }
}
Control and Monitor

33. Monitoring abstraction
class foo::monitor {
include foo::params
monitor::port { "foo_${foo::params::protocol}_
${foo::params::port}":
protocol => "${foo::params::protocol}",
target => "${foo::params::monitor_target_real}",
port => "${foo::params::port}",
enable => "${foo::params::monitor_port_enable}",
tool => "${monitor_tool}",
}
monitor::process { "foo_process":
process => "${foo::params::processname}",
service => "${foo::params::servicename}",
pidfile => "${foo::params::pidfile}",
enable => "${foo::params::monitor_process_enable}",
tool => "${monitor_tool}",
}
[ ... ]
if $my_project {
case $my_project_onmodule {
yes,true: { include "${my_project}::foo::monitor" }
default: { include "foo::monitor::${my_project}" }
Control and Monitor

34. Monitor Meta-module
define monitor::process ( $process, $service,
$pidfile, $tool, $enable ) {
if ($enable != "false") and ($enable != "no") {
if ($tool =~ /munin/) { # TODO }
if ($tool =~ /collectd/) { # TODO }
if ($tool =~ /monit/) {
monitor::process::monit { "$name":
pidfile => "$pidfile",
process => "$process",
service => "$service",
}
}
if ($tool =~ /nagios/) {
monitor::process::nagios { "$name":
process => $process,
}
}
} # End if $enable
}
Control and Monitor

35. Monitor Meta-module
define monitor::process::monit (
$pidfile='',
$process='',
$service=''
) {
# Use for Example42 monit module
monit::checkpid { "${process}":
pidfile => "${pidfile}",
startprogram => "/etc/init.d/${service} start",
stopprogram => "/etc/init.d/${service} stop",
}
# Use for Camptocamp’s monit module (sample)
# monit::config { "${process}":
# ensure => present,
# content => template(“monit/checkprocess.erb”), # To create
# }
# Use for Monit recipe on Puppet’s wiki (sample)
# monit::package { "${process}": }
}
Control and Monitor

36. Coherent naming conventions
Predictable behaviors
A name for Everything

37. Naming conventions
include foo - Installs and runs foo service
# If foo can be client or server
include foo::server - Installs foo server
include foo::client - Installs foo client
# If foo is on every/many host either client or server:
if ($foo_server_local == true) or ($foo_server == "$fqdn") {
include puppet::server
} else {
include puppet::client
}
# If foo has to be disable or removed:
include foo::absent - Remove foo
include foo::disable - Disable foo service
include foo::disableboot - Disable foo service but
do not check if is running
A name for Everything

38. Quick cloning & customization
Coherent Modules Infrastructure
Modules Machine

39. Quick cloning
# Few seds for a script that clones foo module
example42_module_clone.sh
# Creates new module from foo template. Then you:
# - Edit params.pp
# - Add specific classes, defines, types, facts
# - Eventually define modules variables and templates
#
# - Everything else is ready out of the box
Modules Machine

40. Quick customization
# Script that adapts a module for a new project
example42_project_rename.sh
# Prepares custom classes for new project
# foo/manifests/my_project.pp
# foo/manifests/monitor/my_project.pp
# foo/manifests/backup/my_project.pp
#
# - Use the logic that works for your project
# - Choose how to provide your configuration files
Modules Machine

41. Coherent Infrastructure
# Clone is GOOD
# Modules are similar
# They share a coherent naming convention
# They have confined places where to apply
# custom logic
# Fixed files for fixes functions
# Common approach for the Puppet team members
# Reduced risks and better manageability
# Quick update to template improvements
# Could be more easily manageable/autogenerated
# by GUI tools
Modules Machine

43. Centralized settings:
foo::params
Qualified variables
Isolation of diversity
Project classes ||
Project module
Monitoring abstraction
Monitor meta-module
Filtering user variables
Setting default values
Coherent naming
conventions
Predictable behaviors
Quick cloning of foo
module
Quick customization
CrossVendor Reusable Plug & Play
Modules
Machine
A name for
Everything
Dealing
with Users
Control
and Monitor
Adapt and
Customize
Facing
Diversity
small dreams turn easier into reality

44. www.example42.com
github.com/example42/puppet-foo/