Deploying and maintaining (Django) applications professionally using puppet, git and scripting.

1. From Zero to Hero
Job Ganzevoort
Douwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com

2. Outline
● Introduction
● Initial Setup
● Deployment
● Maintenance
● Deployment continued
● Conclusion

3. Introduction

4. ● People have great ideas
● Django is perfect for RAD
○ But...
● How to deploy to production?
● How to keep the system maintainable?
○ Or even…
● When is my application production ready?
Why this presentation?

5. Initial Setup

6. ● Before you do anything related to code:
○ git init
● Make sure you track everything you do in a
VCS
Version Control System

7. ● In the first stage of development, program
only the parts that are the core of your
application
● But how to do this in git?
Define Minimum Viable Product

8. ● Have base/production code in master
branch
● git branch for every code change
● Merge all branches with acceptance
branch before going live (again)
Use branches (1/2)

9. Use branches (2/2)
master
acceptance
time
bi-weekly
release
change #123
branch
mergemerge

10. Deployment

11. ● VPS
● Provision with e.g., Puppet
● Find or create scripts
● Put the scripts in version control!
Server(s)

12. node 'kiezel.gw20e.com' {
class { 'ssh':
server_options => {
'PasswordAuthentication' => 'no',
'PermitRootLogin' => 'no',
},
}
appie::app { "mysite":
envs => ["tst", "acc", "prd"],
secret => "secret",
accountinfo => $gw20e::user_accounts,
accounts => ['ganzevoort', 'vandermeij'],
}
}

13. ● Create a user per environment-layer
combination
○ app-mysite-tst
○ app-mysite-acc
○ app-mysite-prd
● With their own home dir, postgresql DB,
nginx glue
● Check out (and fork) our puppet module:
○ puppet-appie @ GitHub
Separate users

14. ● Use fabric to script deploying to your
servers:
○ fab deploy:layer=tst
● Put the scripts in version control!
● Check out (and fork) our template:
○ templateproject @ GitHub
○ It includes gunicorn and supervisor configurations
Use scripts to deploy

15. ● Don’t just change Django’s settings.py
● Keep dev, tst, acc and prd specific
settings in separate files
● Put the settings in version control!
Different DTAP layers (1/6)

16. ● Deploy to their respective environment
Different DTAP layers (2/6)

17. DEV
● Laboratory setup
● Switch to the branch you’re working on
● Work with dev settings
● Deploy to TST, ACC or PRD environment
Different DTAP layers (3/6)

18. TST
● Real deployment, but alpha
● Work with tst settings
Different DTAP layers (4/6)

19. ACC
● Real deployment, but beta
● Work with acc settings
Different DTAP layers (5/6)

20. PRD
● Real deployment!
● Work with prd settings
● Setup monitoring
○ Nagios
○ Sentry
○ ...
● Setup backup
○ Database
○ Uploaded media
Different DTAP layers (6/6)

21. ● Move settings.py as is to:
settings/core.py
● Create: settings/base.py
○ Add: from settings.core import *
○ Add your own generic settings
● Create: settings/{dev,tst,acc,prd}.
py
○ Add: from settings.base import *
○ Add your own layer specific settings
Django settings (1/4)

22. Django settings (2/4)
settings.py core.py
base.py
LAYER.py
settings/

23. ● Use Fabric to create:
settings/__init__.py
● This command:
○ fab pick_settings:layer=prd
● Results in:
○ from settings.prd import *
● Stackable settings scale
○ Imagine (white) labeling
Django settings (3/4)

24. Django settings (4/4)
__init__.pysettings/
dev.py tst.py acc.py prd.py
base.py
core.py

25. Django settings (5/4)
Database passwords in version control?
from .base import *
read_pgpass('app-mysite-prd')

26. Maintenance

27. ● Scrum, RUP, etc.
○ Any iterative methodology will do
● Basicly:
○ Have periodic deadlines/releases
■ Bi-weekly
○ Deliver each iteration
■ To ACC, if OK, to PRD
○ Release early, release often (MVP)
Iterative development

28. ● Create a ticket for every change (RFC)
● Estimate the ticket
● Have discussion in ticket thread
● Create a code branch per ticket
○ Let git help you
● Deploy to a separate TST environment
○ (with separate database)
(How to) use a ticket system

29. Branching revisited
master
acceptance
time
bi-weekly
release
change #123
branch
mergemerge
iteration

30. ● On your test machine, let each ticket/branch
have its own environment (with database)
● Test implementations individually, care for
code merging later
● Multiple TST environments means multiple
deployments
○ So not only three (= TST, ACC, PRD)
Separate environments for TST (1/5)

31. ● change_123/$ fab deploy
● Is shorthand for:
○ fab deploy:layer=tst,branch=change_123
Separate environments for TST (2/5)

32. ● Isn’t setting up separate webserver(s),
database(s), etc. causing a lot of overhead?
Separate environments for TST (3/5)

33. ● Computer says NO
○ If you do it the right way
Separate environments for TST (4/5)

34. ● Keep your (webserver) configuration files in
version control
○ Use e.g., Django templates to construct environment
specific files
● Keep your database scripts in version
control
○ To copy a new DB from a production fixture
○ To install a new DB from scratch with fixtures
○ Or both!
Separate environments for TST (5/5)

35. Deployment continued

36. ● Have separate deployment settings per
layer:
○ use_https
○ path_to_certificate
○ sitename
○ deploy_user_at_host
○ dir_to_deploy_to
○ ...
Deployment settings (1/4)

37. Deployment settings (2/4)
LAYER.pydeployment/

38. ● Use the Django template engine to construct
configuration files on deployment:
{% if use_https %}
server {
listen 80;
server_name {{ sitename }};
rewrite ^(.*) https://$server_name$request_uri?;
}
...
Deployment settings (3/4)

39. ● Symlink the generated configuration files
from the user homedir into the place they’re
needed
● E.g.
○ ln -s ~/current/etc/nginx.conf
/etc/nginx/sites-enabled/SITENAME.conf
○ service nginx reload
Deployment settings (4/4)

40. ● Create a tag for each PRD release
● Switch to that tag on deploy to PRD
○ For easy rollback
■ Switch to previous tag to rollback
● Run migration scripts after switch
○ Watch out for backwards incompatible changes!
Use Git tagging

41. ● When using Pip
○ pip freeze > frozen.txt
● Add to version control!
● pip install -r frozen.txt
Freeze dependencies

42. Conclusion

43. ● Use a VCS
● Have separate DTAP layers
● Script and store
● Work with RFCs
● Make everything repeatable
○ By your colleague
Conclusion(s)

44. ● github.com/
○ Goldmund-Wyldebeast-Wunderliebe/
■ templateproject
■ puppet-appie
References

45. Thank you!
Job Ganzevoort,
Douwe van der Meij
Goldmund, Wyldebeast & Wunderliebe
{ ganzevoort, vandermeij } @gw20e.com