CloudConf 2020 in Streaming Talk about a sample of Automation Solution from local to production https://2020.cloudconf.it/ https://github.com/bilardi/aws-saving/ https://github.com/bilardi/aws-simple-pipeline/

1. Automation: from local test to production deploy
by Alessandra Bilardi - Data Specialist @ pixartprinting.it
05.11.2020 - CloudConf

2. • Pixartprinting S.p.A.
• Where: Quarto d’Altino, Venezia
• Type: Web2Print of Cimpress group
• Clients: 900.000
• Employees: 900
• Company surface: 35.000 MQ
• Orders per day: 12.000
• Packages sent in 1 year: 4M
• Countries reached: +50
• Product combinations: 3M
• VMs on premise: +200
• DynamoDB: +100
• Lambda: +500
• RDS: +50
Who we are

3. • Solution sample to automate
• Workflow
• TDD – Test Driven Development
• CI – Continuous Integration
• CD – Continuous Delivery
• CD – Continuous Deployment
• Where pay attention
Agenda

4. Solution sample – AWS saving

5. Solution sample – AWS saving

6. Solution sample – AWS saving

7. Solution sample – AWS saving

8. Solution sample – AWS saving

9. • Python package manages resources for
• Stopping
• Pausing
• Starting
• Deleting
Solution sample – AWS saving

10. • Python package manages resources for
• Stopping
• Pausing
• Starting
• Deleting
• Each resources has to have some tags
• Saving = Enabled
• Start = ‘0 8 . . .’
• Stop = ‘0 18 . . .’
• # Delete = ‘0 18 . . .’
Solution sample – AWS saving

11. • Python package manages resources for
• Stopping
• Pausing
• Starting
• Deleting
• Each resources has to have some tags
• Saving = Enabled
• Start = ‘0 8 . . .’
• Stop = ‘0 18 . . .’
• # Delete = ‘0 18 . . .’
Solution sample – AWS saving
• Behaviour
• From a list of services, for each resource,
• If it finds Saving tag Enabled
• if it finds a tag action and the current time fits
• then it runs that tag action

12. • Python package manages resources for
• Stopping
• Pausing
• Starting
• Deleting
• Each resources has to have some tags
• Saving = Enabled
• Start = ‘0 8 . . .’
• Stop = ‘0 18 . . .’
• # Delete = ‘0 18 . . .’
Solution sample – Plan
• Behaviour
• From a list of services, for each resource,
• If it finds Saving tag Enabled
• if it finds a tag action and the current time fits
• then it runs that tag action
• Services supported
• Cloudformation
• RDS / Aurora
• EC2
• S3

13. • Python package
• Importable on your code
• Deployable on AWS lambda
Solution sample – CI

14. • Python package
• Importable on your code
• Deployable on AWS lambda
• Each method has to have
• At least one unit test
• If it has params, also functional tests
Solution sample – CI

15. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
Solution sample – CI

16. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
Solution sample – CI

17. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
Solution sample – CI

18. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
Solution sample – CI

19. Workflow – CI

20. Solution sample – AWS saving

21. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
• For each service
• At least one integration test
• With resources deployed for testing all tag actions
(Stop, Pause, Start, Delete)
Solution sample – CI

22. • Python package
• Importable on your code
• Deployable on AWS lambda
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Each method has to have
• At least one unit test
• If it has params, also functional tests
• For each service
• At least one integration test
• With resources deployed for testing all tag actions
(Stop, Pause, Start, Delete)
Solution sample – CI
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests

23. Demo
Solution sample – CI

24. Workflow – CI

25. Solution sample – AWS saving

26. Solution sample – AWS saving

27. Solution sample – AWS saving

28. Solution sample – AWS saving

29. Solution sample – AWS saving

30. Solution sample – AWS saving

31. Workflow – CI

32. • Deploy your python package versioned
• AWS Developer Tools
• GitHub / GitLab
• Others
• Continuous Delivery
• Manual test (ie: by quality environment)
• In production with Manual Approval
• Continuous Deployment
• Deploy based on tests passed
• In production without Manual Approval
Solution sample – CD
• Best practice for CD
• Same build templates for all environments
• Automate as many steps as possible
• Also automate the merge on the main branch

33. Workflow – CI

34. Workflow – CI

35. Workflow – CI

36. Workflow – CI

37. Workflow – CI

38. Workflow – CI

39. Workflow – CI

40. Workflow – CI

41. Demo
Solution sample – CD

42. Workflow – CD

43. Workflow – CD

44. Workflow – CD

45. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

46. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

47. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

48. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

49. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

50. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

51. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

52. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

53. Workflow
Images: https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

54. AWS Workflow

55. AWS Workflow

56. AWS Workflow

57. AWS Workflow

58. AWS Workflow

59. Where pay attention
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests
• Best practice for CD
• Automate the merge on the main branch
• Same scripts & templates for all environments

60. • Collaboration during the plan of changes
Where pay attention
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests
• Best practice for CD
• Automate the merge on the main branch
• Same scripts & templates for all environments

61. • Collaboration during the plan of changes
• Before Unit / Functional / Integration Tests
Where pay attention
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests
• Best practice for CD
• Automate the merge on the main branch
• Same scripts & templates for all environments

62. • Collaboration during the plan of changes
• Before Unit / Functional / Integration Tests
• Small changes
Where pay attention
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests
• Best practice for CD
• Automate the merge on the main branch
• Same scripts & templates for all environments

63. • Collaboration during the plan of changes
• Before Unit / Functional / Integration Tests
• Small changes
• Automate as many steps as possible
Where pay attention
• Best practice for TDD
• The first code is the test
• At the beginning, the test has to fail
• At the end, the test has to pass
• Best practice for CI
• Small and frequent commits
• Feature flags
• Packaging for all tests
• Best practice for CD
• Automate the merge on the main branch
• Same scripts & templates for all environments

64. pixartprinting.it @abilardi
Thanks!