Pipeline your Pipelines - 2020 All Day DevOps
•Download as PPTX, PDF•
0 likes•101 views
11 Nov 2020 - All Day DevOps Pipeline your Pipelines
Recommended
Recommended
More Related Content
Similar to Pipeline your Pipelines - 2020 All Day DevOps
Similar to Pipeline your Pipelines - 2020 All Day DevOps (20)
More from Giulio Vian
More from Giulio Vian (20)
Recently uploaded
Recently uploaded (20)
Pipeline your Pipelines - 2020 All Day DevOps
- 1. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G N OV E M B E R 1 2 , 2 0 2 0 Giulio Vian Pipeline Your Pipelines (Automate Your Automation)!
- 2. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Hardware spec: 1 KB RAM (16KB after upgrade) 4 KB ROM (8KB after upgrade) First computerPast Companies Communities Giulio Vian Senior DevOps Engineer Past Communities
- 3. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G What if I lose my build infrastructure?
- 4. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Can I deploy fixes in production?
- 5. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G
- 6. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G What if I lose my build infrastructure?
- 7. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Can I rebuild an old version?
- 8. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G
- 9. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G What is a development environment?
- 10. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G What is an development environment?
- 11. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Only environments you can rebuild
- 12. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Bricks, mortar & a plan
- 13. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Usefulness scale Source code Keys Deploy data Testing data Build data Build infrastructure Precious Disposable
- 14. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Infrastructure as Code Version Control Secrets Store Infrastructure Pipeline
- 15. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Version Control Secrets Store Run-time Infrastructure Version Control Secrets Store CI/CD Infrastructure Version Control Secrets Store Application Run-time Fractal
- 16. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Demo time
- 17. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Backup
- 18. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Benefits
- 19. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Less chores
- 20. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Sense of accomplishment
- 21. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Learn new technology
- 22. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G Remove fear of change
- 23. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G The more you automate, the more everything becomes production
- 24. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G http://blog.casavian.eu/tags/pipelines/ https://github.com/giuliov/pipeline-your-pipelines https://github.com/actions/virtual-environments https://github.com/microsoft/azure-pipelines-image-generation https://www.mikaelkrief.com/private-azure-devops-agent/ https://wouterdekort.com/2018/02/25/build-your-own-hosted-vsts-agent-cloud- part-1-build/ https://blogs.blackmarble.co.uk/rfennell/2019/12/21/creating-hyper-v-hosted-azure- devops-private-agents-based-on-the-same-vm-images-as-used-by-microsoft-for- their-hosted-agents/ https://medium.com/velotio-perspectives/using-packer-and-terraform-to-setup- jenkins-master-slave-architecture-7bcc4b014874 Links
- 25. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G THANK YOU TO OUR SPONSORS
- 26. T R A C K : C I / C D C O N T I N U O U S E V E R Y T H I N G
Editor's Notes
- Welcome everyone to my session “Pipeline Your Pipelines” during which we will discuss how to automate your automation My name is Giulio Vian
- (A few words about yours truly) I worked for some companies over the years in quite a few different roles, now I work for Unum, a Fortune 500 insurance company. Recognised by Microsoft with the Most Valuable Professional award in the last 5 years, I like to help communities throughout Europe. When I started coding, assembler was not an uber-geek oddity.
- What if I lose my build infrastructure, temporarily or permanently?
- Am I able to deploy fixes in production? Can I answer to audit request? and to security which is analyzing an attack and wants to know if binaries were tampered? Oh, you say, it won’t happen to me, I am safe, I do not have my CI on premises, we do use a service…
- I am safe, I do not have my CI on premises, we do use a service… well, a few things may go wrong anyway Other things may happen to your CI/CD
- What if I lose my build infrastructure, because we retired that old machine and lost that exact version of JDK or Visual Studio installer? Will I be able to…
- …reconstruct an old software version for an investigation? Your company may be subject to law and regulation If you sell shrink-wrapped software, you may face a customer who hasn’t upgraded in years. Some IoT scenario is similar to shrink-wrapped, old hardware lingering in customers’ premises that you still have to test and patch. Oh, you say, it won’t happen to me, I am safe, I do not have my CI on premises, we do use a service…
- Again, even if you use a service, there is no guarantee that older version will stick around. Here is just a couple of announcements. Hope you agree with me that development environments…
- … are required for coders, thus they are production for them! Developers can forgive short interruptions but not for long: Continuous Integration, quality scan, Continuous Delivery, test environments… These are all front and center of modern development. So we must tweak the question to…
- What is an environment, tout court? A simple definition can be: An IT subsystem that is daily used by a group so the real distinction between environments lies on the fact that …
- …you can rebuild them or not. The photographs portrait the XII (twelfth) century palace in my hometown. It was destroyed by bombs during the second World War (1944). As you see, it was rebuilt. This should be our goal: being able to rebuild our CI/CD environment whenever needed. What you need for rebuilding?
- You need a few elements, some elements are concrete, like bricks and mortal, while other elements are virtual, like the blueprint and know-how. This applies in the software world, just like in the physical world.
- If you analyze the components that make up a CI/CD system, you can lay them on a scale. On one extreme, we have source code repositories: most businesses will shut down if they are lost. They holds your application code but also all build, test and deployment scripts. On the other extreme, the build infrastructure – e.g. the build agents/executors - should be treated as a disposable element. What means rebuilding in this context?
- at the core we find Infrastructure as Code. Note the crucial role of source code and secret keys to build infrastructure. The people icon on top represent both the bootstrap process and the authorization activity. This is the foundation pattern which…
- …is kind of recursive. To bootstrap, we use the source code for IaC and a hand-built CI infrastructure. Through that we produce the CI/CD infrastructure that is required by all other processes. The next step use CI/CD to build and update the infrastructure that is needed by applications. Finally, we can build the application deployment packages and push them through CD to get back all our systems back modulo data restore.
- I think you are now bored enough of chat, so let’s see something
- If you have fully scripted your CI/CD infrastructure, you need only the source code and secrets to rebuild it. Backups are absolutely crucial but you already know this.
- Now, I want to hint at a few positive side-effects of scripting everything.
- Life becomes easier: for example when a new version of Node or JDK or .NET is out, clone and edit the Dockerfile and the pipeline script. With the push of a button you make it available to your developers. Also, recycle often your agents/workers to guarantee that builds do not rely on a previous run.
- Your build and deploy infrastructure is finally documented. Celebrate and sleep well.
- If your engineers are not familiar with Docker & Kubernetes, this is a good chance for learning.
- Now, you rebuild your CI/CD infrastructure on any cloud provider. More options for that disaster recovery plan.
- Wrapping up. Back in time when you released every quarter, few noticed when your Jenkins was down unless it happened during that release window. Nowadays, you are urged to deploy more often, which requires more automation. And the more you release, the more CI/CD become crucial.
- A few pointers that you can leverage to implement the ideas expressed in my presentation.
- Thanks to the sponsors and all people that worked on the backstage to make this event successful.
- This is my final word