This presentation was delivered jointly with a hands-on demo. The presentation briefly discusses how Cloud Foundry enables organizations to continuously deliver high-quality software and highlights an integrated development process built with Jenkins, Artifactory and Cloud Foundry.
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
An Integrated Pipeline for Private and Public Clouds with Jenkins, Artifactory and Pivotal Cloud Foundry by Jamie O’Meara
1. An Integrated Pipeline for Private and Public Clouds with Jenkins,
Artifactory and Pivotal Cloud Foundry
2. #jenkinsconf
Who is Jamie O’Meara?
• Cloud Foundry enthusiast
• 20 years of software engineering experience
• Twitter: @JamieOMeara
• Last week completed two bucket
list items
• Cliff Diving in Maui
• Surfing @ Kihei
6. #jenkinsconf
How Did They Do It?
Culture built around change
• 12-24 releases per year for apps
• Experimental releases, A/B testing, production?
Communicative Process
• Frequent releases delivered to users
• User Feedback Loops and Metrics
Technology and Tools
• Automated Platform Delivery and Services
• Cloud Services
8. #jenkinsconf
What is Cloud Foundry?
1An open source platform
http://cloudfoundry.org/
Portable Across Clouds
Polyglot and API CentricHighly Available and Scalable
2
3 4
9. #jenkinsconf
The Value of Platform
1
Containerization
5
Scaling
7
Log Aggregation
and Metrics
2
Role Management
Services
6
9
IaaS Provisioning
and Configuration
3
Load Balancing
and Routing
4
8
Security
Health
Management
10. #jenkinsconf
Blue/Green Deployment
CF Router Blue
Blue
Green
Blue
Green
CF Router
CF Router
GreenCF Router
1 2
3 4
app.example.com
app.example.com
app-new.example.com
app.example.com
app-new.example.com
app.example.com
app.example.com
13. #jenkinsconf
Best Practices/Lessons
• Use the CB Jenkins CF plugin
• Hash commits and Build numbers for
versioning.
• Canonical Route Names for easy of use and
external access.
• SCM for config details and env variables.
• CD design includes multi-cloud for portability
and scale.
14. #jenkinsconf
Future Designs and Topics
• Jenkins Workflows
• Asynchronous pipelines and check points
• Orchestration of micro services.
• Docker images via Cloud Foundry Diego
• On-premise Cloud scale deployment of
Jenkins on Cloud Foundry.
• CF plugin auto-pilot (Alternative)
• plugins.cloudfoundry.org
15. #jenkinsconf
References and Source
• http://github.com/omearaj/App-Pipeline
1 - No Ordinary Disruption The Four Global Forces Breaking All the
Trends. PublicAffairs 2015
Editor's Notes
We’ve entered an era in software development where the emphasis is on rapid delivery. The companies listed are disruptive in their respective industries for a number of reasons. One thing they share in common is a very active software delivery pipeline. For example, Square and Airbnb have released their iOS applications 9 and 12 times since January of this year respectively.
In 2013, some 450,000 active users were launching the Uber app every week. Here is a marketing blitz from Uber. You can see the impact their application has on my business trips and especially San Francisco. I’ve had many interesting conversations with drivers regarding their appetite for Uber, how its changed the industry and how they are treated by the other cab companies and competitors. Ironically many of them drive for Uber and the competitors based on demand.
A behavior I noticed frequently with Uber is the number of releases to their iOS application. Every time I arrive and use my Uber app I need to download the latest release. BTW this is a feature I wish they’d relax. Changes to client apps come with changes to backend systems. All of these changes need to be orchestrated and automated to make delivery possible.
How do these innovate companies move with this speed? I’m asked this question by customers all the time. We want to be agile like Uber how can we accomplish this change. Its important to note, companies like Uber have built their company and culture to be agile. Its evident from their software release cycles. They’ve conducted 8 releases in 2 months. Other agile companies share the same passion for software releases. They experiment with release, conduct releases like A/B where they can test and gain valuable insight on their software release. Many of them have changed the definition of production release. Can anyone explain what a production release is today?
I believe the transformation in software development and delivery leads to higher quality software. For years, we’ve focused on creating higher quality software. Unit test, functional tests, UI tests and integration tests is where we spent the earlier years of agile software development. We shifted to new methodologies like Test Driven Development to change our mind set about the software we produce. Continuous integration allowed us to feel confident about the software we wrote and validate its ability to pass testing.
Continuous deployment and delivery are ushering in a new set of standards helping us to continue to validate our software builds and ensure the delivery of those builds is as successful as the build phase. This approach needs to work across delivery boundaries. If we deliver to AWS today we must be able to feel confident our delivery to on-premise tomorrow is not compromised by proprietary APIs.
Configuration management is also striving for a better approach to software delivery. Gone are the days of property files and isolated black box deployments. Configuration management is moving to the source control repo. Configuration settings are provided by the platform and can be changed and version controlled.
Techniques like Blue/Green deployments are making organizations reconsider the definition of a production deployment. Using these techniques reduces the risk of a bad deployments, lessens the stress and burden on developers and operators and increases the feedback loop on creating better software.
In my opinion, a key element to continuous delivery is to use Platform as a Service. I think these 4 key elements are an absolute must for evaluating a platform,
A platform should have a strong, vibrant open source community. Cloud Foundry is the most popular open source platform today.
A platform provides a clean abstraction from underlying details of a Cloud or IaaS provider. It should be portable across Cloud/IaaS providers.
The platform should provide a layer to scale at the cloud and application level.
An API centric approach is always good for developers who are truly the owner and stewart to the application.
Platforms provide additional features to the continuous delivery of software. Benefits for the operator and developer. IT Operations has spent the last few years attempting to automate and deliver higher value services to the organization. Platforms make this possible by allowing operations to roll out a Platform as a Service offering to developers who can now build and release w/o the need for operations to get overly involved or script time consuming and complicated solutions. Platforms provide tremendous value for developers too. Automated load balancing, health management, instant scaling, log aggregation and metrics and highly available services like persistent stores and messaging.
A pattern that has emerged for production deployments is the blue/green deployment. Designed to reduce risk of a bad software release, the Blue/Green deployment installs side-by side deployments by pushing the latest release while maintaining the existing release. Traffic is drained to the new release and the old release is maintained in case of roll back. Explain the diagram above from a cloud foundry perspective.
Lets review today’s demonstration to ensure we understand what will be demoed. In my example I have an existing application with a release build of 56, the jenkins job number. The application runs in my datacenter and we push the application to a public provider for burst traffic.
My team has decided we need to add a new “Awesome” button to our application based on feedback from our user community. Our first step is to create the new “Awesome” button and test it on our local development environment. After writing the unit tests and the code we feel confident in our change and merge the code to github.
This merge triggers the continuous deployment pipeline, delivering the software into test and Q/A environments where we will run further functional tests and prepare it for promotion to production.
Demo of basic pipeline using Jenkins, Artifactory and Pivotal CF