Keynote at CloudExpo and DevOps Summit 2014, Silicon Valley
•Download as PPTX, PDF•
14 likes•9,894 views
Purpose-Defined Computing: The Next Frontier in Automation
Recommended
Recommended
More Related Content
What's hot
What's hot (9)
Viewers also liked
Viewers also liked (20)
Similar to Keynote at CloudExpo and DevOps Summit 2014, Silicon Valley
Similar to Keynote at CloudExpo and DevOps Summit 2014, Silicon Valley (20)
Recently uploaded
Recently uploaded (20)
Keynote at CloudExpo and DevOps Summit 2014, Silicon Valley
- 1. Purpose-Defined Computing The Next Frontier in Automation Victoria Livschitz, CEO of Qubell @vlivschitz @qubellinc
- 2. Next 10 years will bring more change than last 50 combined! –2014 survey of 250 global retail CEOs
- 3. AGENDA Dancing Elephants Purpose-Defined Computing The Golden Age of the Developer
- 4. Enabling Enterprises to Compete Teaching the Elephant to Dance
- 5. Core Conflict Speed and Self-service Stability and Control VS
- 6. DevOps 1.0: Embraced the Cultural Divide Bureaucrats (Ops as seen by developers) Cowboys (Developers as seen by Ops)
- 7. DevOps 2.0 Changing Role of IT – Zero to Hero
- 8. “Help Me Help You”
- 9. Kira Makagon EVP of Innovation
- 10. The Making of Magic Buttons Purpose-Defined Computing
- 11. Purpose-Defined Computing The complete computing system emerges in response to a business need.
- 12. Button Makers and Button Users
- 13. Search eCommerce Personalization Payment Gateway
- 14. Search eCommerce Personalization Payment Gateway
- 15. Search eCommerce Personalization Update Index Patch OS Upgrade Schema Payment Gateway
- 16. Search eCommerce Personalization Update Index Patch OS Upgrade Schema Payment Gateway
- 17. Current Design Approach: Fork Scripts Configuration A Configuration B CentOS CentOS EC2 20Mb Data WebLogic API Stub Production Script A Script B Testing RedHat 2Tb Data WebLogic API WebLogic RedHat
- 18. Configuration A Configuration B CentOS CentOS EC2 20Mb Data WebLogic API Stub 2Tb Data WebLogic API A Better Way: Adaptive Configuration Policy A Policy B WebLogic RedHat RedHat Testing Production
- 19. Configuration Change Management vs. Deployment
- 20. Configuration Change Management vs. Deployment
- 21. Instance Configuration (t) = F (Application (t), Environment (t) Policy (t))
- 22. Enabling Creativity and Productivity The Golden Age of the Developer
- 23. Demand for Developers Exceeds Supply “ We will need to increase the number of programmers from 350,000 to 1 million to achieve self-sufficiency. –Nikolai Nikiforov Russian Federation, Communications and Mass Media Minister
- 24. Demand for Developers Exceeds Supply “ We will need to increase the number of programmers from 350,000 to 1 million to achieve self-sufficiency. –Nikolai Nikiforov Russian Federation, Communications and Mass Media Minister “ There are about 18.2 million software developers worldwide, a number that is due to rise to 26.4 million by 2019, a 45% increase.” “ Employment of software developers is projected to grow 22 percent from 2012 to 2022, much faster than the average for all occupations.” –The U.S. Bureau of Labor Statistic –Evans Data Corp, Global Developer Population and Demographic Study
- 25. With Great Power Comes Great Responsibility Lower the barrier to learn new technologies • No luxury of spending “10-years to master C” Shorten time-to-value on new projects • Join by 9:00am, first commit by 5:00pm Faster feedback loop on every change • Discover and fix bugs before commit
- 27. “The Magic of 1 Man to 1 Computer”
- 28. Post-Commit: Automated Quality Control Agile Software Factory
- 29. Post-Commit: Enforcing Release Pipeline Regression Commit CI Upgrade Integration Performance User Acceptance Mobile Staging CI Regression Integration Performance User Acceptance Mobile Staging Dynamic Environments
- 30. Ratnakar Lavu EVP, Digital Services
- 31. Happily Ever After The Marriage of Agility and Governance
- 32. Thank you Victoria Livschitz @vlivschitz @qubellinc
Editor's Notes
- Good morning, CloudExpo! My name is Victoria Livschitz. I am the founder and CEO of Qubell. For those who have not heard about us yet. We are a configuration management startup for web scale applications. Our platform collects configuration information about the web-scale applications - what services they talk to, what components they consist of, the nature of the dependencies between them and the infrastructure on which they run - and then uses that information to make launching, deploying, scaling, patching, upgrading and shutting down these sophisticated online systems as seamless and routine as pressing a button.
- 2
- There are three main points to my talk today: Elephants – traditional enterprise industries – must learn to dance. - Need to become digital, agile and innovative - Able to take fast decisions that change the status quo Short feedback loops, an experimental approach to “what if” Iterate over many ideas concurrently, in small increments, quickly. 2. Innovation must become an industrial process. - Must become agile software factories - Conveyor belt running through them, continuously carrying…. To enable these fully-automated conveyor belts Must embrace and enable what I call “purpose-defined” computing. Just-in-Time creation and decommissioning of computing environments made to-order for a purpose fully automated “supply chain” of infrastructure, services, datasets.. IT operations must enable the automation that makes such dynamic assembly possible. 3. Responsibility for innovation rests withdevelopers. It’s a great time to be a developer, the Golden Age, so to speak. Developer productivity and creativity is greatest asset. - Collective challenge - make developers more creative and productive through self-service access to everything they need, whenever they needed, from anywhere in the world. I’ll suggest ways to think about these three areas. Let’s begin by looking at the challenge of getting elephants to dance…
- 1. Large organizations are on a quest to become more agile. It's one thing for start-ups to be agile, to pivot every other month, to experiment, to throwing in new features and measure customer reaction and see what sticks. Likewise, powerful web company like Amazon on Google Have built an incredibly powerful compute machine Able to use that machine to push out innovation and test the market 3,000 updates/day 2. The rest of the non-digital, non-high-tech world the pharmacies, the hospitals, the universities now required to live and abide by the rule of high-tech. need to compete with both Amazon and start-ups. If you are a retailer, think Apple – need to integrate instantaneously. If you can't let somebody walk in your store and expect to get that integrated experience, if they don't get it with you they’ll go somewhere that they do. 3. 2014 is a watershed year. Last year agility was not making the top 10 list of the CIOs. Now - it’s a top priority. So, why is becoming agile so hard?
- 5
- 1. I would argue that the first generation of Devops started back around 2009 idea that since IT Ops can’t give developers what they want - cloud and self-service - they’ll get these things themselves, bypassing IT. Amazon entered the enterprise through Line-of-Business developers getting the computing they needed from the outside of IT, and throwing Jenkins and Chef scripts on top to deploy their applications. 2. This put IT in a bad place buried under the avalanche of change requests blamed for slowing the train down yet responsible for catching releases thrown at them over the wall to deploy and operate while knowing full well they were written by folks who don’t understand operational concerns and have not tested them on certified configurations. This breeds mistrust and blame.
- 7
- Can IT ops folks in the room relate? Jerry is the Ops guy who spends restless, thankless nights trying to close the tickets opened by the impatient stars of the show, the developers. Ops guys need help to give the developers what they want and need.
- Thank you very much, Kira, for being with us this morning and sharing your perspective. Today at 3:30pm will be a technical session focused on details How RingCentral tackled devops for big data.
- Now let’s look into the principles that will allow us to power the agile software factories with next-generation automation engines.
- The inspiration is to have a “complete computing system emerge…” Need may be Test new build Integrate latest framework upgrade Certify existing apps on a new cloud Connect to new service Push your latest change to production
- We need buttons controlled by the power users that give them what they need As Scott McNealy used to say, they need the car, not the camshaft. What they don’t need is a pile of parts: an engine, a steering wheel, a transmission and a set of doors. They need a complete vehicle - infrastructure, third-party services, datasets, tools. Batteries included! Think of a distinction between the button makes and button users. Our quest for better automation means more buttons, and smarter buttons for power users not the mechanics!! So, let’s take a look at the button-making business
- The first button every organization needs is “Launch”. Launch creates a freshly-minted instance of an application from nothing. Launch replaces IT tickets to create a new development environment, test cluster, customer demo, or production instance. Launch should only take minutes to complete It should produce a fully-constructed, ready-to-use, “batteries included” working instance of an application stuffed with data, loaded with the tools, connected to services. All this assembly and configuration across distributed nodes happens Just-in-Time Dynamically manipulating the supply chain of applications, their components and the resources required to run these components. - The Launch button is given to developers, testers, release engineers, sales engineers, field technicians, IT ops, and support staff as a self-service. The Launch button provides instant gratification. It eliminates infrastructure - and most importantly - the need to know how to configure infrastructure - from being a pipeline bottleneck. It replaces sequential operations with parallel operations. - It liberates any team to have access to any environment at any time - manufacture the kind of environment they need or want on demand. The Launch button is very, very cool.
- No budget covers the cloud bills if there was “Launch” button without a “Destroy” button. Environments cannot be allowed to spraw uncontrollably or stay up forever. In fact, having the “Launch” button requires 2 things: 1. regulate how many environments are allowed to be created on a public cloud or determine who wins the resources when demand exceeds the supply on a private cloud. 2. Auto-destruction of unused environments - either on schedule or after a period of inactivity. Destroy button must clean up the resources return them back to a pool release the locks on any external services. The Destroy button may not be as cool as the Launch button, but it is absolutely vital.
- It is great to be able to create a fresh and up-to-date environment whenever we need one. However, by the time it had been constructed, the environment will likely become old and obsolete. Change is constant. An infrastructure team has just released a new security patch. A dev team down the hall just updated their APIs. The test team just shipped the new version of a test suite. My buddy committed change upstream that breaks my component. It is vitally important to be able to reconfigure the existing environment to make it compliant with the latest changes. Production upgrades are the obvious and frequently talked about special case of the “Reconfigure” button Also important to be able to roll back, or “downgrade”. Without the ability to take the changes back and freely move between versions - up or down - In summary, Reconfigure is really a marvelous button that covers a spread of common use cases, and its use must become much more pervasive, used in all environments all the time. - When used properly, the three magic buttons – Launch, Destroy and Reconfigure – provide a solid foundation for application lifecycle change management in the era of speed and agility.
- There are other buttons you might want to make for convenience, like “Scale”, “Patch” or “Add Tenant”. These are specific for your business and could improve productivity. A word of caution: use buttons responsibly. It is tempting to go nuts make a button for each conceivable workflow create hundreds of buttons for every purpose. You’ll likely to end up breeding complexity just as bad - or worse - that the one we are trying to leave behind. It’s better to have a small number of buttons that work well and are well-supported, than a lot of buttons that are sketchy and unsupported. Let me elaborate
- When someone in the organization presses the “Launch” button the result of what gets launched depends quite a bit on the purpose behind the intent. The same application - say, we are talking about a search engine - has to be configured differently for regression testing, performance testing, staging or production. Purpose affects configuration dramatically. This plurality of configurations for a single application presents a design dilemma. It’s tempting to make as many “Launch” buttons as there are unique purposes, each producing a different configuration. This is, pretty much, what people have been doing Write a bunch of Jenkins jobs that call a bunch of Chef recipes, fork them for every stage of the pipeline and make changes to the scripts that call the scripts. Pretty soon, you have a mountain of scripts that are impossible to test, maintain and support. In an agile organization, purpose changes often - which is what agility really means. So, if your automation is based on a “copy/edit” approach to configuration change management, you will end up breeding complexity on grand scale.
- We have another design choice. Instead of making single-purpose buttons, we can make the configuration and button behavior “adaptive” to the purpose. The path to adaptive configuration lies through policy. Policy codifies the purpose in a set of rules and constraints imposed on the configuration, as this picture depicts. The same “Launch” button applied to our search engine for “Testing” or “Production” will construct two different configurations. No re-coding. No multiple sets of scripts or orchestration workflows to write. The configuration is “computed” from the constraints imposed by the policy. I’d argue that the next-generation devops platforms will deliver adaptive configuration management and behavior as THE preferred way of managing complexity.
- And this brings me to the main point of my talk. - The industry talks a lot about the deployment automation. “Deployment automation” problem is a decoy. A wrong choice of the framing of the problem. We don’t have deployment automation problems. We have configuration change management problem. - Deployment automation would be easy, if not for the need to account for constant change. The best way to think about deployment automation is to think about the printing press. The original Gutenberg kind. The press always prints the same book, quickly and reliably. It’s great for making a thousand copies of the exact same thing. Application blueprints, recipes and scripts are all artefacts of this kind of a printing press. - But what if the content of the book changes from a run to a run? Same book, minor changes - add a footer, fix a typo, change font, insert an epilogue. Change happen often - once every few or few hundred runs. Same book, slightly different versions. Let’s also add another type of a change - a paper might change from a run to a run, too. It may be thinner, thinker, glossier, with watermark, longer sheets, different colors. What effect does that have on the design of the printer? - Quite significant, actually. Now the press must understand the structure of the content, and the structure of the media it prints on. It has to understand the elements of the book and the paper, and the relationships between these elements. It must understand how to apply change management to the configuration. - Factor this is, and 5.5 centuries later you end up with something that looks like this.
- - But that’s not the whole story. Imagine: PRINTED, SOLD, RETROFIT. Mandatory. We don’t know how to design. Replaceable pages.. replacement kits with instructions … “replace these 3 pages” with “these three” kind. … not all changes are compatible… 4 new pages instead of 3, … screw up page numbering - Relationship “Page number -> FOOTER”. When you change one, you must change the other. - Replace vs. upgrade. Need upgrade service. Complexity -> New requirement -> CHANGE. OFTEN. The more frequent the change, the more complex is the problem and more systemic is the response. - If this was a frequent situation, it would change how books are priced, sold, tracked and stored. It is exactly what we are dealing with in the world of applications. We think we have a 1-time printing problem. But that’s just not it. Launch new instance -> automation platform PROVISIONS. DEPLOYS MUST UNDERSTAND elements, dependencies. By POLICIES, PURPOSE Once UP – RECONFIGURED. PATCHED. SCALED. Cannot be performed SAFELY if they VIOLATE existing dependencies. Right answer: UPGRADE vs. REPLACE??
- - This kind of application CONFIGURATION management associated DEPENDENCY management is at the heart of the purpose-defined computing. It is what ALLOWS live applications to ABSORB the constant CHANGE gracefully, and ENABLE continuous upgrades to become safe and routine.
- 22
- First, an obvious but important fact is - there is not enough developers in the world to satisfy the demands for new software. A Russian minister of communications recently went on record that the country urgently needs to TRIPLE DEVELOPER RESERVE to ACHIEVE INDEPENDENCE from the foreign technology vendors. Good luck with that, Mr. Nikiforov.
- On more serious note, look at more relevant facts. Quite simply, DEMAND for programmers GROWING FASTER than any OTHER OCCUPATION, and the demand far outruns the supply. We are going to see more people enter the profession, more countries participate in the global workforce, and still the problem remains … writing MORE SOFTWARE BY BRUTE FORCE IS NOT SUSTAINABLE The great news for everyone in this room - you are becoming increasingly more valuable to your organizations. Yes! But there is always the other side to every coin. You will be EXPECTED TO BECOME MORE PRODUCTIVE YEAR OVER YEAR
- This is the challenge the industry will face in the years to come. Developers must LEARN NEW languages and frameworks quickly. When they are assigned to a new project, they must start BRINGING VALUE QUICKLY. And they are expected to SHIP THINGS THAT WORK. That’s a big step from being asked to throw code over the wall and wait to hear if it works from QA and if it runs from Ops. At the end of the day, developers simply OWN MORE. And with great power comes great responsibility. And the ENABLER OF THESE POWERS is SMART AUTOMATION TECHNOLOGY we already talked about. It moves the game from HORCE RACING to FORMULA 1.
- If developers had a Bill of Rights, they would have an INSALIEBLE RIGHT to press a Launch button at any time and get a private “sandbox” copy of the entire system at our disposal - to experiment with and test against. Traditionally, the developer sandbox was her desktop or laptop. That’s where development happened. But what if the application can’t work without accessing FACEBOOK or TWITTER feed, running queries against the EXTERNAL ENGINE or connecting to the telephony server? This broader application environment is not usually part of the sandbox. To help make a point for just how important a private lab environment is to developers, I’d like to bring up a clip from a Steve Job’s interview in the early days of Apple, back in 1980.
- When Steve was talking about the MAGIC that happens between a person and his own computer - the magic that made computers “personal” and fueled CREATIVITY, PRODUCTIVITY and LEARNING for two generations I realized that the exact SAME MAGIC happens between a developer and his environment. The environment has to be PERSONAL – NOT SHARED between everyone on the team because it has the PERSONAL of the developer’s code and his FAVORITE TOOLS. 34 years after Steve spoke, the personal computer has been AUGMENTED. First by NETWORK, then a CLOUD, and SOFTWARE has evolved into SERVICES. A personal computer had become a PERSONAL ENVIRONMENT, a CLOUD LAB. And the job of automators is to be able MANUFACTURE THAT LAB TO ORDER, based on the PERSON and the PURPOSE.
- Once a change is out of the developer’s hands and committed to the shared source code repository, a different sort of a process takes a center stage. INNOVATION AT SCALE requires an INDUSTRIAL APPROACH to change verification and rollout. Big Releases that happen once every few months are a thing of the past. Instead of batching up hundreds of changes to all parts of the systems into one giant update which will take weeks to test and certify… THE GOAL IS TO INSTALL A CONVEYOR BELT that constantly picks up the latest changes, pushes them through a series of automated tests and rolls them out of production on the other end of the pipeline.
- I call this kind of software operation an Agile Software Factory. An Agile Software Factory brings together two things. First, BUSINESS WORKFLOWS associated with the release pipeline management and promotional decisions. 2. the logistics of on-demand, purpose-defined environment creation and management TOGETHER, you get INTEGRATED , CONTINUOUS INNOVATION PIPELINE CAPABLE OF DELIVERING production UPGRADES DAILY or HOURL. SAFELY and ROUTINELY. To discuss how that marriage happens in practice in traditional enterprises, please help me welcome Ratnakar Lavu, EVP of Digital Services at Kohl’s.
- Content TBD
- In summary, I’d like to RETURN to where I started. - To the marriage of agility and reliability. To making peace between the Cowboys and Bureaucrats. To realizing Steve’s vision in our era. Is it within our reach? I certainly believe so. The mighty web companies are proof – delivering continuous innovation at a grand scale. Now LEADNING COMPANIES in every industry, like RingCentral and like Kohls, are HARD AT WORK turning their development and IT operations teams into Agile Software Factories. That the STORY WE HEARD from Kira and Ratnakar. That’s the bottom line. It’s happening today. The NEXT FRONTIER OF INNOVATION within the devops community is to CONQUIRE APPLICATION CONFIGRATION CHANGE MANAGMENT and BUILD AUTOMATION ENGINES capable of delivering PURPOSE-DEFINED COMPUTING with a single click. Then devops teams can focus on the button-making, and deliver ONE-CLICK, SELF-SERVICE experience to millions of developers, testers and release engineers. Then the elephant will truly dance. Thank you.