AWS Summit - Trends in Advanced Monitoring for AWS environments
•Download as PPTX, PDF•
2 likes•1,222 views
Why you have to rethink your monitoring strategy when moving or building apps for new stack cloud based environments: #1: Why "the old way" of monitoring doesnt work any longer! #2: How the Cloud and New Stack has transformed Dynatrace! #3: How Dynatrace Redefined Monitoring for Cloud Applications
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to AWS Summit - Trends in Advanced Monitoring for AWS environments
Similar to AWS Summit - Trends in Advanced Monitoring for AWS environments (20)
More from Andreas Grabner
More from Andreas Grabner (12)
Recently uploaded
Recently uploaded (20)
AWS Summit - Trends in Advanced Monitoring for AWS environments
- 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Andreas Grabner / @grabnerandi Wednesday April 19th, 4:30-5:00 Trends in Advanced Monitoring for AWS environments
- 2. confidential Dynatrace is the industry-first AI-based monitoring solution Runs on the AWS Cloud, monitoring thousands of servers The biggest companies in the world trust us when it comes to monitoring We are certified for AWS competencies Available via the AWS marketplace
- 3. Monitoring used to be about looking at dashboards …
- 6. .. and about analyzing logs & exceptions …
- 7. confidential
- 8. .. and finding bad code through CPU Sampling
- 10. But the apps and services we build have transformed …
- 11. Develop Ship Deploy Run Scale Compute nodejs mongo db netty cassandra redis ansible jenkins puppet chef docker cloudfoundry rh openshift rh atomic rocket core os rancher kvm busybox mesos marathon kubernetes swarm amazon azure openstack mesosphere calico weave eureka/hystrix A whole new technology stack & polyglot development
- 12. ruxit theme 2014.05.15 Big monolithic application. Small interconnected purpose-built services. Develop
- 13. ruxit theme 2014.05.15 Ship Deploy Big Bang Releases of single special built Small continuous service delivery of standardized delivery
- 14. ruxit theme 2014.05.15 Run One size fits-all operating systems. Slimmed down OS to run containers.
- 15. ruxit theme 2014.05.15 Compute Hardwired datacenters. Datacenter as an API.
- 16. THIS IS WHY monitoring had to transform as well
- 17. 2major releases/year customers deploy & operate on-prem 26 major releases/year 170 prod deployments/day self-service online sales SaaS & Managed 2011 2016 sprint releases (continuous-delivery) 1h: Code -> Prod6months major/minor release
- 18. confidential Monitoring as Pipeline & Platform Feature Dev Perf/Test Ops Biz Faster Innovation with Quality Gates Faster Acting on Feedback Unit Perf Cont. Perf New Deploy New Capability CI CD Remove/Promote Triage/Optimize Update Tests Innovate/Design $$$ Lower Costs Happy Users
- 19. confidential acting as Engineers Role of Dynatrace DevOps Team Dynatrace Managed/SaaS Orchestration Layer DynatracePipeline Visualization Deployment Timeline Log Overview using Dynatrace Log APIJIRA Integrations & Product Managers
- 20. confidential Learnings when scaling DevOps Pipelines Service Team A Service Team B Service Team X Improve “Efficiency” Cloud Ops Ensure “Operational Service” PM/Biz Improve“Business”
- 21. confidential Be proud of your feature! DevOps NoOps
- 22. confidential Dynatrace Transformation by the numbers 26 170 Releases / Year Deployments / Day 31000 60h Unit & Int Tests / hour UI Tests per Build More Quality ~200 340 Code commits / day Stories per sprint More Agile 93% Production bugs found by Dev More Stability 450 99.998% Global EC2 Instances Global Availability
- 23. Confidential, Dynatrace, LLC Monitoring redefined Every user, every app, everywhere. AI powered, full stack, automated. Full lifecycle - development, test, and production
- 24. Here is what we have built …
- 25. confidential OneAgent for all Technologies and Use Cases
- 26. confidential We get to know your environment better than you do small, and bigones …
- 28. We baseline your hosts, processes and services
- 29. … And understand your end users
- 30. confidential And we speak AWS natively
- 31. All this new monitoring requires a new approach to analytics as well …
- 32. confidential Cloud Ops: Automatic Problem, Anomaly and Root Cause Detection
- 33. confidential Biz: Automated Problem, Impact and Root Cause Detection
- 34. confidential Dev: Executing Fact-based Remediation Actions #1: Compare w prev. Timeframe / Release #2: Actionable Diff-View for Devs
- 35. Redefining monitoring doesn’t stop with monitoring and analytics …
- 36. confidential davis – the first digital assistant
- 37. confidential Everybody knows how to ask a question
- 38. confidential How are we doing from an operational perspective? Do I need to adjust the priorities of the team? Anything important for my team? The standup
- 39. confidential The board meeting Are our digital processes working well for customers ? Are we meeting availability, technical requirements and costs? Are customers adopting new functionalities as expected?
- 41. confidential Learn more @ https://www.dynatrace.com/meet-davis/
- 42. confidential AI-Driven Digital Operation Ecosystem DAVIS / REST
- 43. confidential Try what we’ve built @ http://bit.ly/dtsaastrial or visit http://www.dynatrace.com Stalk me @grabnerandi
Editor's Notes
- Instead of talking about what Dynatrace does I want to start with what connects us all: The Cloud has transformed our industry in many ways – and it has and had an impact on all of us. From a monitoring perspective …
- That may have worked well for static environments where you knew what you are looking at
- It may have worked in scenarios where you manually and visually correlated business, frontend and backend metrics to identify why certain issues happend
- If your apps gave you logs you could use log analytics to analyze the log files ->in case you knew what to look for and in case the log messages were actually written We could also correlate logs and exceptions to identify strange patterns
- When we had a performance issues or a resource spike it was most often a case where you then relied on sampled application monitoring data
- All of this worked well in case the applications were rather static – not too large and you had the people that understood how to analyze data provided by different tools BUT – the world has changed
- These is the new technology stack we are dealing with – and it is by far not complete New players coming and going – allowing us to implement new types of apps with new architectural and deployment options
- As you move toward these apps of innovation, not only will the technologies change, but so will your processes. The left side is likely how you currently create your apps of engagement. Multiple-month development cycles, a big group of engineers all working together on a monolithic application. You may be improving those processes with some continuous integration and some aspects of agile development, but there is a big shift when talking about your apps of innovation, represented on the right. In order to be agile and be faster to market for customers, you have to change the way you develop. Here you have smaller, more agile teams like scrum teams working independently on small, interconnected services or micro-services. The services are loosely connected, which means that each team can release on their own without affecting other services. This is a big change for development teams. It represents a change in culture and also enables a shift to DevOps and continuous delivery. Discussion questions: Which side more closely represents the way you’re doing development today? Are you outsourcing some of that development effort?
- Another element in this transformation is how to ship and deploy that code. Classically companies do what we call “big bang” releases, meaning a 6-12 month release cycle and 2 or 3 weeks to get the release ready to work in the real world for customers. You may have some well documented or semi-automated procedures to bring new code into production, but it’s still a time-consuming and painful process that involves a lot of heavy lifting. With apps of innovation you are shipping small micro-services, and you do it at a much higher frequency on a biweekly sprint or even down to a daily basis. In doing this you avoid the risk associated with the big bang deployment, and you also avoid the lengthy firefighting and 24/7 all hands on deck sessions to get your application rolled out. You’re deploying faster and in a more automated fashion. That has a lot of implications for monitoring that we’ll talk about more. Not only does your monitoring now have to be automated but it has to be able to pick up those small changes in real time. Discussion questions: How do you ship and deploy new code today? Does it more closely resemble the left side or the right side? Are you moving toward continuous delivery?
- It’s also about the way you run your applications. Your apps of record and apps of engagement are most likely running on highly standardized platforms and operating systems. The problem with this approach is that it’s not very agile and requires a great deal of management and control. In the containerized world, we’re essentially slimming down to very tiny operating systems containing only the processes and components you need for that service. This brings you new agility and allows you to start your systems in a matter of seconds instead of minutes. Discussion questions: What does your operating environment look like today? Do you see additional abstraction coming with the use of container technology?
- It’s also about how you compute within your data center or cloud. As you move toward apps of innovation you will also move toward software-defined data centers. If you’re using the public cloud everything is soft-wired already, but maybe in your data centers you’re looking toward VXLAN or other means to turn your current hard-wired networking into software. And not only networking, but also how you provision systems, how you make them accessible on a self-serve basis to groups like your engineering teams. In many ways your data center is turning into an API and you’re becoming a cloud provider. Discussion questions: Are you moving toward self-service for your engineering teams? Do you see your data center team as becoming a cloud operations team?
- Because of all these changes Dynatrace, both as software company but also as monitoring vendor, went through a major transformation: both on the product side but also on the engineering side. If you want to learn more about our transformation check out these resources: DevOps Webinar with Bernd Greifeneder (CTO): https://info.dynatrace.com/apm_dtm_ops_17q3_wc_from_enterprise_tocloud_native_na_registration.html DevOps Webinar with Anita Engleder (DevOps Manager): https://info.dynatrace.com/17q3_wc_from_agile_to_cloudy_devops_na_registration.html
- Just as all of your we had to think about our delivery pipeline! We quickly understood that embedding Monitoring into the whole pipeline is the only way to achieve faster innovation as well as reacting faster to feedback. Because without feedback you cant make the right decisions on whether you can push a change from Dev into Production and you are not sure which actions you should take if you don’t know what impact a deployment had. Traditional we looked at monitoring to be a production use case only. But that would mean many missed opportunities along the pipline to provide better and faster feedback to make better and faster informed decisions. This is why we shaped our pipelines and Dynatrace as a product to provide feedback in every stage of the pipeline, and to every individual stake holder
- Our DevOps Team – initially 7 people – now only 3 – are Responsible for “The Delivery Pipeline and the DevOps Tool Chain” Their Customers: The different Dev Teams that want to push features through the pipeline into production
- We also learned a lot when scaling from one dev pipeline to many dev pipelines. That happened when we onboarded more teams to the new development model. We saw that Ops was often the first point where different deployments from different teams came together. Understanding all the dependencies was therefore critical. Because this helps you to understand the Risk when it comes to deploying a new version of a component! Providing good monitoring for the Cloud Ops Teams was essential to ensure “Operational Services” Monitoring as a Service Capacity Planning Risk/Cost Control For the Service / App Teams it was essential to think about how to Improve “Efficiency” of their deliverables. We also talked about “Improving their Performance Signature” Continuous Performance Shift-Left Failure Usage Feedback Product Management and Business on the other side needs data and the capability to improve business Usage Behavior Costs Innovate A/B Testing
- We learned that we need to have self-service in our pipeline. Intuitive Dashboards, Chat Ops and Voice Ops to allow developers to pro-actively react on feedback from the pipeline
- More success numbers of our dynatrace devops transformation – the number we are very proud of is 93%. That’s because of our feedback loops back from production to engineering. Engineering also takes proactive approaches in fixing problems that have been pushed too far in the pipeline!
- Our transformation helped us to build Dynatrace – the product that redefines monitoring! With Dynatrace you can monitor every user, every app, everywhere; the platform is powered by Artificial Intelligence which is leveraged to identify issues and suggest remedies. Dynatrace monitors the full stack – meaning from the network through database and application tiers, into code-specific analysis all the way out to end user devices and third party add-ins. The same holds true for both in house hosted applications as well as private clouds, public clouds and hybrid clouds where containerization, hyper-scale and elastic computing dominate. And all of this is automated – not only initial set up and instrumentation but problem identification, dashboards, ongoing adaptation to environmental and application changes as well as upgrading the Dynatrace platform itself. All automated.
- Let me show you a LIVE DEMO!! In case you cant see the live demo – check out the following slides with screenshots of the highlights we showed in the demo!
- One single agent technology for all your monitoring. Just install it on your host – it will take care of all the rest!
- A single agent that you install with automatically detect all components in your environment (hosts, processes, services, applications) and all its dependencies
- And that also scales to very large envrionments
- We baseline everything: response times, cpu, memory, network connectivity, failure rates, …
- And we also automatically see your end users and capture their user experience from their device. Whether it is a deskop browser, mobile browser or a mobile native applilcation And yes – we baseline these metrics as well: page load time, mobile crashes, user interactions, bounce rates, conversion rates, …
- As also said, we are running ourself on AWS but in addition we have a full visibility into your services, it starts with the EC2 instances, but goes much beyond that covering also RDS, EBS, ELB, DynamoDB, S3 and Lambda functions. Dynatace is building on existing CloudWatch data but is going much much deeper
- One key differentiator of DT is also the way we present you problems. We are not sending you a ticket for each alert, we are actually bundeling those alert to a problem. Within that problem you will see which impact it had on your real users, because that’s what really matters in the end and you will also get already an root-cause analysis from DT, pinpointing you to the root of the problem. With that, you immediately can assign the problem and you do not need to figure out whom to notify. A very nice piece is also the visual resolutionpath showing you, how the problem evolved over time.
- Our Artificial Intelligence automates most of the work you would normally do – namely: #1: Identifying whether there is a problem #2: Identifying the impact -> who is impacted? #3: Root Cause of the problem to fix it fast! In Dynatrace we automatically package this information in a Problem Ticket and give you all the details you need to make the right decision to address this issue
- Also works well for business problems: which application is currently impacing our end users. What is the impact on conversion rate or bounce rate. And what is the actual root cause?
- And underneath the hood -> we have all the details your architects and developers need to fix any application specific issue
- This was now a small snippet of how Dynatrace works, but it even goes beyond that.
- Davis can be seen as your first digital assistant. Davis is pro-active and talks to multiple stakeholders. We thought it is the most intuitive way to deal with all the informaiton, as everybody of us
- knows how to ask a question. That is the most natural thing. There are multiple scenarios within every business where we see Davis intereacting with different stakeholders.
- One can be the daily standup of your team? A question might be: Has there been any user impact due to my last deployment. Did anything important for my team happened within the last x-hours? But it goes actually even beyond that. We see more and more organizational units interacting with Davis.
- a second scenarion is within the board room. How are my customers doing? How does my new digital processes work How is the real user experience
- Talk with Davis through Slack
- Or via VoiceOps – through Alexa (Amazon Echo). Check out the video if you want to see a VoiceOps enabled conversation with Dynatrace!
- To summarize todays session, we at Dynatrace have redefined monitoring. It goes far beyond usual monitoring. Dynatrace stands for every app, every user, every where. AI powered, full stack, automated. We serve multiple Use Cases and scenarios- starting with full stack insights, to cloud migration – where we support our customers already early in the cloud transformation process during cloud migration. In case you are interested, you can find us in the expo area booth 146.