Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NYC Data Amp - Microsoft Azure and Data Services Overview

766 views

Published on

This is a presentation that covered an overview of Microsoft's AI, machine learning, and data services available in Azure.

Published in: Technology
  • Login to see the comments

NYC Data Amp - Microsoft Azure and Data Services Overview

  1. 1. How do you differentiate in the crowded casual dining market? $3TAnnual worldwide consumer food service revenue
  2. 2. Where guests click She’d like another glass of wine Make a suggestion for a good pasta pairing Delight her Receive her praise on social media Ziosk uses big data and table-side tablets to delight diners Capture guest data Predict preferences Deliver customized experiences in real-time Increase customer satisfaction Encourage return and advocacy
  3. 3. Tableside data also powers a real-time manager dashboard Sped average table turnover by 7 min • Reassign servers to relieve back-ups • Respond to guest dissatisfaction before they leave the dining room • Integrate with loyalty programs • Automatically send reports to corporate
  4. 4. Explore new business opportunities with data-driven services Improve visibility to make more accurate predictions with remote monitoring Get the right products to the right places with inventory management Offer customers exactly what they want, when they want it, with personalization Fix problems proactively before they start with predictive maintenance Integrate historic, real-time and predictive insights within your apps
  5. 5.  Instantaneous response times  Robust and scalable security  Intelligence built-in  Flexibility for changes in demand  Powerful analytics  Intuitive visualization tools
  6. 6. Cloud- powered Enjoy the scale, flexibility and affordability of the cloud Layered with security Secure your data and apps with built-in tools Integrated development Leverage open, flexible and extensible cross- platform DevOps tools Built for intelligence Drive decisions with predictive intelligence
  7. 7. STEP 1 Aggregate data STEP 2 Detect patterns STEP 3 Predict outcomes STEP 4 Automate decisions
  8. 8. Always encrypted Row level security Threat detection DATA ISOLATION Azure Active Directory Report generation
  9. 9. Any data Scale to petabytes Run apps anywhereCross-platform
  10. 10. DEVELOPER SERVICES APPLICATION SERVICES .NET JAVASCRIPT PHPNODE.JS C++PYTHON R CORDOVA UNITY Robust development platform for any app Visual Studio Azure SDK Team Project Application Insights Visual Studio + Xamarin Web Apps Mobile Apps API Apps API Management Logic Apps Notification Hubs
  11. 11. Watch Video
  12. 12. Filtering the signal from the noise Rolls-Royce “Our goal is not data for the sake of data, but to embrace the cloud and analytical technologies to deliver more expert insights to the right stakeholders at the right time.“ Nick Farrant Senior Vice President ROLLS-ROYCE Objectives Connect Rolls-Royce jet engine data to Microsoft's intelligent cloud for insights to improve aircraft performance, safety and maintenance. Tactics Use Cortana Intelligence to scale quickly and efficiently, aggregate data across customer fleets and process data in real time. Results • More efficient flight and maintenance plans • Targeted and actionable fuel efficiency insights • Quickly-generated reports and dashboards that tell compelling stories and deliver high-quality insights
  13. 13. Data Factory Files Query Events Push/ Pull Data Ingestion Data Storage Azure Storage DocumentDB HDInsight App Service Power BI Operations Engineering Pilots Weather data Maintenance data Flight Plan data Airplane data Data Visualization Machine Learning HDInsight Data Analysis Security / Identity Azure Key Vault Azure AD MFA Data Analyst Data Scientist Azure SQL Operations / Monitoring App Insights Azure Search Logic App Batch Security Center Governance Data Catalog SQL Data Warehouse Example: Data and Service Architecture Files Query Events Push/ Pull Data Ingestion 0 Data Storage Operations Engineering Pilots Weather data Maintenance data Flight Plan data Airplane data Data Visualization Data Analysis Security / Identity Digital Collaboration Platform Instance Data Analyst Data Scientist Operations / Monitoring Processing Step 1 CopyFormatSummarise Processing Step n Machine Learning Published Data Enriched Data Raw Data Dashboards Reports Analytics Models RBACDevOps Metadata Governance
  14. 14. Business intelligence Advanced analytics Any language, any platform, anywhere Security Structured Unstructured OLTP CRM ERP LOB Graph Social IoT Media Datavirtualization Dataintegration Big data processing Data warehousing Operational data Dashboards Reporting Mobile BI Cubes Predictive Analytics Machine learning Stream analytics Cognitive AI DATA SOURCES DATA INSIGHTSDATA STORAGE AND PROCESSING DEVELOPMENT CONSUMPTION Build powerful big data analytics apps with a comprehensive platform App services Dev services App health Data visualization Developers Customers Sellers Warehouse managers
  15. 15. Business intelligence Advanced analytics Structured Unstructured OLTP CRM ERP LOB Graph Social IoT Media Datavirtualization Dataintegration Big data processing Data warehousing Operational data DATA SOURCES DATA INSIGHTSDATA STORAGE AND PROCESSING DEVELOPMENT CONSUMPTION Build powerful big data analytics apps with a comprehensive platform Azure SQL Data Warehouse Azure Data Lake Azure Machine Learning Azure Stream Analytics Power BI Embedded Visual Studio Enterprise Visual Studio Team Services Cognitive Services Azure Analysis ServicesHDInsight Xamarin Developers Customers Sellers Warehouse managers Any language, any platform, anywhere Security: least vulnerable for the last 7 years 0 50 100 150 200 SQL Server MySQL Oracle IBM DB2 PostgreSQL SAP HANA Vulnerabilities (2010-2016) .NET Azure 3rd
  16. 16. Hyper-scale repository for big data analytics LOB Apps SocialDevices Clickstream Sensors Video Web Relational ADL Store Big Data Stores Data Lake Store SQL Data Warehouse ADL Analytics HDInsight R Spark Machine Learning • A Hadoop Distributed File System for the cloud • No fixed limits on file size • No fixed limits on account size • Unstructured and structured data in their native format • Massive throughput to increase analytic performance • High durability, availability, and reliability • Azure Active Directory access control
  17. 17. Power BI App Service SQL Database SQL Data Warehouse Machine Learning Hadoop Intelligent App Big Data Stores Data Lake Store SQL Data Warehouse • Petabyte scale with massively parallel processing • Independent scaling of compute and storage—in seconds • Transact-SQL queries across relational and non-relational data • Full enterprise-class SQL Server experience • Works seamlessly with Power BI, Machine Learning, HDInsight, and Data Factory Elastic data warehouse with enterprise-class features
  18. 18. Machine Learning and Analytics Predictive analytics solutions with easy deployment HDInsight (Hadoop and Spark) Stream Analytics Data Lake Analytics Machine Learning • Simple, scalable, cutting edge. A fully managed cloud service that enables you to easily build, deploy, and share predictive analytics solutions. • Deploy in minutes. Azure Machine Learning means business. You can deploy your model into production as a web service that can be called from any device, anywhere and that can use any data source. • Publish, share, monetize. Share your solution with the world in the Gallery or on the Azure Marketplace.
  19. 19. Machine Learning and Analytics HDInsight (Hadoop and Spark) Stream Analytics Data Lake Analytics Machine Learning SQL Data Warehouse SQL DB Storage BlobsData Lake Store SQL DB in a VM • Analyze data of any kind and size • Develop faster, debug and optimize smarter • Interactively explore patterns in your data • No learning curve—use U-SQL, Spark, Hive, HBase and Storm • Managed and supported with an enterprise-grade SLA • Dynamically scales to match your business priorities • Enterprise-grade security with Azure Active Directory • Built on YARN, designed for the cloud Big data analytics for every data source Data Lake Analytics
  20. 20. Core Engine Batch Map Reduce Script Pig SQL Hive NoSQL HBase Streaming Storm In-Memory Spark Machine Learning and Analytics HDInsight (Hadoop and Spark) Data Lake Analytics Machine Learning • Scale to petabytes on demand • Process unstructured and semi-structured data • Develop in Java, .NET, and more • Skip buying and maintaining hardware • Deploy in Windows or Linux • Spin up an Apache Hadoop cluster in minutes • Visualize your Hadoop data in Excel • Easily integrate on-premises Hadoop clusters Operational data suite powered by Apache Stream Analytics
  21. 21. Machine Learning and Analytics HDInsight (Hadoop and Spark) Stream Analytics Data Lake Analytics Machine Learning • Scale to petabytes on demand • Process unstructured and semi-structured data • Develop in Java, .NET, and more • Skip buying and maintaining hardware • Deploy in Windows or Linux • Spin up an Apache Hadoop cluster in minutes • Visualize your Hadoop data in Excel • Easily integrate on-premises Hadoop clusters Real-time stream processing in the cloud Event Hubs Blob Storage Stream Analytics SQL Database Blob Storage Event Hubs Table Storage Power BI
  22. 22. Smart applications built to understand people Machine Learning & Analytics • Faces, images, emotion recognition and video intelligence • Spoken language processing, speaker recognition, custom speech recognition • Natural language processing, sentiment and topics analysis, spelling errors • Complex tasks processing, knowledge exploration, intelligent recommendations • Bing engine capabilities for Web, Autosuggest, Image, Video and News Cortana Bot Framework Cognitive Services Language Understanding Intelligent Service Bing Auto Suggest API Speaker Recognition Speech CRIS Text Analytics Bing Speller Web Language Model Linguistic Analysis Academic Knowledge Entity Linking Service Knowledge Exploration Service Recommendations Bing Search API Bing Image Search API Bing Video Search API Bing News Search API Computer Vision Face Emotion Video
  23. 23. Intelligent agents to interact with users Machine Learning & Analytics • Bot Connector Service: A service to register your bot, configure channels and publish to the Bot Directory. Connect your bot(s) seamlessly to text/sms, Office 365 mail, Skype, Slack, Twitter, and more. • Bot Builder SDK: An open source SDK hosted on GitHub. Everything you need to build great dialogs within your Node.js or C# bot • Bot Directory: A public directory of bots registered through the Bot Connector Service. Discover, try, and add bots to conversation experiences Cortana Bot Framework Cognitive Services
  24. 24. Demo Building a Bot in 5 seconds
  25. 25. Machine Learning Event HubsStream Analytics HDInsight Storage SQL Database Live dashboards for interactive business intelligence Dashboards & Visualizations • Analytics for everyone, even non-data experts • Your whole business on one dashboard • Create stunning, interactive reports • Drive consistent analysis across your organization • Embed visuals in your applications • Get real-time alerts when things change Power BI Power BI
  26. 26. Most consistent experience across on-premises and cloud Any data Any platform Any language Most secure database T-SQL Java PHP Node.js C/C++ C#/VB.NET Python Ruby
  27. 27. Redefining commerce through transparency Jet.com “To be one of the best e-commerce destinations in the US, we will have to handle millions of customers, placing tens of thousands of orders a day. That requires a top-class e-commerce system built on a flexible, open cloud platform. That is exactly what we got with Azure.” Mike Hanrahan Chief Technical Officer JET.COM Objectives Create a flexible, scalable data warehouse infrastructure with capabilities to handle an aggressive growth strategy. Tactics Adopted Microsoft SQL Data Warehouse and Azure HD Insight to handle vast amounts of data and streamline the development process. Results • Developed a full-fledged e- commerce marketplace in 12 months • Achieved rapid and flexible scalability enterprise-wide.
  28. 28. Using data to help save lives in the fight against pediatric AIDS Elizabeth Glaser Pediatric AIDS Foundation “Every day we make life-and-death decisions, so we can’t go on gut feelings or opinions. The data is the most important thing we have. Using Microsoft BI and analytics tools helps us make better decisions and save more children’s lives.” Stephanie Bruno Data Architecture Manager – Informatics ELIZABETH GLASER PEDIATRIC AIDS FOUNDATION Objectives Create a robust solution to accelerate the slow, inefficient collection of vital health information across at-risk populations and provide actionable intelligence from the data. Tactics Used SQL Server Reporting Services and Power BI to build a standardized set of reports, capture and summarize data, and roll it up to centralized reporting. Results • Easily accessible reports that helped communicate with donors and partners • Increased data reliability and accuracy • Improved data availability and reporting despite limited network capabilities
  29. 29. Moving from insight to action Rockwell Automation “What we’re talking about is delivering a degree of collaboration and visibility unheard of in the oil and gas industry. With sensors, software and the cloud, these disparate assets can become part of a connected enterprise, powered at its core by a rich flow of data.” Doug Weber Business Manager, Remote Application Monitoring ROCKWELL AUTOMATION Objectives Support high volume data to improve production efficiency, drive better performance and enable rapid innovation and go to market. Tactics Used Microsoft Azure SQL Database and HD Insight to collect sensor data from remote equipment across global supply chains and support real-time insight, analytics and predictive maintenance. Results • Improved access to production and supply chain data worldwide • Supported accelerated business growth with a highly scalable cloud platform • Utilized new features to get to market faster and make development easier
  30. 30. Assess with a partner-led data estate assessment Test with a partner-led proof of concept Get started today Learn and Experiment using self-service templates and training
  31. 31. Harnessing an ocean of data Carnival Maritime Objectives Improve visibility and gain deeper insight into their business operations by centralizing data management for thousands of devices and sensors across a fleet of 26 cruise ships sailing all over the world . Tactics Implemented Azure SQL Data Warehouse to leverage data captured by existing industrial hardware, and utilized the big-data platform to improve operations by analyzing historical data with custom models. Results • Connected thousands of devices and sensors into a centralized data repository • Created a scalable platform to extend, monitor and improve equipment maintenance across the fleet • Used predictive analytics to optimize water consumption, saving an estimated $200,000 a year “To build a big data and analytics strategy, our company needs to better understand what kind of data we can collect on the ships and what kind of data we need to have in the future…we want to use the data to get a better understanding of our operations and to help our ships be more efficient and sustainable.” Alexander Klingelhoefer Director of Continuous Improvement CARNIVAL MARITIME
  32. 32. Appendix
  33. 33. Data management • Azure Data Lake • Azure SQL Data Warehouse • Azure HDInsight Analytics • Azure Machine Learning • Azure Stream Analytics Development • Visual Studio Enterprise • Visual Studio Team Services • Azure Application Insights • Power BI Embedded • Xamarin
  34. 34. • Specifically designed to work with multiple analytics frameworks • Seamlessly scales from a few KBs to several PBs • Data is never lost or unavailable—even under failures • Offers enterprise-grade security for even the most sensitive data DATA MANAGEMENT Azure Data Lake ADL Analytics Spark Azure Machine Learning HDInsight R Azure Data Lake Store LOB Applications Social Clickstream Sensors Video Web Relational Devices A cost-effective cloud repository for big data
  35. 35. True cloud elasticity for freedom with control • Fully managed, Petabyte scale • Provision in minutes, scale in seconds • Query across relational and non-relational data with PolyBase • True cloud savings, with independent scaling of compute and storage, as well as pause & resume feature • Unmatched innovations in security with Threat Detection • Industry leading SLA – guaranteeing 99.9% uptime in GA regions • Availability in more regions than any other cloud data warehouse provider DATA MANAGEMENT Azure SQL Data Warehouse INGEST INSIGHT STORE A data warehouse that grows with your needs
  36. 36. Optimized open source analytics clusters • Fully managed Hadoop and Spark for the cloud • Enterprise-grade security and monitoring • Easy to manage, clusters up and running in minutes • Reliable with the industry’s best enterprise SLA • Familiar BI tools for analysis, or open source notebooks for interactive data science • Cost-effective cloud scale • 63% lower total cost of ownership than deploying your own Hadoop on-premises* • The most comprehensive protection against intellectual property risks with Azure IP Advantage *IDC study “The Business Value and TCO Advantage of Apache Hadoop in the Cloud with Microsoft Azure HDInsight” DATA MANAGEMENT Azure HDInsight
  37. 37. Powerful cloud-based predictive analytics • Easily build, deploy, and share predictive analytics solutions • Hundreds of built-in packages and support for custom code • Interactive, visual workspace – no programming required • Unlimited extensibility and one-click operationalization Collect & manage data Create machine learning models ML STUDIO SAAS APP MACHINE LEARNING SERVICE Run regular algorithms through connected database WEB SERVICE Add intelligence to app or provide insights in BI tools BI VISUALIZATIONS ANALYTICS Azure Machine Learning
  38. 38. Intelligence to enhance the user experience • Add smart API capabilities to enable contextual interactions • Tap into powerful AI algorithms for vision, speech, language, knowledge, and search • Allow your apps to apt to what users want and learn from your data • Map complex data to provide intelligent cross-sell recommendations Improved customization and engagement SaaS app ANALYTICS Microsoft Cognitive Services
  39. 39. Converge digital and physical worlds • Use pre-configured solutions such as Remote Monitoring and Predictive Maintenance to accelerate your IoT projects and jump ahead of the competition • Connect a broad range of existing device types and harness disparate data to create new intelligence • Predict with advanced analytics and machine learning to capture previously impossible insights • Tailor IoT solutions to your company’s business needs to quickly move from proof of concept to broader deployments • Integrate Azure IoT Suite with existing systems to make the best use of the data and processes you already have ANALYTICS Azure IoT Suite
  40. 40. Share code, track work, and ship software—in any language DEVELOPMENT Visual Studio Team Services • Secure code management to effectively create, manage and deliver against your backlog • Unparalleled flexibility for evolving codebases • Code is linked directly to the story, bug, or task • A toolset optimized for QA professionals, providing work style flexibility and team collaboration • Build apps with any tools and languages across multiple platforms • Streamline and automate the workflow between development and IT Ops to deliver higher quality SW more frequently • Customize and extend the Visual Studio platform to create the perfect dev environment
  41. 41. An integrated, end-to-end DevOps solution • Greater productivity for enterprise application development & delivery • Create mobile business applications for Android, iOS and Windows • Manage complexity and close the loop between Development & IT Ops • Plan, execute & monitor your entire testing effort • Full support across the DevOps lifecycle DEVELOPMENT Visual Studio Enterprise iOS Android Windows Visual Studio Enterprise Data Analyst Developer ITDM BDM .NET JAVASCRIPT PHPNODE.JS C++PYTHON R CORDOVA UNITY
  42. 42. Interactive data visualization and compelling reports • Author interactive data reports without writing any code using Power BI Desktop • Choose modern data visualizations out-of-the-box or customize without from scratch • Embed interactive visuals in your app using REST APIs and the Power BI SDK • Use your existing authentication and authorization methods • Speed up time to value without redesigning your existing app • Pay only for what you use with no upfront costs DEVELOPMENT Power BI Embedded
  43. 43. Create amazing cloud-powered mobile apps faster • Scale your apps to millions of customers across multiple geographies • Quickly add authentication, push notifications, and offline data sync • Connect your apps to enterprise systems, in the cloud or on premises • Use infinitely scalable storage to manage app resources • Build powerful apps with integrated tools, cloud services, and mobile SDKs • Integrate your mobile DevOps processes and systems with a few lines of code DEVELOPMENT Xamarin
  44. 44. Sign up for a free trial of Microsoft Azure https://azure.microsoft.com/en-us/free/
  45. 45. Cutting energy costs through machine learning Carnegie Mellon University Objectives Resolve complex big data challenges across fields as diverse as astrophysics and building management. Tactics Used Azure Machine Learning to address the challenge of fault detection and diagnosis for control-system components hidden from visual inspection. Results • Realized estimated energy savings of 20 percent • Simplified and accelerated the time-consuming process of creating and testing machine learning models “We see Azure Machine Learning ushering in an era of self-service predictive analytics for the masses. We can only imagine the possibilities.” Bertrand Lasternas Researcher CARNEGIE MELLON UNIVERSITY
  46. 46. Creating a crystal ball for appliance manufacturing Arçelik A.Ş. Objectives Replace an outdated forecasting system with a new solution to improve accuracy and ensure the right spare parts are available anytime and anywhere they’re needed. Tactics Used Azure Machine Learning to test algorithms and identify the most accurate ones to forecast the needs for spare parts 12 months in advance. Results • Forecasting accuracy increased up to 80% • Inventory turnover expected to climb by 10% • Increased forecasts from 100,000 to all 350,000 spare parts SKUs “With more spare parts in our warehouse, we needed a way to respond to customer needs quickly. We reached that goal by using Azure Machine Learning to increase forecast accuracy.” Burcu Aksoy, Spare Part Team Leader, Customer Care ARCELIK
  47. 47. Filtering the signal from the noise Rolls-Royce “Our goal is not data for the sake of data, but to embrace the cloud and analytical technologies to deliver more expert insights to the right stakeholders at the right time.“ Nick Farrant Senior Vice President ROLLS-ROYCE Objectives Connect Rolls-Royce jet engine data to Microsoft's intelligent cloud for insights to improve aircraft performance, safety and maintenance. Tactics Use Cortana Intelligence and Azure Machine Learning to scale quickly and efficiently, aggregate data across customer fleets and process data in real time. Results • More efficient flight and maintenance plans • Targeted and actionable fuel efficiency insights • Quickly-generated reports and dashboards that tell compelling stories and deliver high-quality insights
  48. 48. Business intelligence turns customers into fans Metro Bank “As we grew, we needed something more dynamic, more visually appealing and more user-friendly…Power BI fits the bill in all of those respects.” Bruce Rioch Head of Business Information and Customer Systems METRO BANK Objectives Innovate customer service by capturing data that is clear and easy to understand, and available to the right person at the right time. Tactics Used Power BI to gather deeper and more detailed information about what customers wants and needs to guide analysis and decision-making. Results • Continuous improvement in customer experience • Expanded offerings built on new capabilities • At-a-glance visualizations of how customers interact with bank services
  49. 49. Using data to help save lives in the fight against pediatric AIDS Elizabeth Glaser Pediatric AIDS Foundation “Every day we make life-and-death decisions, so we can’t go on gut feelings or opinions. The data is the most important thing we have. Using Microsoft BI and analytics tools helps us make better decisions and save more children’s lives.” Stephanie Bruno Data Architecture Manager – Informatics ELIZABETH GLASER PEDIATRIC AIDS FOUNDATION Objectives Create a robust solution to accelerate the slow, inefficient collection of vital health information across at-risk populations and provide actionable intelligence from the data. Tactics Used SQL Server Reporting Services and Power BI to build a standardized set of reports, capture and summarize data, and roll it up to centralized reporting. Results • Easily accessible reports that helped communicate with donors and partners • Increased data reliability and accuracy • Improved data availability and reporting despite limited network capabilities
  50. 50. Redefining commerce through transparency Jet.com “To be one of the best e-commerce destinations in the US, we will have to handle millions of customers, placing tens of thousands of orders a day. That requires a top-class e-commerce system built on a flexible, open cloud platform. That is exactly what we got with Azure.” Mike Hanrahan Chief Technical Officer JET.COM Objectives Create a flexible, scalable data warehouse infrastructure with capabilities to handle an aggressive growth strategy. Tactics Adopted Microsoft SQL Data Warehouse and Azure HD Insight to handle vast amounts of data and streamline the development process. Results • Developed a full-fledged e- commerce marketplace in 12 months • Achieved rapid and flexible scalability enterprise-wide.
  51. 51. Moving from insight to action Rockwell Automation “What we’re talking about is delivering a degree of collaboration and visibility unheard of in the oil and gas industry. With sensors, software and the cloud, these disparate assets can become part of a connected enterprise, powered at its core by a rich flow of data.” Doug Weber Business Manager, Remote Application Monitoring ROCKWELL AUTOMATION Objectives Support high volume data to improve production efficiency, drive better performance and enable rapid innovation and go to market. Tactics Used Microsoft Azure SQL Database and HD Insight to collect sensor data from remote equipment across global supply chains and support real-time insight, analytics and predictive maintenance. Results • Improved access to production and supply chain data worldwide • Supported accelerated business growth with a highly scalable cloud platform • Utilized new features to get to market faster and make development easier

×