Hadoop and Manufacturing
•
9 likes•4,485 views
Manufacturers have an abundance of data, whether from connected sensors, plant systems, manufacturing systems, claims systems and external data from industry and government. Manufacturers face increased challenges from continually improving product quality, reducing warranty and recall costs to efficiently leveraging their supply chain. For example, giving the manufacturer a complete view of the product and customer information integrating manufacturing and plant floor data, with as built product configurations with sensor data from customer use to efficiently analyze warranty claim information to reduce detection to correction time, detect fraud and even become proactive around issues requires a capable enterprise data hub that integrates large volumes of both structured and unstructured information. Learn how an enterprise data hub built on Hadoop provides the tools to support analysis at every level in the manufacturing organization.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to Hadoop and Manufacturing
Similar to Hadoop and Manufacturing (20)
More from Cloudera, Inc.
More from Cloudera, Inc. (20)
Recently uploaded
Recently uploaded (20)
Hadoop and Manufacturing
- 1. Information-Driven Manufacturing Capture Value from Manufacturing Data with an Enterprise Data Hub Speaker name // Speaker title
- 2. 2© Cloudera, Inc. All rights reserved. Trends in Manufacturing Everything that can be measured will be measured. Only increasing... Continuous Improvement in cost and efficiency in all areas of manufacturing operation Now, more than ever, Quality is a top concern both from consumer, dealer and regulatory standpoint Instrumentation Efficency Quality NEED BETTER PICTURE
- 3. 3© Cloudera, Inc. All rights reserved. Manufacturers are collecting data at an exponential rate, yet struggle to derive value from all that data...
- 4. 4© Cloudera, Inc. All rights reserved. : Manufacturing Enterprise Data HUB Provides the ability to store, analyze all the data and quickly uncover new insights, derive value to all phases of the process from initial design to final delivery.
- 5. 5© Cloudera, Inc. All rights reserved. Manufacturing Enterprise Data Hub Overview Keep all the data, whether its people generated, machine generated or external. Statistical and machine learning analyses using advanced analytic tools on all the data (Spark, R, Python,SAS, Matlab) Access to all the data from the enterprise and manufacturing at your fingertips, consolidate silos (Self Service BI, Search) Keep all the data Advanced Analytics Leverage all the data
- 6. 6© Cloudera, Inc. All rights reserved. Where Is the Manufacturing Data? Mapping and Consolidation Are the Tip of the Iceberg for Big Data Devices & Sensors • Device Readings • Device Performance • Device Diagnostics • Battery / Power Consumption • Software Logs • Environmental Interactions • R&D • Quality / Testing Plant & Operations • MES • Sensors • Video / Surveillance • Line Productivity • Machines • Staffing / Scheduling • Quality data Supply Chain & Inventory • ERP • Supplier / Manufacturer • Orders / Receivables • Commodity Supplies / Prices • Chargebacks • Scorecards • Delivery Metrics Marketing & CRM • Transactions • Accounts • Warranties / Aftermarket • Customer Service Logs • Campaigns / Promotions • Website / SEO • Affiliates / Merchants • Surveys • Competitive Intelligence Public & Trade • Market Intelligence • Policy / Regulation • Demographic / Census • Psychographic • Inflation / Macroeconomic • Gas Prices • Labor Statistics • Social / Search • Public Health Data • Clinical Studies • Store Schematics • Journals / Editorial • Seismic / Speculation
- 7. 7© Cloudera, Inc. All rights reserved. A Traditional Architecture: What have we tried Access Data Experiment FastAnalyze Data Enterprise Data Warehouse ImplementData Sources ETLStructured Unstructured Ingest Storage #1, 2, N ELT Store & Process Traditional Architecture EDW Archive ETL Access Data Analyze Data Search Serve Serve Serve Optimize Implement Custom Application Point Solution ELT ELT Statistical Machine Learning SQL Filter? Filter? Filter? Filter? Machine Data Ingest
- 8. 8© Cloudera, Inc. All rights reserved. Enterprise Data Warehouse ImplementData Sources ETLStructured Unstructured Ingest Storage #1, 2, N ELT Store & Process Traditional Architecture EDW Archive ETL Access Data Analyze Data Search Serve Serve Serve Optimize Implement Custom Application Point Solution ELT ELT Statistical Machine Learning SQL Challenges with Traditional Architectures 1) Limited Data 2) Long Time to Value 1 2 2 3) Sub Optimal Decisions 3 Filter? Filter? Filter? Filter? Machine Data Ingest Filter?
- 9. 9© Cloudera, Inc. All rights reserved. The New Way Forward 1) Unlimited Data Access 2) Reduce Time to Value 3) Decision on all data Enterprise Data Warehouse ImplementData Sources ELT Store & Process Modern Architecture Access Data Analyze Data Optimize Implement Custom Application Point Solution Statistical Machine Learning SQL Structured Unstructured EDW ETL Serve ETL Active Ingest Ingest EDH Archive Load Cloudera ELT 3 2 2 3 1 Search Machine Data ETL Active Ingest
- 10. 10© Cloudera, Inc. All rights reserved.©2014 Cloudera, Inc. All rights reserved. Overview on Data Flow in Cloudera EDH 10 3rd party or public Network Equipment Traditional RDMBS EDW Ffffffffff Event Based, Near Real Time • Flume • Spark Streaming • Kafka (coming soon) SQL / Relational • Sqoop – SQL Import including Metadata Web Services/API/Cmd line • Put/Store Copy/Move files • NFS Gateway HUE Web GUI • User Upload • User Copy/Move/Rename Third Part Integrations Ingest/Storage Process/Transformation WORKLOAD MANAGEMENT / Yarn (Resource Management) & Oozie (Workflow Engine) Hadoop File System (HDFS) / Distributed File Storage ELT, ETL, Transform, Cleanse, Pre-aggregate, analyze etc. SQL / Relational • Hive (Batch SQL) • Impala (Interactive SQL) Map Reduce – Java based distributed processing • Machine Learning libraries • Pig – scripting language to perform Map Reduce Spark – In Memory distributed processing • Java, Scala or Python Third Party Integrations SQL / Relational (ODBC/JDBC) • Hive (Batch SQL) • Impala (Interactive SQL) Web Services/API/Cmd line • Get/Move files • NFS Gateway HUE Web GUI • User Download • User Copy/Move/Rename Search Index • Solr Search, full featured with Facets, NLP, etc. Third Part Integrations Raw Data Insight and Value to User Publish/Consume
- 11. 11© Cloudera, Inc. All rights reserved. AUTHENTICATION Guarding access to the system, its data, and its various systems LDAP Kerberos RPC PROTECTION Encryption for data at rest or in motion with full key management Cloudera Navigator: Encrypt & Key Trustee AUTHORIZATION Controlling who or what has access to a resource or service POSIX Permissions Apache Sentry AUDIT Capture a complete and immutable record of all activity Cloudera Navigator SIEM Tools Security Important? Cloudera Enterprise Data Hub provides Enterprise-Grade Security, Audit and Regulatory Compliance Governing Access to and Management of All Data-at-Rest and Data-in-Motion • Cloudera Manager and Navigator automate protections for Hadoop and related projects • Perimeter security • Role-based access control • The only complete policy-based management of sensitive data • Data lineage and discoverability
- 12. 12© Cloudera, Inc. All rights reserved. Core Benefits of a Manufacturing Enterprise Data Hub ©2014 Cloudera, Inc. All rights reserved. • Full-Fidelity Active Archive • Any and All Kinds of Data • Accelerate Time to Insight (Scale) • Unlock Agility and Exploration • Consolidate Silos for 360o View • Enable Pervasive Analytics across the entire Value Chain (Design to Post Sales Delivery and Warranty)
- 13. 13© Cloudera, Inc. All rights reserved. What value is there in Manufacturing Data Hub? • What product issues are paramount? • What are technology trends? • Efficient Parts Utilization—what is the best part for my design? • Is all my machine data being utilized? Design, R&D, PD, Engineering Hadoop Cloudera Secure Scalable Flexible Open Production, Quality, Manufacturing • Diagnose Production problems • What is the cause? People, Parts, Process, Suppliers? • Plant inventory • Resource utilization • Is all my shop floor data being analyzed? Supply Chain, Purchasing • Who are my best Suppliers? • Who are my worst Suppliers? • Consolidated view of the Supply Chain? • Supply Chain disruption impact analysis? • Consolidated Purchasing (360 Supplier view) Manufacturing Data Hub Delivery, Warranty, Support, Service • Review Customer 360 • Analyze Product Launch information • Detect Emerging Warranty Issues • Decrease Correction Times • Increased Accuracy of Warranty Forecast • Knowledge base for After Delivery Service Ask Bigger Questions of all the data
- 14. 14© Cloudera, Inc. All rights reserved. Customer Story
- 15. 15© Cloudera, Inc. All rights reserved. About Vehicle Manufacturer What do we do - Manufacture, Sell and Service Vehicles Who is this Manufacturer A worldwide leading Manufacturer of Vehicles
- 16. 16© Cloudera, Inc. All rights reserved. Our Objectives Store and Analyze worldwide data from Dealers, Customers and Vehicles Better, Deeper Analysis Smarter Predictions, Earlier Detection
- 17. 17© Cloudera, Inc. All rights reserved. The Pre-Hadoop Environment Parts Suppliers Dealers 1 Difficult to connect to multiple sources 1 BI/RDBMS/DW Challenge Claims Machine Data IDLE Vehicle Data WHY? • Volume • Too much to store, let alone query • Variety • Different formats, not all Table Based data ? ? ?
- 18. 18© Cloudera, Inc. All rights reserved. The Pre-Hadoop Environment Parts Suppliers Dealers 2 Impossible to analyze all that data 2 BI/RDBMS/DW Another Challenge Claims Machine Data IDLE Vehicle Data WHY? • It wasn’t even in one system. • Different workloads (Macro vs. Micro Analysis) Advanced Analytics
- 19. 19© Cloudera, Inc. All rights reserved. Vehicle Manufacturer Modern Hadoop Architecture Complete storage of data (structured and unstructured) 1 Improvements Process 1 Store HDFS, HBase Claims Machine Data IDLE Vehicle Data Sqoop Flume, KafkaCopy (XML)
- 20. 20© Cloudera, Inc. All rights reserved. Vehicle Manufacturer Modern Hadoop Architecture Process Data as needed2 Improvements Process 2 Store HDFS, HBase Claims Machine Data IDLE Vehicle Data Sqoop Flume, KafkaCopy (XML) MR,Pig, Spark, ETL Tools on Hadoop Process Complete storage of data (structured and unstructured) 1
- 21. 21© Cloudera, Inc. All rights reserved. Vehicle Manufacturer Modern Hadoop Architecture Analysis and Large Scale Ad-Hoc Queries 3 Improvements Process 3 Store HDFS, HBase Claims Machine Data IDLE Vehicle Data Sqoop Flume, KafkaCopy (XML) MR,Pig, Spark, ETL Tools on Hadoop Process HUE Discover Impala Solr Access BI Hive Process Data as needed2 Complete storage of data (structured and unstructured) 1
- 22. 22© Cloudera, Inc. All rights reserved. Vehicle Manufacturer Modern Hadoop Architecture Analysis and Large Scale Ad-Hoc Queries 3 Improvements Process 4 Store HDFS, HBase Claims Machine Data IDLE Vehicle Data Sqoop Flume, KafkaCopy (XML) MR, Pig, Spark, ETL Tools on Hadoop Process HUE Discover Impala Solr Access BI Hive Process Data as needed2 Complete storage of data (structured and unstructured) 1 R MLlibSpark 4 Advanced Analytics
- 23. 23© Cloudera, Inc. All rights reserved. Business and Technical ROI Technology ROI Business ROI Proactive Quality Assurance Build machine learning algorithms that identify production anomalies prior to field testing and find performance flaws that could not be identified in R&D. Predictive Intervention Combine data streaming from machine data (vehicles, plant floor), diagnostics, and product/engineering data to proactively avoid or address issues and deploy upgrades. Merge together storage systems for simpler management – Active Archive – Retire Legacy Systems Unified access to disparate, Siloed data – Retire single use systems Scale affordably – Grow without destroying the budget Flexible and Agile – IT can focus on Solutions for the Business vs. being a Data Plumber
- 24. 24© Cloudera, Inc. All rights reserved. 24 What happened at the parts supplier that caused a spike in support calls during the past 30 minutes for devices manufactured in Birmingham? How many devices were returned last month? Reduce the time to QC issue resolution from weeks to hours. Drive $15 to $25 million annual savings for each manufacturer. © 2014 Cloudera, Inc. All rights reserved.
- 25. 25© Cloudera, Inc. All rights reserved. Can we predict which chips have the highest likelihood of failure and intervene to proactively prevent manufacturing issues? Which chips most commonly failed last week? Analyses now executable on hundreds of thousands of units in just seconds. 60x faster data reload and 300% query speedup enable real-time debugging. 25© 2014 Cloudera, Inc. All rights reserved.
- 26. 26© Cloudera, Inc. All rights reserved. Thank you
- 27. 27© Cloudera, Inc. All rights reserved. Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 850 World Class Support 24x7 Global Staff Pro-active & Predictive Support Programs Mission Critical Thousands of Enterprise Users Over 500+ Paying Subscription Customers The Largest Ecosystem Over 1450+ Partners Cloudera University Over 100,000+ Trained Open Source Leaders Cloudera Employees are Leading Developers & Contributors Total Capital Raised $1B+ (from Intel, Google, Dell, T. Rowe Price, Accel, Greylock) Mission Help Organizations Leverage the Power of All Their Data to Ask Bigger Questions.
- 28. 28© Cloudera, Inc. All rights reserved. Expanding Data Requires A New Approach What we do Copy Data to Applications What we should do Bring Applications to Data Data Information-centric businesses use all Data: Multi-structured, Internal & external data of all types App App App Process-centric businesses use: • Structured data mainly • Internal data only • “Important” data only • Multiple copies of data App App App Data Data Data Data
- 29. 29© Cloudera, Inc. All rights reserved. Hadoop Changes the Game: Storage & Compute Together ©2014 Cloudera, Inc. All rights The Hadoop WayThe Old Way $30,000+ per TB Expensive & Unattainable • Hard to scale • Network is a bottleneck • Only handles relational data • Difficult to add new fields & data types Expensive, Special purpose, “Reliable” Servers Expensive Licensed Software Network Data Storage (SAN, NAS) Compute (RDBMS, EDW) $300-$1,000 per TB Affordable & Attainable • Scales out forever • No bottlenecks • Easy to ingest any data • Agile data access Commodity “Unreliable” Servers Hybrid Open Source Software Compute (CPU) Memory Storage (Disk) z z
- 30. 30© Cloudera, Inc. All rights reserved. Enabling the “App Store” of Big Data (Large Ecosystem) Data Systems Enterprise Data Hub Security and Administration Unlimited Storage Process Discover Model Serve Applications System Integration Infrastructure More than 1,450 partners ensure compatibility with existing investments, lower skill barriers, and help maximize value from your data.Operational Tools
- 31. 31© Cloudera, Inc. All rights reserved. WEB/MOBILE APPLICATIONS ONLINE SERVING SYSTEM ENTERPRISE DATA WAREHOUSE ENTERPRISE REPORTINGBI / ANALYTICSMACHINE LEARNING CONVERGED APPLICATIONS CLOUDERA MANAGER META DATA / ETL TOOLS ENTERPRISE DATA HUB The Modern Information Architecture Data Architects System Operators Engineers Data Scientists Analysts Business Users Customers & End Users SYS LOGS WEB LOGS FILES RDBMS
- 32. 32© Cloudera, Inc. All rights reserved.©2014 Cloudera, Inc. All rights A High Level View of the Journey Not Only SQL Agile Exploration ETL Acceleration Operational Efficiency (Faster, Bigger, Cheaper) Transformative Applications (New Business Value) Cheap Storage BusinessIT EDW Optimization Pervasive Analytics
Editor's Notes
- The manufacturing sector was an early and intensive user of data to drive quality and efficiency, adopting information technology and automation to design, build, and distribute products since the dawn of the computer era. In the 1990s, manufacturing companies racked up impressive annual productivity gains because of both operational improvements that increased the efficiency of their manufacturing processes and improvements in the quality of products they manufactured. For example, advanced manufactured products such as computers became much more powerful. Manufacturers also optimized their global footprints by placing sites in, or outsourcing production to, low-cost regions. But despite such advances, manufacturing, arguably more than most other sectors, faces the challenge of generating significant productivity improvement in industries that have already become relatively efficient. We believe that big data can underpin another substantial wave of gains. These gains will come from improved efficiency in design and production, further improvements in product quality, and better meeting customer needs through more precisely targeted products and effective promotion and distribution. For example, big data can help manufacturers reduce product development time by 20 to 50 percent and eliminate defects prior to production through simulation and testing. Using real-time data, companies can also manage demand planning across extended enterprises and global supply chains, while reducing defects and rework within production plants. Overall, big data provides a means to achieve dramatic improvements in the management of the complex, global, extended value chains that are becoming prevalent in manufacturing and to meet customers’ needs in innovative and more precise ways, such as through collaborative product development based on customer data.
- No individual record is particularly valuable, but having every record opens the door to extreme value. This sector generates data from a multitude of sources, from instrumented production machinery (process control), to supply chain management systems, to systems that monitor the performance of products that have already been sold (e.g., during a single cross-country flight, a Boeing 737 generates 240 terabytes of data). And the amount of data generated will continue to grow exponentially. The number of RFID tags sold globally is projected to rise from 12 million in 2011 to 209 billion in 2021. IT systems installed along the value chain to monitor the extended enterprise are creating additional stores of increasingly complex data, which currently tends to reside only in the IT system where it is generated. Manufacturers will also begin to combine data from different systems including, for example, computer-aided design, computer-aided engineering, computer-aided manufacturing, collaborative product development management, and digital manufacturing, and across organizational boundaries in, for instance, end-to-end supply chain data.
- Key takeaway: It is not just a BI or analytics challenge, it is the way that data is managed. Keeping 3 main high level objectives of an architecture built for Data Discovery in mind- accessing data, analyzing data, and experimenting and iterating fast- we can examine a traditional architecture and see where organizations might run into issues. Questions for customer: Does this look like your architecture? What limitations are you “living with” today?
- Limited Data Access Data siloes Archived or deleted data No unstructured data Only SQL Long Time to Value Resource intensive ad-hoc ELT, CONVERT TO TABLES (SQL) Inflexible Adding dimensions takes months Slow large scale queries Sub-Optimal Decisions Limits on data sets Guessing? Missing Critical items Frustrated USERS!
- Key takeaway: An EDH provides the foundation to change the way you collect and manage data in order to provide your analyst what they need in less time. No Filter, No missing data! ETL on the fly: Talk to schema-on-write vs schema-on-read (http://www.slideshare.net/awadallah/schemaonread-vs-schemaonwrite). 1) Unlimited Data Access (Active archive, Scalable storage, Unstructured data) 2) Reduce Time to Value (ETL on the fly, Parallel processing, Complete data access, flexible-any schema, any file) 3) Best Decisions (Decisions on all the data)
- Pulling from the “Insights Section”
- Why Hadoop slide content: Even with primarily relational systems, it involved hundreds of sources Getting a BI tool to connect to so many sources is … not fun More times than not, we needed to understand a subset or aggregate of this data - not all of the data! Can use Pig to process, extract, filter the data Can use Hive - a SQL like query language - to query my data
- Why Hadoop slide content: Even with primarily relational systems, it involved hundreds of sources Getting a BI tool to connect to so many sources is … not fun More times than not, we needed to understand a subset or aggregate of this data - not all of the data! Can use Pig to process, extract, filter the data Can use Hive - a SQL like query language - to query my data
- Why Hadoop slide content: Even with primarily relational systems, it involved hundreds of sources Getting a BI tool to connect to so many sources is … not fun More times than not, we needed to understand a subset or aggregate of this data - not all of the data! Can use Pig to process, extract, filter the data Can use Hive - a SQL like query language - to query my data
- Why Hadoop slide content: Even with primarily relational systems, it involved hundreds of sources Getting a BI tool to connect to so many sources is … not fun More times than not, we needed to understand a subset or aggregate of this data - not all of the data! Can use Pig to process, extract, filter the data Can use Hive - a SQL like query language - to query my data
- Link to account record in SFDC: https://na6.salesforce.com/0018000000y2EIt?srPos=0&srKp=001 Omneo, a Division of Camstar, drives $15 to $25 million in annual savings for electronics manufacturers based on its ability to address supply chain issues in near real time. Background: Today’s consumers have high expectations for the products we use everyday, particularly when it comes to our devices. We want new products to come out faster, at lower prices, with more capabilities than before. But we also demand increased reliability. Camstar, a 30-year veteran in the enterprise manufacturing and supply chain space, saw this trend and identified an opportunity. Challenge: Electronic device manufacturers are responsible for delivering millions of products, each comprised of hundreds of components that are sourced from all over the globe, put together, and pushed through distribution channels to customers. There’s a large margin for error. Camstar set out to address this by spinning off a division called Omneo, who set out to build 360-degree view into supply chain and product quality. Solution: After evaluating IBM Netezza, Infobright, Cassandra, MongoDB, and Hadoop, Omneo decided to try out Hadoop based on 3 main factors: Scalability to grow with customers’ needs over time Flexibility to meet the needs of diverse customers and data sets in a multi-tenant environment Low TCO for an efficient big data solution The team downloaded Cloudera Express since it was easy and no one had any prior experience with the technology. After a few months of demonstrating promising results, Omneo decided to perform a TCO analysis of Cloudera vs. IBM Netezza and their legacy (Oracle) data warehouse. Cloudera’s costs came in 75% lower per TB than IBM Netezza and 90% lower per TB than the incumbent. But before moving forward with a Cloudera Enterprise subscription, the team compared the different Hadoop vendors. They ultimately decided to move forward with Cloudera due to 4 main factors: Long-term company strategy and viability Ease of use and maturity of Cloudera Manager Enterprise-grade support Dedication to open source Omneo has deployed a multi-tenant enterprise data hub from Cloudera as the platform behind its supply chain cloud solution, which ingests machine data and existing system data from throughout the manufacturing process, including from clients’ factory data, supplier data, field services, after-market repairs, and re-manufacturing data. The company uses MapReduce to transform and manipulate data into any structure needed; HBase to access specific records in real time; and Cloudera Search to rapidly index all raw data in a way that makes sense for customers. Results: Omneo’s supply chain SaaS delivers a 360-degree view of the supply chain process in seconds, allowing manufacturers to access their data in different ways, on the fly. If something happens at any supplier that drives a sudden increase in quality issues, they can figure out where the issue stems from and why in minutes or hours. In traditional environments, these investigations would take weeks or months. Instead of spending time trying to pinpoint challenges, manufacturers can spend their time resolving them. Omneo’s clients report total annual savings between $15-25 million each, conservatively.
- AMD improves yield predictions with a Cloudera-powered engineering data warehouse. Background: Advanced Micro Devices (AMD) is a multinational semiconductor manufacturer that designs and builds graphics cards and microprocessors powering millions of the world's personal computers, tablets, gaming consoles, embedded devices, and cloud servers. All of the world’s leading PC and major video game console manufacturers have AMD technology inside. AMD relies on manufacturing test data to ensure product quality and perform engineering analysis in order to improve upon its world-class product designs. Challenge: The company wanted to empower its engineers by giving them access to larger data sets at faster speeds. But the incumbent environment only stored less than 30% of available data elements, was built with several different integration tools, had many integration steps and relied on a large IT team to support and maintain this system. In 2011, there was an environment outage that took weeks to recover, so AMD initiated an Engineering Data Warehouse (EngDW) project to find a more agile, cost-effective solution and a simpler, more robust way to store, process, and fetch larger amounts of data for AMD’s engineers. Solution: The semiconductor manufacturer replaced its legacy engineering data warehouse with the Dell Cloudera Solution for Apache Hadoop. AMD runs a 34-node production cluster today, which collects data throughout the manufacturing process. Hundreds of millions of new digital and parametric test readings are loaded to the cluster every day. At the heart of the EngDW project are CDH and HBase. A custom query engine reads from HBase to put the test measurements in the hands of the company’s engineers. Results: AMD's decision to move from an RDBMS to a Hadoop platform that uses Cloudera on Dell servers powered by AMD Opteron processors has resulted in orders of magnitude performance improvement, in terms of both data loads and analytics. Query times have been reduced by up to 300%, running on larger data sets than before. 99% of all queries execute in 15 minutes or less, with a median execution time of just 23 seconds. Queries on hundreds of thousands of units execute two orders of magnitude faster than before. Data reloads at a rate of three months per day, whereas it used to take a full day to reload 1.5 days’ data—that’s 60X faster. Not only has AMD's EngDW project brought significant performance benefits, but it delivers greater functionality and value as well. Query results on EngDW now have an unlimited row limit, compared to the previous limit of just 100,000 rows (which had been set to ensure queries would return results in a given period of time). The EngDW project's Hadoop-based cluster allows AMD to store more than 90% of available data elements spanning 1.5-plus years’ history, whereas the previous system stored less than 30% of data available for only three to four months’ history. Now that AMD engineers can access greater amounts of test data in higher detail and at faster speeds, they can apply insights to debug and make continuous improvements to ensure their products meet customer needs. AMD has also significantly reduced the TCO of its EngDW through lower vendor support costs for relational database management software, less vendor support for data integration tools and software, fewer steps and tools needed for data integration, less vendor support for high-end storage arrays (external SAN storage), and a smaller IT support staff needed for end-to-end management.
- B+
- Today we're in the middle of a shift in how businesses use information. In the past, you'd define a set of business processes, build applications around each of them, and then go about gathering, conforming, and merging the necessary data sets to support those applications. From an infrastructure perspective, you'd be bringing the data over to the compute, often in relational databases. But you'd be leaving quite a lot on the table. The modern realities of business demand a new approach. Today companies need, more than ever, to become information-driven, but given the amount and diversity of information available, and the rate of change in business, it's simply unsustainable to keep moving around and transforming huge volumes of data.
- Pricing Data: Cloudera: HW + SW per-year list prices for Basic thru EDH at various configs Old Way: Various sources. One of note: - Cowen / Goldmacher coverage initiation of Teradata, June 17, 2013 - List price of high-end appliance (which he thinks is more comparable to our solution) is $57K/TB + maintenance for an annual cost of $39K/TB - Prices have likely decreased, but we estimate they are still in excess of $30K/TB/year - List price of their low-end appliance is $12K/TB + maint or $8K per year
- Cloudera partners more broadly and deeply across the Hadoop ecosystem than any other vendor. With over 1200 partners and counting, our partnerships offer: Compatibility with your existing tools and skills 160+ certified on Cloudera 5, including all 12 of the 12 Gartner Business Intelligence Magic Quadrant leaders Flexible deployment options On-premises Public, private, or hybrid cloud Appliances and engineered systems Partnerships you can trust Deep engineering relationships Comprehensive certification program