Apache Hadoop 0.23

•

23 likes•4,481 views

This document summarizes Apache Hadoop release 0.23, which is scheduled to be the first stable release since 0.20 in 2009. Key highlights include improvements to HDFS federation, MapReduce, and high availability. The release aims to support large clusters of thousands of machines with high concurrency. Extensive testing is being done to validate performance gains from changes like MapReduce shuffle reimplementation and optimizations for small jobs. The 0.23 branch is expected in August 2011 with an alpha release in October and production release in late Q1 2012.

Technology

Apache Hadoop 0.23 Arun C. Murthy Hortonworks Founder and Architect @acmurthy (@hortonworks) © Hortonworks Inc. 2011 August 25, 2011

Hello! I’m Arun… Architect & Lead, Apache Hadoop MapReduce Development Team at Hortonworks (formerly at Yahoo!) Apache Hadoop Committer and Member of PMC Full-time contributor to Apache Hadoop since early 2006 Apache HadoopRelease Manager for hadoop-0.23

hadoop-0.23 On track to be first stable, and widely deployed, release since hadoop-0.20 in 2009 All stable releases of Hadoop today are based on hadoop-0.20 Multiple folks and entities collaborating: Hortonworks, Yahoo, Cloudera, EBay etc. hadoop-0.23 branch in Apache hours away! © Hortonworks Inc. 2011 4

Highlights HDFS Federation http://www.hortonworks.com/an-introduction-to-hdfs-federation/ NextGenerationHadoopMapReduce http://www.slideshare.net/hortonworks/nextgen-apache-hadoop-mapreduce Coming soon – HDFS High Availability https://issues.apache.org/jira/browse/HDFS-1623 WIP: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-1623/ © Hortonworks Inc. 2011 5

More… Build - Full Mavenization EditLogs re-write https://issues.apache.org/jira/browse/HDFS-1073 HDFS Write pipeline improvements for Hbase Append/flush etc. Re-implementation of MapReduce Shuffle 30% performance gain Stability using netty rather than jetty Small jobs optimizations … © Hortonworks Inc. 2011 6

Deployment goals Clusters of 6,000machines Each machine with 16 cores, 48G/96G RAM, 24TB/36TB disks 100,000+ concurrent tasks 10,000 concurrent jobs © Hortonworks Inc. 2011 7

Testing Currently tested at reasonable scale - ~500 nodes incl. GridMixv3 Continue to improve on performance benchmarks GridMixv3 Sort Shuffle HDFS Scan HDFS throughput … © Hortonworks Inc. 2011 8

Timelines branch-0.23 – August 2011 Alpha (hadoop-0.23.0) - ~October 2011 Production – late Q1 2012 YMMV!  © Hortonworks Inc. 2011 9

Thank You.@acmurthy © Hortonworks Inc. 2011

What's hot

Hortonworks SmartSense provides proactive recommendations that improve cluster performance, security and operations. And since 30% of issues are configuration related, Hortonworks SmartSense makes an immediate impact on Hadoop system performance and availability, in some cases boosting hardware performance by two times. Learn how SmartSense can help you increase the efficiency of your Hadoop hardware, through customized cluster recommendations. View the on-demand webinar: https://hortonworks.com/webinar/boosts-hadoop-hardware-performance-2x-smartsense/

Double Your Hadoop Hardware Performance with SmartSense

Hortonworks

Managing Enterprise Hadoop Clusters with Apache Ambari

Hortonworks

Hive present-and-feature-shanghai

Yifeng Jiang

View the recording: http://hortonworks.com/webinar/accelerating-real-time-data-ingest-hadoop/ Hadoop didn’t disrupt the data center. The exploding amounts of data did. But, let’s face it, if you can’t move your data to Hadoop, then you can’t use it in Hadoop. The experts from Hortonworks, the #1 leader in Hadoop development, and Attunity, a leading data management software provider, cover: - How to ingest your most valuable data into Hadoop using Attunity Replicate - About how customers are using Hortonworks DataFlow (HDF) powered by Apache NiFi - How to combine the real-time change data capture (CDC) technology with connected data platforms from Hortonworks We discuss how Attunity Replicate and Hortonworks Data Flow (HDF) work together to move data into Hadoop.

Attunity Hortonworks Webinar- Sept 22, 2016

Hortonworks

Webinar Series Part 5 New Features of HDF 5

Hortonworks

Streamline Hadoop DevOps with Apache Ambari

DataWorks Summit/Hadoop Summit

Crossing the Chasm

Hortonworks

Customers are preparing themselves to analyze and manage an increasing quantity of structured and unstructured data. Business leaders introduce new analytical workloads faster than what IT departments can handle. Legacy IT infrastructure needs to evolve to deliver operational improvements and cost containment, while increasing flexibility to meet future requirements. By providing HDP on IBM Power Systems, Hortonworks and IBM are giving customers have more choice in selecting the appropriate architectural platform that is right for them. In this webinar, we’ll discuss some of the challenges with deploying big data platforms, and how choosing solutions built with HDP on IBM Power Systems can offer tangible benefits and flexibility to accommodate changing needs.

Delivering a Flexible IT Infrastructure for Analytics on IBM Power Systems

Hortonworks

Deep learning is useful for enterprises tasks in the field of speech recognition, image classification, AI chatbots, and machine translation, just to name a few. In order to train deep learning/machine learning models, applications such as TensorFlow, MXNet, Caffe, and XGBoost can be leveraged. And sometimes these applications will be used together to solve different problems. To make distributed deep learning/machine learning applications easily launched, managed, and monitored, we introduced, in Apache Hadoop 3.x, YARN native services along with other improvements such as first-class GPU support, container-DNS support, scheduling improvements, etc. These improvements make distributed deep learning/machine learning applications run on YARN as simple as running it locally, which can let machine learning engineers focus on algorithms instead of worrying about underlying infrastructure. Also, YARN can better manage a shared cluster which runs deep learning/machine learning and other services and ETL jobs with these improvements. In this session, we will take a closer look at these improvements and show how to run these applications on YARN with demos. Audiences can start trying running these applications on YARN after this talk. Speakers Wanga Tan, Staff Software Engineer, Hortonworks Sunil Govindan, Staff Engineer, Hortonworks

Deep learning on yarn running distributed tensorflow etc on hadoop cluster v3

DataWorks Summit

Apache Ambari is now the preferred way of provisioning, managing and monitoring Hadoop Clusters. Ambari helps users to manage Hadoop clusters simplifying actions such as upgrades, configuration management, service management, etc. From release 2.0, Ambari started supporting automated Rolling Upgrades. This was further enhanced with release 2.2.0.0 to include support for Express Upgrades, which allows users to upgrade large scale clusters faster but requiring cluster downtime. This talk will cover planning and execution of Hadoop cluster upgrades from an operational perspective. The talk will also cover the internals of the upgrade process including the various stages such as pre-upgrade, backup, service checks, configuration upgrades, and finalization. Finally, the talk will cover troubleshooting upgrade failures, monitoring services during upgrades and post upgrade actions. The presentation will conclude with a case study that will cover how the upgrade process works on a large cluster (including aspects such as planning the upgrade, the amount of time required for the various stages, and troubleshooting)

Apache Ambari - HDP Cluster Upgrades Operational Deep Dive and Troubleshooting

DataWorks Summit/Hadoop Summit

Apache Hive 2.0: SQL, Speed, Scale

DataWorks Summit/Hadoop Summit

Apache NiFi Toronto Meetup

Hortonworks

HDF: Hortonworks DataFlow: Technical Workshop

Hortonworks

Ambari continues on its journey of provisioning, monitoring and managing enterprise Hadoop deployments. With 2.0, Apache Ambari brings a host of new capabilities including updated metric collections; Kerberos setup automation and developer views for Big Data developers. In this Hortonworks Technical Workshop session we will provide an in-depth look into Apache Ambari 2.0 and showcase security setup automation using Ambari 2.0. View the recording at https://www.brighttalk.com/webcast/9573/155575. View the github demo work at https://github.com/abajwa-hw/ambari-workshops/blob/master/blueprints-demo-security.md. Recorded May 28, 2015.

Hortonworks technical workshop operations with ambari

Hortonworks

Apache Hive has been continuously evolving to support a broad range of use cases, bringing it beyond its batch processing roots to its current support for interactive queries with sub-second response times using LLAP. However, the development of its execution internals is not sufficient to guarantee efficient performance, since poorly optimized queries can create a bottleneck in the system. Hence, each release of Hive has included new features for its optimizer aimed to generate better plans and deliver improvements to query execution. In this talk, we present the development of the optimizer since its initial release. We describe its current state and how Hive leverages the latest Apache Calcite features to generate the most efficient execution plans. We show numbers demonstrating the improvements brought to Hive performance, and we discuss future directions for the next-generation Hive optimizer, which include an enhanced cost model, materialized views support, and complex query decorrelation.

An Overview on Optimization in Apache Hive: Past, Present Future

DataWorks Summit/Hadoop Summit

Learn more: http://hortonworks.com/hdf/ Log data can be complex to capture, typically collected in limited amounts and difficult to operationalize at scale. HDF expands the capabilities of log analytics integration options for easy and secure edge analytics of log files in the following ways: More efficient collection and movement of log data by prioritizing, enriching and/or transforming data at the edge to dynamically separate critical data. The relevant data is then delivered into log analytics systems in a real-time, prioritized and secure manner. Cost-effective expansion of existing log analytics infrastructure by improving error detection and troubleshooting through more comprehensive data sets. Intelligent edge analytics to support real-time content-based routing, prioritization, and simultaneous delivery of data into Connected Data Platforms, log analytics and reporting systems for comprehensive coverage and retention of Internet of Anything data.

Log Analytics Optimization

Hortonworks

Apache Hadoop YARN: Past, Present and Future

DataWorks Summit

Apache Ambari - What's New in 2.4

Hortonworks

Hive ACID Apache BigData 2016

alanfgates

Apache NiFi in the Hadoop Ecosystem

DataWorks Summit/Hadoop Summit

What's hot (20)

Double Your Hadoop Hardware Performance with SmartSense

Managing Enterprise Hadoop Clusters with Apache Ambari

Hive present-and-feature-shanghai

Attunity Hortonworks Webinar- Sept 22, 2016

Webinar Series Part 5 New Features of HDF 5

Streamline Hadoop DevOps with Apache Ambari

Crossing the Chasm

Delivering a Flexible IT Infrastructure for Analytics on IBM Power Systems

Deep learning on yarn running distributed tensorflow etc on hadoop cluster v3

Apache Ambari - HDP Cluster Upgrades Operational Deep Dive and Troubleshooting

Apache Hive 2.0: SQL, Speed, Scale

Apache NiFi Toronto Meetup

HDF: Hortonworks DataFlow: Technical Workshop

Hortonworks technical workshop operations with ambari

An Overview on Optimization in Apache Hive: Past, Present Future

Log Analytics Optimization

Apache Hadoop YARN: Past, Present and Future

Apache Ambari - What's New in 2.4

Hive ACID Apache BigData 2016

Apache NiFi in the Hadoop Ecosystem

Viewers also liked

Getting involved with Open Source at the ASF

Hortonworks

Combining IOT, Customer Experience and Real-Time Enterprise Data within Hadoop. What if you could derive real-time insights using ALL of your data? Join us for this webinar and learn how companies are combining “new” real-time data sources (i.e. IOT, Social, Web Logs) with continuously updated enterprise data from SAP and other enterprise transactional systems, providing deep and up-to-the-second analytical insights. This presentation will include a demonstration of how this can be achieved quickly, easily and affordably by utilizing a joint solution from Attunity and Hortonworks.

Enabling the Real Time Analytical Enterprise

Hortonworks

As enterprises around the world bring more of their sensitive data into Hadoop data lakes, balancing the need for democratization of access to data without sacrificing strong security principles becomes paramount. In this webinar, Srikanth Venkat, director of product management for security & governance will demonstrate two new data protection capabilities in Apache Ranger – dynamic column masking and row level filtering of data stored in Apache Hive. These features have been introduced as part of HDP 2.5 platform release.

Dynamic Column Masking and Row-Level Filtering in HDP

Hortonworks

Apache NiFi, Storm and Kafka augment each other in modern enterprise architectures. NiFi provides a coding free solution to get many different formats and protocols in and out of Kafka and compliments Kafka with full audit trails and interactive command and control. Storm compliments NiFi with the capability to handle complex event processing.   Join us to learn how Apache NiFi, Storm and Kafka can augment each other for creating a new dataplane connecting multiple systems within your enterprise with ease, speed and increased productivity. https://www.brighttalk.com/webcast/9573/224063

Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...

Hortonworks

Student performance data is increasingly being captured as part of software-based and online classroom exercises and testing. This data can be augmented with behavioral data captured from sources such as social media, student-professor meeting notes, blogs, student surveys, and so forth to discover new insights to improve student learning. The results transcend traditional IT departments to focus on issues like retention, research, and the delivery of content and courses through new modalities. Hortonworks is partnering with Microsoft to show you how the Hortonworks Data Platform (HDP) running on the Microsoft stack enables you to develop a “single view of a student”.

How Universities Use Big Data to Transform Education

Hortonworks

Hive - 1455: Cloud Storage

Hortonworks

Scaling real time streaming architectures with HDF and Dell EMC Isilon

Hortonworks

S3Guard: What's in your consistency model?

Hortonworks

Hortonworks Data Cloud for AWS

Hortonworks

Innovative mobile operators need to mine the vast troves of unstructured data now available to them to help develop compelling customer experiences and uncover new revenue opportunities. In this webinar, you’ll learn how HDB’s in-database analytics enable advanced use cases in network operations, customer care, and marketing for better customer experience. Join us, and get started on your advanced analytics journey today!

Pivotal - Advanced Analytics for Telecommunications

Hortonworks

Top 5 Strategies for Retail Data Analytics

Hortonworks

Delivering Data-Driven Applications at the Speed of Business: Global Banking AML use case. Chief Data Officers in financial services have unique challenges: they need to establish an effective data ecosystem under strict governance and regulatory requirements. They need to build the data-driven applications that enable risk and compliance initiatives to run efficiently. In this webinar, we will discuss the case of a global banking leader and the anti-money laundering solution they built on the data lake. With a single platform to aggregate structured and unstructured information essential to determine and document AML case disposition, they reduced mean time for case resolution by 75%. They have a roadmap for building over 150 data-driven applications on the same search-based data discovery platform so they can mitigate risks and seize opportunities, at the speed of business.

The path to a Modern Data Architecture in Financial Services

Hortonworks

Edw Optimization Solution

Hortonworks

NextGen Apache Hadoop MapReduce

Hortonworks

Intro to Big Data Hadoop

Apache Apex

Hadoop crashcourse v3

Hortonworks

Big data Overview

Arnon Rotem-Gal-Oz

Introduction to Hadoop and Pig

prash1784

How to shutdown and power up of the netapp cluster mode storage system

Saroj Sahu

The Elephant in the Clouds

DataWorks Summit/Hadoop Summit

Viewers also liked (20)

Getting involved with Open Source at the ASF

Enabling the Real Time Analytical Enterprise

Dynamic Column Masking and Row-Level Filtering in HDP

Hortonworks Data in Motion Webinar Series Part 7 Apache Kafka Nifi Better Tog...

How Universities Use Big Data to Transform Education

Hive - 1455: Cloud Storage

Scaling real time streaming architectures with HDF and Dell EMC Isilon

S3Guard: What's in your consistency model?

Hortonworks Data Cloud for AWS

Pivotal - Advanced Analytics for Telecommunications

Top 5 Strategies for Retail Data Analytics

The path to a Modern Data Architecture in Financial Services

Edw Optimization Solution

NextGen Apache Hadoop MapReduce

Intro to Big Data Hadoop

Hadoop crashcourse v3

Big data Overview

Introduction to Hadoop and Pig

How to shutdown and power up of the netapp cluster mode storage system

The Elephant in the Clouds

Similar to Apache Hadoop 0.23

Hortonworks Presentation at Big Data London

Hortonworks

4apachehadoop 0-23hadoopworld2011-111110151810-phpapp02

gr0Jmmd7Q_qarobo gr0Jmmd7Q_qarobo

4apachehadoop-0-23hadoopworld2011-111110151810-phpapp02

gr0Jmmd7Q_qarobo gr0Jmmd7Q_qarobo

4apachehadoop 0-23hadoopworld2011-111110151810-phpapp02

Nitish Bhardwaj

Don't Let Security Be The 'Elephant in the Room'

Hortonworks

Hortonworks continues to innovate throughout all Hadoop related projects, packaging the most enterprise-ready components, such as Ambari, into the Hortonworks Data Platform (HDP). Please join us in this interactive webinar as we present real-world use cases of Enterprise customers that are finding success with HDP and their Big Data initiatives. We will also introduce new features from version 1.2 of the Hortonworks Data Platform and how it has become the leading 100% open source distribution choice for the Enterprise. In this webinar we will outline how enterprise customers are successfult with HDP and also review some of the newest features in version1.2 including: -How to provision a cluster -How to manage and monitor a cluster using completely open source tools -How to perform diagnostics to identify issues in a cluster

Hadoop Operations, Innovations and Enterprise Readiness with Hortonworks Data...

Hortonworks

In 2012, we released Hortonworks Data Platform powered by Apache Hadoop and established partnerships with major enterprise software vendors including Microsoft and Teradata that are making enterprise ready Hadoop easier and faster to consume. As we start 2013, we invite you to join us for this live webinar where Shaun Connolly, VP of Strategy at Hortonworks, will cover the highlights of 2012 and the road ahead in 2013 for Hortonworks and Apache Hadoop.

State of the Union with Shaun Connolly

Hortonworks

Which Hadoop Distribution to use: Apache, Cloudera, MapR or HortonWorks?

Edureka!

Mrinal devadas, Hortonworks Making Sense Of Big Data

PatrickCrompton

Best Practices for Hadoop Data Analysis with Tableau and Hortonworks Data Pla...

Hortonworks

How do I deploy, manage and monitor Apache Hadoop within my Data Center? Olivier Renault will help you kick start your deployment by sharing the experience that the Hortonworks team have from deployment in the Yahoo Data Center and many other enterprise production deployments. During this talk, we will present Ambari the Open Source management solution by Apache which make deploying and monitoring Hadoop extremely easy.

OSDC 2013 | Introduction into Hadoop by Olivier Renault

NETWAYS

Hortonworks - What's Possible with a Modern Data Architecture?

Hortonworks

Apache Hadoop at 10

Cloudera, Inc.

Cloudera

Ahmed Salman

Bring your Service to YARN

DataWorks Summit

Yahoo! - Arun Murthy - Hadoop World 2010

Cloudera, Inc.

Deploying, configuring, and managing large Hadoop and HBase clusters can be quite complex. Just upgrading one Hadoop component on a 2000-node cluster can take a lot of time and expertise, and there have been few tools specialized for Hadoop cluster administrators. AMBARI is an Apache incubator project to deliver Monitoring and Management functionality for Hadoop clusters. This paper presents the AMBARI tools for cluster management, specifically: Cluster pre-configuration and validation; Hadoop software deployment, installation, and smoketest; Hadoop configuration and re-config; and a basic set of management ops including start/stop service, add/remove node, etc. In providing these capabilities, AMBARI seeks to integrate with (rather than replace) existing open-source packaging and deployment technology available in most data centers, such as Puppet and Chef, Yum, Apt, and Zypper.

Deploying and Managing Hadoop Clusters with AMBARI

DataWorks Summit

Hadoop in adtech

Yuta Imai

Hortonworks Big Data Career Paths and Training

Aengus Rooney

SpringOne Platform 2016 Speaker: Ian Fyfe; Director, Product Marketing, Hortonworks Apache Hadoop is the most powerful and popular platform for ingesting, storing and processing enormous amounts of “big data”. However, due to its original roots as a batch processing system, doing interactive business analytics with Hadoop has historically suffered from slow response times, or forced business analysts to extract data summaries out of Hadoop into separate data marts. This talk will discuss the different options for implementing speed-of-thought business analytics and machine learning tools directly on top of Hadoop including Apache Hive on Tez, Apache Hive on LLAP, Apache HAWQ and Apache MADlib.

Achieving Mega-Scale Business Intelligence Through Speed of Thought Analytics...

VMware Tanzu

Similar to Apache Hadoop 0.23 (20)

Hortonworks Presentation at Big Data London

4apachehadoop 0-23hadoopworld2011-111110151810-phpapp02

4apachehadoop-0-23hadoopworld2011-111110151810-phpapp02

4apachehadoop 0-23hadoopworld2011-111110151810-phpapp02

Don't Let Security Be The 'Elephant in the Room'

Hadoop Operations, Innovations and Enterprise Readiness with Hortonworks Data...

State of the Union with Shaun Connolly

Which Hadoop Distribution to use: Apache, Cloudera, MapR or HortonWorks?

Mrinal devadas, Hortonworks Making Sense Of Big Data

Best Practices for Hadoop Data Analysis with Tableau and Hortonworks Data Pla...

OSDC 2013 | Introduction into Hadoop by Olivier Renault

Hortonworks - What's Possible with a Modern Data Architecture?

Apache Hadoop at 10

Cloudera

Bring your Service to YARN

Yahoo! - Arun Murthy - Hadoop World 2010

Deploying and Managing Hadoop Clusters with AMBARI

Hadoop in adtech

Hortonworks Big Data Career Paths and Training

Achieving Mega-Scale Business Intelligence Through Speed of Thought Analytics...

More from Hortonworks

Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level

Hortonworks

Forrester forecasts* that direct spending on the Internet of Things (IoT) will exceed $400 Billion by 2023. From manufacturing and utilities, to oil & gas and transportation, IoT improves visibility, reduces downtime, and creates opportunities for entirely new business models. But successful IoT implementations require far more than simply connecting sensors to a network. The data generated by these devices must be collected, aggregated, cleaned, processed, interpreted, understood, and used. Data-driven decisions and actions must be taken, without which an IoT implementation is bound to fail. https://hortonworks.com/webinar/iot-predictions-2019-beyond-data-heart-iot-strategy/

IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy

Hortonworks

Getting the Most Out of Your Data in the Cloud with Cloudbreak

Hortonworks

Johns Hopkins - Using Hadoop to Secure Access Log Events

Hortonworks

Cybersecurity today is a big data problem. There’s a ton of data landing on you faster than you can load, let alone search it. In order to make sense of it, we need to act on data-in-motion, use both machine learning, and the most advanced pattern recognition system on the planet: your SOC analysts. Advanced visualization makes your analysts more efficient, helps them find the hidden gems, or bombs in masses of logs and packets. https://hortonworks.com/webinar/catch-hacker-real-time-live-visuals-bots-bad-guys/

Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys

Hortonworks

HDF 3.2 - What's New

Hortonworks

With the growth of Apache Kafka adoption in all major streaming initiatives across large organizations, the operational and visibility challenges associated with Kafka are on the rise as well. Kafka users want better visibility in understanding what is going on in the clusters as well as within the stream flows across producers, topics, brokers, and consumers. With no tools in the market that readily address the challenges of the Kafka Ops teams, the development teams, and the security/governance teams, Hortonworks Streams Messaging Manager is a game-changer. https://hortonworks.com/webinar/curing-kafka-blindness-hortonworks-streams-messaging-manager/

Curing Kafka Blindness with Hortonworks Streams Messaging Manager

Hortonworks

The healthcare industry—with its huge volumes of big data—is ripe for the application of analytics and machine learning. In this webinar, Hortonworks and Quanam present a tool that uses machine learning and natural language processing in the clinical classification of genomic variants to help identify mutations and determine clinical significance. Watch the webinar: https://hortonworks.com/webinar/interpretation-tool-genomic-sequencing-data-clinical-environments/

Interpretation Tool for Genomic Sequencing Data in Clinical Environments

Hortonworks

IBM+Hortonworks = Transformation of the Big Data Landscape

Hortonworks

Premier Inside-Out: Apache Druid

Hortonworks

Accelerating Data Science and Real Time Analytics at Scale

Hortonworks

TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA

Hortonworks

Trimble Transportation Enterprise is a leading provider of enterprise software to over 2,000 transportation and logistics companies. They have designed an architecture that leverages Hortonworks Big Data solutions and Machine Learning models to power up multiple Blockchains, which improves operational efficiency, cuts down costs and enables building strategic partnerships. https://hortonworks.com/webinar/blockchain-with-machine-learning-powered-by-big-data-trimble-transportation-enterprise/

Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...

Hortonworks

For years, the healthcare industry has had problems of data scarcity and latency. Clearsense solved the problem by building an open-source Hortonworks Data Platform (HDP) solution while providing decades worth of clinical expertise. Clearsense is delivering smart, real-time streaming data, to its healthcare customers enabling mission-critical data to feed clinical decisions. https://hortonworks.com/webinar/delivering-smart-real-time-streaming-data-healthcare-customers-clearsense/

Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense

Hortonworks

Making Enterprise Big Data Small with Ease

Hortonworks

Webinewbie to Webinerd in 30 Days - Webinar World Presentation

Hortonworks

Using your data smarter and faster than your peers could be the difference between dominating your market and merely surviving. Organizations are investing in IoT, big data, and data science to drive better customer experience and create new products, yet these projects often stall in ideation phase to a lack of global data management processes and technologies. Your new data architecture may be taking shape around you, but your goal of globally managing, governing, and securing your data across a hybrid, multi-cloud landscape can remain elusive. Learn how industry leaders are developing their global data management strategy to drive innovation and ROI. Presented at Gartner Data and Analytics Summit Speaker: Dinesh Chandrasekhar Director of Product Marketing, Hortonworks

Driving Digital Transformation Through Global Data Management

Hortonworks

Hortonworks DataFlow (HDF) is the complete solution that addresses the most complex streaming architectures of today’s enterprises. More than 20 billion IoT devices are active on the planet today and thousands of use cases across IIOT, Healthcare and Manufacturing warrant capturing data-in-motion and delivering actionable intelligence right NOW. “Data decay” happens in a matter of seconds in today’s digital enterprises. To meet all the needs of such fast-moving businesses, we have made significant enhancements and new streaming features in HDF 3.1. https://hortonworks.com/webinar/series-hdf-3-1-technical-deep-dive-new-streaming-features/

HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features

Hortonworks

Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...

Hortonworks

Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. It provides an end-to-end platform that can collect, curate, analyze, and act on data in real-time, on-premises, or in the cloud with a drag-and-drop visual interface. It’s being used across industries on large amounts of data that had stored in isolation which made collaboration and analysis difficult. Join industry experts from Hortonworks and Attunity as they explain how Apache NiFi and streaming CDC technology provides a distributed, resilient platform for unlocking the value of data in new ways.

Unlock Value from Big Data with Apache NiFi and Streaming CDC

Hortonworks

More from Hortonworks (20)

Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level

IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy

Getting the Most Out of Your Data in the Cloud with Cloudbreak

Johns Hopkins - Using Hadoop to Secure Access Log Events

Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys

HDF 3.2 - What's New

Curing Kafka Blindness with Hortonworks Streams Messaging Manager

Interpretation Tool for Genomic Sequencing Data in Clinical Environments

IBM+Hortonworks = Transformation of the Big Data Landscape

Premier Inside-Out: Apache Druid

Accelerating Data Science and Real Time Analytics at Scale

TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA

Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...

Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense

Making Enterprise Big Data Small with Ease

Webinewbie to Webinerd in 30 Days - Webinar World Presentation

Driving Digital Transformation Through Global Data Management

HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features

Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...

Unlock Value from Big Data with Apache NiFi and Streaming CDC

Recently uploaded

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

How to convert PDF to text with Nanonets

naman860154

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

CNv6 Instructor Chapter 6 Quality of Service

giselly40

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Real Time Object Detection Using Open CV

Khem

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Civil Lines Women Seeking Men

Delhi Call girls

Slack Application Development 101 Slides

praypatel2

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Recently uploaded (20)

Boost Fertility New Invention Ups Success Rates.pdf

🐬 The future of MySQL is Postgres 🐘

Boost PC performance: How more available memory can improve productivity

Advantages of Hiring UIUX Design Service Providers for Your Business

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Data Cloud, More than a CDP by Matt Robison

Finology Group – Insurtech Innovation Award 2024

How to convert PDF to text with Nanonets

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

CNv6 Instructor Chapter 6 Quality of Service

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

Real Time Object Detection Using Open CV

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Axa Assurance Maroc - Insurer Innovation Award 2024

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

The 7 Things I Know About Cyber Security After 25 Years | April 2024

08448380779 Call Girls In Civil Lines Women Seeking Men

Slack Application Development 101 Slides

Presentation on how to chat with PDF using ChatGPT code interpreter

How to Troubleshoot Apps for the Modern Connected Worker

Apache Hadoop 0.23

1. Apache Hadoop 0.23 Arun C. Murthy Hortonworks Founder and Architect @acmurthy (@hortonworks) © Hortonworks Inc. 2011 August 25, 2011

2. Hello! I’m Arun… Architect & Lead, Apache Hadoop MapReduce Development Team at Hortonworks (formerly at Yahoo!) Apache Hadoop Committer and Member of PMC Full-time contributor to Apache Hadoop since early 2006 Apache HadoopRelease Manager for hadoop-0.23

3. hadoop-0.23 On track to be first stable, and widely deployed, release since hadoop-0.20 in 2009 All stable releases of Hadoop today are based on hadoop-0.20 Multiple folks and entities collaborating: Hortonworks, Yahoo, Cloudera, EBay etc. hadoop-0.23 branch in Apache hours away! © Hortonworks Inc. 2011 4

4. Highlights HDFS Federation http://www.hortonworks.com/an-introduction-to-hdfs-federation/ NextGenerationHadoopMapReduce http://www.slideshare.net/hortonworks/nextgen-apache-hadoop-mapreduce Coming soon – HDFS High Availability https://issues.apache.org/jira/browse/HDFS-1623 WIP: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-1623/ © Hortonworks Inc. 2011 5

5. More… Build - Full Mavenization EditLogs re-write https://issues.apache.org/jira/browse/HDFS-1073 HDFS Write pipeline improvements for Hbase Append/flush etc. Re-implementation of MapReduce Shuffle 30% performance gain Stability using netty rather than jetty Small jobs optimizations … © Hortonworks Inc. 2011 6

6. Deployment goals Clusters of 6,000machines Each machine with 16 cores, 48G/96G RAM, 24TB/36TB disks 100,000+ concurrent tasks 10,000 concurrent jobs © Hortonworks Inc. 2011 7

7. Testing Currently tested at reasonable scale - ~500 nodes incl. GridMixv3 Continue to improve on performance benchmarks GridMixv3 Sort Shuffle HDFS Scan HDFS throughput … © Hortonworks Inc. 2011 8

8. Timelines branch-0.23 – August 2011 Alpha (hadoop-0.23.0) - ~October 2011 Production – late Q1 2012 YMMV!  © Hortonworks Inc. 2011 9

Apache Hadoop 0.23

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Apache Hadoop 0.23

Similar to Apache Hadoop 0.23 (20)

More from Hortonworks

More from Hortonworks (20)

Recently uploaded

Recently uploaded (20)

Apache Hadoop 0.23