This session describes how Impala integrates with Kudu for analytic SQL queries on Hadoop and how this integration, taking full advantage of the distinct properties of Kudu, has significant performance benefits.

1. How to use Impala &
Kudu to optimize
performance for
Analytic Workloads
David Alves| david.alves@cloudera.com

2. ‹#›‹#›
Impala: A Modern, Open-Source SQL Engine
• Implementation of an MPP SQL query engine for the Hadoop environment
• Designed for performance: brand-new engine, written in C++
• Maintains Hadoop flexibility by utilizing standard Hadoop components
(HDFS, Kudu, HBase, MetaStore, Yarn)
• Plays well with traditional BI tools:
exposes/interacts with industry-standard interfaces (odbc/jdbc, Kerberos and
LDAP, ANSI SQL)

3. ‹#›‹#›
Kudu
Storage for Fast Analytics on Fast Data
• New updatable column store for
Hadoop
• Currently incubating as an
Apache project
• Beta now available
(kudu.apache.org)Columnar Store
Kudu

4. ‹#›‹#›
Impala - Introduction
4

5. ‹#›‹#›
Impala Architecture: Distributed System
• Daemon process (impalad) runs on every node with data
• Each node handles user requests
• Load balancer configuration for multi-user environments recommended
• Metadata management: catalog service (single node)
• System state repository and distribution: statestore (single node)
• Catalog service and statestore are stateless

6. ‹#›‹#›
Impala Query Execution
• Query execution phases:
• client requests arrive via odbc/jdbc
• query planner turns request into collection of plan fragments
• coordinator initiates execution on remote impala’s
• During execution:
• intermediate results are streamed between query executors
• query results are streamed back to client
• subject to limitation imposed by blocking operators (top-n, aggregation,
sorting)

7. ‹#›‹#›
Impala Query Execution
Request arrives via odbc/jdbc

8. ‹#›‹#›
Impala Query Execution
Planner turns request into collection of plan fragments
Coordinator initiates execution on remote impalad nodes

9. ‹#›‹#›
Impala Query Execution
Intermediate results are streamed between impalad’s
Query results are streamed back to client

10. ‹#›‹#›
Query Planning: Overview
• Two-phase planning process
• single-node plan: tree of plan operators
• partitioning of operator tree into plan fragments for parallel execution
• Parallelization of operators across nodes
• all query operators are fully distributed
• Cost-based join order optimization
• Cost-based join distribution optimization

11. ‹#›‹#›
Impala Execution Engine
• Written in C++ for minimal cycle and memory overhead
• Leverages existing parallel DB research
• data-partitioned parallelism
• pipelined relational operators
• batch-at-a-time runtime
• Focussed on speed and efficiency
• intrinsics/machine code for text parsing, hashing, etc.
• runtime code generation with llvm

12. ‹#›‹#›
Kudu - Introduction
12

13. ‹#›‹#›
• High throughput for big scans (columnar
storage and replication)
Goal: Within 2x of Parquet
• Low-latency for short accesses (primary
key indexes and quorum replication)
Goal: 1ms read/write on SSD
• Database-like semantics (initially single-
row ACID)
• Relational data model
• SQL query
• “NoSQL” style scan/insert/update (Java client)
Kudu Design Goals

14. ‹#›‹#›
Kudu Design Goals
how effectively primary key filters can be pushed down to Kudu.
What do I use Kudu for?
We talked about how Kudu is made for SQL, allows fast scans, and allows fast mutability at
scale. With that in context, let’s look at the variety of use cases done in Hadoop today and see
where Kudu fits in.
If we look at Kudu in the above figure, we will see that many of the traditional SQL use cases

15. ‹#›‹#›
Kudu Usage
• Table has a SQL-like schema
• Finite number of columns (unlike HBase/Cassandra)
• Types: BOOL, INT8, INT16, INT32, INT64, FLOAT, DOUBLE, STRING,
BINARY, TIMESTAMP
• Some subset of columns makes up a possibly-composite primary key
• Fast ALTER TABLE
• Java and C++ “NoSQL” style APIs
• Insert(), Update(), Delete(), Scan()
• Integrations with Impala, Spark, MapReduce
• more to come!
15

16. ‹#›‹#›
Kudu Use Cases
Kudu is best for use cases requiring a simultaneous combination
of sequential and random reads and writes
● Time Series
○ Examples: Stream market data; fraud detection & prevention; risk
monitoring
○ Workload: Insert, updates, scans, lookups
● Machine Data Analytics
○ Examples: Network threat detection
○ Workload: Inserts, scans, lookups
● Online Reporting
○ Examples: ODS
○ Workload: Inserts, updates, scans, lookups

17. ‹#›‹#›
Real-Time Analytics in Hadoop Today
Fraud Detection in the Real World = Storage Complexity
Considerations:
● How do I handle failure
during this process?
● How often do I reorganize
data streaming in into a
format appropriate for
reporting?
● When reporting, how do I see
data that has not yet been
reorganized?
● How do I ensure that
important jobs aren’t
interrupted by maintenance?
New Partition
Most Recent Partition
Historic Data
HBase
Parquet
File
Have we
accumulated
enough data?
Reorganize
HBase file
into Parquet
• Wait for running operations to complete
• Define new Impala partition referencing
the newly written Parquet file
Incoming Data
(Messaging
System)
Reporting
Request
Impala on HDFS

18. ‹#›‹#›
Real-Time Analytics in Hadoop with Kudu
Improvements:
● One system to operate
● No cron jobs or background
processes
● Handle late arrivals or data
corrections with ease
● New data available
immediately for analytics or
operations
Historical and Real-time
Data
Incoming Data
(Messaging
System)
Reporting
Request
Storage in Kudu

19. ‹#›‹#›
Tables and Tablets
• Table is horizontally partitioned into tablets
• Range or hash partitioning
• Each tablet has N replicas (3 or 5), with Raft consensus
• Allow read from any replica, plus leader-driven writes with low MTTR
• Tablet servers host tablets
• Store data on local disks (no HDFS)
19

20. ‹#›‹#›

21. ‹#›‹#›
Tablet design
• Inserts buffered in an in-memory store (like HBase’s memstore)
• Flushed to disk
• Columnar layout, similar to Apache Parquet
• Updates use MVCC (updates tagged with timestamp, not in-place)
• Allow “SELECT AS OF <timestamp>” queries and consistent cross-
tablet scans
• Near-optimal read path for “current time” scans
• No per row branches, fast vectorized decoding and predicate evaluation
• Performance worsens based on number of recent updates
21

22. ‹#›‹#›
Impala & Kudu –
Better together
22

23. ‹#›‹#›
Impala + Kudu Architecture
Impalad
Kudu
Tablet
Server
Impal
ad
…
Statestore
Catalog
Service
Hive
Metastore
Hadoop
Namenode
Impalad
Kudu
Tablet
Server
Impalad
Kudu
Tablet
Server
Kudu
Master

24. ‹#›‹#›
Impala & Kudu integration – User features
• Table Create/Delete
• Advanced partitioning schemes
• Easily load/store data to/from kudu:
– “Create table kudu_table as select * from hdfs_table”
– ”Insert into hdfs_table select * from kudu_table AS PARQUET”

25. ‹#›‹#›
Impala & Kudu integration – User features
• Table Create/Delete
• Advanced partitioning schemes
• Easily load/store data to/from kudu:
– “Create table kudu_table as select * from hdfs_table”
– ”Insert into hdfs_table select * from kudu_table AS PARQUET”

26. ‹#›‹#›
Impala & Kudu integration - Partitioning
• Range partitioning
– PRIMARY KEY (host, metric, timestamp) DISTRIBUTE
BY HASH(timestamp) INTO 100 BUCKETS
• Hash partitioning
– PRIMARY KEY (last_name, first_name)DISTRIBUTE BY
RANGE (last_name, first_name)
• Range + Hash partitioning
– PRIMARY KEY (last_name, first_name)DISTRIBUTE BY
RANGE (last_name, first_name) INTO 100 BUCKETS

27. ‹#›‹#›
Impala & Kudu integration – Runtime Features
• Optimized data layout
• Predicate pushdown
• Data locality
• Tolerance to Kudu faults

28. ‹#›‹#›
Impala & Kudu integration – Roadmap
• Shared memory between impala and Kudu
• Scan Tokens – Forward encoded partitioning information to the Kudu client
• Timetravel scans
• Memory layout matching
• More predicate pushdown (like bloomfilters).

29. ‹#›‹#›
Benchmarks
29

30. ‹#›‹#›
TPC-H (Analytics benchmark)
• 75TS + 1 master cluster
• 12 (spinning) disk each, enough RAM to fit dataset
• Using Kudu 0.5.0, Impala 2.2 with Kudu support, CDH 5.4
• TPC-H Scale Factor 100 (100GB)
• Example query:
• SELECT n_name, sum(l_extendedprice * (1 - l_discount)) as revenue FROM customer,
orders, lineitem, supplier, nation, region WHERE c_custkey = o_custkey AND
l_orderkey = o_orderkey AND l_suppkey = s_suppkey AND c_nationkey = s_nationkey
AND s_nationkey = n_nationkey AND n_regionkey = r_regionkey AND r_name = 'ASIA'
AND o_orderdate >= date '1994-01-01' AND o_orderdate < '1995-01-01’ GROUP BY
n_name ORDER BY revenue desc;
30

31. © Cloudera, Inc. All rights reserved.
- Kudu outperforms Parquet by 31% (geometric mean) for RAM-resident data
- Parquet likely to outperform Kudu for HDD-resident (larger IO requests)

32. ‹#›‹#›
What about Apache Phoenix?
• 10 node cluster (9 worker, 1 master)
• HBase 1.0, Phoenix 4.3
• TPC-H LINEITEM table only (6B rows)
32
2152
219
76
131
0.04
1918
13.2
1.7
0.7
0.15
155
9.3
1.4 1.5 1.37
0.01
0.1
1
10
100
1000
10000
Load TPCH Q1 COUNT(*)
COUNT(*)
WHERE…
single-row
lookup
Time(sec)
Phoenix
Kudu
Parquet

33. ‹#›‹#›
What about NoSQL-style random access? (YCSB)
• YCSB 0.5.0-snapshot
• 10 node cluster
(9 worker, 1 master)
• HBase 1.0
• 100M rows, 10M ops
33

34. © Cloudera, Inc. All rights reserved.
Kudu & Impala @ Xiaomi
Mobile service monitoring and tracing tool
Requirements
u High write throughput
>5 Billion records/day and growing
u Query latest data and quick response
Identify and resolve issues quickly
u Can search for individual records
Easy for troubleshooting
Gather important RPC tracing events from mobile
app and backend service.
Service monitoring & troubleshooting tool.

35. © Cloudera, Inc. All rights reserved.
Benchmark
Environment
u 71 Node cluster
u Hardware:
CPU: E5-2620 2.1GHz * 24 core Memory: 64GB
Network: 1Gb Disk: 12 HDD
u Software: Hadoop2.6/Impala 2.1/Kudu
Data
u 1 D of tracingdata:
~2.6 B rows, ~270
bytes/row
17 columns, 5 key columns
Workload
u Mix of analytical more lookup style queries
u Compared vs Parquet

36. © Cloudera, Inc. All rights reserved.
Benchmark Results
1.4 2.0 2.3
3.1
1.3 0.91.3
2.8
4.0
5.7
7.5
16.7
Q1 Q2 Q3 Q4 Q5 Q6
kudu
parquet
Total Time(s) Throughput(Total) Throughput(per node)
Kudu 961.1 2.8M record/s 39.5k record/s
Parquet 114.6 23.5M record/s 331k records/s
Bulk load using impala (INSERT INTO):
Query latency:
* HDFS parquet file replication = 3 , kudu table replication = 3
* Each query run 5 times then take average

37. ‹#›‹#›
http://kudu.apache.org/
@ApacheKudu
http://impala.io/
@RideImpala
Thank you
David Alves
@dribeiroalves

38. ‹#›‹#›
This an example
segue slide on a blue
background. This could
also be a quote slide.
This is an optional subtitle or space for attribution

39. ‹#›‹#›
This an example
segue slide on a blue
background. This could
also be a quote slide.
This is an optional subtitle or space for attribution

40. ‹#›‹#›
This an example
segue slide on a blue
background. This could
also be a quote slide.
This is an optional subtitle or space for attribution