Teradata joined the Presto community in 2015 and is now a leading contributor to this open source SQL engine, originally created by Facebook. The project has a rapidly growing community of users, including Airbnb, FINRA, Netflix, Twitter, and Uber. Kamil Bajda-Pawlikowski explores the key architectural components that allow querying variety of data sources and make Presto uniquely position to be applied in both Hadoop and Cloud use cases. Along the way, Kamil covers Teradata’s recent enhancements in query performance, security integrations, and ANSI SQL coverage and shares the roadmap for 2017 and beyond.

1. 1
Presto: Distributed SQL on Anything
Strata Hadoop, San Jose 2017
Kamil Bajda-Pawlikowski
Chief Architect
Teradata Center for Hadoop

2. 2
What is Presto?
100% open source distributed ANSI SQL query engine
Originally developed by Facebook
Key Differentiators:
Performance & Scale
Cross platform query capability, not only SQL on Hadoop
Apache licensed, hosted on GitHub
Certified distro & support from Teradata

3. 3
Presto Architecture
Data stream API
Worker
Data stream API
Worker
Coordinator
Metadata
API
Parser/
analyzer
Planner Scheduler
Worker
Client
Data location
API
Pluggable

4. 4
Presto Extensibility
Parser/
analyzer
Planner
Worker
Data location API
HDFS/S3
NoSQL
DBMS
Custom
…
Metadata API
HDFS/S3
NoSQL
DBMS
Custom
…
Data stream API
HDFS/S3
NoSQL
DBMS
Custom
…
Scheduler
Coordinator

5. 5
Presto Connectors
Amazon S3

6. 6
Presto Users
See more at https://github.com/prestodb/presto/wiki/Presto-Users

7. 7
Presto in Action
Multiple clusters (1000s of nodes
total)
300PB in HDFS, MySQL, and Raptor
1000s users, 10-100s concurrent
queries

8. 8
Presto in Action
250+ nodes on AWS
40+ PB stored in S3 (Parquet)
Over 650 users with 6K+ queries
daily

9. 9
Presto in Action
200+ nodes (2 dedicated clusters)
25K+ & 3K+ queries daily

10. 10
Presto in Action
200+ nodes on-premises
Parquet nested data

11. 11
Presto in Action
120+ nodes in AWS
2PB is S3 and 200+ users
supported by Teradata

12. 12
Open Source Community
• Collaboration with Facebook and Presto community
– Joint design and development
– Conference talks, meetups and webinars
• Major commitment from Teradata Labs:
– 20 full-time engineers
– Free and open source contributions
– Enterprise-ready distribution
"A special shout out goes to Teradata — which joined the Presto community this year
with a focus on enhancing enterprise features and providing support — for having
seven of our top 10 external contributors."
— Facebook

13. 13
Implement Integrate Proliferate
• Installer
• Documentation
• Monitoring & Support
Tools
• Management Tool
Integration
• YARN Integration
• ODBC Driver
• JDBC Driver
• BI Certification
• Kerberos
• Connectors
Commercial Support
Phase 1 Phase 2 Phase 3
June 8, 2015 Q4 2015 Q2 2016
Expanding ANSI SQL Coverage
Teradata Initial Contributions to Presto

14. 14
[ WITH with_query [, ...] ]
SELECT [ ALL | DISTINCT ] select_expr [, ...]
[ FROM table1 [[ INNER | OUTER ] JOIN table2 ON (…)]
[ WHERE condition ]
[ GROUP BY expression [, ...] ]
[ HAVING condition]
[ UNION [ ALL | DISTINCT ] select ]
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
[ LIMIT [ count | ALL ] ]
In addition:
• Windowing functions
• UNNEST, TABLESAMPLE
• ROLLUP, CUBE, GROUPING SETS
• EXISTS, EXCEPT, INTERSECT
• Subqueries
ANSI SQL Support

15. 15
BI Tools Certifications

16. 16
• On premises deployments:
– Dedicated clusters (bare metal & virtual)
– Co-located with HDFS
• Cloud deployments:
– AWS / EMR
– Teradata IntelliCloud
– Microsoft Azure
Deployment Types

17. 17
Teradata Planned Contributions to Presto
Expand Optimize Manage
• LDAP
• Secure communication
• New Connectors
• Performance Tweaks
• Cost Based Optimizer
Spill to Disk
• New Connectors
• Performance Tweaks
• Workload Management
• Optimizer Improvements
• New Connectors
• Performance Tweaks
Commercial Support & Services
Phase 4 Phase 5 Phase 6
Q4 2016 Q2 2017 Q4 2017
Expanding ANSI SQL Coverage

18. 18
Certified Distro: www.teradata.com/presto
Website: www.prestodb.io
Presto Users Group: www.groups.google.com/group/presto-users
GitHub:
www.github.com/prestodb/presto
www.github.com/Teradata/presto
More information

19. 19
www.teradata.com/presto