From the Data Work Out event: Performant and scalable Data Science with Dataiku DSS and Snowflake Managing the whole process of setting up a machine learning environment from end-to-end becomes significantly easier when using cloud-based technologies. The ability to provision infrastructure on demand (IaaS) solves the problem of manually requesting virtual machines. It also provides immediate access to compute resources whenever they are needed. But that still leaves the administrative overhead of managing the ML software and the platform to store and manage the data. A fully managed end-to-end machine learning platform like Dataiku Data Science Studio (DSS) that enables data scientists, machine learning experts, and even business users to quickly build, train and host machine learning models at scale, needs to access data from many different sources and can also access data provided by Snowflake. Storing data in Snowflake has three significant advantages: a single source of truth, shorten the data preparation cycle, scale as you go.

1. © 2020 Snowflake Inc. All Rights Reserved

2. © 2020 Snowflake Inc. All Rights Reserved
Introduction to
Snowflake
Dataiku Berlin Meetup
25 February 2020
Harald Erb | Sr. Solutions Engineer

3. © 2020 Snowflake Inc. All Rights Reserved
Quick Intro to Snowflake

4. © 2020 Snowflake Computing Inc. All Rights Reserved
SNOWFLAKE TIMELINE
4
Founded in 2012 by
industry veterans
with over 120
database patents
~$1.5BN in venture
capital funding from
leading investors
~$12.4BN valuation
First customers
2014, general
availability 2015
1.800+ employees
Over 3500+
customers today
Queries processed in
Snowflake per day:
> 300 million
Largest single
table:
> 68 trillion rows
Largest number of
tables single DB:
> 200,000
Single customer
most data:
> 55PB
Single customer
most users:
> 10,000
FUN FACTS
Gartner and
Forrester “Leader”

5. © 2020 Snowflake Inc. All Rights Reserved
On Premises
EDW
1st Gen Cloud
EDW
Data Lake,
Hadoop
Cloud Data
Platform
All Data
All Users
Fast Answers
SQL Database
Value
of Data
Time
You can’t use yesterday’s technology to solve today’s data problems -- and definitely not tomorrow’s.
JOURNEY TO A CLOUD DATA PLATFORM

6. © 2020 Snowflake Inc. All Rights Reserved
A REAL-WORLD PROBLEM
Data Warehouse Appliance
Resource Usage
Heatmap of weekly usage profile showing critical peak usage times,
but also low Avg. CPU usage of 33% per week
Typical 24h usage profile showing the two main workload groups
“competing for Data Warehouse Appliance resources”
And what about Analytics workloads?
How can we support new Data Initatives??
Data Loading, ETL, Aggregation
Reporting, BI
Workload Groups over time

7. © 2020 Snowflake Inc. All Rights Reserved
SNOWFLAKE ARCHITECTURE
11
Scale Out Services
Multi-Cluster Compute
Centralized Storage
Cloud Agnostic Layer

8. © 2020 Snowflake Inc. All Rights Reserved

9. © 2020 Snowflake Inc. All Rights Reserved
Data Science
Data Transformation
Marketing
Analytics / Reporting / BI
XS
S
L
M
Data
Load
Structured &
Semi-Structured
Finance
App
Functional
Architecture
L
Enabling one or multiple
Data Teams/Projects to
drive innovation…

10. © 2020 Snowflake Inc. All Rights Reserved
Data Transformation
Marketing
Analytics / Reporting / BI
XS
S
M
L
Data
Load
Structured &
Semi-Structured
Finance
App
L
L
XL
Data Science
Functional
Architecture
…they are even allowed
to scale-up compute
resources when needed…

11. © 2020 Snowflake Inc. All Rights Reserved
Data Transformation
XS
S
L
Data
Load
Structured &
Semi-Structured
Finance
App
L
L
XL
Data Science
Functional
Architecture
Marketing
Analytics / Reporting / BI
M
M
… without slowing down
other active users of the
Cloud Data Platform.

12. © 2020 Snowflake Inc. All Rights Reserved
Marketing
Analytics / Reporting / BI
M
M
Data Transformation
XS
S
L
Data
Load
Structured &
Semi-Structured
Finance
App
L
Functional
Architecture Secure Sharing &
Collaboration Your Private
Data Exchange
Your
Business
Ecosystem
Public Data
Exchange
Your Employees
M

13. © 2020 Snowflake Inc. All Rights Reserved
Marketing
Analytics / Reporting / BI
M
M
Data Transformation
XS
S
L
Data
Load
Structured &
Semi-Structured
Finance
App
L
Secure Sharing &
Collaboration Your Private
Data Exchange
Your
Business
Ecosystem
Public Data
Exchange
Your Employees
M
Functional
Architecture
Clone
M
Test/Dev

14. © 2020 Snowflake Inc. All Rights Reserved 19
ONE PLATFORM, SHARED DATA, MANY WORKLOADS
Data
Warehouse
Data
Lake
Data
Engineering
Data
Exchange
Data
Applications
Data
Science
Data
Monetization
Operational
Reporting
Ad Hoc
Analysis
Real-time
Analytics
OLTP
Databases
Enterprise
Applications
Third-Party
Web/Log
Data
IoT
Data Sources Data Consumers
ETL,Streaming
Live-Demo!

15. © 2020 Snowflake Inc. All Rights Reserved
Demo

16. © 2019 Snowflake Computing Inc. All Rights Reserved 21
AWS
PrivateLink
AWS Direct
Connect
Frankfurt
CDN
S3
Endpoint
Snowflake
S3 bucket
OCSP
cache
External Stage
(COPY data)
Internal Stage
(GET data, Large Results)
Customer
On-premise
environment
Snowflake Driver/Clients
TYPICAL SNOWFLAKE SETUP & DEMO FLOW
Customer Cloud Snowflake
Customer Account
Data Exploration / Data Science
Python/JupyterLab
4. Use Snowflake Python Connector to
access / prepare Time series data
5. Train TS Model, predict future values
and plot forecast along with historical
data
6. Write the predictions back into new
Snowflake table
Customer
“Data Lake”
Snowflake Web UI
1. Provisioning of a Compute Cluster via
SQL command!
2. Resize Compute Cluster and load data
from external S3 Bucket
3. Analyze data using SQL and prepare a
secure Database View for other users
DEMO FLOW

17. © 2020 Snowflake Inc. All Rights Reserved

18. © 2020 Snowflake Inc. All Rights Reserved

19. © 2019 Snowflake Computing Inc. All Rights Reserved© 2020 Snowflake Inc. All Rights Reserved
Connecting the dots

20. © 2020 Snowflake Computing Inc. All Rights Reserved
SNOWFLAKE REFERENCE ARCHITECTURE FOR ANALYTICS
25

21. © 2020 Snowflake Computing Inc. All Rights Reserved 26
SNOWFLAKE: A SCALABLE + POWERFUL
DATA PROCESSING BACKEND FOR DATAIKU!

22. © 2020 Snowflake Inc. All Rights Reserved
THANK YOU