Crystal is a data nerd, self-taught programmer, and avid non-fiction reader. Having joined GO-JEK over two years ago, she has first-hand experience of the many challenges involved with scaling data-driven teams at Indonesia’s first unicorn startup. She currently leads the strategy and vision of the Business Intelligence team’s internal products and data culture across the company. Her team aims to produce actionable insights for all of the different verticals on the GO-JEK platform. This slide was shared at Tech in Asia Product Development Conference 2017 (PDC'17) on 9-10 August 2017. Get more insightful updates from TIA by subscribing techin.asia/updateselalu

1. Building Data Foundations and
Analytics Tools Across the
Product

2. Who am I?
● Started at GO-JEK in July 2015 as the first “data” hire
First day: Creating a Data Dictionary without any reference tables
Yesterday: Discussions for a more advanced experimentation platform,
prototyping Growth ROI formulas, QAing new data marts

3. Agenda
● Infrastructure for Scale
● Data Model Foundations
● Tools for Business Users

4. Infrastructure

5. GO-JEK Data Today
~27%
*This is only business metrics data collected
by BI
GROWING DATA VOLUME PER MONTH
> 5000
METABASE CARDS AND TABLEAU SHEETS
> 450
AVG DAILY BUSINESS USERS ON
INTERNAL DATA TOOLS

6. 4
FULL TIME DATA WAREHOUSE DEVELOPERS
> 30
BI DATA ANALYSTS
100s
OF MICROSERVICES ACROSS GO-JEK
GO-JEK Data Today

7. “The choices you made were the right choices given the facts
that you had at the time.”
- Ajey Gore, CTO at GO-JEK

8. Storage

9. Storage

10. crontabs are fun

12. Data Modeling

13. More data to more people

14. Staging Layer
RAW Dataset
Integration Layer
Fact / Dimension dataset
Access Layer
Summary and roll-up data
Datamart Layer
Product-specialized dataset
Current Data Architecture

15. Staging Layer
RAW Dataset
Integration Layer
Fact / Dimension Dataset
Access Layer
Summary and roll-up data
Datamart Layer
Product-specialized dataset
Current Data Architecture
Why?
1. Transparency
2. Standardization

17. “Can I get a list of all full-time drivers?
I want to [give them a reward | put them on a beta
group | interview them | … ]”

18. What qualities make a driver a “full-time driver”?
# of days the driver logs into the app in a week
# of minutes a driver spends on a booking
# of bookings a driver does per day on avg in the past X weeks
# of minutes a driver spends logged into the app per day
# of completed bookings a driver does in a particular service
most common hour the driver logs into the app in the past month
Keep the First Data Layer Factual

19. ● Star Schema
● Advantages
○ Clean and structured model
Merchant Dimension
id nama kategori_merchant
1 Warung Bu Iis TRADISIONAL
Customer Dimension
id nama nomor_telepon
123 Jo 628112345678
Driver Dimension
id nama jenis_kelamin
456 Asep M
457 Doni M
458 Siti F
Order Fact
id id_customer id_driver id_merchant
10001 123 458 1
Item Fact
id id_order nama_item harga
101 10001 Nasi Goreng 30000
102 10001 Es Teh Manis 5000
Driver Search Fact
id id_driver nama status
1 456 Asep Rejected
2 457 Doni Rejected
3 458 Siti Accepted
● Disadvantages
○ Difficult to do data discovery for non-technical users
○ Needs a lot of joins, resulting in high computational resource needs

20. App Login Data Bid Data
Completed Booking
Data
Income Data Driver Profile Data
Factual Activity Data
Daily Partition of Driver
Activity and
Profile Data in Denormalized
& Nested Form
The Data Model

21. avg_minutes_online_past_3_days total_minutes_online_past_3_days
avg_minutes_online_past_7_days total_days_active_past_3_days
avg_minutes_online_past_30_days total_orders_completed_past_7_days
avg_income_past_3_days total_orders_completed_past_30_days
avg_income_past_7_days total_services_completed_past_7_days
total_completed_ride_past_7_days total_completed_send_past_7_days
for each driver_id...
… and +200 other data points

23. Tools
for
Scale

25. Lifecycle of a Data Point
One Week Old
One Month Old
3 Months Old

34. Let Analysts Define Events

35. Sample Events to Save on Costs
Better sample that data point...

39. Take Away
● Build for the infrastructure you have, not what you think you’ll have
● Build simple step-by-step data models with transparency
● Build tools that work for all the different stages of the company