Scaling the Data Infrastructure @Spotify

•

5 likes•2,684 views

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2k7JYDP. Mārtiņš Kalvāns and Matti Pehrs overview the Data Infrastructure at Spotify, diving into some of the data infrastructure components, such us Event Delivery, Datamon and Styx. Filmed at qconsf.com. Mārtiņš Kalvāns is a Big Data Engineer at Spotify. Since 2013, he has been working in the Spotify Data Infrastructure Tribe. Matti Pehrs is a Software Engineer at Spotify. He works as a back-end developer in the Spotify Data Infrastructure Tribe. He has worked in the IT industry for more than 20 years and with Java almost from its inception.

Technology

Scaling Data Infrastructure
@ Spotify
matti@spotify.com
kalvans@spotify.com

InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
spotify-data-infrastructure

Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

Mārtiņš Kalvāns
kalvans@spotify.com
Matti Pehrs
matti@spotify.com

Agenda
1. Data at Spotify
2. Summer of 2015
3. Challenges & Victory
○ Datamon
○ Styx
○ GABO

Spotify big-data context
● Over 100 million monthly active users
● Over 30 million song
● Over 2 billion playlists
● Active in 60 markets

Data is at the heart of Spotify
In 2007
- Monthly Royalty Report
In 2016
- Monthly Royalty Report
- Weekly Billboard
- Daily reports to partners
- ...
- AB-Testing
- Discover weekly
- Daily Mix
- ...

Our growth in Data
Users
+50 TB/day
+100M Users
Developers
+60 TB/day
+10k M/R jobs

Hadoop
Autonomy & Dependencies
Team A
Team B
Team C

● A strain of incidents
Summer of Incidents

● A strain of incidents
● War-room
Summer of Incidents

● A strain of incidents
● War-room
● Hadoop on it’s knees
Summer of Incidents

● A strain of incidents
● War-room
● Hadoop on it’s knees
● Event Delivery Catch up
Summer of Incidents

● A strain of incidents
● War-room
● Hadoop on it’s knees
● Event Delivery Catch up
● Reprocessing of data
Summer of Incidents

● A strain of incidents
● War-room
● Hadoop on it’s knees
● Event Delivery Catch up
● Reprocessing of data
● Hard to debug data issues
Summer of Incidents

1. Early Warning Datamon - Data monitoring
Challenges and the path to victory...

1. Early Warning Datamon - Data monitoring
2. Debuggability & Control Styx - Scheduling and control
Challenges and the path to victory...

1. Early Warning Datamon - Data monitoring
2. Debuggability & Control Styx - Scheduling and control
3. Automate Capacity GABO - Event Delivery
Challenges and the path to victory...

● Unified view
○ Alignment between teams
● Ownership
○ Clear ownership of data
● SLA
○ Alert on late data
Early Warning - Datamon

● Define terminology
● Provide metadata language
● Implement a Datamon service
Early Warning - Datamon

- Execution control
- Self service for data users
- Execution information
- Expose debug information
- Execution isolation
- Docker for data jobs
Debuggability & Control - Styx
The river Styx

● Execution control
○ Centralized execution API
Debuggability & Control - Styx

Debuggability & Control - Styx
● Execution control
○ Centralized execution API
○ Backfilling and reprocessing

● Execution control
● Execution information
○ Timeline
Debuggability & Control - Styx

Debuggability & Control - Styx
● Execution control
● Execution information
○ Timeline
○ Google Cloud Logging

Debuggability & Control - Styx
● Execution control
● Execution information
● Execution isolation
○ Docker

● Complex and manual config
Automate Capacity - GABO/Event Delivery

● Complex and manual config
● Pubsub & Dataflow streaming
Automate Capacity - GABO/Event Delivery

● Complex and manual config
● Pubsub & Dataflow streaming
● Pubsubs at scale
Automate Capacity - GABO/Event Delivery

● Complex and manual config
● Pubsub & Dataflow streaming
● Pubsubs at scale
● Dataflow streaming
Automate Capacity - GABO/Event Delivery

● Complex and manual config
● Pubsub & Dataflow streaming
● Pubsubs at scale
● Dataflow streaming :-(
● 2 micro services + 1 Map/Reduce job
Automate Capacity - GABO/Event Delivery

● Complex and manual config
● Pubsub & Dataflow streaming
● Pubsubs at scale
● Dataflow streaming :-(
● 2 micro services + 1 Map/Reduce job
● Autoscaling & The Stuffer
Automate Capacity - GABO/Event Delivery

● Handles at least 10x our load
● Darkloading
● Autoscale everything
● Self service
GABO - WIP

● Make sure you have the right
tools to deal with data incidents
○ Make sure you have time to
implement the tools you need
● Remember that your capacity
model can fail at larger scale
○ Keep track of your scale and
Automate, automate, automate...
Summary

Thank you!
kalvans@spotify.com
matti@spotify.com
Want to join the band?
http://spoti.fi/jobs

Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
spotify-data-infrastructure

Viewers also liked

Growing up with agile - how the Spotify 'model' has evolved

Peter Antman

This was an extended version of the talk that I gave at InfoShare 2016 in GDansk. This version of the talk was presented at ao.com and Think Money in Manchester UK in May 2016. This is a remix of several earlier talks and some new content to tie Spotify's autonomy and continuous improvement culture to it's data-driven product development approach to show the complete picture. As usual, I tend to talk to slides instead of putting a lot of the content into the slides themselves, so sorry if these don't have all the info.

How Spotify Builds Products (Organization. Architecture, Autonomy, Accountabi...

Kevin Goldsmith

Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to...

Big Data Spain

Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneurs and startuppers. Annually it takes place at the beginning of October in Lviv at Arena Lviv stadium. In 2016 the conference gathered more than 1800 participants and over 100 speakers from companies like Microsoft, Philips, Twitter, UBER and IBM. More details about the conference at itarena.lviv.ua.

JavaScript @ Spotify (Felipe Ribeiro Technology Stream)

IT Arena

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2l2Rr6L. Doug Daniels discusses the cloud-based platform they have built at DataDog and how it differs from a traditional datacenter-based analytics stack. He walks through the decisions they have made at each layer, covers the pros and cons of these decisions and discusses the tooling they have built. Filmed at qconsf.com. Doug Daniels is a Director of Engineering at Datadog, where he works on high-scale data systems for monitoring, data science, and analytics. Prior to joining Datadog, he was CTO at Mortar Data and an architect and developer at Wireless Generation, where he designed data systems to serve more than 4 million students in 49 states.

Elastic Data Analytics Platform @Datadog

C4Media

Spotify Teknikdagarna

Sebastian Widlund

Evolution of Spotify's ad architecture (Qcon 2016 Shanghai)

Kinshuk Mishra

Playlists at Spotify - Using Cassandra to store version controlled objects

Jimmy Mårdell

Africa DevOps Day 2015

Danielle Jabin

Docker at Spotify - Dockercon14

dotCloud

How the economist with cloud BI and Looker have improved data-driven decision...

Looker

Music Personalization At Spotify

Vidhya Murali

How data drives spotify

Ali Sarrafi

Spotify Business Case

David Gorgan

Empowering Engineering Talent - an update from Spotify

Kevin Goldsmith

How Apache Drives Music Recommendations At Spotify

Josh Baer

Spotify Company presentation

alifost

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2hgpOEw. Preslav Le talks about how Dropbox’s infrastructure evolved over the years and how it looks today as well the challenges and lessons learned and tips addressing massive scale, consistency, architecture, MySQL, Memcache, and more. Filmed at qconsf.com. Preslav Le has been a software engineer at Dropbox for the past 3 years, contributing to various aspects of Dropbox’s infrastructure including traffic, performance and storage. He was part of the core on-call and storage on-call rotations, dealing with high emergency real world issues from bad code pushes to complete datacenter outages.

Scaling Dropbox

C4Media

If we’re pushing ourselves, innovating, and trying new things, we'll inevitably make some mistakes on our path to a brighter future. Knowing this, how can we make sure that when we do make mistakes that they are moving us in the right direction? In this talk, I'll share some strategies for making "better" mistakes along with specific examples of tactics and techniques we use at Spotify to help us learn better and faster.

Making Better Mistakes Tomorrow

Danielle Jabin

This is a session given by Horia Jurcut at Nordic APIs 2016 Platform Summit on October 25th, in Stockholm Sweden. Description: Spotify is one of the world leaders in media subscriptions and is the largest music streaming service with over 30 million paying subscribers. We currently provide 16 different methods of payment in over 60 countries and we continue to expand our offerings. In this session we are going to talk about engineering an internal API to abstract away from the complexities of multiple payment gateways and multiple clients with potentially long release cycles (iOS, Android). I will share some of the benefits of growing the API organically, and some of the downfalls of premature abstraction.

How Spotify Payments Creates APIs to Manage Complexity (Horia Jurcut)

Nordic APIs

Viewers also liked (20)

Growing up with agile - how the Spotify 'model' has evolved

How Spotify Builds Products (Organization. Architecture, Autonomy, Accountabi...

Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to...

JavaScript @ Spotify (Felipe Ribeiro Technology Stream)

Elastic Data Analytics Platform @Datadog

Spotify Teknikdagarna

Evolution of Spotify's ad architecture (Qcon 2016 Shanghai)

Playlists at Spotify - Using Cassandra to store version controlled objects

Africa DevOps Day 2015

Docker at Spotify - Dockercon14

How the economist with cloud BI and Looker have improved data-driven decision...

Music Personalization At Spotify

How data drives spotify

Spotify Business Case

Empowering Engineering Talent - an update from Spotify

How Apache Drives Music Recommendations At Spotify

Spotify Company presentation

Scaling Dropbox

Making Better Mistakes Tomorrow

How Spotify Payments Creates APIs to Manage Complexity (Horia Jurcut)

More from C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/39NIjLV. Akhilesh Gupta does a technical deep-dive into how Linkedin uses the Play/Akka Framework and a scalable distributed system to enable live interactions like likes/comments at massive scale at extremely low costs across multiple data centers. Filmed at qconlondon.com. Akhilesh Gupta is the technical lead for LinkedIn's Real-time delivery infrastructure and LinkedIn Messaging. He has been working on the revamp of LinkedIn’s offerings to instant, real-time experiences. Before this, he was the head of engineering for the Ride Experience program at Uber Technologies in San Francisco.

Streaming a Million Likes/Second: Real-Time Interactions on Live Video

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2x0Fav8. Jose Nino guides the audience through the journey of Mobile APIs at Lyft. He focuses on how the team has reaped the benefits of API generation to experiment with the network transport layer. He also discusses recent developments the team has made with Envoy Mobile and the roadmap ahead. Filmed at qconlondon.com. Jose Nino works as a Software Engineer at Lyft.

Next Generation Client APIs in Envoy Mobile

C4Media

How do we cope with an environment that has been radically disrupted, where people are suddenly thrust into remote work in a chaotic state? What are the emerging good practices and new ideas that are shaping the way in which software development teams work? What can we do to make the workplace a more secure and diverse one while increasing the productivity of our teams? This report aims to assist technical leaders in making mid- to long-term decisions that will have a positive impact on their organisations and teams and help individual contributors find the practices, approaches, tools, techniques, and frameworks that can help them get a better experience at work - irrespective of where they are working from.

Software Teams and Teamwork Trends Report Q1 2020

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2QCmmJ0. Mark Stoodley examines some of the strengths and weaknesses of the different Java compilation technologies, if one was to apply them in isolation. Stoodley discusses how production JVMs are assembling a combination of these tools that work together to provide excellent performance across the large spectrum of applications written in Java and JVM based languages. Filmed at qconsf.com. Mark Stoodley joined IBM Canada to build Java JIT compilers for production use and led the team that delivered AOT compilation in the IBM SDK for Java 6. He spent the last five years leading the effort to open source nearly 4.3 million lines of source code from the IBM J9 Java Virtual Machine to create the two open source projects Eclipse OMR and Eclipse OpenJ9, and now co-leads both projects.

Understand the Trade-offs Using Compilers for Java Applications

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2y2yPiS. Colin McCabe talks about the ongoing effort to replace the use of Zookeeper in Kafka: why they want to do it and how it will work. He discusses the limitations they have found and how Kafka benefits both in terms of stability and scalability by bringing consensus in house. He talks about their progress, what work is remaining, and how contributors can help. Filmed at qconsf.com. Colin McCabe is a Kafka committer at Confluent, working on the scalability and extensibility of Kafka. Previously, he worked on the Hadoop Distributed Filesystem and the Ceph Filesystem.

Kafka Needs No Keeper

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2SXXXiD. Katharina Probst talks about what it means to act like an owner and why teams need ownership to be high-performing. When team members, regardless of whether they have a formal leadership role or not, act like owners, magical things can happen. She shares ideas that we can apply to our own work, and talks about how to recognize when we don’t live up to our own expectations of acting like an owner. Filmed at qconsf.com. Katharina Probst is a Senior Engineering Leader, Kubernetes & SaaS at Google. Before this, she was leading engineering teams at Netflix, being responsible for the Netflix API, which helps bring Netflix streaming to millions of people around the world. Prior to joining Netflix, she was in the cloud computing team at Google, where she saw cloud computing from the provider side.

High Performing Teams Act Like Owners

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2T04Lw4. Sergey Kuksenko talks about the performance benefits inline types bring to Java and how to exploit them. Inline/value types are the key part of experimental project Valhalla, which should bring new abilities to the Java language. Filmed at qconsf.com. Sergey Kuksenko is a Java Performance Engineer at Oracle working on a variety of Java and JVM performance enhancements. He started working as Java Engineer in 1996 and as Java Performance Engineer in 2005. He has had a passion for exploring how Java works on modern hardware.

Does Java Need Inline Types? What Project Valhalla Can Bring to Java

C4Media

Do you need service meshes in your tech stack? This on-line guide aims to answer pertinent questions for software architects and technical leaders, such as: what is a service mesh?, do I need a service mesh?, how do I evaluate the different service mesh offerings? In software architecture, a service mesh is a dedicated infrastructure layer for facilitating service-to-service communications between microservices, often using a sidecar proxy.

Service Meshes- The Ultimate Guide

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2UgQ3BU. Christie Wilson describes what to expect from CI/CD in 2019, and how Tekton is helping bring that to as many tools as possible, such as Jenkins X and Prow. Wilson talks about Tekton itself and performs a live demo that shows how cloud native CI/CD can help debug, surface and fix mistakes faster. Filmed at qconsf.com. Christie Wilson is a software engineer at Google, currently leading the Tekton project. Over the past decade, she has worked in the mobile, financial and video game industries. Prior to working at Google she led a team of software developers to build load testing tools for AAA video game titles, and founded the Vancouver chapter of PyLadies.

Shifting Left with Cloud Native CI/CD

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2S7lDiS. Sasha Rosenbaum shows how a CI/CD pipeline for Machine Learning can greatly improve both productivity and reliability. Filmed at qconsf.com. Sasha Rosenbaum is a Program Manager on the Azure DevOps engineering team, focused on improving the alignment of the product with open source software. She is a co-organizer of the DevOps Days Chicago and the DeliveryConf conferences, and recently published a book on Serverless computing in Azure with .NET.

CI/CD for Machine Learning

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/36epVKg. Todd Montgomery discusses the techniques and lessons learned from implementing Aeron Cluster. His focus is on how Raft can be implemented on Aeron, minimizing the network round trip overhead, and comparing single process to a fully distributed cluster. Filmed at qconsf.com. Todd Montgomery is a networking hacker who has researched, designed, and built numerous protocols, messaging-oriented middleware systems, and real-time data systems, done research for NASA, contributed to the IETF and IEEE, and co-founded two startups. He currently works as an independent consultant and is active in several open source projects.

Fault Tolerance at Speed

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2FWc5Sk. Ben Sigelman talks about "Deep Systems", their common properties and re-introduces the fundamentals of control theory from the 1960s, including the original conceptualizations of Observability & Controllability. He uses examples from Google & other companies to illustrate how deep systems have damaged people's ability to observe software, and what needs to be done in order to regain control. Filmed at qconsf.com. Ben Sigelman is a co-founder and the CEO at LightStep, a co-creator of Dapper (Google’s distributed tracing system), and co-creator of the OpenTracing and OpenTelemetry projects (both part of the CNCF). His work and interests gravitate towards observability, especially where microservices, high transaction volumes, and large engineering organizations are involved.

Architectures That Scale Deep - Regaining Control in Deep Systems

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/39SddUL. Victor Dibia provides a friendly introduction to machine learning, covers concrete steps on how front-end developers can create their own ML models and deploy them as part of web applications. He discusses his experience building Handtrack.js - a library for prototyping real time hand tracking interactions in the browser. Filmed at qconsf.com. Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. Prior to this, he was a Research Staff Member at the IBM TJ Watson Research Center, New York. His research interests are at the intersection of human computer interaction, computational social science, and applied AI.

ML in the Browser: Interactive Experiences with Tensorflow.js

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2s9T3Vl. Colin Eberhardt looks at some of the internals of WebAssembly, explores how it works “under the hood”, and looks at how to create a (simple) compiler that targets this runtime. Filmed at qconsf.com. Colin Eberhardt is the Technology Director at Scott Logic, a UK-based software consultancy where they create complex application for their financial services clients. He is an avid technology enthusiast, spending his evenings contributing to open source projects, writing blog posts and learning as much as he can.

Build Your Own WebAssembly Compiler

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2S9tOgy. Satyajit Thadeshwar provides useful insights on how Netflix implemented a secure, token-agnostic, identity solution that works with services operating at a massive scale. He shares some of the lessons learned from this process, both from architectural diagrams and code. Filmed at qconsf.com. Satyajit Thadeshwar is an engineer on the Product Edge Access Services team at Netflix, where he works on some of the most critical services focusing on user and device authentication. He has more than a decade of experience building fault-tolerant and highly scalable, distributed systems.

User & Device Identity for Microservices @ Netflix Scale

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2Ezs08q. Justin Ryan talks about Netflix’ scalability issues and some of the ways they addressed it. He shares successes they’ve had from unintuitively partitioning computation into multiple services to get better runtime characteristics. He introduces us to useful probabilistic data structures, innovative bi-directional data passing, open-source projects available from Netflix that make this all possible. Filmed at qconsf.com. Justin Ryan is Playback Edge Engineering at Netflix. He works on some of the most critical services at Netflix, specifically focusing on user and device authentication. Years of building developer tools has also given him a healthy set of opinions on developer productivity.

Scaling Patterns for Netflix's Edge

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2Z4ZJjn. Kilian Valkhof discusses the process of making an Electron app feel at home on all three platforms: Windows, MacOS and Linux, making devs aware of the pitfalls and how to avoid them. Filmed at qconsf.com. Kilian Valkhof is a Front-end Developer & User-experience Designer at Firstversionist. He writes about various topics, from design to machine learning, on his personal website, kilianvalkhof.com and is a frequent contributer to open source software. He is part of the Electron governance team that oversees the development of the Electron framework.

Make Your Electron App Feel at Home Everywhere

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/344PnB1. Steve Klabnik goes over the deep details of how async/await works in Rust, covering concepts like coroutines, generators, stack-less vs stack-ful, "pinning", and more. Filmed at qconsf.com. Steve Klabnik is on the core team of Rust, leads the documentation team, and is an author of "The Rust Programming Language." He is a frequent speaker at conferences and is a prolific open source contributor, previously working on projects such as Ruby and Ruby on Rails.

The Talk You've Been Await-ing For

C4Media

Future of Data Engineering

C4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2rm4hFD. Yevgeniy Brikman talks about how to write automated tests for infrastructure code, including the code written for use with tools such as Terraform, Docker, Packer, and Kubernetes. Topics covered include: unit tests, integration tests, end-to-end tests, dependency injection, test parallelism, retries and error handling, static analysis, property testing and CI / CD for infrastructure code. Filmed at qconsf.com. Yevgeniy Brikman is the co-founder of Gruntwork, a company that provides DevOps as a Service. He is the author of two books published by O'Reilly Media: Hello, Startup and Terraform: Up & Running. Previously, he worked as a software engineer at LinkedIn, TripAdvisor, Cisco Systems, and Thomson Financial.

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

C4Media

More from C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video

Next Generation Client APIs in Envoy Mobile

Software Teams and Teamwork Trends Report Q1 2020

Understand the Trade-offs Using Compilers for Java Applications

Kafka Needs No Keeper

High Performing Teams Act Like Owners

Does Java Need Inline Types? What Project Valhalla Can Bring to Java

Service Meshes- The Ultimate Guide

Shifting Left with Cloud Native CI/CD

CI/CD for Machine Learning

Fault Tolerance at Speed

Architectures That Scale Deep - Regaining Control in Deep Systems

ML in the Browser: Interactive Experiences with Tensorflow.js

Build Your Own WebAssembly Compiler

User & Device Identity for Microservices @ Netflix Scale

Scaling Patterns for Netflix's Edge

Make Your Electron App Feel at Home Everywhere

The Talk You've Been Await-ing For

Future of Data Engineering

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

Recently uploaded

DBX First Quarter 2024 Investor Presentation

Dropbox

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

ICT role in 21st century education and its challenges

rafiqahmad00786416

Whatsapp Number Escorts Call girls 8617370543 Available 24x7 Navi Mumbai Call Girls Service Offer Genuine VIP Model Escorts Call Girls in Your Budget. Navi Mumbai Call Girls Service Provide Real Call Girls Number. Make Your Sexual Pleasure Memorable with Our Navi Mumbai Call Girls at Affordable Price. Top VIP Escorts Call Girls, High Profile Independent Escorts Call Girls, Housewife Women Escorts Call Girl, College Girls Escorts Call Girls, Russian Escorts Call girls Service in Your Budget.

Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model

Deepika Singh

FWD Group - Insurer Innovation Award 2024

The Digital Insurer

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

Zilliz

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

Modernizing Securities Finance: The cloud-native prime brokerage platform transforming capital markets. Madhu Subbu, Managing Director, Head of Securities Finance Engineering Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu

apidays

In the thrilling conclusion to 2023, ransomware groups had a banner year, really outdoing themselves in the "make everyone's life miserable" department. LockBit 3.0 took gold in the hacking olympics, followed by the plucky upstarts Clop and ALPHV/BlackCat. Apparently, 48% of organizations were feeling left out and decided to get in on the cyber attack action. Business services won the "most likely to get digitally mugged" award, with education and retail nipping at their heels. Hackers expanded their repertoire beyond boring old encryption to the much more exciting world of extortion. The US, UK and Canada took top honors in the "countries most likely to pay up" category. Bitcoins were the currency of choice for discerning hackers, because who doesn't love untraceable money?

Ransomware_Q4_2023. The report. [EN].pdf

Overkill Security

MINDCTI Revenue Release Quarter One 2024

MIND CTI

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

Recently uploaded (20)

DBX First Quarter 2024 Investor Presentation

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

ICT role in 21st century education and its challenges

Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model

FWD Group - Insurer Innovation Award 2024

Boost Fertility New Invention Ups Success Rates.pdf

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

How to Troubleshoot Apps for the Modern Connected Worker

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Data Cloud, More than a CDP by Matt Robison

Corporate and higher education May webinar.pptx

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu

Ransomware_Q4_2023. The report. [EN].pdf

MINDCTI Revenue Release Quarter One 2024

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

Scaling the Data Infrastructure @Spotify

1. Scaling Data Infrastructure @ Spotify matti@spotify.com kalvans@spotify.com

2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ spotify-data-infrastructure

3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com

4. Mārtiņš Kalvāns kalvans@spotify.com Matti Pehrs matti@spotify.com

5. Agenda 1. Data at Spotify 2. Summer of 2015 3. Challenges & Victory ○ Datamon ○ Styx ○ GABO

6. Spotify big-data context ● Over 100 million monthly active users ● Over 30 million song ● Over 2 billion playlists ● Active in 60 markets

7. Data is at the heart of Spotify In 2007 - Monthly Royalty Report In 2016 - Monthly Royalty Report - Weekly Billboard - Daily reports to partners - ... - AB-Testing - Discover weekly - Daily Mix - ...

8. Our growth in Data Users +50 TB/day +100M Users Developers +60 TB/day +10k M/R jobs

9. Hadoop Autonomy & Dependencies Team A Team B Team C

10. Autonomy & Dependencies

11. Autonomy & Dependencies

12. Autonomy & Dependencies

13. Summer of Incidents

14. ● A strain of incidents Summer of Incidents

15. ● A strain of incidents ● War-room Summer of Incidents

16. ● A strain of incidents ● War-room ● Hadoop on it’s knees Summer of Incidents

17. ● A strain of incidents ● War-room ● Hadoop on it’s knees ● Event Delivery Catch up Summer of Incidents

18. ● A strain of incidents ● War-room ● Hadoop on it’s knees ● Event Delivery Catch up ● Reprocessing of data Summer of Incidents

19. ● A strain of incidents ● War-room ● Hadoop on it’s knees ● Event Delivery Catch up ● Reprocessing of data ● Hard to debug data issues Summer of Incidents

20. Challenges and the path to victory...

21. 1. Early Warning Datamon - Data monitoring Challenges and the path to victory...

22. 1. Early Warning Datamon - Data monitoring 2. Debuggability & Control Styx - Scheduling and control Challenges and the path to victory...

23. 1. Early Warning Datamon - Data monitoring 2. Debuggability & Control Styx - Scheduling and control 3. Automate Capacity GABO - Event Delivery Challenges and the path to victory...

24. 1. Early Warning Datamon - Data monitoring 2. Debuggability & Control Styx - Scheduling and control 3. Automate Capacity GABO - Event Delivery Challenges and the path to victory...

25. Early Warning - Datamon

26. ● Unified view ○ Alignment between teams ● Ownership ○ Clear ownership of data ● SLA ○ Alert on late data Early Warning - Datamon

27. ● Define terminology ● Provide metadata language ● Implement a Datamon service Early Warning - Datamon

28. 1. Early Warning Datamon - Data monitoring 2. Debuggability & Control Styx - Scheduling and control 3. Automate Capacity GABO - Event Delivery Challenges and the path to victory...

29. - Execution control - Self service for data users - Execution information - Expose debug information - Execution isolation - Docker for data jobs Debuggability & Control - Styx The river Styx

30. ● Execution control ○ Centralized execution API Debuggability & Control - Styx

31. Debuggability & Control - Styx ● Execution control ○ Centralized execution API ○ Backfilling and reprocessing

32. ● Execution control ● Execution information ○ Timeline Debuggability & Control - Styx

33. Debuggability & Control - Styx ● Execution control ● Execution information ○ Timeline ○ Google Cloud Logging

34. Debuggability & Control - Styx ● Execution control ● Execution information ● Execution isolation ○ Docker

35. 1. Early Warning Datamon - Data monitoring 2. Debuggability & Control Styx - Scheduling and control 3. Automate Capacity GABO - Event Delivery Challenges and the path to victory...

36. ● Complex and manual config Automate Capacity - GABO/Event Delivery

37. ● Complex and manual config ● Pubsub & Dataflow streaming Automate Capacity - GABO/Event Delivery

38. ● Complex and manual config ● Pubsub & Dataflow streaming ● Pubsubs at scale Automate Capacity - GABO/Event Delivery

39. ● Complex and manual config ● Pubsub & Dataflow streaming ● Pubsubs at scale ● Dataflow streaming Automate Capacity - GABO/Event Delivery

40. ● Complex and manual config ● Pubsub & Dataflow streaming ● Pubsubs at scale ● Dataflow streaming :-( ● 2 micro services + 1 Map/Reduce job Automate Capacity - GABO/Event Delivery

41. ● Complex and manual config ● Pubsub & Dataflow streaming ● Pubsubs at scale ● Dataflow streaming :-( ● 2 micro services + 1 Map/Reduce job ● Autoscaling & The Stuffer Automate Capacity - GABO/Event Delivery

42. ● Handles at least 10x our load ● Darkloading ● Autoscale everything ● Self service GABO - WIP

43. ● Make sure you have the right tools to deal with data incidents ○ Make sure you have time to implement the tools you need ● Remember that your capacity model can fail at larger scale ○ Keep track of your scale and Automate, automate, automate... Summary

44. Thank you! kalvans@spotify.com matti@spotify.com Want to join the band? http://spoti.fi/jobs

45. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ spotify-data-infrastructure

Scaling the Data Infrastructure @Spotify

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from C4Media

More from C4Media (20)

Recently uploaded

Recently uploaded (20)

Scaling the Data Infrastructure @Spotify