SlideShare a Scribd company logo
1 of 45
Download to read offline
Jitsi Meet
Our tale of blood, sweat, tears and love.
Open Tech Will Save Us 2020
"Calm waters" by Abevil is licensed under CC BY-NC-ND 2.0
It’s calm before the storm
meet.jit.si architecture
📍
📍
📍
📍
📍
📍
📍
Getting Ready
"hope" by siette is licensed under CC BY-NC-ND 2.0
The Italian Flood
"Mexico City - Tourist Bus Ride" by CarlosVanVegas is licensed under CC BY 2.0
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
What went wrong?
"Collapse" by Jon Scally is licensed under CC BY-NC-ND 2.0
Panic Mode ON
😱😱😱🙀🙀🙀
Tens of millions of

dropped packets
Pagers going off non-stop
Users getting 504s

left and right
Tons of ⭐ reviews on

mobile stores
Hunders of support emails

and forum posts
ulimit -n
Too small! We used prlimit to raise it in production.
Ephemeral port exhaustion
Add more IP addresses to our load balancers and HAProxys.
Inter-server traffic
Having so many servers was killing us,

we made each JVB connect only to its shard.
This lasted for days
Standup turned into a Situation Room.
Reacting to Feedback
"File:Feedback Terminal With A Touchscreen.jpg" by Littokki is licensed under CC BY-SA 4.0
Turning it up to 11
Turning it up to 11
We’ll always have Paris
"File:Eiffel Tower - Tour Eiffel (34377053374).jpg" by Nan Palmero from San Antonio, TX, USA is licensed under CC BY 2.0
📍
📍
📍
📍
📍
📍
📍
📍
AWS Global Accelerator
Instrumental to balance the load across regions.
The numbers (so far)
"XYZ" by Marina Lewandowska is licensed under CC BY-NC-ND 4.0
60x
More conferences
108x
More participants
180x
More traffic
1000s of servers
Instead of 10s
Going Forward
Optimize our use of resources
Optimize for transit cost by moving JVBs out of AWS
Move away from

HAProxy stick tables
Split-brains are unavoidable with the current scheme.
Address more feedback
Keep it coming!
Thank you
🙏
Jitsi Meet: our tale of blood, sweat, tears and love

More Related Content

More from Saúl Ibarra Corretgé

Jitsi: state-of-the-art video conferencing you can self-host
Jitsi: state-of-the-art video conferencing you can self-hostJitsi: state-of-the-art video conferencing you can self-host
Jitsi: state-of-the-art video conferencing you can self-hostSaúl Ibarra Corretgé
 
WebRTC: El epicentro de la videoconferencia y IoT
WebRTC: El epicentro de la videoconferencia y IoTWebRTC: El epicentro de la videoconferencia y IoT
WebRTC: El epicentro de la videoconferencia y IoTSaúl Ibarra Corretgé
 
libuv: cross platform asynchronous i/o
libuv: cross platform asynchronous i/olibuv: cross platform asynchronous i/o
libuv: cross platform asynchronous i/oSaúl Ibarra Corretgé
 
Videoconferencias: el santo grial de WebRTC
Videoconferencias: el santo grial de WebRTCVideoconferencias: el santo grial de WebRTC
Videoconferencias: el santo grial de WebRTCSaúl Ibarra Corretgé
 
SylkServer: State of the art RTC application server
SylkServer: State of the art RTC application serverSylkServer: State of the art RTC application server
SylkServer: State of the art RTC application serverSaúl Ibarra Corretgé
 
Escalabilidad horizontal desde las trincheras
Escalabilidad horizontal desde las trincherasEscalabilidad horizontal desde las trincheras
Escalabilidad horizontal desde las trincherasSaúl Ibarra Corretgé
 
libuv, NodeJS and everything in between
libuv, NodeJS and everything in betweenlibuv, NodeJS and everything in between
libuv, NodeJS and everything in betweenSaúl Ibarra Corretgé
 
CDRTool: CDR mediation and rating engine for OpenSIPS
CDRTool: CDR mediation and rating engine for OpenSIPSCDRTool: CDR mediation and rating engine for OpenSIPS
CDRTool: CDR mediation and rating engine for OpenSIPSSaúl Ibarra Corretgé
 
WebRTC enabling your OpenSIPS infrastructure
WebRTC enabling your OpenSIPS infrastructureWebRTC enabling your OpenSIPS infrastructure
WebRTC enabling your OpenSIPS infrastructureSaúl Ibarra Corretgé
 
Building an Open Source VoIP Hardware Phone
Building an Open Source VoIP Hardware PhoneBuilding an Open Source VoIP Hardware Phone
Building an Open Source VoIP Hardware PhoneSaúl Ibarra Corretgé
 

More from Saúl Ibarra Corretgé (20)

Jitsi: Estado de la Unión (2018)
Jitsi: Estado de la Unión (2018)Jitsi: Estado de la Unión (2018)
Jitsi: Estado de la Unión (2018)
 
Jitsi: state-of-the-art video conferencing you can self-host
Jitsi: state-of-the-art video conferencing you can self-hostJitsi: state-of-the-art video conferencing you can self-host
Jitsi: state-of-the-art video conferencing you can self-host
 
WebRTC: El epicentro de la videoconferencia y IoT
WebRTC: El epicentro de la videoconferencia y IoTWebRTC: El epicentro de la videoconferencia y IoT
WebRTC: El epicentro de la videoconferencia y IoT
 
Jitsi: Open Source Video Conferencing
Jitsi: Open Source Video ConferencingJitsi: Open Source Video Conferencing
Jitsi: Open Source Video Conferencing
 
Jitsi: State of the Union
Jitsi: State of the UnionJitsi: State of the Union
Jitsi: State of the Union
 
libuv: cross platform asynchronous i/o
libuv: cross platform asynchronous i/olibuv: cross platform asynchronous i/o
libuv: cross platform asynchronous i/o
 
Videoconferencias: el santo grial de WebRTC
Videoconferencias: el santo grial de WebRTCVideoconferencias: el santo grial de WebRTC
Videoconferencias: el santo grial de WebRTC
 
SylkServer: State of the art RTC application server
SylkServer: State of the art RTC application serverSylkServer: State of the art RTC application server
SylkServer: State of the art RTC application server
 
Escalabilidad horizontal desde las trincheras
Escalabilidad horizontal desde las trincherasEscalabilidad horizontal desde las trincheras
Escalabilidad horizontal desde las trincheras
 
A deep dive into libuv
A deep dive into libuvA deep dive into libuv
A deep dive into libuv
 
Planning libuv v2
Planning libuv v2Planning libuv v2
Planning libuv v2
 
libuv, NodeJS and everything in between
libuv, NodeJS and everything in betweenlibuv, NodeJS and everything in between
libuv, NodeJS and everything in between
 
From SIP to WebRTC and vice versa
From SIP to WebRTC and vice versaFrom SIP to WebRTC and vice versa
From SIP to WebRTC and vice versa
 
Extendiendo SIP con WebRTC
Extendiendo SIP con WebRTCExtendiendo SIP con WebRTC
Extendiendo SIP con WebRTC
 
De SIP a WebRTC y vice versa
De SIP a WebRTC y vice versaDe SIP a WebRTC y vice versa
De SIP a WebRTC y vice versa
 
Python, WebRTC and You (v2)
Python, WebRTC and You (v2)Python, WebRTC and You (v2)
Python, WebRTC and You (v2)
 
CDRTool: CDR mediation and rating engine for OpenSIPS
CDRTool: CDR mediation and rating engine for OpenSIPSCDRTool: CDR mediation and rating engine for OpenSIPS
CDRTool: CDR mediation and rating engine for OpenSIPS
 
WebRTC enabling your OpenSIPS infrastructure
WebRTC enabling your OpenSIPS infrastructureWebRTC enabling your OpenSIPS infrastructure
WebRTC enabling your OpenSIPS infrastructure
 
Building an Open Source VoIP Hardware Phone
Building an Open Source VoIP Hardware PhoneBuilding an Open Source VoIP Hardware Phone
Building an Open Source VoIP Hardware Phone
 
Python, WebRTC and You
Python, WebRTC and YouPython, WebRTC and You
Python, WebRTC and You
 

Recently uploaded

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 

Recently uploaded (20)

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 

Jitsi Meet: our tale of blood, sweat, tears and love