SlideShare a Scribd company logo
1 of 26
by the #s
witter by the
    ing a t alk entitled “T
Giv
                   @twit  terU a t Cal.
Number    s” with
Lots o f ##s!                      ne
                 Tw itter for iPho
1m inute ago via
                     on
Retweet ed by 1 pers
What’s a Tweet?

It’s a short message that's sent through




                                           140 characters
How many are there?
How many are there?   70M
                      60M




                            Today! *




                             *off   the chart
op_oh
                                 Commons from lo
                    der Creative
       Photo used un




47M served                                               70M tweets
  per day                                                  per day
70M tweets
   per day   =   800 tweets
                 per second
Twitter by the Numbers
How big are they?


1 tweet text   =    140
                    characters
               ≈    200 bytes
800 tweets
per second       ≈       160 KB/sec
                 ≈ 9 MB/min
                 ≈ 12 GB/day
      Just tweet text!
MySQL
         Can’t generate IDs fast enough
Centralized and a single point of failure




                                    snowflake
                                    Highly available and uncoordinated (10kqps)
                                    Compatible with the           ecosystem
                                    http://github.com/twitter/snowflake
ampura
                                    Commons from ch
                       der Creative
          Photo used un




1 TB generated                                               8 TB generated
    per day                                                     per day
8 TB
per day
in total
                                                              ≈   100 MB
                                                                  per sec


  Photo used u
               nder Creative C
                              ommons from
                                            Mac Users G
                                                       uide
                                                              =   80 MB
                                                                  per sec
Where do they go?
              Followed by
  Following




                            Asymmetric Digraph
Tweets multiply
1

                   Digraph                           2
            Need to represent this

                                                         4
    1   2      3     4                    3
1




                             Matrix
2




                             Naïve implementation is not scalable
3
4
150M registered users




     2006     2008      2010
Photo used under Creative Commons from jurvetson




          Distributed graph database

flockdb   High rate of CRUD operations
          Complex set arithmetic queries
          http://github.com/twitter/flockdb
@ladygaga
mother mons†er
6.1 million followers

@BarakObama
44th President of the United States
5.3 million followers

@justinbieber
Justin Bieber
5.1 million followers

@raffi
me!
4.1 thousand followers
How do they get out?


6B API calls
    per day    ≈   70,000 calls
                   per second
REST API
         XML/JSON API over HTTP
Poll-based system / pseudo real-time




                               hosebird
                               Streaming API
                               Long poll HTTP
                               Near real-time delivery of Tweets
752%
in 2008
1358%
 in 2009
Where do we want to be?

           Today - 150M people generate ~1000 TPS

Tomorrow - we want to support half the world and all its devices

                  (5B phones and 6B people)
Real challenges in front of us
                 Real time

       Indexing, search, and analytics

             Relevance systems

              Graph databases

                   Storage

          Scalability and efficiency
Questions?   Follow me at
             twitter.com/raffi

More Related Content

What's hot

Algorand Technical Workshop 2021
Algorand Technical Workshop 2021Algorand Technical Workshop 2021
Algorand Technical Workshop 2021DanielBohnemann
 
Blockchain types architecture
Blockchain types architectureBlockchain types architecture
Blockchain types architectureSKMohamedKasim
 
MobiliteaTime #7 : Blockchain
MobiliteaTime #7 : BlockchainMobiliteaTime #7 : Blockchain
MobiliteaTime #7 : BlockchainUSERADGENTS
 
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...Edureka!
 
UX/ユーザビリティ評価法
UX/ユーザビリティ評価法UX/ユーザビリティ評価法
UX/ユーザビリティ評価法Tarumoto Tetsuya
 
Blockchain
BlockchainBlockchain
BlockchainSai Nath
 
Blockchain Technology ppt project.pptx
Blockchain Technology ppt project.pptxBlockchain Technology ppt project.pptx
Blockchain Technology ppt project.pptxSahilBansal648873
 
Blockchain on Azure and Use Cases
Blockchain on Azure and Use CasesBlockchain on Azure and Use Cases
Blockchain on Azure and Use CasesNuri Cankaya
 
Introduction to Azure Functions
Introduction to Azure FunctionsIntroduction to Azure Functions
Introduction to Azure FunctionsCallon Campbell
 
Blockchain Interoperability using Cosmos Interblockchain Communication
Blockchain Interoperability using Cosmos Interblockchain CommunicationBlockchain Interoperability using Cosmos Interblockchain Communication
Blockchain Interoperability using Cosmos Interblockchain CommunicationChjango Unchained
 
Contratos inteligentes y blockchain
Contratos inteligentes y blockchainContratos inteligentes y blockchain
Contratos inteligentes y blockchainkyaalena
 
Block chain technology
Block chain technologyBlock chain technology
Block chain technologyMd. Syful Azam
 
Data Collaboration Stack
Data Collaboration StackData Collaboration Stack
Data Collaboration StackPierre Brunelle
 
Example Mapping in the Wild
Example Mapping in the WildExample Mapping in the Wild
Example Mapping in the WildCeri Shaw
 
Human Values in Requirements Engineering
Human Values in Requirements EngineeringHuman Values in Requirements Engineering
Human Values in Requirements EngineeringJon Whittle
 
Introduction to Ethereum
Introduction to EthereumIntroduction to Ethereum
Introduction to EthereumTerek Judi
 
ユーザビリティテストをやってみよう
ユーザビリティテストをやってみようユーザビリティテストをやってみよう
ユーザビリティテストをやってみようscarletplover
 

What's hot (20)

Algorand Technical Workshop 2021
Algorand Technical Workshop 2021Algorand Technical Workshop 2021
Algorand Technical Workshop 2021
 
Blockchains and databases a new era in distributed computing
Blockchains and databases a new era in distributed computingBlockchains and databases a new era in distributed computing
Blockchains and databases a new era in distributed computing
 
Blockchain
BlockchainBlockchain
Blockchain
 
Blockchain types architecture
Blockchain types architectureBlockchain types architecture
Blockchain types architecture
 
MobiliteaTime #7 : Blockchain
MobiliteaTime #7 : BlockchainMobiliteaTime #7 : Blockchain
MobiliteaTime #7 : Blockchain
 
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...
Blockchain Technology | Blockchain Explained | Blockchain Tutorial | Blockcha...
 
UX/ユーザビリティ評価法
UX/ユーザビリティ評価法UX/ユーザビリティ評価法
UX/ユーザビリティ評価法
 
Blockchain
BlockchainBlockchain
Blockchain
 
Blockchain Technology ppt project.pptx
Blockchain Technology ppt project.pptxBlockchain Technology ppt project.pptx
Blockchain Technology ppt project.pptx
 
Blockchain on Azure and Use Cases
Blockchain on Azure and Use CasesBlockchain on Azure and Use Cases
Blockchain on Azure and Use Cases
 
Introduction to Azure Functions
Introduction to Azure FunctionsIntroduction to Azure Functions
Introduction to Azure Functions
 
Blockchain: Promise, reality or fiction
Blockchain: Promise, reality or fictionBlockchain: Promise, reality or fiction
Blockchain: Promise, reality or fiction
 
Blockchain Interoperability using Cosmos Interblockchain Communication
Blockchain Interoperability using Cosmos Interblockchain CommunicationBlockchain Interoperability using Cosmos Interblockchain Communication
Blockchain Interoperability using Cosmos Interblockchain Communication
 
Contratos inteligentes y blockchain
Contratos inteligentes y blockchainContratos inteligentes y blockchain
Contratos inteligentes y blockchain
 
Block chain technology
Block chain technologyBlock chain technology
Block chain technology
 
Data Collaboration Stack
Data Collaboration StackData Collaboration Stack
Data Collaboration Stack
 
Example Mapping in the Wild
Example Mapping in the WildExample Mapping in the Wild
Example Mapping in the Wild
 
Human Values in Requirements Engineering
Human Values in Requirements EngineeringHuman Values in Requirements Engineering
Human Values in Requirements Engineering
 
Introduction to Ethereum
Introduction to EthereumIntroduction to Ethereum
Introduction to Ethereum
 
ユーザビリティテストをやってみよう
ユーザビリティテストをやってみようユーザビリティテストをやってみよう
ユーザビリティテストをやってみよう
 

Viewers also liked

What we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpWhat we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpKen Mugrage
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Rich Heimann
 
Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Raffi Krikorian
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Raffi Krikorian
 
Aula 5 - Redes Sociais
Aula 5 - Redes SociaisAula 5 - Redes Sociais
Aula 5 - Redes SociaisRebeca Ferrari
 
Sistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosSistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosAricelio Souza
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling TwitterBlaine
 
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17spark-project
 
Fitbit presentation
Fitbit presentationFitbit presentation
Fitbit presentationjryan39
 
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social Singapore
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureP. Taylor Goetz
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephSage Weil
 
Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)We Are Social Singapore
 

Viewers also liked (20)

The Future of Check ins
The Future of Check insThe Future of Check ins
The Future of Check ins
 
Hacking Conway's Law
Hacking Conway's LawHacking Conway's Law
Hacking Conway's Law
 
Digital in 2017 Global Overview
Digital in 2017 Global OverviewDigital in 2017 Global Overview
Digital in 2017 Global Overview
 
What we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can helpWhat we're learning about burnout and how DevOps can help
What we're learning about burnout and how DevOps can help
 
Scalding @ Coursera
Scalding @ CourseraScalding @ Coursera
Scalding @ Coursera
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
 
Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon Re-architecting on the Fly #OReillySACon
Re-architecting on the Fly #OReillySACon
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)
 
Aula 5 - Redes Sociais
Aula 5 - Redes SociaisAula 5 - Redes Sociais
Aula 5 - Redes Sociais
 
Sistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplosSistemas NoSQL, surgimento, características e exemplos
Sistemas NoSQL, surgimento, características e exemplos
 
Scaling Twitter
Scaling TwitterScaling Twitter
Scaling Twitter
 
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17Deep Dive with Spark Streaming - Tathagata  Das - Spark Meetup 2013-06-17
Deep Dive with Spark Streaming - Tathagata Das - Spark Meetup 2013-06-17
 
Fitbit presentation
Fitbit presentationFitbit presentation
Fitbit presentation
 
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
We Are Social's Guide to Social, Digital and Mobile Around the World (Feb 2013)
 
Digital in 2017: South America
Digital in 2017: South AmericaDigital in 2017: South America
Digital in 2017: South America
 
Presentation social network
Presentation social networkPresentation social network
Presentation social network
 
Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm Architecture
 
BlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for CephBlueStore: a new, faster storage backend for Ceph
BlueStore: a new, faster storage backend for Ceph
 
Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)Social, Digital & Mobile Around The World (January 2014)
Social, Digital & Mobile Around The World (January 2014)
 
Big data and Hadoop
Big data and HadoopBig data and Hadoop
Big data and Hadoop
 

Similar to Twitter by the Numbers

Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Raffi Krikorian
 
Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Raffi Krikorian
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Raffi Krikorian
 
Maintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldMaintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldJeremy Edberg
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successStartupfest
 
The Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet ScaleThe Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet Scalegoodfriday
 
Why all payments innovations are rubbish
Why all payments innovations are rubbishWhy all payments innovations are rubbish
Why all payments innovations are rubbishChris Skinner
 
The History and Possible Futures of the Internet
The History and Possible Futures of the InternetThe History and Possible Futures of the Internet
The History and Possible Futures of the InternetLarry Smarr
 
Big Data vs Data Warehousing
Big Data vs Data WarehousingBig Data vs Data Warehousing
Big Data vs Data WarehousingThomas Kejser
 
2012: The End of the World?
2012: The End of the World?2012: The End of the World?
2012: The End of the World?Saumil Shah
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb, Inc.
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Masa Nakatsu
 
The Creativity Machine
The Creativity MachineThe Creativity Machine
The Creativity MachineSal
 
Maq Software Live On Cutting Edge Dream Spark Yatra
Maq Software  Live On Cutting Edge  Dream Spark YatraMaq Software  Live On Cutting Edge  Dream Spark Yatra
Maq Software Live On Cutting Edge Dream Spark YatraMAQ Software
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterouscapotej
 

Similar to Twitter by the Numbers (20)

Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)
 
Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
 
500Startups @ Twitter
500Startups @ Twitter500Startups @ Twitter
500Startups @ Twitter
 
Maintaining reliability in an unreliable world
Maintaining reliability in an unreliable worldMaintaining reliability in an unreliable world
Maintaining reliability in an unreliable world
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator success
 
The Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet ScaleThe Megasite: Infrastructure for Internet Scale
The Megasite: Infrastructure for Internet Scale
 
Why all payments innovations are rubbish
Why all payments innovations are rubbishWhy all payments innovations are rubbish
Why all payments innovations are rubbish
 
Colorado leadership v4
Colorado leadership v4Colorado leadership v4
Colorado leadership v4
 
The History and Possible Futures of the Internet
The History and Possible Futures of the InternetThe History and Possible Futures of the Internet
The History and Possible Futures of the Internet
 
Big Data vs Data Warehousing
Big Data vs Data WarehousingBig Data vs Data Warehousing
Big Data vs Data Warehousing
 
Ngn abridged oct2010
Ngn abridged oct2010Ngn abridged oct2010
Ngn abridged oct2010
 
2012: The End of the World?
2012: The End of the World?2012: The End of the World?
2012: The End of the World?
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?
 
Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?Orb - How Blockchain Industry is evolving now and future?
Orb - How Blockchain Industry is evolving now and future?
 
The Creativity Machine
The Creativity MachineThe Creativity Machine
The Creativity Machine
 
IPv6 Matrix Project
IPv6 Matrix ProjectIPv6 Matrix Project
IPv6 Matrix Project
 
IPv6 Matrix Project - general presentation
IPv6 Matrix Project - general presentationIPv6 Matrix Project - general presentation
IPv6 Matrix Project - general presentation
 
Maq Software Live On Cutting Edge Dream Spark Yatra
Maq Software  Live On Cutting Edge  Dream Spark YatraMaq Software  Live On Cutting Edge  Dream Spark Yatra
Maq Software Live On Cutting Edge Dream Spark Yatra
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterous
 

More from Raffi Krikorian

Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Raffi Krikorian
 
Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Raffi Krikorian
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Raffi Krikorian
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time WebRaffi Krikorian
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4healthRaffi Krikorian
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Raffi Krikorian
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010Raffi Krikorian
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapiRaffi Krikorian
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ ChirpRaffi Krikorian
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time GeostreamsRaffi Krikorian
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"Raffi Krikorian
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Raffi Krikorian
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorRaffi Krikorian
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingRaffi Krikorian
 

More from Raffi Krikorian (20)

Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)Twitter: Engineering for Real-Time (Stanford ACM 2011)
Twitter: Engineering for Real-Time (Stanford ACM 2011)
 
Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
 
#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)
 
Users and Geo
Users and GeoUsers and Geo
Users and Geo
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time Web
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4health
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapi
 
Twitter API Annotations
Twitter API AnnotationsTwitter API Annotations
Twitter API Annotations
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp
 
Energy / Tweet
Energy / TweetEnergy / Tweet
Energy / Tweet
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time Geostreams
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"
 
What's happening here?
What's happening here?What's happening here?
What's happening here?
 
WattzOn @ ETech 2009
WattzOn @ ETech 2009WattzOn @ ETech 2009
WattzOn @ ETech 2009
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth Simulator
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warming
 

Recently uploaded

IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 

Recently uploaded (20)

IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 

Twitter by the Numbers

  • 2. witter by the ing a t alk entitled “T Giv @twit terU a t Cal. Number s” with Lots o f ##s! ne Tw itter for iPho 1m inute ago via on Retweet ed by 1 pers
  • 3. What’s a Tweet? It’s a short message that's sent through 140 characters
  • 4. How many are there?
  • 5. How many are there? 70M 60M Today! * *off the chart
  • 6. op_oh Commons from lo der Creative Photo used un 47M served 70M tweets per day per day
  • 7. 70M tweets per day = 800 tweets per second
  • 9. How big are they? 1 tweet text = 140 characters ≈ 200 bytes
  • 10. 800 tweets per second ≈ 160 KB/sec ≈ 9 MB/min ≈ 12 GB/day Just tweet text!
  • 11. MySQL Can’t generate IDs fast enough Centralized and a single point of failure snowflake Highly available and uncoordinated (10kqps) Compatible with the ecosystem http://github.com/twitter/snowflake
  • 12. ampura Commons from ch der Creative Photo used un 1 TB generated 8 TB generated per day per day
  • 13. 8 TB per day in total ≈ 100 MB per sec Photo used u nder Creative C ommons from Mac Users G uide = 80 MB per sec
  • 14. Where do they go? Followed by Following Asymmetric Digraph
  • 16. 1 Digraph 2 Need to represent this 4 1 2 3 4 3 1 Matrix 2 Naïve implementation is not scalable 3 4
  • 17. 150M registered users 2006 2008 2010
  • 18. Photo used under Creative Commons from jurvetson Distributed graph database flockdb High rate of CRUD operations Complex set arithmetic queries http://github.com/twitter/flockdb
  • 19. @ladygaga mother mons†er 6.1 million followers @BarakObama 44th President of the United States 5.3 million followers @justinbieber Justin Bieber 5.1 million followers @raffi me! 4.1 thousand followers
  • 20. How do they get out? 6B API calls per day ≈ 70,000 calls per second
  • 21. REST API XML/JSON API over HTTP Poll-based system / pseudo real-time hosebird Streaming API Long poll HTTP Near real-time delivery of Tweets
  • 24. Where do we want to be? Today - 150M people generate ~1000 TPS Tomorrow - we want to support half the world and all its devices (5B phones and 6B people)
  • 25. Real challenges in front of us Real time Indexing, search, and analytics Relevance systems Graph databases Storage Scalability and efficiency
  • 26. Questions? Follow me at twitter.com/raffi