Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

4

Share

MILLIONS EVENT DELIVERY WITH CLOUD PUB / SUB

How to use Google Cloud Pub / Sub as PaaS message queue to improve you system architecture

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

MILLIONS EVENT DELIVERY WITH CLOUD PUB / SUB

  1. 1. WHO AM I PHAM PHUONG TU Co-Founder of DYNO (Online email marketing SaaS) Data engineer at MOG Corp (previous: VC Corp) Cloud lover, data lover
  2. 2. FACTS Google: 1.5 B monthly active device. Whatsapp: 1 B monthly active users, 1.6 billion daily photo message Facebook: 1.55 B monthly active users Spotify: 100M monthly active users, moving from Kafka to Google Cloud Pub Sub.
  3. 3. WHY MESSAGE QUEUE SO IMPORTANT Traditional database is not the right approach for: Balancing workload Asynchronous workflows Data stream Reliability improvement Refreshing distributed caches
  4. 4. EARLY PLAYER RabbitMQ ActiveMQ ZeroMQ HornetQ Apollo QPID
  5. 5. WHAT IS CLOUD PUB/SUB A PaaS message queue with Scalability Flexibility Reliability
  6. 6. WHAT IS CLOUD PUB/SUB
  7. 7. PUB / SUB CONCEPTS AND FLOW
  8. 8. PUBLISHER AND SUBSCRIBER ENDPOINTS
  9. 9. BENEFITS AND FEATURES Unified messaging Global presence Flexible delivery options Data reliability
  10. 10. BENEFITS AND FEATURES End-to-end reliability Data security and protection Flow control Simplicity
  11. 11. GCLOUD SDK Setup • curl https://sdk.cloud.google.com | bash • exec -l $SHELL • gcloud init • gcloud components: list / install / update Commands: gcloud alpha pubsub • topics: create / list / publish / list-subscription / delete • subscriptions: create / list / ack / modify-ack-deadline / modify-push-config / delete
  12. 12. API CLIENTS
  13. 13. CREATE CLIENT
  14. 14. CREATE TOPIC
  15. 15. LIST TOPIC
  16. 16. PUSH AND PULL
  17. 17. PUBLISH MESSAGE
  18. 18. SAMPLE MESSAGE
  19. 19. MESSAGE ACKNOWLEDGEMENT DEADLINE
  20. 20. RECEIVING PUSH MESSAGE
  21. 21. RECEIVING PULL MESSAGE
  22. 22. RETRY POLICY Push: If the push endpoint returns an error code, messages are retried for up to 7 days. Pull: Messages that are not acknowledged become available again for pulling when the acknowledgement deadline expires, up to 7 days after publication time.
  23. 23. LIST SUBSCRIPTION
  24. 24. SPOTIFY EVENT DELIVERY - OLD SYSTEM
  25. 25. SPOTIFY EVENT DELIVERY – NEW SYSTEM
  26. 26. SPOTIFY PRODUCER LOAD TEST
  27. 27. SPOTIFY CONSUMER STABILITY TEST
  28. 28. MAKE MORE FUN
  29. 29. DEMO AppEngine & Cloud Pub/Sub • Sample: https://goo.gl/ifMPE6 • Gihub:https://github.com/phamptu/gcp-next2016- pubsub-demo
  30. 30. Q&A
  31. 31. RESOURCES • Gcloud SDK https://cloud.google.com/sdk/ • Identity platform https://developers.google.com/identity/ • API client https://developers.google.com/api-client-library/ • API explore https://developers.google.com/apis-explorer/ • Cloud API https://console.cloud.google.com/apis/library?project=<project_id> • Cloud PubSub https://cloud.google.com/pubsub/overview • Blog https://cloudplatform.googleblog.com/ • Youtube https://www.youtube.com/user/googlecloudplatform
  • alessiostefanetti

    Dec. 13, 2017
  • BenediktBenz

    Jun. 18, 2017
  • gavinzhm

    Dec. 22, 2016
  • phongdle5

    May. 4, 2016

How to use Google Cloud Pub / Sub as PaaS message queue to improve you system architecture

Views

Total views

2,984

On Slideshare

0

From embeds

0

Number of embeds

420

Actions

Downloads

3

Shares

0

Comments

0

Likes

4

×