We're all distributed systems devs now: a crash course in distributed programming
•Download as PPTX, PDF•
2 likes•1,112 views
Going forward, every developer who works in server-side development will be expected to understand the fundamental concepts that drive the design of distributed systems. It's a matter of when, not if. In this talk we'll dive into concepts such as the CAP theorem, eventual consistency, microservices, event-driven architectures – and how to apply each of these tools to build effective, resilient, distributed systems.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to We're all distributed systems devs now: a crash course in distributed programming
Similar to We're all distributed systems devs now: a crash course in distributed programming (20)
More from petabridge
More from petabridge (12)
Recently uploaded
Recently uploaded (20)
We're all distributed systems devs now: a crash course in distributed programming
- 1. We're All Distributed Systems Developers Now By Aaron Stannard, Founder & CEO Petabridge
- 3. The obvious way to scale is wrong.
- 7. Brittle
- 8. Scenario 2: Real-time User Interactivity
- 10. Reality
- 12. Decentralization
- 13. Elastic (Join)
- 14. Elastic (Leave)
- 17. Event and Message Driven Programming
- 21. Gossip: How Nodes Discover Each Other
- 23. Stateful Apps Serve Results from Memory
- 25. State Locality
- 26. State Makes Protocols Work
- 27. CAP Theorem
- 29. CAP Trade-offs
- 32. Fault and Resource Isolation with Microservices
- 35. Try to make CPU / Memory- intensive tasks into stateless services
- 36. Stateful services should increase CPU / memory utilization slowly
Editor's Notes
- This is a talk about the future of back-end systems development and the new realities of our work in this field. We will not be diving into lots of gritty implementation details, nor will we be advocating any specific technologies as part of this presentation. Rather, the goal is to help introduce all of you to the concepts, algorithms, and techniques championed by industry leaders in the field of distributed systems development. And we’ll start our look at those concepts with a quick reality check – taking stock of what’s really happening in our industry right now and why the obvious solutions are doomed to fail.
- The demands on our back-end services have never been higher, thanks to the invention of smartphones and other new types of screens we have to support. Our applications are expected to be available, 24 hours a day 7 days a week. Downtime is an unforgivable sin – users demand availability. Could you imagine Google or Amazon being offline for planned maintenance for even a couple of hours? The world would shut down. That expectation of “always on” availability no longer applies solely to giants like them – it applies to everyone. But it gets even more demanding from there – not only do we always have to be available, but now we need to be available across multiple types of devices like SmartPhones, tablets, and even watches now. And on top of that – we’re expected to provide value immediately in the form of notifications, fresh content, and real-time experiences. And NOT only that –but we’re also expected to collect 100x the data we did before and actually put it to some use through machine learning, analytics, marketing automation. And even more! We might even be expected to put an API for other developers to use on top of our own applications! The point of all of this: our industry is at an inflection point. Our software and our software developers must be able to deliver more, because that’s what customers and users demand.
- And the obvious ways of scaling to support these demands are inherently wrong. Let’s take a look.
- So take our standard enterprise CRUD application – stateless web applications on top of a SQL database with some OLAP for data analysis and some private web apps for things like reporting, BI, and internal administration running behind a firewall. This was the state of the art from the 1990s until the mid-2000s, but it’s still an architecture that works well for smaller applications.
- However, one day this application isn’t so small anymore – the volume, variety, and velocity of user requests has grown and grown and grown over time and now the service has a major problem: contention around its relational database. The problem with relational models is that they inherently require synchronization, which means locks in this case, in order to fulfill all of its isolation, consistency, and transactionality guarantees. That works great for a while, but eventually you reach a point where your SQL database becomes a single point of failure and bottleneck for everything else in the system and you start to become less available and lose business as a result.
- Now I’ve totally stolen this slide from Jonathan Ellis’ keynote at the 2015 Cassandra Summit, but I’m taking it because it illustrates a great point. The obvious solution to this problem was to sub-divide your database into multiple physical and logical regions using a technique called sharding. So you’d have lots of physical clones of our previous master +