In this presentation, we discuss various load balancing algorithms, their benefits, the places where they will screw you over and which one you should be choosing for your use case.

1. Load Balancing
Algorithms
Aditya Kamat

2. Load Balancing?

6. Load Balancing
Algorithms => {}

7. 1. Random

9. Random algorithm:
• Can you “truly randomly” distribute load?
• It is simple to implement and scales well when you have a
distributed load balancer setup.
• Latency is not being accounted for.
• Servers which are underperforming can have more requests
being sent to them.

10. 2. Round Robin

11. 1 1
1

12. 2 2
2

13. 3 3
3

14. Round Robin
algorithm
• Guarantees same number of requests
to all servers.
• We again face the same latency
problem.
• Does not scale well for distributed
Load balancers as the logic of where to
start from needs to be statically
hardcoded.
• What if the backend servers have
different configs?

15. 3. Weighted Round Robin

16. Weights
1
2
3
1
1
2 23
3
456
7
4
56
7

17. Weighted
Round Robin
algorithm
• Servers can take load based on their
H/W configs.
• Can introduce another service which
check for latency and update the
weights dynamically.
• The weights are present on the load
balancer and not the servers, so if it is
distributed, all need to be in sync.
• Also called “Join shortest queue”
algorithm.

18. 4. Least connections

19. Served Requests
1
1
2 23
3
456 7
4
5
6
7
1
3

20. Least
Connections
algorithm
• Routes requests considering the
current load on each servers.
• In a distributed setup, the LBs should
be in sync about the active
connections on each server.
• We can have race condition of all LBs
sending to the same server if all
requests come to the LBs at once.

21. 5. Least connections with
randomness

22. Each LB, when receives a request
chooses a set of random servers.
Then, the least connection algorithm
is applied to those servers and the
backend server is chosen for the
request.

23. Least
connections
with
randomness
• We are trying to solve the problem of race
condition in Least connections algorithm by using
this approach.
• This works only in a distributed LB setup.
• Adds one more layer of complexity to choose
random set of servers.
• The number of random servers that need to be
chosen needs to be hardcoded in each LB. This
number cannot be too close to the actual number
of servers ? And cannot be too less as well?

24. • All the techniques that we have looked at so
far assume that the application is stateless
and any of the servers can serve requests.
• Maintaining persistent connections is a
much more difficult task and shall be
discussed in the future videos.

25. • Join idle queue algorithm – Link
• Source IP hash algorithm, based on consistent hashing –
Link1, Link2
• This is used for maintaining persistent connections.
• Global Server Load Balancing – Link
Further reading:

26. References
• Comparison between basic LB algorithms: Link
• Tyler McMullen - Load Balancing is Impossible: Link
• “Balls into Bins” — A Simple and Tight Analysis - Link
• The Power of Two Choices in Randomized Load Balancing - Link