An introduction to Big O and asymptotic analysis, time complexity, space complexity, memory, arrays, and cache efficiency.

1. SCALABILITY AND MEMORY
CS FUNDAMENTALS SERIES
http://bit.ly/1TPJCe6

2. HOW DO YOU MEASURE
AN ALGORITHM?

3. ???

4. CLOCK TIME?

5. CLOCKTIME?

6. DEPENDS ON
WHO’S COUNTING.

7. ALSO, TOO FLAKY
EVEN ON THE SAME
MACHINE.

8. THE NUMBER OF
LINES?

9. THENUMBEROF
LINES?

10. THIS IS TWO LINES, BUT A WHOLE
LOT OF STUPID.

11. THE NUMBER OF
CPU CYCLES?

12. THENUMBEROF
CPUCYCLES?

14. DEPENDS ON THE
RUNTIME.

15. ALL THESE METHODS SUCK.
NONE OF THEM CAPTURE WHAT WE
ACTUALLY CARE ABOUT.

16. ENTER BIG O!

18. TEXT
ASYMPTOTIC ANALYSIS

19. TEXT
ASYMPTOTIC ANALYSIS
▸ Big O is about asymptotic analysis

20. TEXT
ASYMPTOTIC ANALYSIS
▸ Big O is about asymptotic analysis
▸ In other words, it’s about how an algorithm scales when
the numbers get huge

21. TEXT
ASYMPTOTIC ANALYSIS
▸ Big O is about asymptotic analysis
▸ In other words, it’s about how an algorithm scales when
the numbers get huge
▸ You can also describe this as “the rate of growth”

22. TEXT
ASYMPTOTIC ANALYSIS
▸ Big O is about asymptotic analysis
▸ In other words, it’s about how an algorithm scales when
the numbers get huge
▸ You can also describe this as “the rate of growth”
▸ How fast do the numbers become unmanageable?

23. TEXT
ASYMPTOTIC ANALYSIS

24. TEXT
ASYMPTOTIC ANALYSIS
▸ Another way to think about this is:

25. TEXT
ASYMPTOTIC ANALYSIS
▸ Another way to think about this is:
▸ What happens when your input size is 10,000,000? Will
your program be able to resolve?

26. TEXT
ASYMPTOTIC ANALYSIS
▸ Another way to think about this is:
▸ What happens when your input size is 10,000,000? Will
your program be able to resolve?
▸ It’s about scalability, not necessarily speed

27. TEXT
PRINCIPLES OF BIG O

28. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation

29. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation
▸ In computer science, it means essentially means

30. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation
▸ In computer science, it means essentially means
“the asymptotic rate of growth”

31. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation
▸ In computer science, it means essentially means
“the asymptotic rate of growth”
▸ In other words, how does the running time of this function
scale with the input size when the numbers get big?

32. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation
▸ In computer science, it means essentially means
“the asymptotic rate of growth”
▸ In other words, how does the running time of this function
scale with the input size when the numbers get big?
▸ Big O notation looks like this:

33. TEXT
PRINCIPLES OF BIG O
▸ Big O is a kind of mathematical notation
▸ In computer science, it means essentially means
“the asymptotic rate of growth”
▸ In other words, how does the running time of this function
scale with the input size when the numbers get big?
▸ Big O notation looks like this:
O(n) O(nlog(n)) O(n2
)

34. TEXT
PRINCIPLES OF BIG O

35. TEXT
PRINCIPLES OF BIG O
▸ n here refers to the input size

36. TEXT
PRINCIPLES OF BIG O
▸ n here refers to the input size
▸ Can be the size of an array, the length of a string, the
number of bits in a number, etc.

37. TEXT
PRINCIPLES OF BIG O
▸ n here refers to the input size
▸ Can be the size of an array, the length of a string, the
number of bits in a number, etc.
▸ O(n) means the algorithm scales linearly with the input

38. TEXT
PRINCIPLES OF BIG O
▸ n here refers to the input size
▸ Can be the size of an array, the length of a string, the
number of bits in a number, etc.
▸ O(n) means the algorithm scales linearly with the input
▸ Think like a line (y = x)

39. TEXT
PRINCIPLES OF BIG O

40. TEXT
PRINCIPLES OF BIG O
▸ “Scaling linearly” can mean 1:1 (one iteration per extra
input), but it doesn’t necessarily

41. TEXT
PRINCIPLES OF BIG O
▸ “Scaling linearly” can mean 1:1 (one iteration per extra
input), but it doesn’t necessarily
▸ It can simply mean k:1 where k is constant, like 3:1 or 5:1
(i.e., a constant amount of time per extra input)

42. TEXT
PRINCIPLES OF BIG O

43. TEXT
PRINCIPLES OF BIG O
▸ In Big O, we strip out any coefficients or smaller factors.

44. TEXT
PRINCIPLES OF BIG O
▸ In Big O, we strip out any coefficients or smaller factors.
▸ The fastest-growing factor wins. This is also known as the
dominant factor.

45. TEXT
PRINCIPLES OF BIG O
▸ In Big O, we strip out any coefficients or smaller factors.
▸ The fastest-growing factor wins. This is also known as the
dominant factor.
▸ Just think, when the numbers get huge, what dwarfs
everything else?

46. TEXT
PRINCIPLES OF BIG O
▸ In Big O, we strip out any coefficients or smaller factors.
▸ The fastest-growing factor wins. This is also known as the
dominant factor.
▸ Just think, when the numbers get huge, what dwarfs
everything else?
▸ O(5n) => O(n)

47. TEXT
PRINCIPLES OF BIG O
▸ In Big O, we strip out any coefficients or smaller factors.
▸ The fastest-growing factor wins. This is also known as the
dominant factor.
▸ Just think, when the numbers get huge, what dwarfs
everything else?
▸ O(5n) => O(n)
▸ O(½n - 10) also => O(n)

48. TEXT
PRINCIPLES OF BIG O

49. TEXT
PRINCIPLES OF BIG O
▸ O(k) where k is any constant reduces to O(1).

50. TEXT
PRINCIPLES OF BIG O
▸ O(k) where k is any constant reduces to O(1).
▸ O(200) = O(1)

51. TEXT
PRINCIPLES OF BIG O
▸ O(k) where k is any constant reduces to O(1).
▸ O(200) = O(1)
▸ Where there are multiple factors of growth, the most
dominant one wins.

52. TEXT
PRINCIPLES OF BIG O
▸ O(k) where k is any constant reduces to O(1).
▸ O(200) = O(1)
▸ Where there are multiple factors of growth, the most
dominant one wins.
▸ O(n4 + n2 + 40n) = O(n4)

53. TEXT
PRINCIPLES OF BIG O

54. TEXT
PRINCIPLES OF BIG O
▸ If there are two inputs (say you’re trying to find all the
common substrings of two strings), then you use two
variables in your Big O notation => O(n * m)

55. TEXT
PRINCIPLES OF BIG O
▸ If there are two inputs (say you’re trying to find all the
common substrings of two strings), then you use two
variables in your Big O notation => O(n * m)
▸ Doesn’t matter if one variable probably dwarfs the other.
You always include both.

56. TEXT
PRINCIPLES OF BIG O
▸ If there are two inputs (say you’re trying to find all the
common substrings of two strings), then you use two
variables in your Big O notation => O(n * m)
▸ Doesn’t matter if one variable probably dwarfs the other.
You always include both.
▸ O(n + m) => this is considered linear

57. TEXT
PRINCIPLES OF BIG O
▸ If there are two inputs (say you’re trying to find all the
common substrings of two strings), then you use two
variables in your Big O notation => O(n * m)
▸ Doesn’t matter if one variable probably dwarfs the other.
You always include both.
▸ O(n + m) => this is considered linear
▸ O(2n + log(m)) => this is considered exponential

58. TEXT
COMPREHENSION TEST

59. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!

60. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!
▸ O(3n + 5)

61. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!
▸ O(3n + 5)
▸ O(n + 1/5n2)

62. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!
▸ O(3n + 5)
▸ O(n + 1/5n2)
▸ O(log(n) + 5000)

63. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!
▸ O(3n + 5)
▸ O(n + 1/5n2)
▸ O(log(n) + 5000)
▸ O(2m3
+ 50 + ½n)

64. TEXT
COMPREHENSION TEST
Convert each of these to their appropriate Big O form!
▸ O(3n + 5)
▸ O(n + 1/5n2)
▸ O(log(n) + 5000)
▸ O(2m3
+ 50 + ½n)
▸ O(nlog(m) + 2m2
+ nm)

65. ▸ What should n be for this function?

66. For each character in the string…
Unshift them into an array…
And then join the array together.
Let’s break it down.
Make an empty array.

67. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ Initialize an empty array => O(1)
▸ Then, split the string into an array of characters => O(n)
▸ Then for each character => O(n)
▸ Unshift into an array => O(n)
▸ Then join the characters into a string => O(n)
We’ll see later why this is.
Make an empty array.

68. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ Initialize an empty array => O(1)
▸ Then, split the string into an array of characters => O(n)
▸ Then for each character => O(n)
▸ Unshift into an array => O(n)
▸ Then join the characters into a string => O(n)
Make an empty array.

69. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ Initialize an empty array => O(1)
▸ Then, split the string into an array of characters => O(n)
▸ Then for each character => O(n)
▸ Unshift into an array => O(n)
▸ Then join the characters into a string => O(n)
These multiply. => O(n2
)
Make an empty array.

70. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ Initialize an empty array => O(1)
▸ Then, split the string into an array of characters => O(n)
▸ Then for each character => O(n)
▸ Unshift into an array => O(n)
▸ Then join the characters into a string => O(n)
These multiply. => O(n2
)
Make an empty array.

71. For each character in the string…
Unshift them into an array…
And then join the array together.
Make an empty array.

72. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ O(n2 + 2n) = O(n2)
Make an empty array.

73. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ O(n2 + 2n) = O(n2)
▸ This algorithm is quadratic.
Make an empty array.

74. For each character in the string…
Unshift them into an array…
And then join the array together.
▸ O(n2 + 2n) = O(n2)
▸ This algorithm is quadratic.
▸ Let’s see how badly it sucks.
Make an empty array.

75. Benchmark
away!
(showSlowReverse.js)

76. TEXT
TIME COMPLEXITIES WAY TOO FAST

77. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable

78. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search

79. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration

80. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)

81. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort

82. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort
Cubic O(n3
) triply nested looping, matrix multiplication

83. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort
Cubic O(n3
) triply nested looping, matrix multiplication
Polynomial O(nk
) all “efficient” algorithms

84. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort
Cubic O(n3
) triply nested looping, matrix multiplication
Polynomial O(nk
) all “efficient” algorithms
Exponential O(2n
) subsets, solving chess

85. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort
Cubic O(n3
) triply nested looping, matrix multiplication
Polynomial O(nk
) all “efficient” algorithms
Exponential O(2n
) subsets, solving chess
Factorial O(n!) permutations

86. TEXT
TIME COMPLEXITIES WAY TOO FAST
Constant O(1)
math, pop, push, arr[i], property access,
conditionals, initializing a variable
Logarithmic O(logn) binary search
Linear O(n) linear search, iteration
Linearithmic O(nlogn) sorting (merge sort, quick sort)
Quadratic O(n2
) nested looping, bubble sort
Cubic O(n3
) triply nested looping, matrix multiplication
Polynomial O(nk
) all “efficient” algorithms
Exponential O(2n
) subsets, solving chess
Factorial O(n!) permutations

88. TIME TO IDENTIFY
TIME COMPLEXITIES

89. OPTIMIZATIONS DON’T
ALWAYS MATTER

91. BOTTLENECKS

92. BOTTLENECKS
▸ A bottleneck is the part of your code where your algorithm
spends most of its time.

93. BOTTLENECKS
▸ A bottleneck is the part of your code where your algorithm
spends most of its time.
▸ Asymptotically, it’s wherever the dominant factor is.

94. BOTTLENECKS
▸ A bottleneck is the part of your code where your algorithm
spends most of its time.
▸ Asymptotically, it’s wherever the dominant factor is.
▸ If your algorithm is has an O(n) part and an O(50) part,
the bottleneck is the O(n) part.

95. BOTTLENECKS
▸ A bottleneck is the part of your code where your algorithm
spends most of its time.
▸ Asymptotically, it’s wherever the dominant factor is.
▸ If your algorithm is has an O(n) part and an O(50) part,
the bottleneck is the O(n) part.
▸ As n => ∞, your algorithm will eventually spend 99%+ of
its time in the bottleneck.

96. BOTTLENECKS

97. BOTTLENECKS
▸ When trying to optimize or speed up an algorithm, focus
on the bottleneck.

98. BOTTLENECKS
▸ When trying to optimize or speed up an algorithm, focus
on the bottleneck.
▸ Optimizing code outside the bottleneck will have a
minuscule effect.

99. BOTTLENECKS
▸ When trying to optimize or speed up an algorithm, focus
on the bottleneck.
▸ Optimizing code outside the bottleneck will have a
minuscule effect.
▸ Bottleneck optimizations on the other hand can easily be
huge!

100. BOTTLENECKS

101. BOTTLENECKS
▸ If you cut down non-bottleneck code, you might be able to
save .01% of your runtime.

102. BOTTLENECKS
▸ If you cut down non-bottleneck code, you might be able to
save .01% of your runtime.
▸ If you cut down on bottleneck code, you might be able to
save 30% of your runtime.

103. BOTTLENECKS
▸ If you cut down non-bottleneck code, you might be able to
save .01% of your runtime.
▸ If you cut down on bottleneck code, you might be able to
save 30% of your runtime.
▸ Better yet, try to lower the time complexity altogether if
you can!

104. BOTTLENECK
EXERCISE

105. SPACE
COMPLEXITY

106. SPACE COMPLEXITY

107. SPACE COMPLEXITY
▸ Same thing, except now with memory instead of time.

108. SPACE COMPLEXITY
▸ Same thing, except now with memory instead of time.
▸ Do you take linear extra space relative to the input?

109. SPACE COMPLEXITY
▸ Same thing, except now with memory instead of time.
▸ Do you take linear extra space relative to the input?
▸ Do you allocate new arrays? Do you have to make a copy
of the original input? Are you creating nested data
structures?

110. COMPREHENSION CHECK

111. COMPREHENSION CHECK
▸ What is the space complexity of:

112. COMPREHENSION CHECK
▸ What is the space complexity of:
▸ max(arr)

113. COMPREHENSION CHECK
▸ What is the space complexity of:
▸ max(arr)
▸ firstFive(arr)

114. COMPREHENSION CHECK
▸ What is the space complexity of:
▸ max(arr)
▸ firstFive(arr)
▸ substrings(str)

115. COMPREHENSION CHECK
▸ What is the space complexity of:
▸ max(arr)
▸ firstFive(arr)
▸ substrings(str)
▸ hasVowel(str)

116. SO WHAT THE HELL
IS MEMORY ANYWAY

117. TO UNDERSTAND MEMORY, WE
NEED TO UNDERSTAND HOW A
COMPUTER IS STRUCTURED.

118. Immediate workspace. A CPU usually has 16 of these.
Data Layers
1 cycle

119. Immediate workspace. A CPU usually has 16 of these.
Data Layers
A nearby reservoir of useful data we’ve recently read. Close-by.
1 cycle
~4 cycles

120. Immediate workspace. A CPU usually has 16 of these.
Data Layers
A nearby reservoir of useful data we’ve recently read. Close-by.
More nearby data, but a little farther away.
1 cycle
~4 cycles
~10 cycles

121. Immediate workspace. A CPU usually has 16 of these.
Data Layers
A nearby reservoir of useful data we’ve recently read. Close-by.
More nearby data, but a little farther away.
~800 cycles. Getting pretty far now.
It’s completely random-access, but takes a while.
1 cycle
~4 cycles
~10 cycles

122. Immediate workspace. A CPU usually has 16 of these.
Data Layers
A nearby reservoir of useful data we’ve recently read. Close-by.
More nearby data, but a little farther away.
~800 cycles. Getting pretty far now.
It’s completely random-access, but takes a while.
1 cycle
~4 cycles
~10 cycles
On an SSD, you’re looking at ~5,000 cycles.
This is pretty much another country.
And on a spindle drive, it’s more like 50,000.

123. SO ALL DATA TAKES A JOURNEY
UP FROM THE HARD DISK TO
EVENTUALLY LIVE IN A REGISTER.

124. WHAT DOES MEMORY
ACTUALLY LOOK LIKE?

125. IT’S JUST A BUNCH OF CELLS
WITH SHIT IN ‘EM.

126. IT’S ALL BINARY DATA.
STRINGS, FLOATS, OBJECTS, THEY’RE
ALL STORED AS BINARY.

127. AND IT’S ALL STORED CONTIGUOUSLY.
THIS IS VERY IMPORTANT WHEN IT
COMES TO ARRAYS.

128. ARRAYS ARE JUST
CONTIGUOUS BLOCKS OF
MEMORY.

129. THAT’S WHY
THEY’RE SO FAST.

131. Garbage Also garbage

132. Garbage Also garbage
Assume each of these cells are 8 bytes (64-bits)

133. Garbage Also garbage
Assume each of these cells are 8 bytes (64-bits)
Let’s imagine they’re addressed like so…

134. Assume each of these cells are 8 bytes (64-bits)
Let’s imagine they’re addressed like so…
832968 833032 833096 833160 833224 833288 833352 833416 833480 833544
this.startAddr = 833096;

135. Assume each of these cells are 8 bytes (64-bits)
Let’s imagine they’re addressed like so…
832968 833032 833096 833160 833224 833288 833352 833416 833480 833544
Each cell is offset by exactly 64 in the address space
this.startAddr = 833096;

136. Assume each of these cells are 8 bytes (64-bits)
Let’s imagine they’re addressed like so…
832968 833032 833096 833160 833224 833288 833352 833416 833480 833544
Each cell is offset by exactly 64 in the address space
Meaning you can easily derive the address of any index
this.startAddr = 833096;

137. 832968 833032 833096 833160 833224 833288 833352 833416 833480 833544
function get(i) {
return this.startAddr + i * 64;
}
this.startAddr = 833096;

138. 832968 833032 833096 833160 833224 833288 833352 833416 833480 833544
function get(i) {
return this.startAddr + i * 64;
}
this.startAddr = 833096;
get(3) = 833096 + 3 * 64 = 83306 + 192 = 833288

139. THIS IS POINTER
ARITHMETIC.

140. THIS IS WHAT MAKES
ARRAY LOOKUPS O(1)

141. AND IT’S WHY ARRAYS ARE BY
FAR THE FASTEST DATA
STRUCTURE

142. LET’S WRAP UP BY
TALKING ABOUT CACHE
EFFICIENCY.

143. CACHES ARE
DUMB.

144. When the CPU needs data, it first looks in the cache.

145. When the CPU needs data, it first looks in the cache.
Say it’s not in the cache. This is called a cache miss.

146. When the CPU needs data, it first looks in the cache.
Say it’s not in the cache. This is called a cache miss.
The cache then loads the data the CPU requested from RAM…

147. When the CPU needs data, it first looks in the cache.
Say it’s not in the cache. This is called a cache miss.
The cache then loads the data the CPU requested from RAM…
But the cache guesses that if the CPU wanted this data, it probably will also want
other nearby data eventually. It would be stupid to have to make multiple round trips.

148. In other words, the cache assumes that
related data will be stored around the same
physical area.

149. In other words, the cache assumes that
related data will be stored around the same
physical area.
The cache assumes locality of data.

150. So the cache just loads a huge
contiguous chunk of data around
the address the CPU asked for.

151. OK. SO?

152. Remember this?
Loading from memory is slow as shit.

153. Remember this?
Loading from memory is slow as shit.
We really want to minimize cache misses.

154. SO KEEP YOUR DATA LOCAL AND
YOUR DATA STRUCTURES
CONTIGUOUS.

155. ARRAYS ARE KING, BECAUSE ALL OF
THE DATA IS LITERALLY RIGHT NEXT
TO EACH OTHER IN MEMORY!

156. An algorithm that jumps around in memory
or follows a bunch of pointers to other objects
will trigger lots of cache misses!

157. An algorithm that jumps around in memory
or follows a bunch of pointers to other objects
will trigger lots of cache misses!
Think linked lists, trees, even hash maps.

158. IDEALLY, YOU WANT TO WORK
LOCALLY WITHIN ARRAYS OF
CONTIGUOUS DATA.

159. LET’S DO A QUICK
EXERCISE.

160. QUESTIONS?

161. I AM
HASEEB QURESHI
You can find me on Twitter: @hosseeb
You can read my blog at: haseebq.com

162. PLEASE DONATE IF YOU GOT
SOMETHING OUT OF THIS
<3
Ranked by GiveWell as the most
efficient charity in the world!