SlideShare a Scribd company logo
1 of 41
Download to read offline
PROGRAM EFFICIENCY & COMPLEXITY
ANALYSIS
Lecture 03-04
By: Dr. Zahoor Jan
1
ALGORITHM DEFINITION
A finite set of statements that guarantees an optimal
solution in finite interval of time
2
GOOD ALGORITHMS?
 Run in less time
 Consume less memory
But computational resources (time complexity) is usually
more important
3
MEASURING EFFICIENCY
 The efficiency of an algorithm is a measure of the amount of
resources consumed in solving a problem of size n.
 The resource we are most interested in is time
 We can use the same techniques to analyze the consumption of other
resources, such as memory space.
 It would seem that the most obvious way to measure the
efficiency of an algorithm is to run it and measure how much
processor time is needed
 Is it correct ?
4
FACTORS
 Hardware
 Operating System
 Compiler
 Size of input
 Nature of Input
 Algorithm
Which should be improved?
5
RUNNING TIME OF AN ALGORITHM
 Depends upon
 Input Size
 Nature of Input
 Generally time grows with size of input, so running time of an
algorithm is usually measured as function of input size.
 Running time is measured in terms of number of
steps/primitive operations performed
 Independent from machine, OS
6
FINDING RUNNING TIME OF AN ALGORITHM /
ANALYZING AN ALGORITHM
 Running time is measured by number of steps/primitive
operations performed
 Steps means elementary operation like
 ,+, *,<, =, A[i] etc
 We will measure number of steps taken in term of size of
input
7
SIMPLE EXAMPLE (1)
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N)
{
int s=0;
for (int i=0; i< N; i++)
s = s + A[i];
return s;
}
How should we analyse this?
8
SIMPLE EXAMPLE (2)
9
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N){
int s=0;
for (int i=0; i< N; i++)
s = s + A[i];
return s;
}
1
2 3 4
5
6 7
8
1,2,8: Once
3,4,5,6,7: Once per each iteration
of for loop, N iteration
Total: 5N + 3
The complexity function of the
algorithm is : f(N) = 5N +3
SIMPLE EXAMPLE (3) GROWTH OF 5N+3
Estimated running time for different values of N:
N = 10 => 53 steps
N = 100 => 503 steps
N = 1,000 => 5003 steps
N = 1,000,000 => 5,000,003 steps
As N grows, the number of steps grow in linear proportion to N for
this function “Sum”
10
WHAT DOMINATES IN PREVIOUS EXAMPLE?
What about the +3 and 5 in 5N+3?
 As N gets large, the +3 becomes insignificant
 5 is inaccurate, as different operations require varying amounts of time and
also does not have any significant importance
What is fundamental is that the time is linear in N.
Asymptotic Complexity: As N gets large, concentrate on the
highest order term:
 Drop lower order terms such as +3
 Drop the constant coefficient of the highest order term i.e. N
11
ASYMPTOTIC COMPLEXITY
 The 5N+3 time bound is said to "grow asymptotically"
like N
 This gives us an approximation of the complexity of the
algorithm
 Ignores lots of (machine dependent) details, concentrate
on the bigger picture
12
COMPARING FUNCTIONS: ASYMPTOTIC
NOTATION
 Big Oh Notation: Upper bound
 Omega Notation: Lower bound
 Theta Notation: Tighter bound
13
BIG OH NOTATION [1]
If f(N) and g(N) are two complexity functions, we say
f(N) = O(g(N))
(read "f(N) is order g(N)", or "f(N) is big-O of g(N)")
if there are constants c and N0 such that for N > N0,
f(N) ≤ c * g(N)
for all sufficiently large N.
14
BIG OH NOTATION [2]
15
O(F(N))
16
EXAMPLE (2): COMPARING FUNCTIONS
 Which function is better?
10 n2 Vs n3
0
500
1000
1500
2000
2500
3000
3500
4000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
10 n^2
n^3
17
COMPARING FUNCTIONS
 As inputs get larger, any algorithm of a smaller order will
be more efficient than an algorithm of a larger order
18
Time(steps)
Input (size)
3N = O(N)
0.05 N2 = O(N2)
N = 60
BIG-OH NOTATION
 Even though it is correct to say “7n - 3 is O(n3)”, a better
statement is “7n - 3 is O(n)”, that is, one should make the
approximation as tight as possible
 Simple Rule:
Drop lower order terms and constant factors
7n-3 is O(n)
8n2log n + 5n2 + n is O(n2log n)
19
BIG OMEGA NOTATION
 If we wanted to say “running time is at least…” we use Ω
 Big Omega notation, Ω, is used to express the lower bounds on a
function.
 If f(n) and g(n) are two complexity functions then we can say:
f(n) is Ω(g(n)) if there exist positive numbers c and n0 such that 0<=f(n)>=cΩ (n) for all n>=n0
20
BIG THETA NOTATION
 If we wish to express tight bounds we use the theta notation, Θ
 f(n) = Θ(g(n)) means that f(n) = O(g(n)) and f(n) = Ω(g(n))
21
WHAT DOES THIS ALL MEAN?
 If f(n) = Θ(g(n)) we say that f(n) and g(n) grow at the same
rate, asymptotically
 If f(n) = O(g(n)) and f(n) ≠ Ω(g(n)), then we say that f(n) is
asymptotically slower growing than g(n).
 If f(n) = Ω(g(n)) and f(n) ≠ O(g(n)), then we say that f(n) is
asymptotically faster growing than g(n).
22
WHICH NOTATION DO WE USE?
 To express the efficiency of our algorithms which of the
three notations should we use?
 As computer scientist we generally like to express our
algorithms as big O since we would like to know the
upper bounds of our algorithms.
 Why?
 If we know the worse case then we can aim to improve it
and/or avoid it.
23
PERFORMANCE CLASSIFICATION
f(n) Classification
1 Constant: run time is fixed, and does not depend upon n. Most instructions are executed once, or
only a few times, regardless of the amount of information being processed
log n Logarithmic: when n increases, so does run time, but much slower. Common in programs which
solve large problems by transforming them into smaller problems. Exp : binary Search
n Linear: run time varies directly with n. Typically, a small amount of processing is done on each
element. Exp: Linear Search
n log n When n doubles, run time slightly more than doubles. Common in programs which break a problem
down into smaller sub-problems, solves them independently, then combines solutions. Exp: Merge
n2 Quadratic: when n doubles, runtime increases fourfold. Practical only for small problems; typically
the program processes all pairs of input (e.g. in a double nested loop). Exp: Insertion Search
n3 Cubic: when n doubles, runtime increases eightfold. Exp: Matrix
2n Exponential: when n doubles, run time squares. This is often the result of a natural, “brute force”
solution. Exp: Brute Force.
Note: logn, n, nlogn, n2>> less Input>>Polynomial
n3, 2n>>high input>> non polynomial
24
SIZE DOES MATTER[1]
25
What happens if we double the input size N?
N log2N 5N N log2N N2 2N
8 3 40 24 64 256
16 4 80 64 256 65536
32 5 160 160 1024 ~109
64 6 320 384 4096 ~1019
128 7 640 896 16384 ~1038
256 8 1280 2048 65536 ~1076
COMPLEXITY CLASSES
Time(steps)
26
SIZE DOES MATTER[2]
 Suppose a program has run time O(n!) and the run time for
n = 10 is 1 second
For n = 12, the run time is 2 minutes
For n = 14, the run time is 6 hours
For n = 16, the run time is 2 months
For n = 18, the run time is 50 years
For n = 20, the run time is 200 centuries
27
STANDARD ANALYSIS TECHNIQUES
 Constant time statements
 Analyzing Loops
 Analyzing Nested Loops
 Analyzing Sequence of Statements
 Analyzing Conditional Statements
28
CONSTANT TIME STATEMENTS
 Simplest case: O(1) time statements
 Assignment statements of simple data types
int x = y;
 Arithmetic operations:
x = 5 * y + 4 - z;
 Array referencing:
A[j] = 5;
 Array assignment:
j, A[j] = 5;
 Most conditional tests:
if (x < 12) ...
29
ANALYZING LOOPS[1]
 Any loop has two parts:
 How many iterations are performed?
 How many steps per iteration?
int sum = 0,j;
for (j=0; j < N; j++)
sum = sum +j;
 Loop executes N times (0..N-1)
 4 = O(1) steps per iteration
 Total time is N * O(1) = O(N*1) = O(N) 30
ANALYZING LOOPS[2]
 What about this for loop?
int sum =0, j;
for (j=0; j < 100; j++)
sum = sum +j;
 Loop executes 100 times
 4 = O(1) steps per iteration
 Total time is 100 * O(1) = O(100 * 1) = O(100) = O(1)
31
ANALYZING LOOPS – LINEAR LOOPS
 Example (have a look at this code segment):
 Efficiency is proportional to the number of iterations.
 Efficiency time function is :
f(n) = 1 + (n-1) + c*(n-1) +( n-1)
= (c+2)*(n-1) + 1
= (c+2)n – (c+2) +1
 Asymptotically, efficiency is : O(n)
32
ANALYZING NESTED LOOPS[1]
 Treat just like a single loop and evaluate each level of nesting as
needed:
int j,k;
for (j=0; j<N; j++)
for (k=N; k>0; k--)
sum += k+j;
 Start with outer loop:
 How many iterations? N
 How much time per iteration? Need to evaluate inner loop
 Inner loop uses O(N) time
 Total time is N * O(N) = O(N*N) = O(N2) 33
ANALYZING NESTED LOOPS[2]
 What if the number of iterations of one loop depends on the
counter of the other?
int j,k;
for (j=0; j < N; j++)
for (k=0; k < j; k++)
sum += k+j;
 Analyze inner and outer loop together:
 Number of iterations of the inner loop is:
 0 + 1 + 2 + ... + (N-1) = O(N2)
34
HOW DID WE GET THIS ANSWER?
 When doing Big-O analysis, we sometimes have to compute a
series like: 1 + 2 + 3 + ... + (n-1) + n
 i.e. Sum of first n numbers. What is the complexity of this?
 Gauss figured out that the sum of the first n numbers is always:
35
SEQUENCE OF STATEMENTS
 For a sequence of statements, compute their complexity
functions individually and add them up
 Total cost is O(n2) + O(n) +O(1) = O(n2)
36
CONDITIONAL STATEMENTS
 What about conditional statements such as
if (condition)
statement1;
else
statement2;
 where statement1 runs in O(n) time and statement2 runs in O(n2)
time?
 We use "worst case" complexity: among all inputs of size n, what is
the maximum running time?
 The analysis for the example above is O(n2)
37
DERIVING A RECURRENCE EQUATION
 So far, all algorithms that we have been analyzing have been non
recursive
 Example : Recursive power method
 If N = 1, then running time T(N) is 2
 However if N ≥ 2, then running time T(N) is the cost of each step taken plus time
required to compute power(x,n-1). (i.e. T(N) = 2+T(N-1) for N ≥ 2)
 How do we solve this? One way is to use the iteration method.
38
ITERATION METHOD
 This is sometimes known as “Back Substituting”.
 Involves expanding the recurrence in order to see a pattern.
 Solving formula from previous example using the iteration method
:
 Solution : Expand and apply to itself :
Let T(1) = n0 = 2
T(N) = 2 + T(N-1)
= 2 + 2 + T(N-2)
= 2 + 2 + 2 + T(N-3)
= 2 + 2 + 2 + ……+ 2 + T(1)
= 2N + 2 remember that T(1) = n0 = 2 for N = 1
 So T(N) = 2N+2 is O(N) for last example.
39
SUMMARY
 Algorithms can be classified according to their
complexity => O-Notation
 only relevant for large input sizes
 "Measurements" are machine independent
 worst-, average-, best-case analysis
40
REFERENCES
Introduction to Algorithms by Thomas H. Cormen
Chapter 3 (Growth of Functions)
41

More Related Content

What's hot

Asymptotic notations
Asymptotic notationsAsymptotic notations
Asymptotic notationsNikhil Sharma
 
Fundamentals of the Analysis of Algorithm Efficiency
Fundamentals of the Analysis of Algorithm EfficiencyFundamentals of the Analysis of Algorithm Efficiency
Fundamentals of the Analysis of Algorithm EfficiencySaranya Natarajan
 
Asymptotic Notation and Complexity
Asymptotic Notation and ComplexityAsymptotic Notation and Complexity
Asymptotic Notation and ComplexityRajandeep Gill
 
Algorithm analysis
Algorithm analysisAlgorithm analysis
Algorithm analysissumitbardhan
 
Data Structure: Algorithm and analysis
Data Structure: Algorithm and analysisData Structure: Algorithm and analysis
Data Structure: Algorithm and analysisDr. Rajdeep Chatterjee
 
Dinive conquer algorithm
Dinive conquer algorithmDinive conquer algorithm
Dinive conquer algorithmMohd Arif
 
Lecture 4 asymptotic notations
Lecture 4   asymptotic notationsLecture 4   asymptotic notations
Lecture 4 asymptotic notationsjayavignesh86
 
Algorithms Lecture 3: Analysis of Algorithms II
Algorithms Lecture 3: Analysis of Algorithms IIAlgorithms Lecture 3: Analysis of Algorithms II
Algorithms Lecture 3: Analysis of Algorithms IIMohamed Loey
 
Bruteforce algorithm
Bruteforce algorithmBruteforce algorithm
Bruteforce algorithmRezwan Siam
 
Divide and Conquer - Part 1
Divide and Conquer - Part 1Divide and Conquer - Part 1
Divide and Conquer - Part 1Amrinder Arora
 
Analysis of algorithms
Analysis of algorithmsAnalysis of algorithms
Analysis of algorithmsGanesh Solanke
 
Algorithms Lecture 7: Graph Algorithms
Algorithms Lecture 7: Graph AlgorithmsAlgorithms Lecture 7: Graph Algorithms
Algorithms Lecture 7: Graph AlgorithmsMohamed Loey
 
Data Structures and Algorithm Analysis
Data Structures  and  Algorithm AnalysisData Structures  and  Algorithm Analysis
Data Structures and Algorithm AnalysisMary Margarat
 

What's hot (20)

Asymptotic notations
Asymptotic notationsAsymptotic notations
Asymptotic notations
 
Analysis of algorithm
Analysis of algorithmAnalysis of algorithm
Analysis of algorithm
 
Fundamentals of the Analysis of Algorithm Efficiency
Fundamentals of the Analysis of Algorithm EfficiencyFundamentals of the Analysis of Algorithm Efficiency
Fundamentals of the Analysis of Algorithm Efficiency
 
Asymptotic Notation and Complexity
Asymptotic Notation and ComplexityAsymptotic Notation and Complexity
Asymptotic Notation and Complexity
 
Algorithm analysis
Algorithm analysisAlgorithm analysis
Algorithm analysis
 
Data Structure: Algorithm and analysis
Data Structure: Algorithm and analysisData Structure: Algorithm and analysis
Data Structure: Algorithm and analysis
 
Daa notes 1
Daa notes 1Daa notes 1
Daa notes 1
 
Dinive conquer algorithm
Dinive conquer algorithmDinive conquer algorithm
Dinive conquer algorithm
 
Lecture 4 asymptotic notations
Lecture 4   asymptotic notationsLecture 4   asymptotic notations
Lecture 4 asymptotic notations
 
Algorithms Lecture 3: Analysis of Algorithms II
Algorithms Lecture 3: Analysis of Algorithms IIAlgorithms Lecture 3: Analysis of Algorithms II
Algorithms Lecture 3: Analysis of Algorithms II
 
Time complexity
Time complexityTime complexity
Time complexity
 
Unit 1 chapter 1 Design and Analysis of Algorithms
Unit 1   chapter 1 Design and Analysis of AlgorithmsUnit 1   chapter 1 Design and Analysis of Algorithms
Unit 1 chapter 1 Design and Analysis of Algorithms
 
Time andspacecomplexity
Time andspacecomplexityTime andspacecomplexity
Time andspacecomplexity
 
Bruteforce algorithm
Bruteforce algorithmBruteforce algorithm
Bruteforce algorithm
 
Divide and Conquer - Part 1
Divide and Conquer - Part 1Divide and Conquer - Part 1
Divide and Conquer - Part 1
 
Asymptotic Notation
Asymptotic NotationAsymptotic Notation
Asymptotic Notation
 
Analysis of algorithms
Analysis of algorithmsAnalysis of algorithms
Analysis of algorithms
 
Algorithms Lecture 7: Graph Algorithms
Algorithms Lecture 7: Graph AlgorithmsAlgorithms Lecture 7: Graph Algorithms
Algorithms Lecture 7: Graph Algorithms
 
Data Structures and Algorithm Analysis
Data Structures  and  Algorithm AnalysisData Structures  and  Algorithm Analysis
Data Structures and Algorithm Analysis
 
chapter 1
chapter 1chapter 1
chapter 1
 

Viewers also liked

Algorithm chapter 11
Algorithm chapter 11Algorithm chapter 11
Algorithm chapter 11chidabdu
 
Gate mathematics questions all branch by s k mondal
Gate mathematics questions all branch by s k mondalGate mathematics questions all branch by s k mondal
Gate mathematics questions all branch by s k mondalAashishv
 
Digital Electronics Notes
Digital Electronics Notes Digital Electronics Notes
Digital Electronics Notes Srikrishna Thota
 
Computer Networks Foundation - Study Notes
Computer Networks Foundation - Study NotesComputer Networks Foundation - Study Notes
Computer Networks Foundation - Study NotesMarius FAILLOT DEVARRE
 
Made easy notes of Digital electronics Part-3
Made easy notes of Digital electronics Part-3Made easy notes of Digital electronics Part-3
Made easy notes of Digital electronics Part-3ranjeet kumar singh
 
Made easy notes of Digital electronics Part-2
Made easy notes of Digital electronics Part-2Made easy notes of Digital electronics Part-2
Made easy notes of Digital electronics Part-2ranjeet kumar singh
 
Time and space complexity
Time and space complexityTime and space complexity
Time and space complexityAnkit Katiyar
 
Time complexity (linear search vs binary search)
Time complexity (linear search vs binary search)Time complexity (linear search vs binary search)
Time complexity (linear search vs binary search)Kumar
 
Made easy notes of Digital electronics Part-1
Made easy notes of Digital electronics Part-1Made easy notes of Digital electronics Part-1
Made easy notes of Digital electronics Part-1ranjeet kumar singh
 
Digital notes
Digital notesDigital notes
Digital notesstivengo2
 
Advance data structure & algorithm
Advance data structure & algorithmAdvance data structure & algorithm
Advance data structure & algorithmK Hari Shankar
 
Advanced Computer Architecture Chapter 123 Problems Solution
Advanced Computer Architecture Chapter 123 Problems SolutionAdvanced Computer Architecture Chapter 123 Problems Solution
Advanced Computer Architecture Chapter 123 Problems SolutionJoe Christensen
 
Lecture 3
Lecture 3Lecture 3
Lecture 3Mr SMAK
 
Computer architecture kai hwang
Computer architecture   kai hwangComputer architecture   kai hwang
Computer architecture kai hwangSumedha
 
Algorithm Analysis and Design Class Notes
Algorithm Analysis and Design Class NotesAlgorithm Analysis and Design Class Notes
Algorithm Analysis and Design Class NotesKumar Avinash
 

Viewers also liked (20)

Tsp branch and-bound
Tsp branch and-boundTsp branch and-bound
Tsp branch and-bound
 
Algorithm chapter 11
Algorithm chapter 11Algorithm chapter 11
Algorithm chapter 11
 
Gate mathematics questions all branch by s k mondal
Gate mathematics questions all branch by s k mondalGate mathematics questions all branch by s k mondal
Gate mathematics questions all branch by s k mondal
 
Kai hwang solution
Kai hwang solutionKai hwang solution
Kai hwang solution
 
Gate mathematics
Gate mathematicsGate mathematics
Gate mathematics
 
Digital Electronics Notes
Digital Electronics Notes Digital Electronics Notes
Digital Electronics Notes
 
Computer Networks Foundation - Study Notes
Computer Networks Foundation - Study NotesComputer Networks Foundation - Study Notes
Computer Networks Foundation - Study Notes
 
Made easy notes of Digital electronics Part-3
Made easy notes of Digital electronics Part-3Made easy notes of Digital electronics Part-3
Made easy notes of Digital electronics Part-3
 
Made easy notes of Digital electronics Part-2
Made easy notes of Digital electronics Part-2Made easy notes of Digital electronics Part-2
Made easy notes of Digital electronics Part-2
 
Time and space complexity
Time and space complexityTime and space complexity
Time and space complexity
 
Time complexity (linear search vs binary search)
Time complexity (linear search vs binary search)Time complexity (linear search vs binary search)
Time complexity (linear search vs binary search)
 
Made easy notes of Digital electronics Part-1
Made easy notes of Digital electronics Part-1Made easy notes of Digital electronics Part-1
Made easy notes of Digital electronics Part-1
 
Digital notes
Digital notesDigital notes
Digital notes
 
Advance data structure & algorithm
Advance data structure & algorithmAdvance data structure & algorithm
Advance data structure & algorithm
 
Complexity of Algorithm
Complexity of AlgorithmComplexity of Algorithm
Complexity of Algorithm
 
Advanced Computer Architecture Chapter 123 Problems Solution
Advanced Computer Architecture Chapter 123 Problems SolutionAdvanced Computer Architecture Chapter 123 Problems Solution
Advanced Computer Architecture Chapter 123 Problems Solution
 
Lecture 3
Lecture 3Lecture 3
Lecture 3
 
Computer architecture kai hwang
Computer architecture   kai hwangComputer architecture   kai hwang
Computer architecture kai hwang
 
Algorithm Analysis and Design Class Notes
Algorithm Analysis and Design Class NotesAlgorithm Analysis and Design Class Notes
Algorithm Analysis and Design Class Notes
 
Design & Analysis Of Algorithm
Design & Analysis Of AlgorithmDesign & Analysis Of Algorithm
Design & Analysis Of Algorithm
 

Similar to how to calclute time complexity of algortihm

Lec03 04-time complexity
Lec03 04-time complexityLec03 04-time complexity
Lec03 04-time complexityAbbas Ali
 
Unit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfUnit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfAmayJaiswal4
 
DAA-Unit1.pptx
DAA-Unit1.pptxDAA-Unit1.pptx
DAA-Unit1.pptxNishaS88
 
Daa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmsDaa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmssnehajiyani
 
Basic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusBasic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusNANDINI SHARMA
 
Analysis Of Algorithms I
Analysis Of Algorithms IAnalysis Of Algorithms I
Analysis Of Algorithms ISri Prasanna
 
lecture 1
lecture 1lecture 1
lecture 1sajinsc
 
Introduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationIntroduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationAmrinder Arora
 
01 - DAA - PPT.pptx
01 - DAA - PPT.pptx01 - DAA - PPT.pptx
01 - DAA - PPT.pptxKokilaK25
 
Brief introduction to Algorithm analysis
Brief introduction to Algorithm analysis Brief introduction to Algorithm analysis
Brief introduction to Algorithm analysis Anantha Ramu
 
DS Unit-1.pptx very easy to understand..
DS Unit-1.pptx very easy to understand..DS Unit-1.pptx very easy to understand..
DS Unit-1.pptx very easy to understand..KarthikeyaLanka1
 
CS8451 - Design and Analysis of Algorithms
CS8451 - Design and Analysis of AlgorithmsCS8451 - Design and Analysis of Algorithms
CS8451 - Design and Analysis of AlgorithmsKrishnan MuthuManickam
 
Computational Complexity.pptx
Computational Complexity.pptxComputational Complexity.pptx
Computational Complexity.pptxEnosSalar
 
Data Structure & Algorithms - Mathematical
Data Structure & Algorithms - MathematicalData Structure & Algorithms - Mathematical
Data Structure & Algorithms - Mathematicalbabuk110
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Vai Jayanthi
 
04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptxarslanzaheer14
 
Lecture 2 data structures and algorithms
Lecture 2 data structures and algorithmsLecture 2 data structures and algorithms
Lecture 2 data structures and algorithmsAakash deep Singhal
 

Similar to how to calclute time complexity of algortihm (20)

Lec03 04-time complexity
Lec03 04-time complexityLec03 04-time complexity
Lec03 04-time complexity
 
Unit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfUnit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdf
 
DAA-Unit1.pptx
DAA-Unit1.pptxDAA-Unit1.pptx
DAA-Unit1.pptx
 
Daa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmsDaa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithms
 
Lec1
Lec1Lec1
Lec1
 
analysis.ppt
analysis.pptanalysis.ppt
analysis.ppt
 
Basic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusBasic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV Syllabus
 
Analysis Of Algorithms I
Analysis Of Algorithms IAnalysis Of Algorithms I
Analysis Of Algorithms I
 
lecture 1
lecture 1lecture 1
lecture 1
 
Introduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationIntroduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic Notation
 
01 - DAA - PPT.pptx
01 - DAA - PPT.pptx01 - DAA - PPT.pptx
01 - DAA - PPT.pptx
 
Brief introduction to Algorithm analysis
Brief introduction to Algorithm analysis Brief introduction to Algorithm analysis
Brief introduction to Algorithm analysis
 
3 analysis.gtm
3 analysis.gtm3 analysis.gtm
3 analysis.gtm
 
DS Unit-1.pptx very easy to understand..
DS Unit-1.pptx very easy to understand..DS Unit-1.pptx very easy to understand..
DS Unit-1.pptx very easy to understand..
 
CS8451 - Design and Analysis of Algorithms
CS8451 - Design and Analysis of AlgorithmsCS8451 - Design and Analysis of Algorithms
CS8451 - Design and Analysis of Algorithms
 
Computational Complexity.pptx
Computational Complexity.pptxComputational Complexity.pptx
Computational Complexity.pptx
 
Data Structure & Algorithms - Mathematical
Data Structure & Algorithms - MathematicalData Structure & Algorithms - Mathematical
Data Structure & Algorithms - Mathematical
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1
 
04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx
 
Lecture 2 data structures and algorithms
Lecture 2 data structures and algorithmsLecture 2 data structures and algorithms
Lecture 2 data structures and algorithms
 

More from Sajid Marwat

Trusted computing introduction and technical overview
Trusted computing introduction and technical overviewTrusted computing introduction and technical overview
Trusted computing introduction and technical overviewSajid Marwat
 
Digital Rights Management and Trusted Computing Base
Digital Rights Management and Trusted Computing BaseDigital Rights Management and Trusted Computing Base
Digital Rights Management and Trusted Computing BaseSajid Marwat
 
Automata definitions
Automata definitionsAutomata definitions
Automata definitionsSajid Marwat
 
Thr cellular concept
Thr cellular conceptThr cellular concept
Thr cellular conceptSajid Marwat
 
Computer System Overview,
Computer System Overview, Computer System Overview,
Computer System Overview, Sajid Marwat
 
Lec 17 heap data structure
Lec 17 heap data structureLec 17 heap data structure
Lec 17 heap data structureSajid Marwat
 
4 g LTE, LTE Advance
4 g LTE, LTE Advance 4 g LTE, LTE Advance
4 g LTE, LTE Advance Sajid Marwat
 
Advance algorithm hashing lec II
Advance algorithm hashing lec IIAdvance algorithm hashing lec II
Advance algorithm hashing lec IISajid Marwat
 
Advance algorithm hashing lec I
Advance algorithm hashing lec IAdvance algorithm hashing lec I
Advance algorithm hashing lec ISajid Marwat
 
GSM Network 3G Technologies
GSM Network 3G TechnologiesGSM Network 3G Technologies
GSM Network 3G TechnologiesSajid Marwat
 
WiMAX (IEEE 802.16)
WiMAX (IEEE 802.16)WiMAX (IEEE 802.16)
WiMAX (IEEE 802.16)Sajid Marwat
 
Radio over Fiber Technology for WiMAX Systems
 Radio over Fiber Technology for WiMAX Systems Radio over Fiber Technology for WiMAX Systems
Radio over Fiber Technology for WiMAX Systems Sajid Marwat
 
Knowledge Representation & Reasoning
Knowledge Representation & ReasoningKnowledge Representation & Reasoning
Knowledge Representation & ReasoningSajid Marwat
 
top level view of computer function and interconnection
top level view of computer function and interconnectiontop level view of computer function and interconnection
top level view of computer function and interconnectionSajid Marwat
 

More from Sajid Marwat (15)

Trusted computing introduction and technical overview
Trusted computing introduction and technical overviewTrusted computing introduction and technical overview
Trusted computing introduction and technical overview
 
Digital Rights Management and Trusted Computing Base
Digital Rights Management and Trusted Computing BaseDigital Rights Management and Trusted Computing Base
Digital Rights Management and Trusted Computing Base
 
Manet
ManetManet
Manet
 
Automata definitions
Automata definitionsAutomata definitions
Automata definitions
 
Thr cellular concept
Thr cellular conceptThr cellular concept
Thr cellular concept
 
Computer System Overview,
Computer System Overview, Computer System Overview,
Computer System Overview,
 
Lec 17 heap data structure
Lec 17 heap data structureLec 17 heap data structure
Lec 17 heap data structure
 
4 g LTE, LTE Advance
4 g LTE, LTE Advance 4 g LTE, LTE Advance
4 g LTE, LTE Advance
 
Advance algorithm hashing lec II
Advance algorithm hashing lec IIAdvance algorithm hashing lec II
Advance algorithm hashing lec II
 
Advance algorithm hashing lec I
Advance algorithm hashing lec IAdvance algorithm hashing lec I
Advance algorithm hashing lec I
 
GSM Network 3G Technologies
GSM Network 3G TechnologiesGSM Network 3G Technologies
GSM Network 3G Technologies
 
WiMAX (IEEE 802.16)
WiMAX (IEEE 802.16)WiMAX (IEEE 802.16)
WiMAX (IEEE 802.16)
 
Radio over Fiber Technology for WiMAX Systems
 Radio over Fiber Technology for WiMAX Systems Radio over Fiber Technology for WiMAX Systems
Radio over Fiber Technology for WiMAX Systems
 
Knowledge Representation & Reasoning
Knowledge Representation & ReasoningKnowledge Representation & Reasoning
Knowledge Representation & Reasoning
 
top level view of computer function and interconnection
top level view of computer function and interconnectiontop level view of computer function and interconnection
top level view of computer function and interconnection
 

Recently uploaded

In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxKatherine Villaluna
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice documentXsasf Sfdfasd
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxSaurabhParmar42
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesMohammad Hassany
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational PhilosophyShuvankar Madhu
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 

Recently uploaded (20)

In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice document
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptx
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming Classes
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational Philosophy
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 

how to calclute time complexity of algortihm

  • 1. PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS Lecture 03-04 By: Dr. Zahoor Jan 1
  • 2. ALGORITHM DEFINITION A finite set of statements that guarantees an optimal solution in finite interval of time 2
  • 3. GOOD ALGORITHMS?  Run in less time  Consume less memory But computational resources (time complexity) is usually more important 3
  • 4. MEASURING EFFICIENCY  The efficiency of an algorithm is a measure of the amount of resources consumed in solving a problem of size n.  The resource we are most interested in is time  We can use the same techniques to analyze the consumption of other resources, such as memory space.  It would seem that the most obvious way to measure the efficiency of an algorithm is to run it and measure how much processor time is needed  Is it correct ? 4
  • 5. FACTORS  Hardware  Operating System  Compiler  Size of input  Nature of Input  Algorithm Which should be improved? 5
  • 6. RUNNING TIME OF AN ALGORITHM  Depends upon  Input Size  Nature of Input  Generally time grows with size of input, so running time of an algorithm is usually measured as function of input size.  Running time is measured in terms of number of steps/primitive operations performed  Independent from machine, OS 6
  • 7. FINDING RUNNING TIME OF AN ALGORITHM / ANALYZING AN ALGORITHM  Running time is measured by number of steps/primitive operations performed  Steps means elementary operation like  ,+, *,<, =, A[i] etc  We will measure number of steps taken in term of size of input 7
  • 8. SIMPLE EXAMPLE (1) // Input: int A[N], array of N integers // Output: Sum of all numbers in array A int Sum(int A[], int N) { int s=0; for (int i=0; i< N; i++) s = s + A[i]; return s; } How should we analyse this? 8
  • 9. SIMPLE EXAMPLE (2) 9 // Input: int A[N], array of N integers // Output: Sum of all numbers in array A int Sum(int A[], int N){ int s=0; for (int i=0; i< N; i++) s = s + A[i]; return s; } 1 2 3 4 5 6 7 8 1,2,8: Once 3,4,5,6,7: Once per each iteration of for loop, N iteration Total: 5N + 3 The complexity function of the algorithm is : f(N) = 5N +3
  • 10. SIMPLE EXAMPLE (3) GROWTH OF 5N+3 Estimated running time for different values of N: N = 10 => 53 steps N = 100 => 503 steps N = 1,000 => 5003 steps N = 1,000,000 => 5,000,003 steps As N grows, the number of steps grow in linear proportion to N for this function “Sum” 10
  • 11. WHAT DOMINATES IN PREVIOUS EXAMPLE? What about the +3 and 5 in 5N+3?  As N gets large, the +3 becomes insignificant  5 is inaccurate, as different operations require varying amounts of time and also does not have any significant importance What is fundamental is that the time is linear in N. Asymptotic Complexity: As N gets large, concentrate on the highest order term:  Drop lower order terms such as +3  Drop the constant coefficient of the highest order term i.e. N 11
  • 12. ASYMPTOTIC COMPLEXITY  The 5N+3 time bound is said to "grow asymptotically" like N  This gives us an approximation of the complexity of the algorithm  Ignores lots of (machine dependent) details, concentrate on the bigger picture 12
  • 13. COMPARING FUNCTIONS: ASYMPTOTIC NOTATION  Big Oh Notation: Upper bound  Omega Notation: Lower bound  Theta Notation: Tighter bound 13
  • 14. BIG OH NOTATION [1] If f(N) and g(N) are two complexity functions, we say f(N) = O(g(N)) (read "f(N) is order g(N)", or "f(N) is big-O of g(N)") if there are constants c and N0 such that for N > N0, f(N) ≤ c * g(N) for all sufficiently large N. 14
  • 15. BIG OH NOTATION [2] 15
  • 17. EXAMPLE (2): COMPARING FUNCTIONS  Which function is better? 10 n2 Vs n3 0 500 1000 1500 2000 2500 3000 3500 4000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 10 n^2 n^3 17
  • 18. COMPARING FUNCTIONS  As inputs get larger, any algorithm of a smaller order will be more efficient than an algorithm of a larger order 18 Time(steps) Input (size) 3N = O(N) 0.05 N2 = O(N2) N = 60
  • 19. BIG-OH NOTATION  Even though it is correct to say “7n - 3 is O(n3)”, a better statement is “7n - 3 is O(n)”, that is, one should make the approximation as tight as possible  Simple Rule: Drop lower order terms and constant factors 7n-3 is O(n) 8n2log n + 5n2 + n is O(n2log n) 19
  • 20. BIG OMEGA NOTATION  If we wanted to say “running time is at least…” we use Ω  Big Omega notation, Ω, is used to express the lower bounds on a function.  If f(n) and g(n) are two complexity functions then we can say: f(n) is Ω(g(n)) if there exist positive numbers c and n0 such that 0<=f(n)>=cΩ (n) for all n>=n0 20
  • 21. BIG THETA NOTATION  If we wish to express tight bounds we use the theta notation, Θ  f(n) = Θ(g(n)) means that f(n) = O(g(n)) and f(n) = Ω(g(n)) 21
  • 22. WHAT DOES THIS ALL MEAN?  If f(n) = Θ(g(n)) we say that f(n) and g(n) grow at the same rate, asymptotically  If f(n) = O(g(n)) and f(n) ≠ Ω(g(n)), then we say that f(n) is asymptotically slower growing than g(n).  If f(n) = Ω(g(n)) and f(n) ≠ O(g(n)), then we say that f(n) is asymptotically faster growing than g(n). 22
  • 23. WHICH NOTATION DO WE USE?  To express the efficiency of our algorithms which of the three notations should we use?  As computer scientist we generally like to express our algorithms as big O since we would like to know the upper bounds of our algorithms.  Why?  If we know the worse case then we can aim to improve it and/or avoid it. 23
  • 24. PERFORMANCE CLASSIFICATION f(n) Classification 1 Constant: run time is fixed, and does not depend upon n. Most instructions are executed once, or only a few times, regardless of the amount of information being processed log n Logarithmic: when n increases, so does run time, but much slower. Common in programs which solve large problems by transforming them into smaller problems. Exp : binary Search n Linear: run time varies directly with n. Typically, a small amount of processing is done on each element. Exp: Linear Search n log n When n doubles, run time slightly more than doubles. Common in programs which break a problem down into smaller sub-problems, solves them independently, then combines solutions. Exp: Merge n2 Quadratic: when n doubles, runtime increases fourfold. Practical only for small problems; typically the program processes all pairs of input (e.g. in a double nested loop). Exp: Insertion Search n3 Cubic: when n doubles, runtime increases eightfold. Exp: Matrix 2n Exponential: when n doubles, run time squares. This is often the result of a natural, “brute force” solution. Exp: Brute Force. Note: logn, n, nlogn, n2>> less Input>>Polynomial n3, 2n>>high input>> non polynomial 24
  • 25. SIZE DOES MATTER[1] 25 What happens if we double the input size N? N log2N 5N N log2N N2 2N 8 3 40 24 64 256 16 4 80 64 256 65536 32 5 160 160 1024 ~109 64 6 320 384 4096 ~1019 128 7 640 896 16384 ~1038 256 8 1280 2048 65536 ~1076
  • 27. SIZE DOES MATTER[2]  Suppose a program has run time O(n!) and the run time for n = 10 is 1 second For n = 12, the run time is 2 minutes For n = 14, the run time is 6 hours For n = 16, the run time is 2 months For n = 18, the run time is 50 years For n = 20, the run time is 200 centuries 27
  • 28. STANDARD ANALYSIS TECHNIQUES  Constant time statements  Analyzing Loops  Analyzing Nested Loops  Analyzing Sequence of Statements  Analyzing Conditional Statements 28
  • 29. CONSTANT TIME STATEMENTS  Simplest case: O(1) time statements  Assignment statements of simple data types int x = y;  Arithmetic operations: x = 5 * y + 4 - z;  Array referencing: A[j] = 5;  Array assignment: j, A[j] = 5;  Most conditional tests: if (x < 12) ... 29
  • 30. ANALYZING LOOPS[1]  Any loop has two parts:  How many iterations are performed?  How many steps per iteration? int sum = 0,j; for (j=0; j < N; j++) sum = sum +j;  Loop executes N times (0..N-1)  4 = O(1) steps per iteration  Total time is N * O(1) = O(N*1) = O(N) 30
  • 31. ANALYZING LOOPS[2]  What about this for loop? int sum =0, j; for (j=0; j < 100; j++) sum = sum +j;  Loop executes 100 times  4 = O(1) steps per iteration  Total time is 100 * O(1) = O(100 * 1) = O(100) = O(1) 31
  • 32. ANALYZING LOOPS – LINEAR LOOPS  Example (have a look at this code segment):  Efficiency is proportional to the number of iterations.  Efficiency time function is : f(n) = 1 + (n-1) + c*(n-1) +( n-1) = (c+2)*(n-1) + 1 = (c+2)n – (c+2) +1  Asymptotically, efficiency is : O(n) 32
  • 33. ANALYZING NESTED LOOPS[1]  Treat just like a single loop and evaluate each level of nesting as needed: int j,k; for (j=0; j<N; j++) for (k=N; k>0; k--) sum += k+j;  Start with outer loop:  How many iterations? N  How much time per iteration? Need to evaluate inner loop  Inner loop uses O(N) time  Total time is N * O(N) = O(N*N) = O(N2) 33
  • 34. ANALYZING NESTED LOOPS[2]  What if the number of iterations of one loop depends on the counter of the other? int j,k; for (j=0; j < N; j++) for (k=0; k < j; k++) sum += k+j;  Analyze inner and outer loop together:  Number of iterations of the inner loop is:  0 + 1 + 2 + ... + (N-1) = O(N2) 34
  • 35. HOW DID WE GET THIS ANSWER?  When doing Big-O analysis, we sometimes have to compute a series like: 1 + 2 + 3 + ... + (n-1) + n  i.e. Sum of first n numbers. What is the complexity of this?  Gauss figured out that the sum of the first n numbers is always: 35
  • 36. SEQUENCE OF STATEMENTS  For a sequence of statements, compute their complexity functions individually and add them up  Total cost is O(n2) + O(n) +O(1) = O(n2) 36
  • 37. CONDITIONAL STATEMENTS  What about conditional statements such as if (condition) statement1; else statement2;  where statement1 runs in O(n) time and statement2 runs in O(n2) time?  We use "worst case" complexity: among all inputs of size n, what is the maximum running time?  The analysis for the example above is O(n2) 37
  • 38. DERIVING A RECURRENCE EQUATION  So far, all algorithms that we have been analyzing have been non recursive  Example : Recursive power method  If N = 1, then running time T(N) is 2  However if N ≥ 2, then running time T(N) is the cost of each step taken plus time required to compute power(x,n-1). (i.e. T(N) = 2+T(N-1) for N ≥ 2)  How do we solve this? One way is to use the iteration method. 38
  • 39. ITERATION METHOD  This is sometimes known as “Back Substituting”.  Involves expanding the recurrence in order to see a pattern.  Solving formula from previous example using the iteration method :  Solution : Expand and apply to itself : Let T(1) = n0 = 2 T(N) = 2 + T(N-1) = 2 + 2 + T(N-2) = 2 + 2 + 2 + T(N-3) = 2 + 2 + 2 + ……+ 2 + T(1) = 2N + 2 remember that T(1) = n0 = 2 for N = 1  So T(N) = 2N+2 is O(N) for last example. 39
  • 40. SUMMARY  Algorithms can be classified according to their complexity => O-Notation  only relevant for large input sizes  "Measurements" are machine independent  worst-, average-, best-case analysis 40
  • 41. REFERENCES Introduction to Algorithms by Thomas H. Cormen Chapter 3 (Growth of Functions) 41