Lecture 03: Induction and Sorting
COSC 311 Algorithms, Fall 2022
$ \def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } $
Accountability Group Survey
Office Hours
- Evening TA Sessions
- Sunday, 5–7pm in SCCE A131
- Wednesday, 7–9pm in SCCE A131
- Drop-in Office Hours
- Thursdays 11–12, 2:30–3:30
- King & Wieland Quad, Tent 2 (South of SCCE)
- Office Hours by Appointment, starting next week
- Wednesdays 3–4pm
- Another time too…
Today
- Induction
- Selection Sort, Revisited
- Efficiency
Last Time
Selection Sort!
01 SelectionSort(a):
02 n <- size(a)
03 for j = 1 to n - 1 do
04 min <- j
05 for i = j+1 to n do
06 if compare(a, min, i)
07 min <- i
08 endif
09 endfor
10 swap(a, j, min)
11 endfor
Arguing Correctness
Goal. Logically deduce that algorithm succeeds on all inputs.
To do:
- specify task: sorting
- specify allowed operations and effects: $\compare$, $\swap$
- specify algorithm: pseudocode
- demonstrate that on all possible inputs, algorithm output satisfies task specification
- use induction!
Induction
A Bit of Formalism
A logical predicate $P$ is a statement that can be true or false
Examples.
- $P = $ “it is raining today”
- $P = $ “$57$ is a prime number”
- $P = $ “the output of
SelectionSort([5,2,7,1])is sorted”
Sequences of Predicates
A logical principle to establish the truth of a sequence of predicates
Example. $a$ an array of numbers of size $n$:
- $P(1)$: $a[1] \leq a[2]$
- $P(2)$: $a[2] \leq a[3]$
- $P(3)$: $a[3] \leq a[4]$
…
- $P(n-1)$: $a[n-1] \leq a[n]$
Question. How does “sortedness” relate to the predicates $P(1), P(2), \ldots, P(n-1)$?
Principle of Induction
Setup. $P(1), P(2), P(3), \ldots$ a sequence of predicates
Hypotheses. Suppose:
- $P(1)$ is true (base case)
- If $P(i)$ is true, then $P(i+1)$ is true (inductive step)
Conclusion. All of $P(1), P(2),\ldots$ are true
- “For all $i$, $P(i)$”
- logical notation: $\forall i, P(i)$
Ice Cream Example
Selection Sort Pseudocode
Proposition. The output of SelectionSort(a) is sorted.
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
10 swap(a, i, min)
11 endfor
12 end
Induction and Selection Sort
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
10 swap(a, i, min)
11 endfor
12 end
Claim 1 (inner loop invariant). After iteration $j$ of the inner loop, min stores the index of the smallest value in $a[i..j]$.
Claim 2 (outer loop invariant). After iteration $i$ of the outer loop,
- $a[1..i]$ is sorted, and
- for every $k > i$, $a[k] \geq a[i]$
Strategy
Claim 1 (inner loop invariant). After iteration $j$ of the inner loop, min stores the index of the smallest value in $a[i..j]$.
Claim 2 (outer loop invariant). After iteration $i$ of the outer loop,
- $a[1..i]$ is sorted, and
- for every $k > i$, $a[k] \geq a[i]$
Argument overview:
- Use induction to argue Claim 1.
- Use Claim 1 and induction to argue Claim 2.
- Correctness of $\SelectionSort$ follows from Claim 2.
- why?
Induction and Claim 1
Claim 1 (inner loop invariant). After iteration $j$ of the inner loop, min stores the index of the smallest value in $a[i..j]$.
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
Use Induction:
- Base Case. Show Claim 1 holds for $j = i + 1$
- Inductive step. Show that if Claim 1 holds for $j$, then it also holds for $j+1$.
Conclusion: Claim 1 holds for all $j$
Proof of Claim 1
Claim 1 (inner loop invariant). After iteration $j$ of the inner loop, min stores the index of the smallest value in $a[i..j]$.
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
Proof of Base Case.
Proof of Inductive Step.
Establishing Claim 2
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
10 swap(a, i, min)
11 endfor
12 end
Claim 2 (outer loop invariant). After iteration $i$ of the outer loop,
- $a[1..i]$ is sorted, and
- for every $k > i$, $a[k] \geq a[i]$
Argue by induction!
Apply Conclusion of Claim 1
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
.. min <- index of smallest value in a[i..n]
10 swap(a, i, min)
11 endfor
12 end
Claim 2 (outer loop invariant). After iteration $i$ of the outer loop,
- $a[1..i]$ is sorted, and
- for every $k > i$, $a[k] \geq a[i]$
Base case.
Apply Conclusion of Claim 1
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
.. min <- index of smallest value in a[i..n]
10 swap(a, i, min)
11 endfor
12 end
Claim 2 (outer loop invariant). After iteration $i$ of the outer loop,
- $a[1..i]$ is sorted, and
- for every $k > i$, $a[k] \geq a[i]$
Inductive Step.
Conclusion
$\SelectionSort$ sorts every possible array!
Swap Efficiency?
How many $\swap$ operations does $\SelectionSort$ use? Could we do better?
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
10 swap(a, i, min)
11 endfor
12 end
Compare Efficiency?
How many $\compare$ operations does $\SelectionSort$ use? Could we do better?
01 SelectionSort(a):
02 n <- size(a)
03 for i = 1 to n - 1 do
04 min <- i
05 for j = i+1 to n do
06 if compare(a, min, j)
07 min <- j
08 endif
09 endfor
10 swap(a, i, min)
11 endfor
12 end
Next Time
Sorting by “Divide and Conquer”
- MergeSort
To read: notes on “Big O” notation (forthcoming)