# Lecture 02: Sorting and Induction

### COSC 311 Algorithms, Fall 2022

$\def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} }$

## Announcements

1. Accountability groups (message today)
2. Office hours
• Evening TA sessions Sunday, Wednesday (TBD)
• My drop-in: Thursday 11-12, 2-3 (?)
• By appointment: TBD
3. Emails: subject includes [COSC 311]
4. Section enrollment
5. Lecture ticket reminder (read solutions!)

## Today

2. Insertion Sort
3. Induction

Input:

• Sequence $a$ of $n$ numbers
• e.g., $a = 17, 7, 5, 2, 3, 19, 5, 13$

Output:

• A sorted sequence $s$ of same elements as $a$
• $s$ contains same elements with same multiplicities as $a$
• $s_1 \leq s_2 \leq \cdots \leq s_n$
• e.g., $s = 2, 3, 5, 5, 7, 13, 17, 19$

## So Far

• Why?
1. representation
2. supported operations

Examples:

• stack of exams
• array of numbers

Each may support different operations & require different techniques to solve efficiently

## Going Forward

Spend ~2 weeks on sorting

• Elementary algorithms
• argue correctness
• mathematical induction
• argue running time
• big O notation
• Divide-and-conquer algorithms
• argue running time
• “master method”

## Sorting Arrays

Representation:

• $a$ an array of size $n$
• $a, a,\ldots, a[n]$

Supported Operations

• $\compare(a, i, j)$
• return $\true$ if $a[i] > a[j]$ and $\false$ otherwise
• $\swap(a, i, j)$
• before $a[i] = x$ and $a[j] = y$
• after $a[i] = y$ and $a[j] = x$

## Example

$a = [17, 7, 5, 2, 3, 19, 5, 13]$

• $\compare(a, 2, 6)$?
• $\swap(a, 2, 5)$?

## Lecture Ticket

Express “selection sort” in pseudocode

• find smallest element and put it at index 1
• find second smallest element and put it at index 2
• find third smallest element and put it at index 3

## Example

• Sorting a small array:

\begin{align*} a &= [5, 2, 1, 3, 4]\\ &{}\\ &\to [1, 2, 5, 3, 4]\\ &{}\\ &\to [1, 2, 5, 3, 4]\\ &{}\\ &\to [1, 2, 3, 5, 4]\\ &{}\\ &\to [1, 2, 3, 4, 5] \end{align*}

## SelectionSort in Pseudocode

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor


## Arguing Correctness

Goal. Logically deduce that algorithm succeeds on all inputs.

To do:

• specify allowed operations and effects
• specify algorithm
• demonstrate that on all possible inputs, algorithm output satisfies task specification

## A Remark

It may be “obvious” to you that SelectionSort works.

• give formal analysis of algorithm here
• introduce tools that will help when things become less obvious

Input. Array $a$ of numbers

Output. Sorted array $s$:

1. $s$ contains the same elements as $a$
2. $s$ is sorted: $s \leq s \leq \cdots \leq s[n]$
• for every index $i < n$, $s[i] \leq s[i+1]$

## Allowed Operations

• $\compare(a, i, j)$: return $\true$ if $a[i] > a[j]$
• $\swap(a, i, j)$:
• before $\swap$ have $a[i] = x$ and $a[j] = y$
• after $\swap$ have $a[i] = y$ and $a[j] = x$

Observation. If $s$ is array formed from $a$ by any sequence of $\swap$ operations, then $s$ and $a$ contain the same elements.

• Item (1) from sorting task is satisfied for any procedure that only modifies the array with swaps

## Next Step

Claim. The output of SelectionSort(a) is sorted.

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor
12  end


Question. Why does iteration $j$ select $j$th smallest element in the array?

## Inductive Reasoning

Question. Why does iteration $j$ select $j$th smallest element in the array?

04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)


Reason. (informal)

1. Loop in lines 5-9 selects smallest value in a[j..n]
2. Previous steps moved smaller values to a[1..j-1]

Moral. Step j succeeds because steps 1, 2,...,j-1 succeeded

• inductive reasoning

# Induction

## A Bit of Formalism

A logical predicate $P$ is a statement that can be true or false

Examples.

• $P =$ “it is raining today”
• $P =$ “$57$ is a prime number”
• $P =$ “the output of SelectionSort([5,2,7,1]) is sorted”

## Sequences of Predicates

A logical principle to establish the truth of a sequence of predicates

Example. $a$ an array of numbers of size $n$:

• $P(1)$: $a \leq a$
• $P(2)$: $a \leq a$
• $P(3)$: $a \leq a$

• $P(n-1)$: $a[n-1] \leq a[n]$

Question. How does “sortedness” relate to the predicates $P(1), P(2), \ldots, P(n-1)$?

## Principle of Induction

Setup. $P(1), P(2), P(3), \ldots$ a sequence of predicates

Hypotheses. Suppose:

1. $P(1)$ is true (base case)
2. If $P(i)$ is true, then $P(i+1)$ is true (inductive step)

Conclusion. All of $P(1), P(2),\ldots$ are true

• “For all $i$, $P(i)$”
• logical notation: $\forall i, P(i)$

## Selection Sort Pseudocode

Claim. The output of SelectionSort(a) is sorted.

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor
12  end


## Induction and Selection Sort

Main Claim. After iteration $j$:

1. $a[1..j]$ is sorted, and
2. for every $k > j$, $a[k] \geq a[j]$

Proof by Induction. Must show

1. Base case ($j = 1$)
2. Inductive step (if claim holds for $j$, then claim holds for $j+1$)

## A Smaller Claim

Sub-claim. Consider the inner loop:

04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor


When the loop terminates, min is the index of the minimum value of a[j..n].

Why?

## Main Claim, Base Case

After iteration $j$, $a[1..j]$ is sorted, and for every $k > j$, $a[k] \geq a[j]$.

04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)


## Main Claim, Inductive Step

After iteration $j$, $a[1..j]$ is sorted, and for every $k > j$, $a[k] \geq a[j]$.

04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)


## Induction Continued

Main Claim. After iteration $j$:

1. $a[1..j]$ is sorted, and
2. for every $k > j$, $a[k] \geq a[j]$
• Inductive Step:

## Conclusion

The claim holds for all $j$: after iteration $j$

1. $a[1..j]$ is sorted, and
2. for $k > j$, $a[k] \geq a[j]$

In particular, when $j = n-1$, $a$ is sorted. (Why?)

## Next Time

1. Running time analysis & big O notation
2. Divide and Conquer Strategy