Lecture 05: Sorting by Divide and Conquer

COSC 311 Algorithms, Fall 2022

$\def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } \def\Merge{ {\mathrm{Merge}} } \def\MergeSort{ {\mathrm{MergeSort}} }$

Overview

1. Efficiency of Sorting
2. Review of Big O
3. Sorting by Divide and Conquer: MergeSort

Previously

Careful analysis of correctness of SelectionSort

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor


Today

Focus on efficiency

• How many elementary operations does a procedure perform?

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor


How many comparisons?

How many swaps?

How many “elementary” operations (including arithmetic)?

Abstracting Away Unknowns

Uncertain Parameters:

1. precise running times of elementary operations
2. overhead associated with executing algorithm

Assume:

1. elementary operations take constant (but unknown) time
2. overhead of starting computation is negligible for large inputs

Focus on asymptotic growth of # of operations as function of input size

Big O Notation

Qualitative measure of function growth:

1. ignore constant factors
2. ignore values on small inputs

Formally: $f$, $g$ functions, write $f = O(g)$ if

• exists $C > 0$ and $N \geq 1$ such that
• $f(n) \leq C g(n)$ for all $n \geq N$

Properties of O

1. all constants are $O(1)$
2. if $f(n) \leq g(n)$ for all $n$, then $f = O(g)$
• if $a \leq b$ then $n^a = O(n^b)$
• if $a < b$ then $n^b \neq O(n^a)$
3. if $f = O(g)$, then $a \cdot f = O(g)$
4. if $f = O(g)$ and $g = O(h)$, then $f = O(h)$
5. if $f, g = O(h)$, then $f + g = O(h)$
6. if $f_1 = O(g_1)$ and $f_2 = O(g_2)$ then $f_1 \cdot f_2 = O(g_1 \cdot g_2)$

Running time of SelectionSort in O?

01  SelectionSort(a):
02    n <- size(a)
03    for j = 1 to n - 1 do
04      min <- j
05      for i = j+1 to n do
06        if compare(a, min, i)
07          min <- i
08        endif
09      endfor
10      swap(a, j, min)
11    endfor


Divide and Conquer

Divide and Conquer Strategy

• smaller instances of same task
3. Combine solutions to solve original task

Sorting by D&C

1. Divide by index
• MergeSort
2. Divide by value
• QuickSort
3. Divide by bit representation

Dividing by Index: MergeSort

As before, $a$ an array of size $n$

• access/assignment by index (not just compare/swap)

MergeSort algorithm:

1. divide $a$ into halves $m = (n+1) / 2$
2. sort $a[1..m-1]$ recursively
3. sort $a[m..n]$ recursively
4. merge $a[1..m-1]$ and $a[m..n]$ to form sorted array

Pseudocode

# sort values of a between indices i and j-1
MergeSort(a, i, j):
if j - i = 1 then
return
endif
m <- (i + j) / 2
MergeSort(a,i,m)
MergeSort(a,m,j)
Merge(a,i,m,j)


Correctness of MergeSort

Establish two claims:

Claim 1 (merge). If $a[i..m-1]$ and $a[m..j]$ are sorted, then after $\Merge(a, i, m, j)$, $a[i..j]$ is sorted.

• Argued on lecture ticket!

Claim 2. For any indices $i < j$, after calling $\MergeSort(a, i, j)$, $a[i..j]$ is sorted.

• How to argue this? (assume Claim 1 is true…)

Pseudocode Again

00  # sort values of a between indices i and j-1
01  MergeSort(a, i, j):
02    if j - i = 1 then
03      return
04    endif
05    m <- (i + j) / 2
06    MergeSort(a,i,m)
07    MergeSort(a,m,j)
08    Merge(a,i,m,j)


Inductive Claim

Consider $\MergeSort(a, i, j)$, define $k = j - i$ to be size

$P(k)$: for every $k’ \leq k$, $\MergeSort(a, i, j)$ with size $k’$ succeeds

Base case $k = 1$:

Inductive step $P(k) \implies P(k+1)$:

Question

How efficient is MergeSort?

00  # sort values of a between indices i and j-1
01  MergeSort(a, i, j):
02    if j - i = 1 then
03      return
04    endif
05    m <- (i + j) / 2
06    MergeSort(a,i,m)
07    MergeSort(a,m,j)
08    Merge(a,i,m,j)


Analyzing Running Time

00  # sort values of a between indices i and j-1
01  MergeSort(a, i, j):
02    if j - i = 1 then
03      return
04    endif
05    m <- (i + j) / 2
06    MergeSort(a,i,m)
07    MergeSort(a,m,j)
08    Merge(a,i,m,j)


Observation 1. Let $k = j - i$ be the size of the method call $\MergeSort(a, i, j)$. Then running time is $O(k) +$ running time of recursive calls on lines 6-7.

Observation 2. Recursive calls have size $k / 2$.

Recall Logarithms (base 2)

Define $\log$ by

• $\log a = b \iff 2^b = a$

Another way

• $\log a$ is # times $a$ can be divided by $2$ to get (at most) $1$.

Facts.

1. For every constant $c > 0$, $\log n = O(n^c)$.
2. $\log n \neq O(1)$.

A Final Calculation

• Running time $T(n)$ of $\MergeSort(a, 1, n+1)$:

\begin{align*} T(n) &= 2 T(n/2) + O(n)\\ &= 4 T(\frac n 4) + 2 O(\frac n 2) + O(n)\\ &= 8 T(\frac n 8) + 4 O(\frac n 4) + 2 O(\frac n 2) + O(n)\\ &\vdots\\ &= n T(1) + \frac n 2 O(2) + \cdots + 8 O(\frac n 8) + 4 O(\frac n 4) + 2 O(\frac n 2) + O(n)\\ &= O(n) + O(n) + \cdots + O(n) + O(n) + O(n)\\ &{}\\ &= \end{align*}

Coming Up

1. More Divide and Conquer Sorting!
2. Can we sort even faster?!