Lecture 09: Lower Bounds and RadixSort

COSC 311 Algorithms, Fall 2022

$\def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } \def\Merge{ {\mathrm{Merge}} } \def\MergeSort{ {\mathrm{MergeSort}} } \def\QuickSort{ {\mathrm{QuickSort}} } \def\Split{ {\mathrm{Split}} }$

Overview

1. Sorting Lower Bound

Last Time

Can we sort $n$ elements faster than $O(n \log n)$?

I claimed: not really?

1. Decision trees
• encode executions of algorithms on set of inputs
• “decision” = response to $\compare$ operation
2. Sorting task requires that algorithm distinguishes all pairs of permutations
• decision tree must have many leaves
3. Conclude: sorting requires $\Omega(n \log n)$ $\compare$ operations

Decision Trees Again

Follow execution of $A$ on all inputs from $S_n$

Define a binary tree:

1. each node corresponds to a single $\compare$ operation
2. each node has two children corresponding to two possible outcomes of $\compare$

Form this tree for all comparisons made on all inputs in $S_n$

• label each node with inputs consistent with all $\compare$ outcomes

Decision Tree Example, n = 3

  for i = 2 to n do
| j <- i
| while j > 1 and compare(a, j-1, j) do
| | swap(a, j, j-1)


Features of Decision Trees

1. Depth of tree $=$ max # of $\compare$ operations on any input
2. Algorithm distinguishes inputs $a$ and $b \iff a$ and $b$ label different leaves

Indistinguishability Claim

Claim. If $A$ does not distinguish permutations $a$ and $b$ with $a \neq b$, then $A$ does not sort both $a$ and $b$.

Why?

Indistinguishability Consequence

Consequence. If $A$ sorts all arrays of size $n$, then every leaf of $A$’s decision tree is labeled with a single permutation array.

Why?

How Big is Decision Tree?

How many leaves must a correct decision tree have?

How deep must decion tree be?

Putting it All Together

1. Consider any sorting algorithm $A$
2. Fix inputs $S_n =$ permutations of size $n$
3. Decision tree must have at least $\mid S_n \mid = n!$ leaves
4. Decision tree must have depth at least $\log n!$
5. $A$ must perform at least $\log n!$ comparisons

Claim. $\log n! = \Omega(n \log n)$

Conclusion

Theorem. Any algorithm that sorts all permutations of size $n$ using only $\compare$ and $\swap$ operations requires $\Omega(n \log n)$ comparisons.

Question

The $\Omega(n \log n)$ lower bound critically assumes that array is only accessed and modified with $\compare$ and $\swap$.

• What if we have more refined access?

• What if we see binary representation of elements?

Sorting Binary Values

Observation. If $a$ consists of only $0$s and $1$s, we can sort $a$ in $O(n)$ time.

How?

Sorting Numbers Represented in Binary

Assume $a$ consists of $n$ numbers, each with binary representation of $B$ bits

• $a[i] =$ number at index $i$
• $a[i][j] = j$th bit of $a[i]$

A Sorting Idea

Inspired by QuickSort:

• split array by “value”
• rather than comparing values, compare individual bits

How to perform first split?

Bit Split Subroutine

Input:

• array $a$
• indices $i < j$
• bit index $b$

Behavior:

• return value $m$ with $i \leq m \leq j$
• split values of $a[i..j-1]$ such that
• $b$th bit of $a[i..m-1]$ is $0$
• $b$th bit of $a[m..j-1]$ is $1$

BitSplit Pseudocode

  BitSplit(a, i, j, b):
left <- i, right <- j
while left < right do:
if a[left][b] = 1 and a[right][b] = 0 then
swap(a, left, right)
left++, right--
else
if a[left][b] = 0 then left++
if a[right][b] = 1 then right--
endif
endwhile
if a[right][b] = 0 then return right+1 else return right


RadixSort(a, B):                # B is number of bits

if j - i <= 1 then
return
endif
m <- BitSplit(a, i, j, b)