Lecture 09: Lower Bounds and RadixSort

COSC 311 Algorithms, Fall 2022

$ \def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } \def\Merge{ {\mathrm{Merge}} } \def\MergeSort{ {\mathrm{MergeSort}} } \def\QuickSort{ {\mathrm{QuickSort}} } \def\Split{ {\mathrm{Split}} } $

Overview

Sorting Lower Bound
Binary Radix Sort

Last Time

We asked:

Can we sort $n$ elements faster than $O(n \log n)$?

I claimed: not really?

Decision trees
- encode executions of algorithms on set of inputs
- “decision” = response to $\compare$ operation
Sorting task requires that algorithm distinguishes all pairs of permutations
- decision tree must have many leaves
Conclude: sorting requires $\Omega(n \log n)$ $\compare$ operations

Decision Trees Again

Follow execution of $A$ on all inputs from $S_n$

Define a binary tree:

each node corresponds to a single $\compare$ operation
each node has two children corresponding to two possible outcomes of $\compare$

Form this tree for all comparisons made on all inputs in $S_n$

label each node with inputs consistent with all $\compare$ outcomes

Decision Tree Example, n = 3

  for i = 2 to n do
  | j <- i
  | while j > 1 and compare(a, j-1, j) do
  | | swap(a, j, j-1)

Features of Decision Trees

Depth of tree $=$ max # of $\compare$ operations on any input
Algorithm distinguishes inputs $a$ and $b \iff a$ and $b$ label different leaves

Indistinguishability Claim

Claim. If $A$ does not distinguish permutations $a$ and $b$ with $a \neq b$, then $A$ does not sort both $a$ and $b$.

Why?

Indistinguishability Consequence

Consequence. If $A$ sorts all arrays of size $n$, then every leaf of $A$’s decision tree is labeled with a single permutation array.

Why?

How Big is Decision Tree?

How many leaves must a correct decision tree have?

How deep must decion tree be?

Putting it All Together

Consider any sorting algorithm $A$
Fix inputs $S_n = $ permutations of size $n$
Decision tree must have at least $\mid S_n \mid = n!$ leaves
Decision tree must have depth at least $\log n!$
$A$ must perform at least $\log n!$ comparisons

Claim. $\log n! = \Omega(n \log n)$

Conclusion

Theorem. Any algorithm that sorts all permutations of size $n$ using only $\compare$ and $\swap$ operations requires $\Omega(n \log n)$ comparisons.

Question

The $\Omega(n \log n)$ lower bound critically assumes that array is only accessed and modified with $\compare$ and $\swap$.

What if we have more refined access?
What if we see binary representation of elements?

Sorting Binary Values

Observation. If $a$ consists of only $0$s and $1$s, we can sort $a$ in $O(n)$ time.

How?

Binary Split Illustration

Sorting Numbers Represented in Binary

Assume $a$ consists of $n$ numbers, each with binary representation of $B$ bits

$a[i] = $ number at index $i$
$a[i][j] = j$th bit of $a[i]$

A Sorting Idea

Inspired by QuickSort:

split array by “value”
rather than comparing values, compare individual bits

How to perform first split?

Bit Split Subroutine

Input:

array $a$
indices $i < j$
bit index $b$

Behavior:

return value $m$ with $i \leq m \leq j$
split values of $a[i..j-1]$ such that
- $b$th bit of $a[i..m-1]$ is $0$
- $b$th bit of $a[m..j-1]$ is $1$

BitSplit Pseudocode

  BitSplit(a, i, j, b):
    left <- i, right <- j
    while left < right do:
      if a[left][b] = 1 and a[right][b] = 0 then
        swap(a, left, right)
        left++, right--
      else
        if a[left][b] = 0 then left++
        if a[right][b] = 1 then right--
      endif
    endwhile
    if a[right][b] = 0 then return right+1 else return right

RadixSort

RadixSort(a, B):                # B is number of bits
  RadixSort(a, 1, size(a)+1, B)

RadixSort(a, i, j, b):
  if j - i <= 1 then
    return
  endif
  m <- BitSplit(a, i, j, b)
  RadixSort(a, i, m, b-1)
  RadixSort(a, m, j, b-1)