Lecture 09: Lower Bounds and RadixSort

COSC 311 Algorithms, Fall 2022

$ \def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } \def\Merge{ {\mathrm{Merge}} } \def\MergeSort{ {\mathrm{MergeSort}} } \def\QuickSort{ {\mathrm{QuickSort}} } \def\Split{ {\mathrm{Split}} } $


  1. Sorting Lower Bound
  2. Binary Radix Sort

Last Time

We asked:

Can we sort $n$ elements faster than $O(n \log n)$?

I claimed: not really?

  1. Decision trees
    • encode executions of algorithms on set of inputs
    • “decision” = response to $\compare$ operation
  2. Sorting task requires that algorithm distinguishes all pairs of permutations
    • decision tree must have many leaves
  3. Conclude: sorting requires $\Omega(n \log n)$ $\compare$ operations

Decision Trees Again

Follow execution of $A$ on all inputs from $S_n$

Define a binary tree:

  1. each node corresponds to a single $\compare$ operation
  2. each node has two children corresponding to two possible outcomes of $\compare$

Form this tree for all comparisons made on all inputs in $S_n$

  • label each node with inputs consistent with all $\compare$ outcomes

Decision Tree Example, n = 3

  for i = 2 to n do
  | j <- i
  | while j > 1 and compare(a, j-1, j) do
  | | swap(a, j, j-1)

Features of Decision Trees

  1. Depth of tree $=$ max # of $\compare$ operations on any input
  2. Algorithm distinguishes inputs $a$ and $b \iff a$ and $b$ label different leaves

Indistinguishability Claim

Claim. If $A$ does not distinguish permutations $a$ and $b$ with $a \neq b$, then $A$ does not sort both $a$ and $b$.


Indistinguishability Consequence

Consequence. If $A$ sorts all arrays of size $n$, then every leaf of $A$’s decision tree is labeled with a single permutation array.


How Big is Decision Tree?

How many leaves must a correct decision tree have?

How deep must decion tree be?

Putting it All Together

  1. Consider any sorting algorithm $A$
  2. Fix inputs $S_n = $ permutations of size $n$
  3. Decision tree must have at least $\mid S_n \mid = n!$ leaves
  4. Decision tree must have depth at least $\log n!$
  5. $A$ must perform at least $\log n!$ comparisons

Claim. $\log n! = \Omega(n \log n)$


Theorem. Any algorithm that sorts all permutations of size $n$ using only $\compare$ and $\swap$ operations requires $\Omega(n \log n)$ comparisons.


The $\Omega(n \log n)$ lower bound critically assumes that array is only accessed and modified with $\compare$ and $\swap$.

  • What if we have more refined access?

  • What if we see binary representation of elements?

Sorting Binary Values

Observation. If $a$ consists of only $0$s and $1$s, we can sort $a$ in $O(n)$ time.


Binary Split Illustration

Sorting Numbers Represented in Binary

Assume $a$ consists of $n$ numbers, each with binary representation of $B$ bits

  • $a[i] = $ number at index $i$
  • $a[i][j] = j$th bit of $a[i]$

A Sorting Idea

Inspired by QuickSort:

  • split array by “value”
  • rather than comparing values, compare individual bits

How to perform first split?

Bit Split Subroutine


  • array $a$
  • indices $i < j$
  • bit index $b$


  • return value $m$ with $i \leq m \leq j$
  • split values of $a[i..j-1]$ such that
    • $b$th bit of $a[i..m-1]$ is $0$
    • $b$th bit of $a[m..j-1]$ is $1$

BitSplit Pseudocode

  BitSplit(a, i, j, b):
    left <- i, right <- j
    while left < right do:
      if a[left][b] = 1 and a[right][b] = 0 then
        swap(a, left, right)
        left++, right--
        if a[left][b] = 0 then left++
        if a[right][b] = 1 then right--
    if a[right][b] = 0 then return right+1 else return right


RadixSort(a, B):                # B is number of bits
  RadixSort(a, 1, size(a)+1, B)

RadixSort(a, i, j, b):
  if j - i <= 1 then
  m <- BitSplit(a, i, j, b)
  RadixSort(a, i, m, b-1)
  RadixSort(a, m, j, b-1)


Why Does RadixSort Work?

What is RadixSort Running Time?

Next Time


  • multiplication