# Lecture 10: Multiplication

### COSC 311 Algorithms, Fall 2022

$\def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } \def\BubbleSort{ {\mathrm{BubbleSort}} } \def\SelectionSort{ {\mathrm{SelectionSort}} } \def\Merge{ {\mathrm{Merge}} } \def\MergeSort{ {\mathrm{MergeSort}} } \def\QuickSort{ {\mathrm{QuickSort}} } \def\Split{ {\mathrm{Split}} } \def\Multiply{ {\mathrm{Multiply}} } \def\Add{ {\mathrm{Add}} }$

## Announcements

1. Homework 2 Finalized Today (1 additional question)
2. No reading/lecture ticket for Monday
3. Thoughts on Reading

## Overview

1. Recap of Binary Radix Sort
2. Binary Arithmetic
3. Multiplication via Divide and Conquer

## Last Time

Binary Radix Sort

RadixSort(a, B):                # B is number of bits
RadixSort(a, 1, size(a)+1, B)

RadixSort(a, i, j, b):
if j - i <= 1 then
return
endif
m <- BitSplit(a, i, j, b)
RadixSort(a, i, m, b-1)
RadixSort(a, m, j, b-1)


## Conclusion

Lower Bound. Any algorithm that sorts all permutations of size $n$ using only $\compare$ and $\swap$ operations requires $\Omega(n \log n)$ comparisons.

Caveat. If values are all represented with $B$ bits, then RadixSort sorts $n$ using $O(B n)$ bit-wise comparisons.

# Binary Arithmetic

## Multiplication in Binary

Example. Compute $10110 * 1011$.

## Multiplication Procedure

  Multiply(a, b):
product <- 0
shifted <- a    # copy of a we will shift
for i = 1 to size b do
if b[i] = 1 do
product <- Add(product, shifted)
endif
shifted << 1
endfor


## Question

  Multiply(a, b):
product <- 0
shifted <- a    # copy of a we will shift
for i = 1 to size b do
if b[i] = 1 do
product <- Add(product, shifted)
endif
shifted << 1
endfor


If $a$ and $b$ are represented with $n$ bits, what is the running time of $\Multiply(a,b)$?

## Another Question

Why did we previously assume arithmetic takes $O(1)$ time?

## Multiplication via Divide and Conquer

Idea. Break numbers up into parts

• Assume $a$ and $b$ are both represented with $n = 2B$ bits, $n$ power of $2$
• Write:
• $a = a_1 a_0 = a_1 2^B + a_0$
• $b = b_1 b_0 = b_1 2^B + b_0$
• Rewrite multiplication
• $a b = (a_1 2^B + a_0)(b_1 2^B + b_0)$

## Does This Help?

$a b = (a_1 2^B + a_0)(b_1 2^B + b_0) = a_1 b_1 2^{2B} + (a_1 b_0 + a_0 b_1) 2^B + a_0 b_0$

• Replaced $1$ product with $n$ bit numbers with $4$ products of $n/2$ bits

## A Magic Trick

$a b = a_1 b_1 2^{2B} + (a_1 b_0 + a_0 b_1) 2^B + a_0 b_0$

• Define:

\begin{align*} c_2 = a_1 b_1 c_1 = a_1 b_0 + a_0 b_1 c_0 = a_0 b_0 \end{align*}
• Now: $a b = c_2 2^{2B} + c_1 2^B + c_0$

• Consider: $c^* = (a_1 + a_0)(b_1 + b_0) = a_1 b_1 + a_1 b_0 + a_0 b_1 + a_0 b_0$

Question. How do $c_1, c_2, c_3$ relate to $c^*$?

## Counting Products

• Standard multiplication:
• $c_2 = a_1 b_1$
• $c_1 = a_1 b_0 + a_0 b_1$
• $c_0 = a_0 b_0$
• Tricky multiplication
• $c_2 = a_1 b_1$
• $c_0 = a_0 b_0$
• $c^* = (a_1 + a_0) (b_1 + b_0)$
• $c_1 = c^* - c_2 - c_0$

## Progress?

By using $c^*$ to compute $a b$:

• $c_2 = a_1 b_1$
• $c_0 = a_0 b_0$
• $c^* = (a_1 + a_0) (b_1 + b_0)$

Compute

• $a b = c_2 2^{2B} + (c^* - c_2 - c_0) 2^B + c_0$

Computing $ab$ uses:

• $3$ multiplications of size $n/2$
• $O(1)$ additions/subtractions/shifts of size $O(n)$

## Karatsuba Multiplication

  KMult(a, b):
n <- size(a) (= size(b))
if n = 1 then return a*b
a = a1 a0
b = b1 b0
c2 <- KMult(a1, b1)
c0 <- KMult(a0, b0)
c <- KMult(a1 + a0, b1 + b0)
return (c2 << n) + ((c - c2 - c0) << (n/2)) + c0


## Efficiency of Karatsuba

At depth $k$:

• $3^k$ calls to KMult
• size of each call is $n / 2^k$
• depth of recursion is $\log n$

Total running time:

• $O(n) + \frac 3 2 O(n) + \left( \frac 3 2\right)^2 O(n) + \cdots + \left(\frac 3 2 \right)^{\log n} O(n)$

Can show:

• This expression is $O(3^{\log n})$

Simplify:

## Final Running Time

Result. The running time of Karatsuba multiplication is $O(n^{\log 3}) \approx O(n^{1.58})$

• when $n$ is reasonbly large, $n^{1.58} \ll n^2$
• E.g., $1,000^2 = 1,000,000$ vs $1,000^{1.58} \approx 55,000$

## Next Time

• More Divide and Conquer
• Solving General Recurrences