Lecture 01: Course Introduction

COSC 311 Algorithms, Fall 2022

$ \def\compare{ {\mathrm{compare}} } \def\swap{ {\mathrm{swap}} } \def\sort{ {\mathrm{sort}} } \def\insert{ {\mathrm{insert}} } \def\true{ {\mathrm{true}} } \def\false{ {\mathrm{false}} } $

Overview

What this Course is About
Course Structure and Policies
First Problem: Sorting

What this Course is About

What is an Algorithm?

Common Examples of Algorithms?

Why do we Care?

Five Questions

What is the task to be peformed?
How can we devise a procedure to perform the task?
Why does your procedure perform the task?
How efficient is your procedure?
Could a more efficient procedure be developed?

Some Historical Notes

2500 BCE Arithmetic (Babylon)
300 BCE Euclid’s Elements
- Euclidean algorithm
~900 CE Code Breaking (Al-Kindi)
1736 Bridges of Koenigsberg (Euler)
~1850 Analytic Engine (Babbage & Lovelace)
1936 Formal Definition of Algorithm/Universal Computer (Turing)
1945 ENIAC
1990’s MATCHING and PageRank

This Course

Fundamental tasks performed by computers
- sorting
- string algorithms
- graph algorithms
Algorithmic paradigms
- divide and conquer
- greedy
- dynamic programming
Formal analysis
- correctness
- running times
- reductions

Course Aims

learn to design and analyze algorithms
develop familiarity with common paradigms and applications
understand and exploit relationships between computational problems
clearly communicate solutions

Not This Course

Programming

Abstract Problem Solving

Tradeoffs of Abstraction

The Good:

Focus on structure of problem/solution
Solutions indepdent of soft/hardware
Formal proofs of correctness/efficiency
- under explicit assumptions
See connections between problems

The Bad:

May overlook (important) implementation details
Don’t get computer/empirical feedback

Perspective on Implementations

[We prove] the correctness of the described algorithms and going from a correct algorithm to a correct program is tedious and time-consuming, but hardly an intellectual challenge. So we thought. We now think differently.

– Mehlhorn & Naeher

Course Structure and Policies

Meetings

Lectures on MWF

50 minutes “guided discussion”
preparation:
- readings
- lecture ticket (due at beginning of class)

Homework

assigned readings
lecture tickets (every class)
- single problem to prepare for class
- late tickets not accepted
homework assignments (every 2 weeks)
- multiple deeper questions
- start early
- 7 day late policy
submission through Gradescope

Exams

Midterm 1: Friday, 10/07 in class
Midterm 2: Wednesday, 11/16 in class
Final: 3 hrs, time/location TBD

Textbooks

Option 1. Algorithms Illuminated Volumes 1–4 by Roughgarden

More accessible option (less mathematically formal)

Option 2. Algorithm Design by Kleinberg and Tardos

More expansive, mathematically sophisticated (on reserve)

Option 3. Algorithms in Action by Savvich

Most concise. (digital copy @AC library)

Option 4. Choose your own adventure.

Talk to me first

Work Expectations

Come to every lecture prepared
- readings
- lecture ticket
- questions
~10 hours per week minimum
Work/study collaboratively
- write-up individually

Accountability Groups

Assigned accountability groups of 3-4 students

semi-random assignment
can request members/non-members

Expected:

meet once a week to discsuss course material
- suggested: no phones, no computers, no internet
first contact in case of absence, etc

Attendance

Come to class unless you are sick/cannot attend
- all materials available online
- non-attendance is punishment enough
Follow college policies for infectious illness
- if you are actively sick, do not come to class (even for exams)
- if unsure, take COVID test, even if negative, wear mask

Communication

Moodle discussion forum
- questions about course material, etc
Email use subject tag [COSC 311]
- primarily for personal requests
- response pertains to others, response -> general announcement
- goal response time: 1 working day

Getting Help

Accountability Group
Evening TA Session
My office hours
- drop in
- by appointment (15min slots)
Peer tutors

General Advice

Well-being vs Comfort

Attendance
Mental exercise
Studying
Academic integrity

First Problem: Sorting

Task: Sorting

Input:

Sequence $a$ of $n$ numbers
e.g., $a = 17, 7, 5, 2, 3, 19, 5, 13$

Output:

A sorted sequence $s$ of same elements as $a$
- $s$ contains same elements with same multiplicities as $a$
- $s_1 \leq s_2 \leq \cdots \leq s_n$
e.g., $s = 2, 3, 5, 5, 7, 13, 17, 19$

So Far

Sorting task is underspecified!

Why?

representation
supported operations

Examples:

stack of exams
array of numbers
tasks by deadline

Each may support different operations & require different techniques to solve efficiently

Going Forward

Spend ~2 weeks on sorting

Elementary algorithms
- argue correctness
  - mathematical induction
- argue running time
  - big O notation
Divide-and-conquer algorithms
- algorithms: MergeSort, QuickSort, RadixSort
- argue running time
  - “master method”

Sorting Arrays

Representation:

$a$ an array of size $n$
$a[1], a[2],\ldots, a[n]$

Supported Operations

$\compare(a, i, j)$
- return $\true$ if $a[i] > a[j]$ and $\false$ otherwise
$\swap(a, i, j)$
- before $a[i] = x$ and $a[j] = y$
- after $a[i] = y$ and $a[j] = x$

Example

$a = [17, 7, 5, 2, 3, 19, 5, 13]$

$\compare(a, 2, 6)$?

$\swap(a, 2, 5)$?

Central Tenet

Break a large task into smaller subtasks.

A Sorting Strategy

Sort array incrementally

sort first element (done!)
sort first two elements
sort first three elements
…

Example

Sorting
\[\begin{align*} a &= [17, 7, 5, 2, 3, 19, 5, 13]\\ &\to [7, 17, 5, 2, 3, 19, 5, 13]\\ &\to [5, 7, 17, 2, 3, 19, 5, 13]\\ &\to [2, 5, 7, 17, 3, 19, 5, 13]\\ &\to [2, 3, 5, 7, 17, 19, 5, 13]\\ &\to [2, 3, 5, 5, 7, 17, 19, 13]\\ &\to [2, 3, 5, 5, 7, 17, 19, 13]\\ &\to [2, 3, 5, 5, 7, 13, 17, 19]\\ \end{align*}\]

Performing a Single Step

How to get from one array to the next?

$[2, 5, 7, 17, 3, 19, 5, 13] \to [2, 3, 5, 7, 17, 19, 5, 13]$

Insertion Subroutine

Setup:

$a[1..i-1]$ is already sorted
must “insert” $a[i]$ so that $a[1..i]$ is sorted
- can only use $\compare$ and $\swap$ operations

Insertion:

start with $j = i$
repeat until $a[j] \geq a[j-1]$:
- $\swap(a, j, j-1)$
- decrement $j$

Example Insertion

Inserting at index 5 ($a[1..4]$ is sorted):

\[\begin{align*} a &= [2, 5, 7, 17, 3, 19, 5, 13]\\ &{}\\ &\to [2, 5, 7, 3, 17, 19, 5, 13]\\ &{}\\ &\to [2, 5, 3, 7, 17, 19, 5, 13]\\ &{}\\ &\to [2, 3, 5, 7, 17, 19, 5, 13] \end{align*}\]

Insert Subroutine in Pseudocode

Insert(a, i):
  j <- i
  while j > 1 and a[j-1] > a[j] do
    swap(a, j-1, j)
    j <- j-1
  endwhile

Claim. If $a[1..i-1]$ is sorted, then after calling $\insert(a, i)$, $a[1..i$] is sorted.

Why?

Insertion Sort

Pseudocode:

InsertionSort(a):
  for i = 2 up to size(a) do:
    Insert(a, i)
  endfor
  
Insert(a, i):
  j <- i
  while j > 1 and compare(a, j-1, j) do
    swap(a, j-1, j)
    j <- j-1
  endwhile

Insertion Sort (No Subroutine)

Pseudocode:

InsertionSort(a):
  for i = 2 up to size(a) do:
    j <- i
    while j > 1 and compare(a, j-1, j) do
      swap(a, j-1, j)
      j <- j-1
    endwhile
  endfor

Question. How convinced are you that this procedure will always work?

Next Time

Formal analysis of correctness
- induction
Analysis of running time

Before next class:

read “Pseudocode”
do Lecture 02 Ticket
- submit on Gradescope before class

Algorithm Design and Analysis

Algorithm Design:

break a task down into a sequence of “elementary” operations

Algorithm Analysis:

argue that the prescribed sequence of operations performs the task
determine the number of operations used by the algorithm

Computational Complexity

argue that a certain number of operations are necessary for any algorithm