Lecture 19: Pools and Queues

Overview

1. Finishing Up Lists
2. Pools & Queues
3. A Blocking Queue
4. A Lock-free Queue

Last Time: Lock-free linked list

Use AtomicMarkableReference<Node> for fields

• mark indicates logical removal

For add/remove:

1. Find location
2. Validate and modify
   • (first logically remove if remove)
   • use compareAndSet to atomically
     1. check that predecessor node has not been removed
     2. update next field of predecessor

For contains:

• Just traverse the list!

Common Functionality

Traverse are remove:

1. Find pred and curr where item lives
2. Physically remove logically removed nodes along the way

Return: a Window

    class Window {
	public Node pred;
	public Node curr;

	public Window (Node pred, Node curr) {
	    this.pred = pred;
	    this.curr = curr;
	}
    }

Finding and Removing in Code

    public Window find(Node head, long key) {
	Node pred = null;
	Node curr = null;
	Node succ = null;
	boolean[] marked = {false};
	boolean snip;

	retry: while(true) {
	    pred = head;
	    curr = pred.next.getReference();
	    while(true) {
		succ = curr.next.get(marked);
		while(marked[0]) {
		    snip = pred.next.compareAndSet(curr, succ, false, false);
		    if (!snip) continue retry;
		    curr = pred.next.getReference();
		    succ = curr.next.get(marked);
		}
		if (curr.key >= key) {
		    return new Window(pred, curr);
		}
		pred = curr;
		curr = succ;
	    }
	}

Removal in Code

    public boolean remove(T item) {
	int key = item.hashCode();
	boolean snip;
	while (true) {
	    Window window = find(head, key);
	    Node pred = window.pred;
	    Node curr = window.curr;

	    if (curr.key != key) {
		return false;
	    }

	    Node succ = curr.next.getReference();
	    snip = curr.next.compareAndSet(succ, succ, false, true);
	    if (!snip) {
		continue;
	    }
	    pred.next.compareAndSet(curr, succ, false, false);
	    return true;
	}
    }

Test the Code

Single Thread Performance

Runtimes: 1M Operations

Performance, 8 Threads

Runtimes: 1M Operations

Note: fewer elements $\implies$ greater contention

The Moral

Many ways of achieving correct behavior

Tradeoffs:

• Conceptual simplicity vs complexity/sophistication
• Best performance depends on usage case
  • very high contention: coarse locking is best
    • simplest
  • moderately high contention: non-blocking is best
    • most sophisticated
    • hardest to reason about
    • easiest to mess up
  • lower contention: lazy is best
    • moderately sophisticated

More Data Structures

Recently:

• Thread pools to simplify thread/task management

Questions:

• What is a pool?
• How are they implemented?

A Pool<T> Interface

Want to model producer/consumer problem

• producers create things (e.g., tasks)
• consumers consume things (e.g., complete tasks)

A pool stores items between production and consumption:

public interface Pool<T> {
    void put(T item); // add item to the pool
    T get();          // remove item from pool
}

Pool Properties

1. Bounded vs unbounded
2. Fairness
   • FIFO queue
   • LIFO stack
   • Something else?
3. Method specifications
   • partial: threads wait for conditions
     • get from empty pool waits for a put
     • put to full pool waits for a get
   • total: no calls wait for others
   • synchronous: e.g. every put call waits for get

Implementing a Pool

Can use a queue to implement a pool:

• enq implements put
• deq implements get
• LIFO priority

Two queue implementations today

1. Unbounded total queue (blocking)
2. Lock-free unbounded total queue

Unbounded Total Queue (Blocking)

• Use linked list implementation of queue
• Store:
  • Node head sentinal
    • deq returns head.next value (if any), updates head
  • Node tail
    • enq updates tail.next, updates tail
• Locks:
  • enqLock locks enq operation
  • deqLock locks deq operation
  • individual Nodes are not locked

Unbounded Queue in Pictures

Dequeue 1: Aquire deqLock

Dequeue 2: Get Element (or Exception)

Dequeue 3: Update head

Dequeue 4: Release Lock

Enqueue 1: Make Node

Enqueue 2: Acquire enqLock

Enqueue 3: Update tail.next

Enqueue 4: Update tail

Enqueue 5: Release Lock

UnboundedQueue in Code

public class UnboundedQueue<T> implements SimpleQueue<T> {
    final ReentrantLock enqLock;
    final ReentrantLock deqLock;
    volatile Node head;
    volatile Node tail;

    public UnboundedQueue() {
	head = new Node(null);
	tail = head;
	enqLock = new ReentrantLock();
	deqLock = new ReentrantLock();
    }

    public T deq() throws EmptyException {
	T value;
	deqLock.lock();
	try {
	    if (head.next == null) {
		throw new EmptyException();
	    }
	    value = head.next.value;
	    head = head.next;
	} finally {
	    deqLock.unlock();
	}

	return value;
    }

    public void enq (T value) {
	enqLock.lock();
	try {
	    Node nd = new Node(value);
	    tail.next = nd;
	    tail = nd;
	} finally {
	    enqLock.unlock();
	}
    }
    
    class Node {
	final T value;
	volatile Node next;

	public Node (T value) {
	    this.value = value;
	}
    }
}

Unbounded Logic

Easy to reason about:

• concurrent calls to enq
  • one acquires enqLock
  • others wait
• concurrent calls to deq
  • one acquires deqLock
  • others wait

What about concurrent calls to enq/deq?

• Ever an issue?