Lecture 20: Lock-free and Bounded Queues

Last Time

1. Introduced Pool interface:
   • void put(T item)
   • T get()
2. Gave lock-based queue implementation
   • lock enq and deq methods (not nodes)

Today

1. Finish up lock-based queue
2. Implement lock-free queue
3. Discuss a bounded queue

Unbounded Queue in Pictures

Dequeue 1: Aquire deqLock

Dequeue 2: Get Element (or Exception)

Dequeue 3: Update head

Dequeue 4: Release Lock

Enqueue 1: Make Node

Enqueue 2: Acquire enqLock

Enqueue 3: Update tail.next

Enqueue 4: Update tail

Enqueue 5: Release Lock

Unbounded Logic

Easy to reason about:

• concurrent calls to enq
  • one acquires enqLock
  • others wait
• concurrent calls to deq
  • one acquires deqLock
  • others wait

What about concurrent calls to enq/deq?

• Ever an issue?

Concurrent enq/deq

• Okay if queue is not empty
  • head and tail refer to different nodes
  • no concurrent modification
• What if queue is empty?

Concurrent Subtlety

Concurrent Subtlety

• deq checks for head.next
• enq updates tail.next (= head.next)

Who wins?

Linearizing enq()

    public void enq (T value) {
	enqLock.lock();
	try {
	    Node nd = new Node(value);
	    tail.next = nd;
	    tail = nd;
	} finally {
	    enqLock.unlock();
	}
    }

Linearizing deq()

    public T deq() throws EmptyException {
	T value;
	deqLock.lock();
	try {
	    if (head.next == null) {
		throw new EmptyException();
	    }
	    value = head.next.value;
	    head = head.next;
	} finally {
	    deqLock.unlock();
	}

	return value;
    }

Something to Consider

enq linearizes to

tail.next = nd

• this happens before tail is updated

deq linearizes to

head.next == null

• this happens before head is updated

So:

• head and tail don’t always point to first (predecessor) and last items in the queue

Why is this not a problem?

Why is this not a problem?

head and tail don’t always point to first (predecessor) and last items in the queue?

Not a problem because

• Nodes lock enq/deq operations
• tail/head updated before lock released
• only enq cares about tail value
  • other enqueuers don’t modify until after tail updated
• only deq cares about head value
  • other dequeuers don’t modify until after head udpated
• crisis averted

What is the Next Question?

…we’ve got a queue with locks…

Can We Make a Lock-free Queue?

Use AtomicReferences for head/tail/next

• can atomically verify/update fields with compareAndSet
• e.g. for enq

  Node nd = new Node(item);
  Node last = tail.get();
  Node next = last.next.get();
  if (next == null) {
    if (last.next.compareAndSet(next, node)) {
        tail.compareAndSet(last, node);	 
    }
  }
  

The subtelty

• Cannot modify both tail and tail.get().next atomically
• At some point, tail will not refer to last node in list
• Without locks, other threads will be able to see this!

The Challenge

enq and deq must function properly even if:

1. head and tail don’t point where they should
2. other enq and deq operations are in progress
   • partially complete method calls

An Idea

Clean up each others’ messes!

• Call to enq can detect if tail isn’t correct
  • tail.get().next != null
• If this occurs, update tail:

  Node last = tail.get();
  Node next = last.next.get();
  if (next != null) {
    tail.compareAndSet(last, next);
  }
  
  

Threads helping each other!

LockFreeQueue

public class LockFreeQueue<T> implements SimpleQueue<T> {
    private AtomicReference<Node> head;
    private AtomicReference<Node> tail;
    
    public LockFreeQueue() {
	Node sentinel = new Node(null);
	this.head = new AtomicReference<Node>(sentinel);
	this.tail = new AtomicReference<Node>(sentinel);
    }

    public void enq(T item) {...}

    public T deq() throws EmptyException {...}
    
    class Node {
	public T value;
	public AtomicReference<Node> next;
    
	public Node(T value) {
	    this.value = value;
	    this.next  = new AtomicReference<Node>(null);
	}
    }
}

Lock-free Enqueue Method

    public void enq(T item) {
	if (item == null) throw new NullPointerException();
	
	Node node = new Node(item);

	while (true) {
	    Node last = tail.get();
	    Node next = last.next.get();

	    if (last == tail.get()) {

		if (next == null) {

		    if (last.next.compareAndSet(next, node)) {
			tail.compareAndSet(last, node);
			return;
		    }
		    
		} else {
		    
		    tail.compareAndSet(last, next);
		    
		}
	    }
	}
    }

Lock-free Dequeue Method

    public T deq() throws EmptyException {
	
	while (true) {
	    
	    Node first = head.get();
	    Node last = tail.get();
	    Node next = first.next.get();
	    
	    if (first == head.get()) {
		if (first == last) {
		    if (next == null) {
			throw new EmptyException();
		    }
		    
		    tail.compareAndSet(last, next);
		} else {
		    T value = next.value;
		    if (head.compareAndSet(first, next))
			return value;
		}
	    }
	}
    }

Cool.

Let’s test the implementations!