By Joshua Nogales, Alexander Shkapsky, Chris Ishisoko

Again, for review, the 4 necessary conditions that must be present for Deadlock to occur are:

1. Mutual Exclusion: Tasks claim exclusive control of the resources they require (example - no two threads may have instruction pointers in the same segment of code at the same time)
2. Hold & Wait: Tasks hold resources already allocated to them while waiting for additional resources (example - a thread may block or spin while holding onto a lock)
3. No lock preemption: Resources cannot be forcibly removed from the tasks holding them until the resources are used to completion (example - a lock can only be released if a thread does so voluntarily)
4. Circular wait: A circular chain of tasks exists, such that each task holds one or more resources that are being requested by the next task in the chain (example - process 1 is waiting for process 2 who is waiting on process 1)

Definitions from System Deadlocks (E G Coffman, M Elphick, A Shoshani, 1971) . The 4 necessary conditions were first stated here. This paper outlines deadlock, the necessary conditions and some suggestions of how it can be prevented/avoided/recovered from. It also does a good job of relating deadlock to something we're all familiar with - traffic.

The first case discussed is with two processes waiting on each other. Such a case can happen in a swap function.

swap(pipebuf_t *p, pipebuf_t *q){ //This is an atomic swap operation
    acquire(&p->lock);
    acquire(&q->lock);
    //swap the variables, using simple swap
    release(&q->lock);
    release(&p->lock);
}

If the swap function above is called by two processes, one with parameters and (x,y) and the other with (y,x), there is a possibility of deadlock. Suppose the two swaps are scheduled as follows:

Thread 1 will never be able to acquire the lock for y because Thread 2 is holding onto it and waiting to acquire the lock for x. Thread 2 will never be able to acquire the lock for x because Thread 1 has it and is waiting to acquire the lock for y. Since neither thread will release their locks until the other thread does, deadlock results! This condition is known as circular wait. Circular wait can be represented in a wait-for graph.

The following wait-for graph is describes the situation above:

In a wait-for graph, locks are represented as rectangles and threads as ovals. T1 will acquire the x lock, T2 will acquire the y lock, and both threads will try to acquire the other lock but will be unable to do so.

It is possible to have a 1 process deadlock. swap(x,x) is an example of this case. The thread will acquire the lock for the first x and then keep on trying to acquire the lock for the second x. Since it is already holding onto the lock for the first x (which is the same lock as the second x), it will be actually be waiting for itself to release the lock, which it cannot do with the current version of the code. It may sound trivial, but unfortunately it is common in practice. A solution to this problem is discussed below.

Now, lets take a look at swap(x,y), swap(y,z), swap(z,x). Processes trying to acquire locks and getting themselves into a circular wait isn't just reserved for 1 process trying to acquire the same lock, or for 2 processes trying to obtain the 2 of the same locks. It can also happen when more than 2 processes are involved.

Should 'unfortunate' scheduling occur, this situation will also bring about a circular wait!

In the case of swap(x,x), how are we supposed to know that we are going to cause a circular wait?

Nope, nothing here to help us...

struct mutex {
    unsigned val;
} mutex_t;

These guys aren't much help either...

acquire(mutex_t *m) {
    while (t_a_s(&m->val, 1))
        /* do nothing */
}

release(mutex_t *m) {
    m->val = 0;
}

And looking back on the other mutexes we've designed, we never stop anyone from stealing a lock either. A thread can call release() and take the lock.

steal_lock(mutex_t *m) {
    release(m);
}

How can we prevent this? By adding an owner to the mutex!

struct mutex {
    unsigned val;
    procdescriptor_t *owner; //mutex owner!
} mutex_t;

Our acquire() and release() must also change.

acquire(mutex_t *m) {
    assert(m->owner != current); //Now a thread can't acquire a lock twice!
    while(t_a_s(&m->val, 1)) {}
      /*do nothing*/

    m->owner = current;          //Set owner of lock
}

release (mutex_t *m) {
    assert(m->owner == current); //Now other threads can't steal the lock!  They have to be the owner to release it.
    m->owner = NULL;             //Clear owner of lock
    m->val = 0;
}

*Note - owner has to be set to NULL before val is set to 0 because we want to release the lock AFTER we're done changing the state of the mutex. Otherwise, the lock will be released, and conceivably another thread could grab the lock before the owner is reset.

Now we're faced with a new issue, caused by our recent change. In the case of swap(x,x), the single process trying to access a critical section that it has locked, now gets an error because it fails the assert condition. The job of a mutex is to enforce a critical section. Thus, theoretically, a thread holding a mutex should be able to reacquire the mutex without violating the atomicity of the critical section. How can we fix this issue? With a recursive mutex!

A recursive mutex allows a thread to acquire a mutex multiple times.

struct mutex{
    unsigned val;
    procdescriptor_t *owner;
    int count;                 //recursive mutex support.  How many times the owner has locked the mutex recursively.
} recurmutex_t;

Now we can keep track of how many times the lock owner has requested the lock via our mutex structure.

acquire(recurmutex_t *m) {
    if (m->owner == current) {    
        m->count++;            //increment the count for each time after the first that the owner requests the lock
        return;
    } else {
      while (t_a_s(&m->val, 1)) {}
          /*do nothing*/
      m->owner = current;
    }
}

*Note that we no longer assert() that a different owner must be the requestor of the lock. Instead we check if the requestor is already the owner and if they are, we increment the count of recursive lock requests and we're done. Otherwise, it's business as usual with a call to test_and_set(,) until the lock is available.

release(recurmutex_t *m) {
    assert(m->owner == current);
    if(m->count > 0)
        m->count--;           //decrement the lock until there's 0 recursive locks 'open'
    else {
        m->owner = NULL;
        m->val = 0;
    }
}

*Note that count balances the number of calls to acquire with the number of calls to release the owner has requested. If the count never gets back to 0, the lock will never be freed by the owner. And again, the order of owner set to NULL before val is set to 0 is necessary as we mentioned above.

A conservative mechanism to prevent all deadlocks involving mutexes.

Total Orders

A Total Order of some set X will have the following properties:

If (a,b are elements of X) then
    a = b
    or a < b
    or a > b
Transitivity
   if (a,b,c are elements of X and a < b, b < c)
       then a < c

What wikipedia says on Total Orders.

Ordering function O

0(mutex_t *m) produces values that are in a total order.

Given mutexes m, n:
  either m == n
  or O(m) < O(n)
  or O(n) < O(m)

If we only acquire mutexes in order, we can never have a circular wait! In other words, if a process calls acquire(m) and then acquire(n), and O(m) < O(n), and there is no circular wait.

Using our swap(,) function, we'll have:

swap(pipebuf_t *p, pipebuf_t *q) {
  if (p < q) {
    acquire(&p->lock);
    acquire(&q-lock);
  } else {
    acquire(&q->lock);
    acquire(&p-lock);
  }
  //do the swap here
  //do the release
}

With this a lock ordering methodology, our earlier circular wait case won't appear:

*Note: the ordering must be shared by all processes

Professor Kohler told us about an interesting deadlock problem he encountered with an early version of Yahoo! Messenger. It was designed with two processes: a GUI process that the user would interact with, and a helper process to handle sending messages and any other processes that could be blocked. The processes were able to communicate with each other using pipes. By separating the two, the user's GUI would always be available and unaffected by potential errors or blocks. Despite this strategy, he found that if the GUI process sent large enough messages, the entire application would block forever. If 8 1000-byte messages were sent, everything was fine. But, if 9 1000-byte messages were sent, the application would deadlock.

It turns out the cause was that both pipe buffers, each with space of 8192 bytes, would fill up when the messages were large/numerous enough. When the pipes reached capacity, neither process could do anything because they were waiting on the other process to read from its read end of the pipe. The GUI was waiting for the Helper to read from the GUI -> Helper pipe and the Helper was waiting for the GUI to read from the Helper -> GUI pipe. The application entered a state of deadlock!

In this case, it wasn't mutexes that caused deadlock, but buffer space!

Starvation - a process can wait forever to acquire some lock, even if that lock is released infinitely often.

Does a spinlock suffer from starvation? Possibly yes -- it depends on the scheduling order.

It is possible that a thread keeps getting passed over and is blocked when the lock they wanted is released. By the time the thread is awakened, the lock is already acquired by another thread. The other threads play 'keep away' and the thread is starved.

In this case, thread 2 never gets to go because the lock is passed between threads 1 and 3. If this keeps up forever, thread 2 is starved.

Also, thread 2 might have been blocked before thread 3 even started running. Is it fair that thread 2 gets passed over and thread 3 gets the lock? It certainly doesn't sound fair. And it sounds even less fair that thread 2 now gets starved.

And so, to avoid starvation and to be fair we need to service locks in order!

From E. W. Dijkstra:

Suppose that process 1 is in its critical section and that process 2 will be the next one to enter it. Now there are two possible cases.

a) process 1 will have done "exit" before process 2 has tried to "enter"; in that case no sleeping occurs

b) process 2 tries to "enter" before process I has done "exit"; in that case process 2 has to go sleep temporarily until is woken up as a side-effect of the "' exit" done by process t.

When both occurrences have taken place, i.e. when process 2 has succesfully entered its critical section it is no longer material whether we had case a) or case b). In that sense we are looking for primitives (for "enter" and "exit") that are commutative. What are the simplest commutative operations on common variables that we can think of ? The simplest operation is inversion of a common boolean, but that is too simple for our purpose: then we have only one operation at our disposal and lack the possibility of distinguishing between "enter" and "exit". The next simplest commutative operations are addition to (and subtraction from) a common integer. Furthermore we observe that "enter" and "exit" have to compensate eachother: if only the first process passes its critical section the common state before its "enter" equals the common state after its "exit" as far as the mutual exclusion is concerned. The simplest set of operations we can think of are increasing and decreasing a common variable by 1 and we introduce the special synchronizing primitives

P(s):s:=s--t

and

V(s): s :=s + t

special in the sense that they are "indivisible" operations: if a number of P- and V-operations on the same common variable are performed "simultaneously" the net effect of them is as if the increases and decreases are done "in some order".

Now we are very close to a solution: we have still to decide how we wish to characterize that a process may go to sleep. We can do this by making the P- and V-operations operate not on just a common variable, but on a special purpose integer variable, a so-called semaphore, whose value is by definition non-negative; i.e. s=>0.

Quote from pages 9-10, Hierarchical Ordering of Sequential Processes (E. W. Dijkstra, Acta Informatica, June 1971)

P(s) - probeer te verlagen => try & decrease (while the semaphore's value is < 0, it blocks then and decrements once it gets to go.)

V(s) - verhoog => increase (s++)

P requests are serviced in FIFO order; no starvation occurs. Semaphores are fair! (taken at face value)

Semaphore Mutex

struct smutex {
    semaphore_t s = 0; //initialize to 0
} smutex_t;

acquire(smutex_t *m) {
    P(&m->s);
}

release(smutex_t *m) {
    V(&m->s);
}

if s < 0, then it is locked.

By changing the initial value from 0 to 1 or 2, we allow 2 or 3 processes, respectively, to lock.

Now we can implement our bounded buffers w/ semaphores

struct pipebuf {
    sempahore_t lock=0;
    sempahore_t size=-1;
    sempahore_t space=N-1;
    unsigned head;
    unsigned tail;
    char buf[N];
} pipebuf_t;

And reading from and writing to our buffer now looks like:

void writec(pipebuf_t *p, char c) {
    P(&p->space);                //writing reduces available space in buffer
    P(&p->lock);
    p->buf[p->tail %N] = c;
    p->tail++;
    V(&p->lock);
    V(&p->size);                 //writing increases size of buffer
}

char readc(pipebuf_t *p) {
    P(&p->size);                 //reading reduces size of buffer
    P(&p->lock);
    int c = p->buf[p->head % N];
    p->head++;
    V(&p->lock);
    V(&p->space);                //writing increases availabe space in buffer
    return c;
}

*Note - in writec() space is reduced before the P(&p->lock) and size is increased after the V(&p->lock). The inverse situation is also present in readc() with size being reduced before P(&p->lock) and space being increased after the V(&p->lock). This is all possible because if there is no available space to reduce in writec() or available size to reduce in readc(), the semaphore will handle the blocking until there is room. The p->lock serves as the lock for the other shared state that can not be represented with semaphores such as the head and tail pointers and the contents of the buffer.

Idea: To improve utilization by allowing concurrent readers (similarly to lab 2). Multiple processes can read simultaneously, but if one is writing, then no one else can read or write.

struct rwlock {
    mutex_t m;
    int nr;
    condvar_t noreaders; //Number of readers condition variable
}

acquire_read(rwlock_t* l) {
    acquire(&l->m);
    l->nr++;
    release(&l->m);
}

release_read(rwlock_t* l) {
    acquire(&l->m);
    l->nr--;
    if (l->nr == 0)
        notify(&l->noreaders);
    release(&l->m);
}

acquire_write(rwlock_t* l) {
    acquire(&l->m);
    while(l->nr > 0)
        wait(&l->noreaders, &l->m);
}

release_write() needs to release &l->m.