Lecture 10 Scribe Notes

by Terry Heinrich, David Wickeraad, Andrew Pryor-Miller

At the end of Lecture 9, we examined a Bounded Buffer in which we switched in mutexes for all the spinlocks. The result of this is better utilization (in the sense that we don't poll for locks). However, utilization is not optimal as we are waiting for a condition to be true; in particular, we are either waiting for the bounded buffer to be nonempty or nonfull.

Let's take a look back at that bounded buffer implementation.

#define N 8
 
typedef struct {
   char buf[N]; /* N = 8 */
   unsigned head;
   unsigned tail;
   mutex_t l;
} pipebuf_t;
 
void writec(pipebuf_t *p, char c) {
   while (1) {
       bacquire(&p->l);  
       if (p->tail - p->head < N) {
           p->buf[p->tail % N] = c;
           p->tail++;
           brelease(&p->l);
           return;
       }
       brelease(&p->lock);
   }
}
 
char readc(pipebuf_t *p) {
   while (1) {
       bacquire(&p->l);                  
       if (p->tail != p->head) {
           char c = p->buf[p->head % N];
           p->head++;
           brelease(&p->l);
           return c;
       }                                   
       brelease(&p->lock);
   }
}

As we recall, the mutex gives us blocking and no starvation (good utilization), but there are some work loads that lead to bad utilization. In both our writec and readc functions, we are polling for the buffer to not be full or empty, respectively.

Let's now take a step inside readc(). Our use of bacquire() will block until no other process is in the critical section. Simply put, we need to do more. We want the function to block until:

1. No other process is in the critical section, AND
   
2. The pipe buffer is nonempty.

Therefore, in our case, we want to block until p->head != p->tail. To best notice that this expression is true, we turn it into a variable, specifically, a condition variable.

Condition Variable
A variable or object that stands for a condition.
(Ex. A mutex stands for a lock, where the lock variable stands for the lock condition).

There are several functions that are involved with condition variables; the following are examples in pseudo-code:

condvar_t cv;    //declaration of a condition variable
 
void cond_wait(condvar_t *cv, mutex_t *m) {
   //1. release mutex
   //2. block until cond_notify(cv) is called
   //THE FIRST TWO OPERATIONS OCCUR ATOMICALLY AS ONE
   //3. acquire mutex
}
 
void cond_notify(condvar_t *cv) {   //cond_notify will be used every time a condition might be true
   //wake up first process blocked in cond_wait
}
 
void cond_broadcast(condvar_t *cv) {
   //wake up all the processes blocked in cond_wait
}

Why is there a need to have a mutex in cond_wait()?

We need to avoid sleep/wake up races. With the mutex, we protect the condvar_t.

Now that we have our functions, we need to figure out how to implement them with our buffer. Before we actually do that, we need to create our condition variables.

What are our conditions?

• nonempty (for readc())
• nonfull (for writec())

We will include these in the structure for the pipe buffer.

typedef struct {
   char buf[N]; /* N = 8 */
   unsigned head;
   unsigned tail;
   mutex_t l;
   condvar_t nonempty;    //indicates the buffer is not empty
   condvar_t nonfull;     //indicates the buffer is not full
} pipebuf_t;

OK, simple enough, but what about cond_wait and cond_notify? Taking a look at our implementation of readc, we know that we are concerned about blocking when we know that the buffer is empty. Obviously, we know a buffer is empty if we don't enter the core if statement. We do need to be aware, though, that cond_wait will acquire a mutex, so if we put it after the brelease we will encounter a deadlock as the while loop will try to acquire as soon as we loop back up. Instead of putting cond_wait around the release, let's replace it, which means that we should also put bacquire() outside the while loop.

char readc(pipebuf_t *p) {
   bacquire(&p->l);                      //acquire a lock outside the while loop
   while (1) {                  
       if (p->tail != p->head) {
           char c = p->buf[p->head % N];
           p->head++;
           brelease(&p->l);
           return c;
       }                                   
       cond_wait(&p->nonempty, &p->l);   //block because the buffer is empty
   }
}

As mentioned earlier, cond_notify() needs to be used every time a condition might be true. And how do we check if the buffer is not empty? We have to look in the first place in which the buffer will be more nonempty, which would be writec(). Specifically, we know the buffer is not empty when the tail and head pointers are not equal, so as soon as we increment the tail, we can say that the condition might be true.

void writec(pipebuf_t *p, char c) {
   while (1) {
       bacquire(&p->l);  
       if (p->tail - p->head < N) {
           p->buf[p->tail % N] = c;
           p->tail++;
           cond_notify(&p->nonempty);   //we know the condition is true!
           brelease(&p->l);
           return;
       }
       brelease(&p->lock);
   }
}

Great! So now we know that we block when the buffer is empty and we are notified when we can wake up the read! As writec and readc are symmetric equations, we can make the same changes to both with nonfull.

void writec(pipebuf_t *p, char c) {
   bacquire(&p->l);                         //acquire the lock outside the while loop
   while (1) {
       if (p->tail - p->head < N) {
           p->buf[p->tail % N] = c;
           p->tail++;
           cond_notify(&p->nonempty);   
           brelease(&p->l);
           return;
       }
       cond_wait(&p->nonfull, &p->lock);    //block because the buffer is full
   }
}
 
char readc(pipebuf_t *p) {
   bacquire(&p->l);
   while (1) {                  
       if (p->tail != p->head) {
           char c = p->buf[p->head % N];
           p->head++;
           cond_notify(&p->nonfull);        //we know that the buffer is not full!
           brelease(&p->l);
           return c;
       }                                   
       cond_wait(&p->nonempty, &p->l);   
   }
}

Awesome! Our utilization is much better, and we know that blocking is fully implemented! There are a few things to keep in mind with condition variables involving locks(mutexes). Here are two scenarios to ponder:

• What happens if (in either readc or writec) we put the cond_notify after the brelease inside the core if statement?
  • We create a sleep/wake up race. cond_notify does not acquire a mutex, so it must be protected before it is entered.
• Assume cond_wait did not release and acquire a mutex. If we placed a release and acquire around the cond_wait() call, wouldn't that work?
  • Not quite. cond_wait's release occurs atomically with the act of setting the block. The release outside the function call is not atomic, and therefore, cond_notify could acquire the lock in the time that it is released before cond_wait is called.

Semaphores were invented by Edsger Dijkstra in the 1960's. A semaphore is a synchornized integer with two atomic operations:

P(s): probeer te verlagen (try to release)
This function blocks while the value of s < 0, then decrements s when it becomes unblocked.

V(s): verhoog (increase)
This function increments s.

Semaphores are similar to locks, except that semaphores give us more versatility. They provide a means of giving locks to more than one thread. Semaphores are commonly used when protecting multiple indistinct resources. For example, one may use them to prevent a queue from overflowing its bounds. It is possible to use semaphores anywhere that a mutex is used. They are not better than mutexes in terms of starvation or utilization, they just provide us with a different way of thinking.
Ideas taken from Developer Connection: Semaphores.

How to Implement a Mutex with a Semaphore
V(s) maps to a release of a mutex and P(s) maps to an aquire of a mutex. In order to model a mutex s is initially set to 0.

struct mutex_t {
    semaphore_t s = 0; //initialize to 0
} smutex_t;
 
bacquire(mutex_t *m) {
    P(&m->s);          //attempt to decrease
}
 
brelease(mutex_t *m) {
      V(&m->s);        //increase s
}

Semaphores block when s < 0, because this is its locked state. This property gives us extra versatility that isn't possible with normal mutexex. We are able to initialize the semaphore in a locked state by setting it to -1, or we can allow two items in the critical section, by initializing s to 1. By this property we can allow n processes to acquire the lock by initializing s to n - 1.

This new functionality can be seen in the following implementation of our bounded buffers using semaphores:

typedef  struct pipebuf {
      sempahore_t lock  = 0;
      sempahore_t nonempty  = -1;    //initially locked
      sempahore_t nonfull = N-1;     //only want first N to succeed
      unsigned head;
      unsigned tail;
      char buf[N];
  } pipebuf_t;

The semaphore variable lock serves as the lock for all of the shared state variables of the buffer, while nonempty and nonfull handle the blocking until a character is available to read or space in the buffer is available for writing respectively.
Reading from and writing to our buffer now looks like:

  void writec(pipebuf_t *p, char c) {
      P(&p->nonfull);                //block until there is space in the buffer
      P(&p->lock);
      p->buf[p->tail %N] = c;
      p->tail++;
      V(&p->lock);
      V(&p->nonempty);                 //increment because there is now an item in the buffer
  }
 
  char readc(pipebuf_t *p) {
      P(&p->nonempty);                 //block until there is something to read
      P(&p->lock);
      int c = p->buf[p->head % N];
      p->head++;
      V(&p->lock);
      V(&p->nonfull);                //increment becasue we opened space for another character in the buffer
      return c;
  }

This implementation of writec and readc using semaphores rather than mutexes is not better in terms of starvation or utilization, but semaphores allow us to combine claiming space and blocking into one variable type.

There are four necessary conditions that must be present for Deadlock to occur:

1. Mutual Exclusion: Locks-tasks claim exclusive control of the resources they require (example - no two threads may have instruction pointers in the same segment of code at the same time)
2. No Preemption: Tasks hold locks, or rights to resources, until they voluntarily release it. Resources cannot be forcibly removed from the tasks holding them.
3. Hold & Wait: Tasks hold resources that were previously allocated to them while waiting for, or acquiring additional resources (example - a thread may block or spin while holding onto a lock).
4. Circular Wait: A circular chain of tasks exists, such that each task holds one or more resource that is being requested by the next task in the chain (example - process 1 is waiting on a resource that process 2 is waiting for, which is already acquired by process 1. This causes the two processes to wait for each other.

A simple example of deadlock where two processes are waiting on eachother can be seen in the swap function:

swap(pipebuf_t *p, pipebuf_t *q){ //This is an atomic swap function
   char tmp [N];                    //1
   bacquire(&p->l);                 //2
   baquire(&q->l);                  //3
   int s = size(p);                 //4
   for (int i = 0; i < size; i++)   //5
      tmp[i] = readc(p);            //6
   int t = size(q);
   for(int i = 0; i < t; i++) 
      writec(p, readc(q));
   for(int i = 0; i < s; i++)
      writec(q, buf[i]);
   brelease(&p->l);
   brelease(&q->l);
}
 
int size(pipebuf_t *p) {
   return p->tail - p->head;
}

The simplest form of deadlock is if the above swap function is called by one process, but the same pipe buffer is passed in as both parameters: swap(a,a). This results in line 2 acquiring the lock for a, then when line 3 is run, it tries to acquire a lock for a, which blocks, resulting in deadlock.

Waits-For-Graphs are a useful way to see if circular wait has occurred, resulting in deadlock. In order to create a waits-for-graph, first draw a circle to represent each process. Second, draw a rectangle to represent every resource that the processes are attempting to acquire. Third, draw an arrow for every lock that has been acquired from the resource to the process. Lastly, draw an arrow from the process to the resource for every process that is waiting(blocking) to acquire the lock. If a circular loop is created by performing these four steps, then the circular wait property has been fulfilled and deadlock will occur. An example of a waits-for-graph for the deadlock situation described above is shown below:

As you can see in the figure above, the process has aqcuired a lock for resource A, but then requests a second lock. The second acquire causes the process to block, creating deadlock. Implementing an algorithm that checks for deadlock is as easy as implementing a recursive algorithm that traverses a waits-for-graph and reporting any circular patterns before allowing a process to acquire, or wait to acquire a lock.

In the figure above, a circular wait on three resources is shown.

• Does not allow the same process to acquire a lock on itself.
• Allows for more than one process to acquire a lock.

typedef struct recmutex
{
   unsigned locked;
   procdescriptor_t *owner;
}recmutex_t;
 
racquire(recmutex_t *r)
{
   assert(r->owner != current);  //Ensures that a process won't block while waiting on itself
 
   while(r->owner != current && r->owner != NULL)
   {
      //block
   }
   r->owner = current;
   r->locked++;
}
 
rrelease(recmutex_t *r)
{
   r->locked--;
   if(r->locked ==0)
      r->owner = NULL;
}

• Conservative method for preventing deadlocks on all mutexes
• Every mutex has a different order
• Acquire mutexes only in increasing order

The following example shows how to use ordered mutexes using the swap method

//The purpose of this function is to help eliminate circular waiting
//This implementation doesn't work for the case where Process 1 calls swap, 
//and passes in the same pointer into *p and *q
void swap(pipebuf_t *p, pipebuf_t *q)
{
   char tmp[N];
   if(p<q)   
   {
      bacquire(&p->l);  
      bacquire(&q->l);
   }
   else
   {
      bacquire(&q->l);
      bacquire(&p->l);
   }
   int s = size(p);
   for(int i =0; i<s; i++)
   {
      tmp[i] = readc(p);
   }	
   int t = size(q);
   for(int i =0; i<t; i++);
   {
      writec(p,readc(q));
   }
   for(int i = 0; i<s, i++)
   {
      writec(q, tmp[i]);
   }
   brelease(&p->l);
   brelease(&q->l);
}