Lecture 7: Scheduling II

Presented By: Brian Miller, Delwin Yu, and Hemang Thakkar

Before we dive into the different types of schedulers, first let's look at the motivation for schedulers. In previous lectures, we've looked at many different ways to virtualize all aspects of the computer. One particular virtualization came in the form of processes, which are basically virtual machines or programs that run in isolation from one another. A modern computer today runs many processes nearly simultaneously, and the order in which these processes are executed is determined by the kernel. This problem of deciding which processes to run and for how is the driving force for this lecture's main topic, Scheduling. Looking into the variety of different schedulers and analyzing their pros and cons will give us insight as to what scheduling methods should be chosen to efficiently and fairly execute multiple processes.

By the end of these notes, you should have a firm grasp on the following schedulers and ideas:

• First Come First Serve Scheduler
• Shortest Job First Scheduler
• Round Robin Scheduling
• Priority Scheduling

First let us refresh your memory and introduce some new key terms that will be useful for these notes:

• Distance from arrival time to first run

• Distance from the arrival to completion.

• No job is interrupted until it completes, and the process yields control of the CPU willingly.

A scheduler can cause starvation if there exists a set of requests where some requests are never serviced. Here is a list of schedulers and whether or not they cause starvation.

• FCFS → does not cause starvation, because there are finite processes
• SJF → can have starvation if "short" jobs keep arriving without stopping
• RRS → does not cause starvation if implemented correctly, processes may take forever to finish, but they will eventually finish. If implemented badly (new processes are scheduled first) then it is theoretically possible if processes suddenly keep coming without stopping that the processes will all be in starvation and never finish.
• Scheduling with Priority → low priority (high priority value) processes will starve
• Priority with Aging → implemented correctly does not cause starvation, because priority will be shifted to allow all processes to run at some point.

Now that we have these key terms defined, we may begin our journey into the world of scheduling by first examining the most basic of schedulers...

As continued from previous lecture, this is an example of a First Come First Serve Scheduler (FCFS), which simply executes the processes in the order that they are created/queued.

In this example, there are four processes that need to run, each with a different starting time and service time. As just mentioned, in a FCFS scheduler, we will simply execute the processes as they come, without examining any other information on the processes. Look at the Gnatt Chart below to see how long each process runs, and when they start and stop.

Notice how each process runs for the length of their service time. But there is an additional time factor that needs to be examined, namely, how long it takes for the processor to perform a context switch. A context switch, represented by the symbol χ, represents the amount of overhead time it takes for a processor to switch from one process to the other. Context switches do not take that much time to execute (when compared to the service time), but they are significant if many context switches are made in a relatively short amount of time. This overhead time accounts for the fact that after a process finishes, it must yield control back to the kernel, which then schedules the next task to be performed. Now take a look at the chart below to see how much waiting time and service time is needed to execute this particular ordering of the processes.

In order to calculate Waiting time, just do the following:

1. Take the arrival time of a process and subtract it from the time it was first run.

For example:

• Process A: arrival time = -4, first run time = 0 → Wait time = 0 - (-4) = 4
• Process B: arrival time = -3, first run time = 5 + the time it takes the processor to switch from process A to Process B (context switch) → Wait time = 5 - (-3) + χ = 8 + χ.
• Process C: arrival time = -2, first run time = 7 + 2 context switches → Wait time = 7 - (-2) + 2χ = 9 + 2χ
• Process D: arrival time = -1, first run time = 16 + 3 context switches → Wait time = 16 - (-1) + 3χ = 17 + 3χ
• Average Waiting time = (4 + 5 + χ + 9 + 2χ + 17 + 3χ) / 4 = 9.5 + 1.5χ

(just a reminder, the waiting time for a process is the distance from the arrival time (which may be a negative number, as seen in the first example) to the first run of that particular process.)

Now lets see how to calculate Turnaround Time:

1. Take the arrival time of a process and subtract it from the time it completed.

For example:

• Process A: arrival time = -4, completion time = 5 → TT time = 5 - (-4) = 9
• Process B: arrival time = -3, completion time = 7 + 1 context switch → TT time = 7 - (-3) + χ = 10 + χ.
• Process C: arrival time = -2, completion time = 16 + 2 context switches → TT time = 16 - (-2) + 2χ = 18 + 2χ
• Process D: arrival time = -1, completion time = 20 + 3 context switches → TT time = 20 - (-1) + 3χ = 21 + 3χ
• Average TT time = (9 + 10 + χ + 18 + 2χ + 21 + 3χ) / 4 = 14.5 + 1.5χ

(just a reminder, the turn around time for a process is the distance from the arrival time to the completion of that particular process.)

From the previous example:

• The utilization for this FCFS scheduler is: ρ = 20 / (20 + 3χ) (ρ is the symbol used to represent utilization)
  • (20 is the total number of time units, and 3χ is the number of context switches)

It should be noted that this is the best possible utilization for this set of processes. In fact, FCFS scheduling has the maximum possible utilization a scheduler can have! Why you ask? Because this type of scheduler performs the least number of context switches required to complete these tasks. Even though this scheduler has the best possible utilization, it doesn't necessarily mean it is the best scheduler for these processes. From the diagram above, it is easy to see that this type of scheduler is unfair to certain processes (like process D - it has to wait the most amount of time to get started, even though it has a relatively small service time of 4). Is there a scheduler that is more fair than a FCFS scheduler? Yes, be we'll discuss this later. First, lets look at some sample code of a FCFS scheduler.

	Struct process_t {
		int state;
		int arrival;
	}
 
	process_t *fcfs_schedule() {
 
		process_t  *min = NULL;
 
		for each process descriptor p {
			if ( p -> state = = RUNNABLE)
				if (min_p || min_p -> arrival > p -> arrival)
					min_p = p;
		}
		return min_p;
	}

The efficiency of this scheduler is O(n) (linear time), because it will go through each of the processes once. Later in the notes, we'll see a type of scheduler that can be implemented in O(1), or constant time.

Now let us look at how our schedule is affected by incoming requests. Lets say that from time t to time t + Δ, we gain requests with a total service time of T. The number of requests coming in will determine whether our turnaround time (TT) is bounded or not. Lets look at different scenarios and see the affects:

• If T > Δ then the scheduler is over-committed, meaning there isn’t enough time to process all the requests, because the average turnaround time grows without bound. The general solution to this problem is simply to purchase new hardware so that the scheduler can process all the requests.
• In the opposite case where T <= Δ the scheduler is under-committed, and the turnaround time stays bounded. This is important because in the real world processes tend to come in bursts. For example, a scheduler will receive 10 or so requests within the time frame of perhaps 20 cycles and then have a period of time perhaps 500 cycles where no requests are sent. As long as the scheduler is able to finish the 10 requests during the period of time where no requests are made, processes will generally finish in a decent amount of time. If they are unable to finish in time (as in the over-committed case) then processes will begin to take longer and longer then they normally would to finish.

In terms of waiting time, it's useful to look at the worst possible schedule, which is C – A – D – B (using the same times as above):

The reason the above scheduling is indubitably horrible lies in the fact that C is the longest process and yet it is placed first in the process line. Since it is ordered in this manner, the average wait time is maximized, meaning that each process has to wait a very long time before it is allowed to execute itself. If a computer always chose this kind a "worst" scheduling, your computer would lag up more often than it ever could have before. If this ordering of the processes maximized the wait time, what type of scheduling would be fairer and could minimize the average wait time?

The next possible scheduling method is Shortest job first. In the shortest job first algorithm, the scheduler picks the request with the smallest service time ( τ ), and queues up all other processes in order of ascending τ.

Using the same set of processes from our FCFS scheduler, we can get an idea of the affects this scheduler has on WT and TT:

Here is the Gnatt Chart for our SJF scheduler. Compare this to the FCFS scheduler and notice how the wait times all of processes have decreased!

The waiting time of 7.25 + 1.5χ is the best possible time for cooperative scheduling. This makes intuitive sense because in a SJF scheduler, each job waits less time on other processes than in any other arrangement of the processes.

Again, the shortest job first scheduler always gives the best waiting time. Please read the following section to see the proof that proves the shortest job first scheduler will always give the best waiting time.

Before we prove that a SJF has the lowest average waiting time for a schedule of processes, it should be noted that this is a proof by contradiction. Now let's start the proof.

Assume that there is a shortest job first (SJF) schedule S, where the optimal schedule (i.e., the schedule with the shortest average waiting time) is a non-SJF schedule S’. Without loss of generality, we assume that all jobs arrived at the same time, and that S and S’ schedule the same jobs at first (say A-F). Then at some point S’ picks a job different from the job picked by S:

S	A	B	C	D	E	F   G [....] K  ...
S’	A	B	C	D	E	F   K [...] G   ...

Since S is SJF, we know that G has a service time less than that of K. Now switch K and G in S’ we will become the following schedule S’’:

S’’	A	B	C	D	E	F   G [...] K   ...

Looking at the timeline above, it is clear that S’’ has a better average wait time than S’! (Why? Processes A-F have the same wait times as before, as do all processes after the G...K block. Within the G...K block, though, the wait times for X, Y, and Z went down, and the sum of the waiting times for G and K went down too! Thus the average waiting time went down.) But this contradicts our assumption that S’ was optimal!! Therefore, any schedule optimal for waiting time must be equivalent to some shortest job first schedule.

Our shortest job first scheduler is not without problems. First of all, its hard to detect how long a process will run (i.e. what the service time will be for that process). Also, one of the major problems of the SJF scheduler is that it can lead to starvation if “short” processes keep arriving faster than the schedule can process the current processes. What other scheduling techniques can be used to deal with these problems of starvation and fairness? (Hint: Take a look at the next section...)

A another scheduling algorithm is the Round Robin Scheduling, in which there is a scheduling quantum Q, which is the amount of time between timer interrupts. In other words, Q is the maximum amount of time a process can run before yielding the processor to the next process.

Example: Q = 2

Notice how a process only runs for 2 units of time, then yields control so the next process can run. Since no single process hogs the computer's resources, this scheduler can be said to be much fairer than a FCFS scheduler or even a SJF scheduler (just because C takes the longest to complete doesn't mean it should have to wait the longest).

In this case, the utilization is ρ = 20 / (20 + 10χ). As you can see, there are many more context switches in the round robin scheduler, meaning there will be a lower overall utilization (when compared to FCFS scheduler). This doesn't necessarily mean the scheduler is bad...there is just a trade-off of performance for robustness in this case. Robustness is enhanced since a process has less of an affect on when a different process will run. As in the case of a FCFS scheduler, a very long process could delay when other processes ran, which, in a slight way, violates process isolation.

It should be well noted that the Round Robin Scheduling will not suffer from starvation as long as the new jobs are added at the end of the round robin order. If new jobs were put in the front of the ordering, these new processes would be the first to gain control, and if there was a constant stream of new processes, then older processes may starve! Let's look at an example where the quantum number is bumped up to 3. Notice how the average wait time increases, but the overall utilization and context switches decrease (TT also increased, but this isn't always the case).

Ex. Q = 3 total context switches goes from 10 to 7.

• Average Turnaround Time = 16.25 + 4.5χ
• Average waiting time = 6.5 + 1.5χ
• Utilization ρ = 20 / (20 + 7χ)

Note that the general trend with RR schedulers is that a higher Q will yield a higher ρ. With a higher Q the Round Robin Scheduler becomes to look more and more like a FCFS Scheduler finally reaching a point where Q is so large that it is as large or larger than the longest process in which case the Round Robin Scheduler has become a First Come First Serve Scheduler.

One of the key applications where a good scheduler is needed is a web server, which has to process thousands of requests. Here is some code of a web server:

	Struct web_request {
		Int conn_fd;
		Char *filename;
	}
 
	web_request_t * scheduler(){
		….
		….
	}

Here are some questions related to the code that a server administrator might ask, to select the best possible scheduler:

1. Is it a file or a dynamic request?
2. Is the file in the cache?
3. How long did the last request take?
4. What is the file size?

Another possible scheduling method is the scheduling with priority, in which each request is assigned a priority p. Then the processes are scheduled in round robin fashion, where the process with the minimum priority value is scheduled to run (note it is the minimum value because in operating systems, a smaller the priority value means the process has higher priority). Another way to think of these values is in the informal term of niceness. Processes who have a high priority value are very nice, as they allow for all other processes to go before them. Conversely, those who have a very low priority value are mean (the opposite of being nice) and want to execute as soon as possible. Take a look at the example below that uses priority scheduling.

Example: Q = 3

• Average Turnaround Time = 17.5 + 4.75χ
• Average waiting time = 11 + 3.5χ

This method can also suffer from starvation, because low priority processes can “starve” if high priority processes keep arriving. It should also be noted that the TT and WT of schedulers with priority are generally higher (not very good) because of the fact that certain processes MUST be run sooner then others. In this example, the longest process runs first and continues to run until completion. This system can cause starvation and is unfair to high priority processes. What is a possible fix or improvement that we can make in order to increase fairness and take away starvation?

One answer lies in a priority algorithm called Priority with Aging. In this kind of scheduling, starvation can be solved if there is a time attached to each request. Priority with Aging takes priority scheduling and implements dynamic priority, thus allowing the priority of individual processes to change based on time. This is done by either one of two methods:

1. Reducing priority value for priorities that have been waiting a long time or
2. Increasing priority value for processes that have been running a long time.

This in practice is a very good solution, and there are many different algorithms out there to determine how fast or slow certain processes' priorities should change. In the Linux version of Priority with Aging, the algorithm runs in O(1), or constant time!

Now that we've found a good scheduler, is there anything else to worry about? What if one process with a certain priority is waiting on another process of a different priority? What happens in this case? This leads us into our next issue, which is called Priority Inversion.

Besides starvation, the priority method also faces issues of priority inversion, which occurs when a process with high priority is depended on a process that has a lower priority. Priority inversion is the term coined to the phenomenon that happens when a process with high priority effectively runs at a lower priority due to another process. In order to see how this can happen, check out the example below.

P1 => Process 1 with priority value 20 (very //low// priority!)
P2 => Process 2 with priority value -20 (very //high// priority!)
[linux@host]$ P1 | P2

In this example, P2 effectively runs at P1’s priority level, because the pipe never fills unless P1 fills it! So even though P2 is a very important process, it runs as if it is the least important (lowest priority) process.

The best solution for priority inversion is dynamic performance, meaning we should change the priority of the process being waited on (i.e. P1) to the same priority value as P2 (or even to the minimum priority value). In other words, while P2 waits for P1, we should set P1’s priority to the minimum value. Once P1 has run, it's priority value should be set back to it's normal priority. This last step is important, since we don't want the low priority process to be running as a high priority process unless it absolutely needs to run (as in the case of Priority Inversion).

The schedulers that have been introduced so far are considered best effort schedulers, as they strive to run processes as best as they see fit. The only problem with these kinds of schedulers is that they do not guarantee when a process will run. While this does not matter is most situations, there are times when it is important for a process to run at a particular time, or times when a process should be denied altogether. Another array of schedulers that account for these situations are called real time schedulers, and these schedulers guarantee when a request/process will run.

Here are two types of reat time schedulers and associated examples of when they might be necessary:

• hard real time: airplanes, fuel injection → there are serious consequences if the scheduler fails or places these tasks out of order.
• soft real time: cd player → there are less serious consequences if the scheduler fails.

There are a couple different real time schedulers in today's working world. One of them is the admissions control, which is a scheduling algorithm that can deny a particular request altogether. This kind of scheduler is used by NASA when a shuttle is in it's launching sequence: while those rockets are being fired, there shouldn't be any process that can interrupt it. Another kind of scheduler is the deadline scheduling, where every request has a service time τ and a deadline D, and each request must be completed by time D. The example below shows how a earliest deadline first scheduler should work.

Earliest Deadline First, as the title infers, is a type of scheduling that is given both the general time it would take to service an item and the deadline by which the process must be completed. In general, the scheduler looks at the deadline of each process and sees which needs to be finished first; then it schedules them in that order. If, however, it is impossible for the scheduler to finish the process by the given deadline, it will tell the processor that it is impossible and will not schedule the said process.

In summary, we have looked at various forms of scheduling of processes. From the most simple (FCFS) to those who account for more options (Priority with Aging). We understand that if we don't implement a good scheduler, it is possible that processes will starve and never be able to complete. Let's look one last time at the different kinds of schedulers, just for good measure:

• First Come First Serve Scheduling the most simple scheduler does not starve, but it suffers from long Wait Times and Turnaround Times. In reality though as processes usually come in bursts as long as the processes can be scheduled and finished within bursts the penalty of WT and TT in FCFS is minimal.

• Round Robin Scheduling if implemented correctly (New processes go at the end) also does not suffer from starvation, it however suffers from major Turnaround Time. The Waiting Time however is very low as processes are allowed to start, but do not finish until much later. It is good to note that as Q increases a RRS becomes to look more like a FCFS until Q is greater then the service time of the longest process and then the RRS is exactly the same as the FCFS

• Strict Priority Scheduling is the big change to allow for both lower Waiting Time and Turnaround Time. The scheduler assigns a priority value to each process and schedules them accordingly. However if this scheduler is implemented incorrectly it allows for starvation of processes.

• Priority Scheduling with Aging if implemented correctly allows for the benefits of Priority Scheduling as well as having no starvation. Most operating system use some form of this scheduling technique.

Here are the key terms and idea’s from lecture that were covered. If you still don’t understand these terms, go back and read them over in these notes!

• First Come First Serve (FCFS) Scheduling
• Cooperative Scheduling
• Starvation
• Shortest Job First
• Turnaround Time
• Waiting Time
• Round Robin Scheduling
• Priority

Here are a few links that dive further into the topics we've covered in these notes. Click on any of these links for your viewing pleasure:

• Common Types of Schedulers
• Starvation
• First Come First Serve Scheduling (Also known as First In First Out)
• Shortest Job First
• Round Robin Scheduling
• Linux Process Scheduler

That's all for scheduling folks. Please tune in to the next lecture's notes, as they will begin to tackle synchronization and coordination issues.