Lecture 14 Scribe Notes: More File Systems, Virtual Memory

By Nathaniel Dang, Konstantin Rogachyov, and Brian Wallerstein

Last lecture, we discussed the scenario in which we couldn't move a file from one directory to another without potentially losing the file. This was due to the fact that if the PG&E company were to cut the power during the move operation, this could result in the violation of one of the file system invariants, in particular one of the top three invariants (the fourth invariant is a "weak" invariant, resulting only in a space leak). This is a major robustness problem!

Reiterating the file system invariants again:
1) Every block must be used for a single purpose.
2) Every referenced block is initialized.
3) No referenced block is free.
4) Every unreferenced block is free (weak) (counts at most the number of links)

File System Ordering:

Sample Inode Block:

Sample File System Setup:

Let's try removing a file, the proper operations are:
A) Clear directory entry
B) Decrement link count
C) If the link count is 0, then mark the blocks as free.

Example: Removing hello.txt

A) Clear Directory Entry

Example: Removing hello.txt:

B/C) Decrement Link Count and Free Blocks (if Number of Links = 0)

Example: Removing hello.txt:

After the commit is performed, the file is removed from the file system!

Question: What would have happened if we had performed step C) first and then PG&E decided to cut the power?
Answer: We would have violated the third invariant! We can see that we must be careful in choosing the order of operations.

Now, our goals in moving a file from one directory to another are:
1) Always preserve invariants 1-3.
2) No matter when a crash might occur, we want the file to exist under at least the old or new file name.


Steps for properly moving a file from one directory to another
A) Increment link count
B) Create new directory entry for the inode
C) Delete the old directory entry
D) Decrement the link count


Example: Moving hello.txt

A) Increment Link Count

Example: Moving hello.txt

B) Create New Directory Entry for the Inode

Example: Moving hello.txt

C) Delete the Old Directory Entry

Example: Moving hello.txt

D) Decrement the Link Count

Example: Moving hello.txt

After the commit is performed, hello.txt has been moved from the root directory to root/subdir!

One Idea: Recover lost space by running a file system checker program.

1. Examine the file system.
2. Search for any invariant violations.
3. Fix these violations.

Another Idea: Put the file systems operations in a LOG.

GOLDEN RULE OF ATOMICITY: Never modify the only copy of a file.

To create atomicity for file system operations, MAKE A COPY!

Write Ahead Logging:

• Make a copy of a file system operation
  1. Write results of operation to a LOG (or Journal).
  2. Write a COMMIT RECORD to the log.
     • The preceding portion of the log is a faithful copy of the file system after the operation.
     • We are not allowed to write the COMMIT RECORD until we know that the log is right.
  3. Copy the results into the main file system.
     • We are not allowed to write into the main file system until we know that the commit record is right.
  4. Cancel the commit record.
     • We are not allowed to commit the record until the results are copied into the main file system.
• After we reboot from a crash:
  1. Ignore all of the log entries that do not have a commit record.
  2. Replay all of the log entries that do have a commit record.
• Disadvantage:
  1. Twice the number of writes are needed!
• Advantage:
  1. Most of the writes are contiguous!
• Example:
  1. An example of write ahead logging can be seen above in the Removing a File section and/or the Moving a File section.

IDEMPOTENT Operations: The result is the same no matter how many times the operation is applied.

• Example: x = 1 will always have the same result no matter how many times it is applied, whereas x++ will have a different result each time.

Example:

Step One:

Step Two:

Step Three:

Even if a crash occurs, it's okay because the correct results are in the log anyway!

We want to ensure process memory isolation!

Robustness

1. Segmentation
   • Help ensure isolation by telling architecture the BASE (low bound) and the SIZE (high bound) that the process is allowed access to.
   • The architecture has BASE and SIZE registers that delimit allowed memory accesses.
     • Dangerous instructions – changing the BASE or the SIZE!!
   • Disadvantage: external fragmentation
     • RT11 system solves this with compaction but compaction is difficult because of pointers.
       • After compaction all pointers become invalid.
2. Paged Virtual Memory
   • Allocate memory to processes in fixed size blocks called pages.
   • Hardware introduces a translation from process address space to physical memory.
     • That translation is controlled by the kernel!
     • It is called virtual memory because we are virtualizing the addresses.
   • Operation system defines a PAGE MAP FUNCTION
     • Physical Address = PageMap(Virtual Address)
     • Hardware uses this function whenever a process accesses memory
       • Example:
         • Process P wants to access address a
         • Physical Address(a) = PageMap(floor(a/PGSIZE)*PGSIZE) + (a mod PGSIZE)
     • On x86 – PGSIZE is 2^12 or 4KB.
     • Dangerous instruction – changing the page map function!
   • Example:
     • X86-like 2-level page table
       • Reg %cr3 holds the physical address of page directory.
       • The page directory (4096 bytes) is like the doubly indirect block in that it points to the page tables.
       • The page table is like the indirect block in that it directly points to 4096-byte pages.