Keeping Track of Modified Pages (Simulating "Dirty" Bits)
An operating system that provides a page for an application program to use often wants to keep track of whether that page has been modified since the OS last obtained it (perhaps from disk or network) or saved a copy of it. Unmodified ("clean") pages may be quietly discarded, since they can easily be recovered from a file system if they're ever needed again.

In OS parlance the modified pages are called "dirty," and the OS must take care of them until the application program exits or the dirty page is cleaned by being saved away to backing store.

To help out with this process, it is common for CISC CPUs to maintain a bit in the memory-resident page table indicating that a write operation to the page has occurred.

The MIPS CPU does not support this feature, even in the TLB entries. The D bit of the page table (found in the EntryLo register) is a write-enable and of course is used to flag read-only pages.

So here's the trick:

1) When writable page is first loaded intomemory, you mark its page table entry with D clear (leaving it read-only).

2) When any write is attempted to the page, a trap will result; system software will recognize this as a legitimate write but will use the event to set a "modified" bit in the memory resident tables—which, since it's in the EntryLo(D) position, permits future writes to be done without an exception.

3) You will also want to set the D bit in the TLB entry so that the write can proceed (but since TLB entries are randomly and unpredictably replaced, this would be useless as a way of remembering the modified state).

How the Kernel Services a TLB Refill Exception
MIPS's TLB refill exception has always had its own unique entry point (at least as long as the CPU wasn't already in exception mode; in Linux, that would be a fatal error, and isn't considered further).

When the exception routine is called, the hardware has set up EntryHi (VPN2) to the page number of the location we just couldn't translate: EntryHi is set exactly to what is required to create a new TLB entry to map the offending address.

The hardware has also set up a bunch of other address-related fields, but we're not going to use any of them. In particular, we're not going to use the convenient "MIPS standard" way of using Context to find the relevant page table entry.

The MIPS standard way requires that a (notionally very long) linear page table is constructed in kseg2 (it won't really take up excessive space, because kseg2 is mapped and the vast empty spaces of the table would never be mapped to real memory). But Linux really does not like to have the unique-to thread-group kernel mappings that would require.

Instead, Linux's TLB refill page tables are organized as a three-level hierarchy of tables (called "global," "middle," and just "PTE"). But cunning use of C macros allows the middle level to disappear completely without changing the code — a two-level structure is enough for 32-bit MIPS. (However, all three are required for the extended virtual memory space available with 64-bit MIPS.)

So you can find the data for any TLB entry you want - if it exists at all - by following the links, as shown in Figure 14.4 below.

Figure 14.4 Linux two-level page table (32-bit MIPS design).

This structure is reasonably economical of kernel memory: A largish Linux program with a 50-MB virtual address space has about 12-K 4-KB pages, each needing four bytes of PTE: That makes 48 KB or 12 pages.

That's an underestimate, because the address space components have holes in them, but a reasonable-size thread group's mapping needs take only 15 or so pages. Kernel (kseg2) mappings take more, but they're common to all memory maps, so there is only one set of PTEs for kernel mappings.

Most 32-bit CPUs are restricted to a 32-bit physical address, too: Where that's so, there are six unused high-order bits in the EntryLo0-1 registers. Linux recycles those to remember some software state about the page table entries.

This does mean that the TLB refill handler for this 32-bit needs to do two index calculations and three loads. It's certainly longer than the tiny instruction sequence of the original magic MIPS scheme (where the index calculation is done by magic by the way the Context register works, and the only loads are from the final page table), but it's not bad.

You may have noticed that this description is very particular to a certain sort of MIPS CPU. Do all Linux systems have a unique kernel?

Well, no: But in the current Linux/MIPS kernel certain critical routines  - including the TLB miss exception handler - are binaries generated by table-driven kernel software during start-up and tailored to the requirements of this particular CPU.