MIPS CPU with Some or All Interrupts Off
As we'll see later in this series, an interrupt routine exits exception mode but continues to run with at least some interrupts disabled.

Running with all interrupts disabled is a costly but effective way of getting a single CPU to be nonpreemptive (the longest time software spends with interrupts disabled determines your worst-case interrupt latency, and every device driver with a real-time constraint must budget for it). And of course it doesn't prevent re-entrance where there's a second CPU at work.

The simplest, shortest kind of ISR may opt to run to completion without ever re-enabling interrupts - Linux can support this and calls it a fast interrupt handler. You get that behavior by setting the flag SA INTERRUPT when registering the ISR. But most run for a while with higher-priority interrupts enabled.

Potentially, you can get a stack of interrupts interrupting interrupts. Infinite recursion (and stack overflow and an inevitable crash) can't happen because Linux makes sure you can stack up at most one entry at each distinct interrupt level. The amount of data saved at each level must be small enough that the maximum stack of interrupt save information will not overfill a thread's kernel stack.

Interrupt Context. After an interrupt, even after the interrupt handler has re-enabled most interrupts and built a full C environment, interrupt code is still limited because it's borrowing the state (and kernel stack) of whichever thread happened to be interrupted.

Servicing an interrupt is someone's business, certainly, but it has no systematic relationship with the thread that is executing when the interrupt happens. An interrupt borrows the kernel stack of its victim thread and runs parasitically on that thread's environment. The software is in interrupt context, and to prevent unreasonable disruption, interrupt-context code is restricted in what it can do.

One vital job done by the kernel is the scheduler, which determines which thread the OS should run next. The scheduler is a subroutine, called by a thread; in some cases it's called by a thread in interrupt context. Once the interrupt context part of an interrupt handler can get to the point where the hardware's immediate needs are met, it can (and often does) schedule a thread that will complete the interrupt-handling job, this time in thread context.

Executing the Kernel in Thread Context
You can arrive in the kernel in thread context either when an application has made a voluntary system call or a forced call for resources on a virtual memory exception (and the system call or VM exception has emerged from its lower layers), or as a result of a reschedule - which is, in turn, always either caused by an interrupt or by another thread voluntarily rescheduling itself because it's waiting for some event.

System calls are a sort of "subroutine call with security checks." But a range of other exceptions - notably virtual memory maintenance exceptions - are very much the same, even though the application didn't know this particular system call was necessary until it got the exception.

Not every thread is an application thread. Special threads with no attached application can be used to schedule work in the kernel in process context for device management and other kernel functions.

Thread context is the "normal" state of the kernel, and much effort is spent making sure that most kernel execution time is spent in this mode. An interrupt handler's "bottom half " code, which is scheduled into a work queue (as noted earlier), is in thread context, for example.

Dominic Sweetman is a software/hardware boundary expert based in London, England, who previously served as managing director at Algorithmics Ltd.