Operating System (2026 Fall)
  • Home
  • Policies
Operating System · CS3423 · Fall 2026

Worksheet 5

Last week each process had its own private memory. This week you will see how two processes exchange data without accessing each other's memory. They use a buffer that the kernel owns. That buffer is a pipe. You will see how a parent process uses dup2 and exec to connect two programs without modifying their source code. You will also study two ways a pipe can cause a program to block without an error message. A reader may wait forever, and a writer may fill the pipe. Read the worksheet, complete the three walkthroughs, and use the self-check at the end to prepare for the Oct. 14 quiz.

Nini the cat sitting in a box

Part 0Watch the lecture first

Watch: IPC & Pipe

The lecture covers file descriptors, dup(), dup2(), pipe(), and how the shell builds ls | wc. This worksheet assumes you have studied these topics. It focuses on problems that can occur when programs use pipes.

Part 1Two processes, one pipe

$ cat scores.csv | grep niko
niko,95

Recall the Unix principle from Worksheet 3, Part 6. Each program does one thing well, and multiple programs work together. Here cat produces lines and grep filters them. cat and grep are different processes. Since one process cannot read another process's memory, how does grep get data from cat?

The data pass through the kernel. Before starting the two programs, the shell requests a pipe from the kernel. A pipe is a small buffer in kernel memory with two ends. One end is for writing and the other is for reading. cat writes to the write end, and grep reads from the read end. The kernel copies the bytes into and out of the buffer. The pipe is not a shared array in either program. It is a kernel object, and each process accesses it through an entry in its own descriptor table, just as it accesses a file or the terminal. The processes therefore remain isolated. Every write() and read() is a system call, and only the kernel accesses the buffer.

cat · Writer private memory: code, stack, heap FD TABLE 0 → terminal 1 → pipe, write end 2 → terminal grep · Reader private memory: code, stack, heap FD TABLE 0 → pipe, read end 1 → terminal 2 → terminal user space above · kernel below KERNEL write end → [ m e o w ] → read end
Each process accesses the pipe through a descriptor number in its own fd table. The numbers differ (1 and 0), but both entries refer to the same pipe.

Asking for a pipe

A pipe is created by int pipe(int pipefd[2]) (see pipe(2)). The argument is an array of two integers that the caller provides. The kernel creates the buffer and then writes two new descriptor numbers into the array.

Meaning Typical value
pipefd, the argument An array of two int. The caller allocates it, and the kernel fills it. Nothing is passed in; the array is only a place for the two results. int fd[2]; on the stack
pipefd[0], after the call A new entry in the caller's descriptor table that refers to the read end. read() on this number takes bytes out of the buffer. 3, the lowest free number when 0, 1 and 2 are taken
pipefd[1], after the call A new entry that refers to the write end. write() on this number puts bytes into the buffer. 4, the next free number

One way to remember the two indexes is that 0 is for input, like stdin, and 1 is for output, like stdout. The example below creates a pipe and uses both ends from one process:

int fd[2];
pipe(fd);                    /* fd[0]: the read end · fd[1]: the write end */
write(fd[1], "meow", 4);   /* into the kernel's buffer */
read(fd[0], buf, 4);      /* out again: buf now holds "meow" */

A process writes bytes to fd[1] and reads them from fd[0]. The pipe delivers the bytes in the same order, and each byte is delivered once. A pipe carries data in one direction. The single-process example above does not exchange data between processes. fork() copies the descriptor table, so after a fork two processes hold both ends. One process can then write data that the other reads. The lecture showed this with meow between a parent and a child. The next program shows the full sequence the shell uses for cat | grep.

Redirecting a descriptor with dup2()

The second system call the program needs is dup2(). Its prototype is int dup2(int oldfd, int newfd) (see dup2(2)). The two arguments are both descriptor numbers in the calling process's own table, and they play different roles.

Meaning In dup2(fd[1], 1)
oldfd, the first argument An entry that already refers to the thing you want. It is the source. It is not changed by the call. fd[1], which is 4, the write end of the pipe.
newfd, the second argument The number that you want to refer to the same thing. It is the target. If entry newfd is already open, the kernel closes it first, so whatever it referred to before is dropped. 1, stdout. It referred to the terminal before the call.
return value newfd on success. On failure it returns −1 and sets errno, for example when oldfd is not an open descriptor. 1

After dup2(fd[1], 1) returns, entries 1 and 4 of the calling process both refer to the write end of the pipe. The order of the arguments is the common mistake. dup2(1, fd[1]) would do the opposite. It would make entry 4 refer to the terminal and drop the write end, so this process would no longer hold the write end. Just remember that dup2 copies oldfd onto newfd, in the same way that cp old new copies a file onto a new name.

Because entry 4 still refers to the write end after the call, the program closes it right afterwards. That close removes a number, not the write end. Entry 1 keeps the write end alive.

The shell's job for one |, in six system calls

The following C program performs the same steps as the shell for cat scores.csv | grep niko. Read the program carefully and notice the order of the six system calls you already know.

// 01-pipeline.c
/* Part 1: `cat scores.csv | grep niko`, built by hand.
 * pipe() -> fork() -> dup2() -> execlp(), twice. The shell does exactly
 * these steps for you every time you type a `|`. */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>

static void die(const char *msg) { perror(msg); exit(1); }

int main(void)
{
    int fd[2];                                  /* fd[0]: read end, fd[1]: write end */
    if (pipe(fd) == -1) die("pipe");

    pid_t p1 = fork();
    if (p1 == -1) die("fork");
    if (p1 == 0) {                              /* child 1: cat scores.csv */
        if (dup2(fd[1], 1) == -1) die("dup2");  /* stdout now goes into the pipe */
        close(fd[0]);
        close(fd[1]);                           /* fd 1 still refers to the pipe */
        execlp("cat", "cat", "scores.csv", (char *)NULL);
        die("execlp cat");
    }

    pid_t p2 = fork();
    if (p2 == -1) die("fork");
    if (p2 == 0) {                              /* child 2: grep niko */
        if (dup2(fd[0], 0) == -1) die("dup2");  /* stdin now comes from the pipe */
        close(fd[0]);                           /* fd 0 still refers to the pipe */
        close(fd[1]);
        execlp("grep", "grep", "niko", (char *)NULL);
        die("execlp grep");
    }

    close(fd[0]);                               /* parent: hold no pipe ends, */
    close(fd[1]);                               /* or grep never sees EOF     */
    if (wait(NULL) == -1) die("wait");
    if (wait(NULL) == -1) die("wait");
    return 0;
}
On the line with dup2(fd[1], 1)dup2(fd[1], 1) changes what descriptor number 1 refers to in this child's table. When cat writes to fd 1 to print something to the terminal it actually outputs into the pipe.
On the line with execlp("cat", …)execlp() belongs to the exec family. The l means the arguments are listed one by one. The p means the function searches for the program in PATH. Like every exec, it replaces the calling process's code and memory but keeps the fd table (except descriptors marked close-on-exec, which this program does not use). Because exec() preserves the descriptor table, the parent can connect programs without modifying their source code. The child sets up its descriptors between fork() and exec(), and the new program runs with those descriptors.
🔗
Walkthrough 1: Connecting two programs. Explains the 01-pipeline.c program above.
Open walkthrough 1 ▶ 21 steps · use ← → keys · press E for the explanation
Predict. When cat starts, what do fd 0, 1, and 2 refer to? What about grep? After both forks, which pipe ends should the parent still keep open?
Answer

In cat, fd 0 is the terminal, fd 1 is the pipe's write end, and fd 2 is the terminal. Its fds 3 and 4 were closed before exec.

In grep, fd 0 is the pipe's read end, and fd 1 and fd 2 are the terminal. Its fds 3 and 4 were also closed.

cat always writes to fd 1. Because fd 1 refers to the pipe's write end, the kernel puts the bytes into the pipe, and grep reads them from its fd 0. Neither program contains any code about the other. The connection exists only in their fd tables.

The parent should close both ends. It created the pipe only so that its children could get copies of the descriptors through fork(). The parent itself never reads from or writes to the pipe. If the parent forgets to close the write end, grep would never finish. It would keep waiting for more input that would not come.

Question. Does cat have to finish before grep starts?
Answer

No. After the parent's two forks, cat and grep are two runnable processes, and the scheduler decides who runs when. On a multi-core machine they run at the same time. If grep reads before cat has written, its read() finds an empty pipe and blocks. The kernel wakes it as soon as bytes arrive (walkthrough 1, step 20). If cat writes before grep reads, the bytes wait in the kernel's buffer. Either way the output is the same. This is also why a pipeline runs its stages concurrently. grep can process the first line while cat is still producing the last.

Case study: git starts its own pager

git log shows its output one screen at a time, through less. git (written in 2005) starts less (written in 1984) as a child process, and less reads from its stdin without any knowledge of git. There is one important difference from 01-pipeline.c. In 01-pipeline.c the parent only connects two children and produces no output itself. Here the parent is also the producer. After starting the child, git calls dup2() to redirect its own stdout. Every line git prints then goes into the pipe. The excerpt below is from git's source file pager.c

Optional reading: the dup2() calls inside git
/* git, pager.c, in setup_pager() */
    /* spawn the pager */
 prepare_pager_args(&pager_process, pager);
    pager_process.in = -1;
    if (start_command(&pager_process))
        die("unable to execute pager '%s'", pager);

    /* original process continues, but writes to the pipe */
   old_fd1 = dup(1);
 dup2(pager_process.in, 1);
  if (isatty(2)) {
        old_fd2 = dup(2);
        dup2(pager_process.in, 2);
    }
   close(pager_process.in);

pager_process.in = -1 asks git's start_command() to create a pipe, fork, connect the read end to the child's stdin with dup2(fd[0], 0), and exec less. Then the parent redirects its own stdout, and its stderr too when stderr is a terminal, into the write end. dup(1) keeps a copy of the original stdout so that git can restore it later. Exercise 5 traces these calls on a real git log.

Part 2stderr (fd = 2) prints debugging info

Suppose you give cat two files. You can read the first one, /etc/passwd. You cannot read the second one, /etc/shadow, because only root can read it. The output of cat goes through a pipe to grep:

$ cat /etc/passwd /etc/shadow | grep nini
Predict. cat cannot open /etc/shadow, so it prints an error message. What does grep receive through the pipe? Does it receive the error message too?
Answer
$ cat /etc/passwd /etc/shadow | grep nini
cat: /etc/shadow: Permission denied
nini:x:1001:1001::/home/nini:/bin/bash

grep receives only the contents of /etc/passwd. The shell called dup2() only for fd 1 of cat, so fd 1 refers to the pipe. fd 2, stderr, still refers to the terminal. cat writes the file contents to fd 1 and the error message to fd 2. The error message does not contain nini, but it still appears on the screen. This shows that it never went through grep. (The second line appears only if your system has a user named nini.)

cat · Writer FD TABLE 0 → terminal 1 → pipe, write end 2 → terminal (unchanged) grep · Reader FD TABLE 0 → pipe, read end 1 → terminal 2 → terminal KERNEL · THE PIPE "root:x:0:0:…" THE TERMINAL cat: /etc/shadow: Permission denied nini:x:1001:1001::/home/nini:/bin/bash
The pipe replaced only entry 1 of cat. Entry 2 still refers to the terminal, so the error message never enters the pipe. grep writes its matching line to the terminal through its own entry 1.

Programs on Linux typically write data to fd 1 and error and debugging messages to fd 2. Only the data written to fd = 1 is sent to the next command, while messages written to fd 2 still appear in the terminal.

Part 3Capturing a child's output

We often need to run a program and capture its output. For example, in a C programming course, the TA's online judge needs to capture the output of your C program to judge its correctness. The following Python code runs a command and captures its output:

import subprocess
r = subprocess.run(["./01-pipeline"], capture_output=True)
print(r.stdout)   # b'niko,95\n'

Internally, subprocess.run() does something similar to 01-pipeline.c. It creates pipes and forks a child process. The child redirects its file descriptors with dup2(), then starts the new program with execve(). Meanwhile, the parent reads the child's output from the pipe and waits for the child to finish.

This sounds straightforward, but the parent must handle the pipe carefully. Two common mistakes can cause the program to hang:

  • Bug 1. The parent forgets to close its own copy of the write end of the pipe. As long as this write end is still open, read() does not see end-of-file, even after the child has exited. The parent may therefore keep waiting for more data.
  • Bug 2. The parent calls wait() before reading from the pipe. If the child produces more output than the pipe can hold, the child blocks while trying to write. At the same time, the parent is waiting for the child to exit. Neither process can make progress.

Bug 1: the parent never sees end of file

A parent that collects a child's output must know when the output has ended. The following code shows you a common pattern to read all the data from a file (fd is an opened file descriptor):

char buf[1024];
ssize_t n;

while ((n = read(fd, buf, sizeof buf)) > 0) {
    /* Process the n bytes in buf. */
}
if (n == -1)
    perror("read");

Each successful read() returns the number of bytes copied into buf. The loop processes those bytes, then calls read() again. When read() returns 0, the loop stops because it has reached end of file, usually written EOF. The loop also stops if read() returns -1, which means an error occurred.

EOF is not a byte stored in the file. For a regular file, read() returns 0 when the file offset reaches the file's size.

A pipe does not have a file size that the kernel can use to detect EOF. Instead, read() returns 0 when the buffer is empty, and no process holds a descriptor for the write end. If any write descriptor remains open, more data may arrive. Therefore, when the pipe is empty, read() blocks instead of returning 0. The table below shows the three possible outcomes of read() on a pipe.

Buffer Write ends still held by anyone? What read() does
has bytes (doesn't matter) copies up to n bytes out and returns how many
empty yes blocks, because more bytes may still come
empty no returns 0, meaning end of file. Nothing can ever arrive

The kernel counts entries that refer to the pipe's write end across all processes' fd tables. It does not know which process will actually write. If at least one write descriptor remains open, more data may still arrive. Therefore, read() blocks when the pipe is empty and the count is above zero.

In the 04-capture.c code below, the parent uses a loop to collect the worker's output. The loop can end only when read() returns 0. This is made possible by the highlighted line, close(fd[1]). The correct program prints two lines and exits:

$ ./04-capture 1000
worker: wrote 1000 bytes, exiting
parent: child exited with status 0, captured 1000 bytes
The parent never writes, so leaving its copy of the write end open is ok, right?
// 04-capture.c (lines 12–44 of the file)


int main(int argc, char **argv)
{
    if (argc != 2) { fprintf(stderr, "usage: %s N\n", argv[0]); return 2; }

    int fd[2];
    if (pipe(fd) == -1) die("pipe");

    pid_t pid = fork();
    if (pid == -1) die("fork");
    if (pid == 0) {                             /* child: stdout -> pipe, run the worker */
        if (dup2(fd[1], 1) == -1) die("dup2");
        close(fd[0]);
        close(fd[1]);
        execl("./04-worker", "04-worker", argv[1], (char *)NULL);
        die("execl ./04-worker");
    }

    close(fd[1]);                               /* parent: only reads. REMOVE THIS LINE AND THE PROGRAM NEVER FINISHES */

    long captured = 0;                          /* read to EOF first ... */
    char buf[4096];
    ssize_t n;
    while ((n = read(fd[0], buf, sizeof buf)) > 0)
        captured += n;
    if (n == -1) die("read");

    int status;                                 /* ... then collect the exit status */
    if (wait(&status) == -1) die("wait");

    printf("parent: child exited with status %d, captured %ld bytes\n",
           WIFEXITED(status) ? WEXITSTATUS(status) : -1, captured);
    return 0;
}

In fact, commenting out that line will create a bug. The exercise repository contains 04-capture-noclose.c, which is 04-capture.c with that single line replaced by a comment. You can try running the buggy version.

Predict. What does ./04-capture-noclose 1000 print, and does it exit?
Answer

It prints the worker's line and nothing else, and it never exits. The parent reads the 1 000 bytes, and its loop calls read() again. The buffer is empty, the worker has exited, but the parent's own fd[1] still refers to the write end. The kernel counts one write end held, so it blocks the parent instead of returning 0. Only the parent could close that entry, and the parent is blocked inside read(). The program waits for itself, using no CPU, and wait() is never reached. In the correct version the count drops to zero when the worker exits, and the next read() returns 0 immediately.

🔚
Walkthrough 2: Empty versus finished. Follow 04-capture-noclose and see who holds each pipe end at every step, then the same run with the missing close restored.
Open walkthrough 2 ▶ 19 steps · use ← → keys · press E for the explanation

Closing the last write descriptor does not discard buffered data. If the worker writes its bytes and exits before the parent reads anything, the bytes stay in the buffer. The parent first receives those bytes, and only the next read() returns 0.

Bug 2: waiting before reading

A pipe has a capacity. The kernel's buffer is not unlimited. On Linux a pipe holds 65 536 bytes by default. 04-capacity calls fcntl(fd, F_GETPIPE_SZ) and prints the number for your machine. While there is room, write() copies the bytes in and returns immediately. When the buffer is full, write() blocks until a reader takes some bytes out. A blocked process uses no CPU. It waits until the kernel wakes it.

The 04-capture-broken.c below is just 04-capture.c but with the read loop and the wait() swapped. The parent calls wait() before reading the worker's output. Consider whether this order works for both small and large amounts of output:

// 04-capture-broken.c (lines 29–40 of the file)
    close(fd[1]);                               /* parent: only reads */

    int status;
    fprintf(stderr, "parent: calling wait() before reading the pipe ... "
                    "(hangs if N > pipe capacity; Ctrl-C, or run under timeout)\n");
    if (wait(&status) == -1) die("wait");

    long captured = 0;
    char buf[4096];
    ssize_t n;
    while ((n = read(fd[0], buf, sizeof buf)) > 0)
        captured += n;

Small output: fine, large output: deadlock

04-worker N writes N bytes to its stdout. With ./04-capture-broken 1000, the output fits in the buffer, so the worker can exit. Then wait() returns, and the parent reads 1 000 bytes followed by EOF (0). The program looks correct.

With ./04-capture-broken 1000000, the worker fills the buffer after 65 536 bytes. The worker blocks in write() until a reader removes data. The parent is blocked in wait() until the worker exits. Each process is waiting for the other. Nothing crashes, but the program remains blocked and never finishes. This is a deadlock. The small test case does not reveal the deadlock.

Deadlocks are common in large systems and can be difficult to debug. For example, a study of 105 real concurrency bugs in MySQL, Apache, Mozilla, and OpenOffice found that 31 were deadlocks (Lu et al., ASPLOS 2008).

The pipe deadlock above also occurs in real software. In 2026, AgentScope, a framework for building AI agents, had this bug (issue 1255, issue 2838). An agent ran a shell command and captured its output, but the parent waited for the command to exit before reading the output. Commands that produced little output worked. However, when a command printed more than 64 KiB, such as a large JSON document, the agent hung without an error message. When a time limit was set, the agent reported a timeout, which misled the developer to believe that the command was too slow. But it is not the command's problem. It is the parent's problem.

🔁
Walkthrough 3: The pipe deadlock.
Open walkthrough 3 ▶ 21 steps · use ← → keys · press E for the explanation

How to fix the deadlock: drain first before you wait

04-capture.c reverses the order of reading and waiting. The parent reads until read() returns 0, and only then calls wait(). The worker can continue writing because the parent removes data whenever the buffer fills. When the worker exits, its write end closes. After reading the remaining buffered data, the parent receives EOF, and wait() returns immediately. 04-capture.py shows the same bug and fix in Python. Calling p.wait() before p.stdout.read() causes the program to block on large output. Calling p.communicate() reads the output while waiting for the child to finish.

If a program captures a child's stdout and stderr through two pipes, it must read from both pipes. Otherwise, the child can block when the pipe that the parent is not reading becomes full. communicate() handles both pipes, so Python's documentation recommends using it.

Part 4Stopping a process

So far a process ended when its program returned from main or called exit(). But what if the program is stuck in an infinite loop or a deadlock? The process may no longer be able to stop itself. Unix provides signals so that the kernel or another process can notify it, interrupt it, or terminate it.

A signal is a small notification delivered to a process by the kernel. Each signal has a number and a symbolic name, such as SIGINT or SIGTERM. Some signals are generated automatically by the kernel. For example, writing to a pipe after every read end has been closed generates SIGPIPE. A process can also ask the kernel to send a signal to another process by calling kill(). The shell command kill just calls this system call to send a signal to another process.

When a signal is delivered, what happens next depends on the signal and how the program has chosen to handle it. A program can install a signal handler, which is a function that runs when a particular signal arrives. Some signals can also be ignored. If the program has not chosen either of these behaviors, the kernel performs the signal's default action. Depending on the signal, the default action may terminate the process, stop it, or do nothing. Some signals, such as SIGKILL, are special: a program cannot catch or ignore them.

Ctrl-C in the terminalSIGINT (2) kill PID · timeout · docker stopSIGTERM (15) kill -9 PIDSIGKILL (9) THE KERNEL delivers the signalto the target process the target process handler installed? the program's own function runs,then the program continues or exits no handler: default action the kernel terminates the process exit status seen by the shell: 128 + n SIGKILL: always this path
Three common signals.

There are three common signals:

Signal Sent by Default action Can a program catch it?
SIGINT (2) The terminal, when you press Ctrl-C. It goes to every process in the foreground pipeline, not only one. terminate Yes.
SIGTERM (15) kill PID, timeout, docker stop, systemd, and any other tool that wants a process to stop. terminate Yes. A program may catch it to finish writing a file or to remove temporary files, and then exit.
SIGKILL (9) kill -9 PID, or the same tools after a SIGTERM was not acted on in time. terminate No. The kernel ends the process without running any of its code.

A program can catch SIGTERM and use the signal as a request to shut down. This gives the program a chance to clean up resources, save state, or finish other work before exiting. In contrast, a program cannot catch or ignore SIGKILL. When the kernel delivers SIGKILL, the process must die now.

For this reason, process-management tools usually try SIGTERM first, giving the program a chance to exit cleanly. If the process does not respond, the tools can then send SIGKILL to force it to stop.

Real use case: timeout

Sometimes a program must finish within a fixed amount of time. Online judges are a common example. If your C program runs for too long, the judge stops it and reports TLE (Time Limit Exceeded). On Linux, the timeout command provides similar behavior. For example, timeout 10 ./prog runs ./prog with a 10-second time limit.

Conceptually, timeout starts the target program as a child process and then waits. At the same time, it keeps track of the time limit. Two outcomes are possible:

  • If ./prog exits before the time limit, timeout normally returns the same exit status as ./prog.
  • If the time limit expires first, timeout sends SIGTERM to the program. By default, timeout then reports exit status 124, which tells the caller that the command timed out. If you add -k 5, for example timeout -k 5 10 ./prog, timeout waits another 5 seconds after sending SIGTERM. If the program is still running, it sends SIGKILL.

Pattern: First send SIGTERM and give the program a chance to clean up and exit. Use SIGKILL only if the program does not stop.

Mom catches you playing computer games at midnight. First Mom says nicely, "Please save your game and turn it off." You can save and exit, or you can say "one more minute". This is SIGTERM. (>_<) If you are still playing ten minutes later, Mom does not ask again. She unplugs the computer. You cannot say anything, and your game is not saved. This is SIGKILL. (>x<) 

You should thank mom for the 10 minute grace period, because life doesn't usually give you a second chance.

Optional reading: the SIGTERM, then SIGKILL logic in coreutils timeout
/* coreutils, src/timeout.c, in the handler that runs when the timer expires */
if (sig == SIGALRM)
  {
    timed_out = 1;
    /* … */
    initialize_exit_failure (EXIT_TIMEDOUT);     /* 124 */
    sig = term_signal;                         /* SIGTERM unless -s was given */
  }
if (0 < monitored_pid)
  {
    if (kill_after)
      {
        /* Start a new timeout after which we'll send SIGKILL.  */
        term_signal = SIGKILL;
        settimeout (kill_after, false);
        kill_after = 0; /* Don't let later signals reset kill alarm.  */
      }
    /* … */
    send_sig (monitored_pid, sig);

timeout also receives a signal. It asks the kernel for an alarm. When the alarm expires, the kernel sends timeout a SIGALRM, which this handler catches. The handler sends the signal stored in term_signal to the child; the default is SIGTERM. If -k was given, the handler changes term_signal to SIGKILL and starts a second alarm. When the second alarm expires, the same function runs again and sends SIGKILL. send_sig() calls the kill() system call.

Optional reading: the same pattern in Docker, Kubernetes, and systemd

Larger tools that manage processes stop them in the same way as timeout. They send SIGTERM, wait for a fixed time called the grace period, and then send SIGKILL.

  • Docker runs a program inside a container, an isolated environment with its own files, network, and process IDs. docker stop sends SIGTERM to the main process of the container, waits 10 seconds, and then sends SIGKILL.
  • Kubernetes runs containers on many machines at once. It groups one or more containers into a pod. When it removes a pod, it sends SIGTERM, waits 30 seconds by default (terminationGracePeriodSeconds), and then sends SIGKILL.
  • systemd is the first process (PID 1) on most Linux systems. It starts and stops services, which are programs that run in the background, such as a web server or the SSH server. When it stops a service, it sends SIGTERM, waits 90 seconds by default (TimeoutStopSec), and then sends SIGKILL.

A well-written server catches SIGTERM, finishes the requests it is handling, saves its state, and exits within the grace period.

SIGPIPE, the signal a pipe sends

SIGPIPE occurs when a process writes to a pipe with no open read descriptors. The write cannot succeed, so the kernel sends the writer SIGPIPE (13). The default action terminates the writer. This is why yes | head -3 ends even though yes would otherwise run forever. It also explains why quitting less ends git log in exercise 5. A process downstream of the terminated process does not receive a signal for this event. Instead, the downstream process receives EOF after reading any remaining data, because the terminated process's write end closed when it exited. Exercise 8 shows both outcomes.

Part 5Practice in the terminal

All the programs below are in the exercise repository sys-nthu/os26-w5. File names start with the exercise number, so 04-capture.c corresponds to exercise 4.

Open in GitHub Codespaces

Run ./setup.sh to install the software dependencies, then run make to build everything. Several steps deliberately hang. Run them under the timeout command as shown. timeout 3 ./prog runs ./prog and stops it after 3 seconds if it has not finished, and the shell then reports exit status 124. You can also press Ctrl-C.

  1. Your program versus the shell. Run 01-pipeline and compare it with the shell command, then trace it. Match each traced call to a line of 01-pipeline.c:
    $ ./01-pipeline
    $ cat scores.csv | grep niko
    $ strace -f -e trace=pipe2,dup2,close,execve,clone,wait4 ./01-pipeline 2>&1 | grep -v ENOENT
    (The grep hides the execve … ENOENT lines, which appear because execlp tries each directory in PATH until it finds cat.) On Linux, fork() appears as clone in the trace, and pipe() appears as pipe2. Which process calls dup2(…, 1), and which process calls dup2(…, 0)?
  2. Same pipe, different numbers. You do not need to build a program for this exercise. Every process was created by a fork() in some other process, so each one has a chain of ancestors. pstree -ps $$ prints that chain for your shell ($$ is the shell's PID, -s asks for the ancestors, -p shows PIDs). In the Codespace, the chain ends with the VS Code server, its terminal host, and your bash. The server did to your shell exactly what 01-pipeline did to cat. It forked, set up descriptors, and called exec.
    $ pstree -ps $$
    systemd(1)───node(2311)───node(2402)───bash(2498)───pstree(2561)     # your PIDs will differ
    Linux also shows every process's fd table as a directory. /proc/PID/fd holds one symbolic link per entry, named after the descriptor number and pointing to the object that the entry refers to. Start a pipeline that stays alive for a while, find the two PIDs (pgrep -x name prints the PIDs of processes with exactly that name), and list both tables. Predict first: will the two processes use the same number for the pipe, and will they show the same pipe:[…] identifier?
    $ sleep 1000 | wc -l &
    $ pgrep -x sleep; pgrep -x wc
    $ ls -l /proc/$(pgrep -x sleep)/fd       # sleep: the writer
    lrwx------ 1 nini nini 64 Oct 14 10:00 0 -> /dev/pts/0
    l-wx------ 1 nini nini 64 Oct 14 10:00 1 -> pipe:[48213]
    lrwx------ 1 nini nini 64 Oct 14 10:00 2 -> /dev/pts/0
    $ ls -l /proc/$(pgrep -x wc)/fd          # wc: the reader
    $ kill %1
    Which entry number refers to the pipe in each process? Do the two pipe:[…] numbers match? The number in brackets is the pipe's inode number, which identifies that pipe in the kernel. What do the r and w permission letters on the two links tell you?
  3. A debug print on the wrong descriptor. Many programs communicate with their parent through stdin and stdout, just like 02-numbers and 02-sum. An editor communicates with a language server this way. An AI assistant also communicates with a tool server this way using the Model Context Protocol. The protocol specification allows a server to write only protocol messages to stdout. This exercise shows why. Run the two programs first without DEBUG, then with DEBUG set. Then edit 02-numbers.c so the debugging message goes to stderr (the two lines are marked), rebuild, and run again:
    $ ./02-numbers 5 | ./02-sum
    $ DEBUG=1 ./02-numbers 5 | ./02-sum
    $ make 02-numbers && DEBUG=1 ./02-numbers 5 | ./02-sum
    After the fix the debugging message still reaches your screen. Through which descriptor, and why did the pipe not capture it?
  4. Capturing a child's output, two bugs. First the missing close. Run it under timeout (exit status 124 means it was killed after 3 seconds), and while it hangs look at the parent's fd table from a second terminal:
    $ ./04-capture 1000
    $ timeout 3 ./04-capture-noclose 1000; echo "exit status $?"
    $ ./04-capture-noclose 1000 & sleep 1; ls -l /proc/$!/fd; ps -o pid,stat,wchan:12,cmd -p $!; kill %1
    Which entry of which process keeps the second read() waiting? Then the full pipe. Find the pipe capacity, then run the broken capture program with small and with large output. While the large run hangs, look at both processes from a second terminal. The wchan column names the kernel function where each process is blocked. Then run the fixed version, in C and in Python:
    $ ./04-capacity
    $ ./04-capture-broken 1000
    $ timeout 5 ./04-capture-broken 1000000; echo "exit status $?"
    $ ./04-capture-broken 1000000 & sleep 1; ps -o pid,stat,wchan:16,cmd -p $!,$(pgrep -x 04-worker); kill %1
    $ ./04-capture 1000000
    $ python3 04-capture.py 1000 broken; timeout 5 python3 04-capture.py 1000000 broken; echo "exit status $?"
    $ python3 04-capture.py 1000000 fixed
    What is the state letter of the two hung processes, and what are the two wchan values? Why does the 1 000-byte run succeed with the same code?
  5. git pipes itself into less. You do not need to build a program for this exercise. git log shows its output through a pager. Trace how:
    $ strace -f -o git.trace -e trace=pipe2,dup2,execve git log     # press q to leave the pager
    $ grep -E "pipe2|dup2|execve" git.trace
    The trace goes to a file because git only starts a pager when its stdout is a terminal. If you pipe its output into grep, git does not start a pager. Which process calls dup2(…, 0) and which process calls dup2(…, 1)? Note that the parent redirects its own stdout (and stderr) into the pipe. Here the parent is also the producer. One consequence is that if you quit less early, git is still writing into a pipe whose only reader is gone. The kernel then sends git the SIGPIPE signal, which terminates it without any message. That is why quitting a pager never leaves a producer running.
  6. Three signals, three exit statuses. You do not need to build a program for this exercise. Start a long-running process in the background three times, stop it with a different signal each time, and read the exit status that wait returns. Then print the table of signal numbers:
    $ sleep 1000 &
    $ kill -INT %1; wait %1; echo "exit status $?"
    $ sleep 1000 &
    $ kill -TERM %1; wait %1; echo "exit status $?"
    $ sleep 1000 &
    $ kill -KILL %1; wait %1; echo "exit status $?"
    $ kill -l
    When a signal terminates a process, the shell reports exit status 128 + the signal number. Subtract 128 from each status and find the number in the table. bash also prints a word for each, Interrupt, Terminated or Killed. Then run sleep 1000 | cat in the foreground, press Ctrl-C, and check $?. Both processes are gone. Which signal did they receive, and from whom?
  7. timeout sends a signal when the time limit expires. You do not need to build a program for this exercise. The -v option makes timeout say what it sends:
    $ timeout -v 2 sleep 10; echo "exit status $?"
    timeout: sending signal TERM to command ‘sleep’
    exit status 124
    $ timeout -s KILL 2 sleep 10; echo "exit status $?"
    $ timeout -k 2 -v 2 sleep 10; echo "exit status $?"
    Which signal does sleep receive in the first and in the second run? (Use the 128 + n rule from exercise 6 to read the 137.) In the third run, does the SIGKILL ever get sent, and why not?
  8. Optional: SIGPIPE upstream, end of file downstream. You do not need to build a program for this exercise. Run a three-stage pipeline in the foreground, kill the middle process from a second terminal, and look at the three exit statuses:
    $ yes | cat | wc -l          # in a second terminal:  pkill -x cat
    $ echo "${PIPESTATUS[@]}"
    141 143 0
    cat was terminated by pkill (143 = 128 + 15). yes was terminated by SIGPIPE (141 = 128 + 13) the next time it wrote. wc was not signalled at all. It read end of file, printed its count, and exited with 0. Why did wc see end of file instead of waiting for more input?

Self-checkLearning goals

Check each item that you can complete without consulting the worksheet. The Oct. 14 quiz covers these topics. (Progress is stored only in this browser.)

Reset checklist

AttributionCredits

  • The course illustrations of Nini are NTHU CS Operating Systems course material (Tony Chen / Yun-Chih Chen), licensed under CC BY 4.0. The cat art is from Freepik and ChatGPT. See images/CREDITS.md for sizes and sources.
  • The Model Context Protocol rule mentioned in exercise 3 is from its specification, section Transports: stdio.
  • The code excerpts in the case studies are from git, pager.c (GPL-2.0) and coreutils, src/timeout.c (GPL-3.0), lightly trimmed as noted. The grace periods are from the documentation of docker stop, Kubernetes pod termination and systemd.service(5). Signal numbers and default actions are from signal(7).
  • The pipe capacity and F_GETPIPE_SZ are described in pipe(7) and fcntl(2). Parts 3 and 4 draw on chapter 5 (Interlude: Process API) of Arpaci-Dusseau and Arpaci-Dusseau, Operating Systems: Three Easy Pieces, in the worksheet's own words.

Cat Left

Made with ❤️ by Tony, (CC BY 4.0)
Cat source: Freepik and ChatGPT

Cat Right