diff --git a/README.md b/README.md index 8da429c..e157875 100644 --- a/README.md +++ b/README.md @@ -15,10 +15,16 @@ web server. > > - Every behaviour described here is exercised by drills against a fixture > host. That is SYNTHETICALLY TESTED. -> - Nothing in this code has yet been run on a real host or registered with -> GitHub. That verification is the next stage, and its plan is in -> [docs/verification.md](docs/verification.md). -> - Do not run it on a machine you care about yet. +> - The full lifecycle has also run on **one real host**: an Ubuntu 24.04 +> aarch64 VM with two real GitHub runners. It went from install to +> registration, jobs, pool, busy guard, reboot, deregistration and +> retirement ([docs/acceptance/2026-10-arm64.md](docs/acceptance/2026-10-arm64.md)). +> - RunnerSlice's own drills ran there as GitHub Actions jobs, on runners +> RunnerSlice installed and managed. +> - **Not verified:** x86_64, other distributions, and GitHub-hosted CI for +> this repository. GitHub-hosted CI is currently blocked by the +> organisation's billing. +> - No release exists. Use a machine you can afford to rebuild. ## Why @@ -94,9 +100,10 @@ fleet with the runners idle, and register the new runner. ## Supported hosts Ubuntu with systemd 249 or later, cgroup v2 and x86_64 or aarch64 is what -preflight accepts. **No combination has been verified with this code yet.** -[docs/compatibility.md](docs/compatibility.md) is the only list of what has -been. +preflight accepts. What is supported is narrower: **verified so far, Ubuntu +24.04 on aarch64 (systemd 255), on one VM.** Preflight warns on any systemd +other than 255. [docs/compatibility.md](docs/compatibility.md) is the only list of +what has been verified. ## Documentation diff --git a/docs/acceptance/2026-10-arm64.md b/docs/acceptance/2026-10-arm64.md new file mode 100644 index 0000000..f952c4e --- /dev/null +++ b/docs/acceptance/2026-10-arm64.md @@ -0,0 +1,107 @@ +# Acceptance record: ARM64, 2026-10-15 to 2026-10-16 + +This is the first real-host run of the lifecycle acceptance defined in +[verification.md](../verification.md). Raw logs are kept by the maintainer +outside the repository, with credentials excluded. + +## Environment + +- **VM:** a disposable Ubuntu 24.04 aarch64 VM. Lima 2.2.1, Apple + Virtualization, 2 vCPU, 3 GiB RAM, 25 GiB disk, NAT networking, no shared + folders, no ports forwarded. Kernel 6.8.0, systemd 255. +- **Runners:** two repository-scoped runners, `rs-arm64-dogfood-01` (GitHub id + 2) and `rs-arm64-dogfood-02` (id 22). Both serve a private dogfood + repository and form one pool. Neither was used for anything else. +- **Inventory:** + - `runners.slice`, with 100% CPU, MemoryHigh 1G and MemoryMax 1280M (from + 1536M), 512 tasks, CPUWeight 50, and a deny on `1.1.1.1/32`; + - one masked directory of another service, and one protected secret file; + - the arm64 2.338.0 seed, with `apt-dependency liblttng-ust1t64`. +- **Registration and removal tokens** were obtained and entered by the + owner. They travelled through a pipe into `ACTIONS_RUNNER_INPUT_TOKEN`, and + were never shown, logged or stored. + +## Commits + +Each observation is recorded against the commit installed when it was made. +The installed tree was compared with `git archive` of that commit, file by +file. + +| Commit | What it was used for | +|---|---| +| 6adadd3 | First install. Preflight refused the group-writable tree, which led to #16. | +| c52630f, 364a9c3 | Preflight, first apply, slice limits, deny on reload, killed download and unpack, retiring an unregistered runner. | +| f28b6a5 | The registration of runner 1, by the command it printed. | +| 9208151 | Runner 1 and runner 2 live, all dogfood jobs, pool, busy guard and reboot. | +| 2bc81c0 | Final retirement, after both runners were deregistered. | +| 7f84c01 | The #18 follow-up below. No runner was registered. | + +## Dogfood jobs + +All ran on the self-hosted runners, with GitHub-hosted minutes exhausted and a +$0 Actions budget in place. **Billing did not stop self-hosted jobs.** + +| Run | Mode | Runner (id) | Result | +|---|---|---|---| +| 38895433749 | drills | 2 | success: shellcheck 20 clean, 436 cases, commit 9208151, tree 57413b0 | +| 38895647330 | boundary | 2 | success, 12/12 | +| 38899375262 | drills, `target=dogfood-2` | 22 | success, 436 cases | +| 38899378922 | drills, pool, concurrently | 2 | success, 436 cases | +| 38899791941, 38899805349 | boundary on each runner | 22, 2 | success, 14/14 each | +| 38899906623, 38899920558 | hold, during the busy-guard test | 22, 2 | success: not interrupted | +| 38901272065, 38901282745, 38901413328 | boundary after the guest reboot | 22, 22, 2 | success, 14/14 each | +| 38901401020 | hold, during retirement | 22 | success: retirement waited for it | + +## Defects found on the real host, and fixed + +Each fix landed with a drill case that fails without it. + +1. **The release archive unpacked group-writable**, and root's preflight + refused it (#16). +2. **`systemctl show` prints `RestrictNamespaces=yes`, not an empty value.** + A correct host would have failed its own check (#17). +3. **The printed registration used `--replace`**, which silently takes over + another host's runner of the same name (#17). +4. **`config.sh` checks its libraries by `./bin` paths before it changes + directory.** Run from elsewhere, the check passed vacuously, with `ldd: … + No such file` (#17, `env -C`; actions/runner #945 and #4460). +5. **`config.sh remove` rejects `--unattended`** (#17). + +## Follow-up: the post-acceptance review (#18), 2026-10-16 + +The same VM, after the acceptance had ended and the host matched its +baseline. Commit 7f84c01 (tree ae6c64a) was installed. All 49 files in the +archive were compared with the commit and are identical; root owns the whole +tree. No runner was registered. Every case used a new test-only profile, +`probe`, which was never registered, and new test-only objects. Every +temporary host change was undone. Afterwards, `/etc/passwd`, `/etc/group` +and `/etc/login.defs` matched their checksums from before, byte for byte, +and the host listing matched its state from before. + +| Case | Observed | +|---|---| +| Preflight on systemd 255 | `ok systemd 255`. No "unverified" warning, because 255 is the supported version. | +| The runner directory during the unpack | 232 samples, in this order: 750 (created by apply), then 700 (for the whole unpack), then 750 (after chmod). The archive's 0775 never appeared. | +| A symlinked `bin/` in the runner directory | `check`: `cannot determine the installed runner version`. With the real `bin/`: `runner version 2.338.0`. | +| A real process running as the account (`Uid: 997 997 997 997`) | `retire` refused, named the pid, and removed nothing. | +| A process holding the account only as its saved uid (`Uid: 0 0 997 0`) | `pgrep -u`, the old check, found nothing (exit 1). `retire` refused, named the pid, and removed nothing. | +| `userdel` failing, with `/etc/passwd` locked | `userdel could not remove rs-probe (exit 1); the account is still there`. The account, group, home, runtime root, marker and unit were all still there. | +| The account removed, but an immutable file in the runtime root | `could not remove all of /var/lib/rs-probe`. The unit and the marker were kept. After `chattr -i`, the next `retire` removed the root, then the unit, and exited 0. | +| `USERGROUPS_ENAB no` | `userdel` left the group. RunnerSlice removed it as the account's own unused group (gid 987). Exit 0. | +| The group with a member, and a home not owned by the account | `userdel` exited 12. The leftover home was reported, and not removed. The group was left, because it has a member. Exit 1. On a second run, both were reported again. The group was now "without its account", and was still not removed. After a manual cleanup, as the recovery table describes, `retire` exited 0. | + +Not produced on the real host: an unreadable `/proc` (root can always read +it), and a `.runner` that is a FIFO, oversized or deeply nested. Those remain +SYNTHETICALLY TESTED. The drills ran inside the VM as `nobody`: 504 cases, +0 failed. The release drill skipped, because it needs a git checkout. + +## Not verified here + +- Unknown job state on a real host. It cannot be produced safely as root on a + healthy host, so it remains SYNTHETICALLY TESTED. +- A misregistration on a real host. It remains SYNTHETICALLY TESTED. +- A runner self-update. +- `runner-slice status` on the real host. The VM had no authenticated `gh`, so + GitHub's view was read from the maintainer's machine instead. The command + itself was not run there. +- x86_64, and any other distribution or provider. diff --git a/docs/compatibility.md b/docs/compatibility.md index 5d2522c..504bd2c 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -9,14 +9,37 @@ actually been verified, and at what level ([verification.md](verification.md)). | Distribution | Architecture | systemd | Status | |---|---|---|---| | Ubuntu 24.04 | x86_64 | 255 | **Planned** for the first verification VM. Not yet verified. | -| Ubuntu 24.04 | aarch64 | 255 | **Planned** for the ARM64 verification VM. Not yet verified. | -| Ubuntu 22.04 | aarch64 | 249 | The code RunnerSlice was extracted from has run here in production (see [field-report.md](field-report.md)). **This code** has not been run there. | +| Ubuntu 24.04 | aarch64 | 255 | **Verified** on a disposable VM (Lima, Apple Virtualization), at RUNTIME VERIFIED and REBOOT VERIFIED. See [acceptance/2026-10-arm64.md](acceptance/2026-10-arm64.md). The lifecycle was run with two real GitHub runners, ending in deregistration and retirement. One VM; no other aarch64 host verified. | +| Ubuntu 22.04 | aarch64 | 249 | The code RunnerSlice was extracted from has run here in production (see [field-report.md](field-report.md)). **This code** has not been run there. Not supported. | | Any other distribution | | | Refused by preflight. | +## systemd: accepted is not supported + +Two ranges differ, on purpose, and neither stands in for the other. + +| | Versions | Meaning | +|---|---|---| +| **Supported** | 255 | Verified on a real host by the acceptance above. | +| **Accepted, unverified** | 249–254, and 256 or later | Preflight lets these through, and warns, naming the version as unverified. Nothing about them has been observed on a host. | +| **Refused** | older than 249 | Preflight refuses. | + +Why the code accepts more than is supported. The floor of 249 is what the +units rely on: `ProtectHome=tmpfs` with `BindPaths`, and a slice's +`IPAddressDeny` merged into every unit inside it. The one value `check` was +found to depend on in a version-specific way is how `systemctl show` prints +`RestrictNamespaces` (`yes`, `no`, or a list). That printing code +(`src/shared/bus-print-properties.c`) was read, not run, in the v249, v252 +and v255 sources and is the same in all three. That is a source review. It +is not verification, and it covers only that one property. Raising the floor +to 255 would refuse hosts that nothing shows are broken. Calling them +supported would claim what nothing shows works. So they are accepted with a +warning, and only 255 is listed as supported. + Preflight refuses: - a distribution other than Ubuntu; -- systemd older than 249; +- systemd older than 249 (249–254 and 256 or later pass with a warning that + they are unverified); - a host without the cgroup v2 unified hierarchy, or without its `cpu`, `memory` or `pids` controller; - an architecture other than the release seed's. diff --git a/docs/contract.md b/docs/contract.md index dc1607a..d009265 100644 --- a/docs/contract.md +++ b/docs/contract.md @@ -151,7 +151,8 @@ is. It remains a candidate for its own project (decision 0004). Only what has been verified is listed, in [compatibility.md](compatibility.md). Preflight refuses anything other than Ubuntu with systemd 249 or later and -cgroup v2. +cgroup v2. It accepts more than is supported: any systemd other than 255 +passes with a warning that it is unverified. ## Security position diff --git a/docs/installation.md b/docs/installation.md index 90b6377..e3436b1 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,16 +1,16 @@ # Installation -> **Status: pre-release.** These steps follow what the code does and are -> exercised by drills against a fixture host. They have not yet been carried -> out on a real host. That is the next verification stage -> ([verification.md](verification.md)). Until it is done, use a disposable -> machine. +> **Status: pre-release.** These steps were carried out on one real host: an +> Ubuntu 24.04 aarch64 VM ([acceptance/2026-10-arm64.md](acceptance/2026-10-arm64.md)). +> No x86_64 host has been verified. Use a machine you can afford to rebuild. ## Requirements What preflight refuses to work without: -- **Ubuntu**, with systemd 249 or later. +- **Ubuntu**, with systemd 249 or later. Only systemd 255 is supported, that + is, verified. Preflight warns on any other accepted version + ([compatibility.md](compatibility.md)). - **The cgroup v2 unified hierarchy**, with the `cpu`, `memory` and `pids` controllers available. - **x86_64 or aarch64**, matching the `runner-platform` of the release seed. @@ -103,4 +103,10 @@ To roll back, move the symlink back and apply again. 1. Retire every runner, deregistering each from GitHub ([operations.md](operations.md)). 2. Remove `/usr/local/bin/runner-slice` and `/opt/runner-slice-`. 3. Remove `/etc/runner-slice` and the slice unit in `/etc/systemd/system`. + The slice unit is the one thing retirement leaves on purpose, because it is + shared by every runner. Once the last profile is gone, `runner-slice` + refuses an empty inventory and will not remove it for you. 4. Run `systemctl daemon-reload`. + +Packages installed for `apt-dependency` stay installed. RunnerSlice never +removes packages, because other software may need them. diff --git a/docs/operations.md b/docs/operations.md index 2a68794..afe382f 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1,8 +1,9 @@ # Operations -> **Status: pre-release.** The commands and outputs below are what the code -> does under its drills. They have not yet been observed on a real host -> ([verification.md](verification.md)). +> **Status: pre-release.** The lifecycle below was run end to end on one real +> Ubuntu 24.04 aarch64 host with two GitHub runners +> ([acceptance/2026-10-arm64.md](acceptance/2026-10-arm64.md)). Unknown job state +> and misregistration are exercised by drills only. The command forms used throughout: @@ -53,17 +54,31 @@ Every `apply` line is one of: never handles it: ```sh - export ACTIONS_RUNNER_INPUT_TOKEN="$(gh api -X POST /repos///actions/runners/registration-token --jq .token)" - sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u runner-app -H \ - env HOME=/home/runner-app /var/lib/ci-app/runner/config.sh \ - --url https://github.com// \ - --name --labels \ - --work /var/lib/ci-app/work --unattended --replace - unset ACTIONS_RUNNER_INPUT_TOKEN + ( + ACTIONS_RUNNER_INPUT_TOKEN="$(gh api -X POST /repos///actions/runners/registration-token --jq .token)" || exit + export ACTIONS_RUNNER_INPUT_TOKEN + sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u runner-app -H \ + env -C /var/lib/ci-app/runner HOME=/home/runner-app ./config.sh \ + --url https://github.com// \ + --name --labels \ + --work /var/lib/ci-app/work --unattended + ) sudo chmod 600 /var/lib/ci-app/runner/.runner /var/lib/ci-app/runner/.credentials /var/lib/ci-app/runner/.credentials_rsaparams ``` Copy the exact command `apply` printed. It has your names and labels in it. + The parentheses matter: the token stays in that subshell and is gone when + it ends, whatever `config.sh` does. + + `env -C` runs `config.sh` in its own directory. `config.sh` checks its + native libraries with `./bin/...` paths before it changes directory, so run + from anywhere else that check finds no files, prints `ldd: … No such file + or directory`, and passes without checking anything (actions/runner #945, + #4460). + + There is no `--replace`. If a runner of that name is already registered to + the repository, `config.sh` refuses, because that runner may be another + host's. Find out which it is before re-registering with `--replace`. The token travels in the environment, never as an argument. A command line is logged by sudo and shown by `ps` to every account, and anyone holding @@ -124,8 +139,31 @@ each step a person takes. 3. **Deregister.** It stops with exit code 10 until a person has deregistered the runner (the command is printed). If GitHub is unreachable, delete the runner on GitHub and remove the registration files by hand. -4. **Remove.** It removes the unit, the runtime root and the account. Only - things RunnerSlice created are removed. +4. **Remove.** It checks everything first: the unit is RunnerSlice's, the + runtime root carries this profile's marker, the account has RunnerSlice's + shape, and no process runs as the account. Any uid counts: real, + effective, saved or filesystem. If `/proc` cannot be read, it removes + nothing. Then it removes, in this order: + - the account, its home, and its own group; + - the runtime root, with its marker last; + - the unit. + + It also refuses, removing nothing, while any filesystem is mounted at or + below the runtime root, or while the mount table cannot be read. + + Removal is not atomic: + - if the account, or the runtime root, cannot be removed, the steps after + it are not taken; + - a leftover home or group is reported, but the root and the unit are + still removed. + + Either way the run exits 1, names what is left, and running `retire` again + continues from there. Only things RunnerSlice created are removed. + + Between a partial failure and its retry, the files left behind are owned + by a uid and gid that no longer exist. The next system account created on + the host may be given the same ids. Finish the retirement before creating + other accounts. Then delete the profile and apply the remaining runners. Their units drop the mask for the retired root. @@ -143,5 +181,37 @@ mask for the retired root. | `check`: `NOT masked in namespace` | The running service does not have a mask its unit declares. Usually it was started before the unit changed. | `apply` with the runner idle restarts it with the current unit. | | `check`: `differs from the rendering`, `unexpected drop-in` | Someone edited a unit or added a drop-in. | Remove the drop-in, then `apply`. | | `check`: `runner version ... below the seed` | Self-update is not working. | Re-seed by hand, with the runner idle (`apply` prints how). Registration files survive an unpack. | +| `retire`: `processes of ... are still running` | A job left a process behind, possibly one that only kept the account as its saved uid. | Stop it (`ps -eo pid,ruid,euid,suid,cmd`), then `retire` again. Nothing was removed. | +| `retire`: `cannot tell whether any process runs as ...` | `/proc` could not be read. | Find out why. Nothing was removed. | +| `retire`: `userdel could not remove ...` | The account is still there. | Nothing else was removed. The unit and runtime root still identify the runner. Fix the cause, then `retire` again. | +| `retire`: `could not remove all of ...` | The account is gone; part of the runtime root is not, for example an immutable file. | The unit and the root's marker are kept. If the last `rmdir` failed, the marker is written back. Fix the cause, then `retire` again; it finishes the root, then the unit. | +| `retire`: `a filesystem is mounted at or below ...` | Something is mounted in the runtime root, for example a larger disk at `work/`. | Nothing was removed. Unmount it, and keep what is on it if you need it, then `retire` again. | +| `retire`: `cannot read the mount table` / `cannot look up account ...` | `/proc/self/mountinfo` was unreadable, or the account lookup failed rather than answering "no such account". | Nothing was removed. Find out why, then `retire` again. | +| `retire`: `group ... is left: this host also resolves accounts outside local files` | Accounts also come from a directory service (`nsswitch.conf`), so not every user of the gid can be listed. | Check that nothing uses the gid, then `groupdel `. | +| `retire`: `could not remove /etc/systemd/system/...` | The unit file could not be deleted, for example on a read-only `/etc`. | Fix the cause, then `retire` again. | +| `retire`: `/home/... is still there without its account` | `userdel` removed the account but not its home. | Look at what is in it, then remove it by hand. Until then, `apply` refuses to create the account again. | +| `retire`: `group ... exists without its account` | The account is already gone, so RunnerSlice can no longer prove the group was its own. This is found on a later run, after the group was not removed on the run that removed the account. | If nothing else uses it (`getent group `, `awk -F: '$4 == ' /etc/passwd`), remove it by hand: `groupdel `. | +| `retire` or `apply`: `refusing to remove ...` / `... exists and was not created by RunnerSlice` (runtime root) | The root has no marker, or the marker names another profile or is a symlink. RunnerSlice cannot prove the directory is this runner's, so it never takes it over or removes it. | See "A runtime root without its marker" below. | +| `apply`: `group ... exists without an account of that name` or `/home/... exists without an account` | A leftover of an earlier install, or someone else's. | If it is a leftover and nothing uses it, remove it by hand (`groupdel`, `rm -rf`), then `apply`. | | `status`: `remote runner is not online` | GitHub does not see the runner. | Look at the service journal: `journalctl -u `. | | After a reboot | Units are enabled and start with the host. | Run `check`. Then run `status`. | + +### A runtime root without its marker + +`/var/lib/-/.runner-slice-root` is root-owned and names its +profile. Without it, RunnerSlice will not adopt or remove the directory. +That holds even if RunnerSlice created it, for example when someone deleted +the marker or restored the directory from a backup without it. Nothing +recreates the marker automatically. Writing a marker is a claim that the +whole directory is this runner's, and it is a person's decision: + +1. Check that the runner is stopped: `systemctl status `. +2. Look at what is in the directory. If any of it is not this runner's, move + that out first, or rename the profile and leave the directory alone. +3. If the runner was registered, deregister it on GitHub. Its `.credentials` + are in this directory. +4. Either remove the directory by hand, or restore the marker. To restore + it, run `printf '%s\n' | sudo tee /var/lib/-/.runner-slice-root`, + then `sudo chown root:root` and `sudo chmod 0644` on the marker. The id + is the profile's file name without `.conf`. +5. Run `apply` or `retire` again. diff --git a/docs/threat-model.md b/docs/threat-model.md index 8b79707..7b79e52 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -42,7 +42,7 @@ Actions Runner Controller, or GARM with a VM provider. | Everything else on the host: CPU, memory and process count | One slice for all runners with declared `CPUQuota`, `MemoryMax`, optional `MemoryHigh`, and `TasksMax` | The installed slice is compared with the rendering. systemd's effective values are compared with the declaration. | | Declared network destinations (`slice-ip-deny`) | `IPAddressDeny=` on the slice. systemd merges it into every runner's own IP filter. | systemd's effective deny list includes each entry. No allow list exists on the runner, the slice or any slice above it. | | What root runs and reads | Preflight refuses a configuration or tool tree that root does not own exclusively: other owners, group- or world-writable entries, symlinks, writable parent directories. | This is checked before every apply and check. | -| Units, accounts and directories RunnerSlice did not create | Units carry a first-line marker. Runtime roots carry a root-owned marker naming their profile. Accounts are adopted only if they are system accounts with the expected home. A group or home that predates its account is never adopted. | Apply stops before creating anything next to a foreign unit or directory. Retirement checks every precondition first, and removes nothing unless all hold. | +| Units, accounts and directories RunnerSlice did not create | Units carry a first-line marker. Runtime roots carry a root-owned marker naming their profile. Accounts are adopted only if they are system accounts with the expected home. A group or home that predates its account is never adopted. | Apply stops before creating anything next to a foreign unit or directory. Retirement checks every precondition first, and removes nothing unless all hold. A group is removed only when it is the account's own, unused primary group, read before the account goes, and only when every account comes from local files. Nothing is removed while a filesystem is mounted in the runtime root. | | Running jobs | No unit is replaced and no runner is restarted while a job is running or its state is unknown. | The job state is read from the service's cgroup, whose path comes from systemd, and from `/proc`. | Every runner unit also sets: @@ -116,13 +116,22 @@ observed on a real host before it counts. it, stores it or prints it. It prints the command for a person to run. That command passes the token through the environment, never as an argument: sudo logs command lines, and `ps` shows them to every account. + It is printed inside `( ... )`, so the token lives in a subshell and is gone + when the subshell ends, even if `config.sh` fails or is interrupted. A `gh` + that fails ends the subshell before `config.sh` runs. - RunnerSlice never runs what the runner account can write. The installed version is read from the runner's dependency manifest as data. Files the - runner writes are opened without following symlinks, only if they are - small regular files. + runner writes (`.runner`, `bin/Runner.Listener.deps.json`) are reached by + opening the runner directory, which sits in a root-owned runtime root, and + then each component below it with `O_NOFOLLOW`. `O_NOFOLLOW` alone covers + only the last component, so the walk is what refuses a symlinked `bin/`. + Only a small regular file is read. Anything that cannot be read or parsed + that way, including a parser error such as deep nesting, is "cannot tell" + (exit 2), never a pass. - A registered runner's directory is never unpacked into again. A first - unpack goes into a directory root has just created, closed to the runner - account until it is handed over. + unpack goes into a directory root has just created, mode 0700 for the whole + unpack (`tar --no-overwrite-dir` keeps the archive's own mode off it), and + handed to the runner account only after. - `.runner` holds the runner's name, repository and work folder, and no credential. RunnerSlice reads it to refuse a runner registered as something else. diff --git a/docs/verification.md b/docs/verification.md index fa72a1d..ebf22a5 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -62,37 +62,21 @@ survived and why. ### Assumptions the fixtures encode The drills prove that the scripts judge their inputs correctly. They do not -prove that a real host produces those inputs. Each of these must be observed on -the integration VM: - -1. The runner registers and runs jobs under `RestrictNamespaces=yes`. -2. What `systemctl show` prints for: - - `RestrictNamespaces`, restricted and not; - - `SupplementaryGroups`; - - `CPUQuotaPerSecUSec`; - - `IPAddressDeny`; - - `IPAddressAllow`; - - `MemoryHigh` when it is unset. -3. Changed slice limits and a changed `IPAddressDeny` reach running runners - through a daemon reload, with no restart. This has to be shown by the - runners' cgroup files and IP filter programs, not by the reload's exit - code. -4. `ProtectHome=tmpfs` with `BindPaths` produces the mount table the check - expects. - - `InaccessiblePaths` masks appear as mounts whose root is - `/systemd/inaccessible/...`, the field the check reads. - - They resolve as the check expects, including `/var/run` aliases. -5. Registration and the files it reads: - - `.runner` has the fields `agentName`, `gitHubUrl` and `workFolder`, and - a UTF-8 BOM; - - `config.sh` takes the token from `ACTIONS_RUNNER_INPUT_TOKEN` and - accepts the printed command; - - `bin/Runner.Listener.deps.json` names the installed release as - `Runner.Listener/`, after self-updates too. RunnerSlice reads - the version there and never executes the runner. -6. Which packages, if any, the runner needs on the target Ubuntu release. -7. `useradd --system` gives a uid below `UID_MIN`, with home `/home/NAME`. -8. GitHub's "remove all custom labels" endpoint drains a runner. +prove that a real host produces those inputs. Each assumption below was +checked on the ARM64 acceptance VM +([acceptance/2026-10-arm64.md](acceptance/2026-10-arm64.md)). An x86_64 host +has not been checked. + +| # | Assumption | Result on Ubuntu 24.04 aarch64, systemd 255 | +|---|---|---| +| 1 | The runner registers and runs jobs under `RestrictNamespaces=yes` | **Holds.** Real jobs ran, and `unshare -U` was refused inside them. | +| 2 | What `systemctl show` prints | **Partly wrong, fixed.** `RestrictNamespaces` prints `yes`, `no` or a list, not an empty value; the check now expects that. `CPUQuotaPerSecUSec=1s`, byte values for memory, `/32` on `IPAddressDeny`, and empty `IPAddressAllow` and `SupplementaryGroups` are as assumed. `MemoryHigh` when unset was not observed. | +| 3 | Slice changes reach running processes on reload, without a restart | **Holds**, for `IPAddressDeny` and `memory.max`. It was shown on a live process in the slice by its connections and cgroup files, with the same PID before and after. The live runners carried the deny from their start, and real jobs could not reach it. | +| 4 | The mount table the check reads | **Holds.** Masks are mounts rooted at `/systemd/inaccessible/…`. `/home` is a tmpfs with only the account's own home bound. Read from the real Listener process. | +| 5 | Registration files and inputs | **Holds.** `.runner` has a UTF-8 BOM and the fields `agentName`, `gitHubUrl` and `workFolder`. `config.sh` reads `ACTIONS_RUNNER_INPUT_TOKEN`, for both registration and removal. `deps.json` names `Runner.Listener/2.338.0`. A runner self-update was not observed. | +| 6 | Packages the runner needs | **`liblttng-ust1t64`** is the only one missing from the Ubuntu 24.04 cloud image. | +| 7 | `useradd --system` gives uid < `UID_MIN` with home `/home/NAME` | **Holds** (uids 995 and 997). | +| 8 | Removing a runner's custom labels drains it | **Holds** as far as observed: after removal, only `self-hosted`, `Linux` and `ARM64` remained. That no job was then assigned was not separately tested. | ## Lifecycle acceptance @@ -103,24 +87,42 @@ recorded commit. Each item is marked: - **synthetic**: covered by drills only; - **unverified**: not done yet. -| # | Step | Status | +| # | Step | Status (ARM64 VM) | |---|---|---| -| 1 | Install on a clean VM, from the release archive | unverified (synthetic: release drill) | -| 2 | Preflight accepts the VM and refuses a broken declaration | unverified (synthetic) | -| 3 | Profile validation | unverified (synthetic) | -| 4 | Slice and service applied | unverified (synthetic) | -| 5 | Real GitHub registration, with identity confirmed by apply | unverified | -| 6 | RunnerSlice's drills run as a job on its own runner | unverified | -| 7 | Account, groups, masks and home namespace, as `check` and by hand | unverified (synthetic) | -| 8 | Resource limits in effect: the cgroup files under load | unverified | -| 9 | A second runner as a pool member, and jobs spread across it | unverified (synthetic) | -| 10 | A unit change while a job runs is deferred, and applied after it | unverified (synthetic) | -| 11 | Re-apply changes nothing, and restarts nothing | unverified (synthetic) | -| 12 | Reboot: units start, `check` passes, limits and filters are present | unverified | -| 13 | Failure and recovery: an interrupted unpack, a killed apply, a misregistration | unverified (synthetic) | -| 14 | Deregistration | unverified | -| 15 | Retirement removes exactly what RunnerSlice created | unverified (synthetic) | -| 16 | Nothing left behind: units, accounts, directories, and runners on GitHub | unverified | +| 1 | Install on a clean VM, from the release archive | **PASS.** Found #16 (the archive was group-writable); fixed | +| 2 | Preflight accepts the VM and refuses a broken declaration | **PASS** | +| 3 | Profile validation | **PASS** | +| 4 | Slice and service applied | **PASS** | +| 5 | Real GitHub registration, with identity confirmed by apply | **PASS.** Two runners | +| 6 | RunnerSlice's drills run as a job on its own runner | **PASS.** 436 cases, on each runner | +| 7 | Account, groups, masks and home namespace, as `check` and by hand | **PASS.** `check`, the live process, and boundary jobs | +| 8 | Resource limits in effect: the cgroup files under load | **PASS.** Both runners busy, held to one CPU and MemoryHigh | +| 9 | A second runner as a pool member, and jobs spread across it | **PASS.** Jobs observed on each; assignment is GitHub's | +| 10 | A unit change while a job runs is deferred, and applied after it | **PASS.** Real hold jobs; exactly one restart afterwards | +| 11 | Re-apply changes nothing, and restarts nothing | **PASS** | +| 12 | Reboot: units start, `check` passes, limits and filters are present | **PASS.** Guest reboot; boundary jobs after it | +| 13 | Failure and recovery: an interrupted unpack, a killed apply, a misregistration | **PASS** for the killed download and the killed unpack. A misregistration stays SYNTHETIC | +| 14 | Deregistration | **PASS.** Found a rejected `--unattended`; fixed | +| 15 | Retirement removes exactly what RunnerSlice created | **PASS** | +| 16 | Nothing left behind: units, accounts, directories, and runners on GitHub | **PASS.** Only the shared slice unit remains, by design | + +Unknown job state could not be produced safely on a healthy host as root, so +it remains SYNTHETICALLY TESTED. x86_64 is not verified. + +The retirement and reader changes from the post-acceptance review (#18) were +run on the same VM, at RUNTIME VERIFIED, with a test-only profile that was +never registered. That run covered: + +- leftover processes, by effective and by saved uid; +- `userdel` failing; +- a runtime root that could only be partly removed, and the retry after it; +- `USERGROUPS_ENAB no`; +- a group in use, and a leftover home; +- a symlinked `bin/`; +- the unpack mode. + +The results are in [the acceptance record](acceptance/2026-10-arm64.md). An +unreadable `/proc`, and malformed runner files, remain SYNTHETICALLY TESTED. Each run records: diff --git a/fabric/fabric-apply.sh b/fabric/fabric-apply.sh index d4a0775..19bb2d8 100755 --- a/fabric/fabric-apply.sh +++ b/fabric/fabric-apply.sh @@ -303,10 +303,12 @@ if [ "$MODE" = retire ]; then if [ -f "$FS_RUNNER_DIR/.runner" ] || [ -f "$FS_RUNNER_DIR/.credentials" ]; then todo "deregister $RUNNER_NAME from $RUNNER_REPO, as a person; the files here are its credentials:" note "" - note " export ACTIONS_RUNNER_INPUT_TOKEN=\"\$(gh api -X POST /repos/$RUNNER_REPO/actions/runners/remove-token --jq .token)\"" - note " sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u $RUNNER_USER -H \\" - note " env HOME=$RUNNER_HOME $RUNNER_DIR/config.sh remove --unattended" - note " unset ACTIONS_RUNNER_INPUT_TOKEN" + note " (" + note " ACTIONS_RUNNER_INPUT_TOKEN=\"\$(gh api -X POST /repos/$RUNNER_REPO/actions/runners/remove-token --jq .token)\" || exit" + note " export ACTIONS_RUNNER_INPUT_TOKEN" + note " sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u $RUNNER_USER -H \\" + note " env -C $RUNNER_DIR HOME=$RUNNER_HOME ./config.sh remove" + note " )" note "" note " if GitHub cannot be reached from here, delete the runner on GitHub instead and then" note " sudo rm -f $RUNNER_DIR/.runner $RUNNER_DIR/.credentials $RUNNER_DIR/.credentials_rsaparams" @@ -353,28 +355,76 @@ if [ "$MODE" = retire ]; then fi # A job's leftover process (KillMode=process lets one outlive the # service) would keep files open and make the account removal fail. - if leftover=$(pgrep -u "$RUNNER_USER" 2>/dev/null | paste -sd' ' -) && [ -n "$leftover" ]; then - fail "processes of $RUNNER_USER are still running ($leftover)" - note " stop them, then run --retire $PROFILE_ID again" - removable=0 - fi + # Every uid the kernel keeps is compared, and a process table that + # cannot be read is a refusal: "could not look" is never "none". + prc=0 + leftover=$(host_account_processes "$RUNNER_USER") || prc=$? + case "$prc" in + 0) fail "processes of $RUNNER_USER are still running ($(paste -sd' ' - <<<"$leftover"))" + note " stop them, then run --retire $PROFILE_ID again" + removable=0 ;; + 1) ok "no process runs as $RUNNER_USER" ;; + *) fail "cannot tell whether any process runs as $RUNNER_USER" + removable=0 ;; + esac + # rm --one-file-system skips a mount only below the argument it was + # given, so a filesystem mounted at work/ or runner/ would be emptied. + # Anything mounted in the root is refused before anything is removed. + mrc=0 + mounts=$(host_mounts_under "$FS_INSTALL_ROOT") || mrc=$? + case "$mrc" in + 0) ;; + 1) fail "a filesystem is mounted at or below $INSTALL_ROOT: $mounts" + note " unmount it, and move what is on it if it is to be kept" + removable=0 ;; + *) fail "cannot read the mount table; cannot tell whether anything is mounted below $INSTALL_ROOT" + removable=0 ;; + esac if [ "$removable" -ne 1 ]; then note "nothing was removed" host_report_apply fi - if host_remove_file "$UNIT_DIR/$SERVICE" "retired runner" && [ "$CHECK_ONLY" -eq 0 ]; then - systemctl daemon-reload + # Removal is not atomic, so it is ordered to leave a retryable state + # at every step. The account goes first: while it exists, the unit + # and the marked root still identify it as this runner's, and a + # second run removes the rest exactly as the first would have. The + # unit goes last, so a profile half-way through still names it. + if ! host_remove_user "$RUNNER_USER"; then + note "nothing else was removed; $INSTALL_ROOT and $SERVICE are left as they were" + note "fix the cause above, then run --retire $PROFILE_ID again" + host_report_apply fi + # The marker is removed last: a removal that stops part-way must leave + # the proof that the rest is RunnerSlice's, or no retry could finish. + # If the final rmdir fails, the marker is written back for the same + # reason. --preserve-root=all also refuses any child that is a mount + # point, should one appear after the check above. if [ -e "$FS_INSTALL_ROOT" ]; then change "remove $INSTALL_ROOT" - [ "$CHECK_ONLY" -eq 0 ] && rm -rf --one-file-system -- "$FS_INSTALL_ROOT" + if [ "$CHECK_ONLY" -eq 0 ] && + ! { find "$FS_INSTALL_ROOT" -mindepth 1 -maxdepth 1 ! -name .runner-slice-root \ + -exec rm -rf --one-file-system --preserve-root=all -- {} + && + rm -f -- "$FS_INSTALL_ROOT/.runner-slice-root" && + { rmdir -- "$FS_INSTALL_ROOT" || + { printf '%s\n' "$PROFILE_ID" > "$FS_INSTALL_ROOT/.runner-slice-root"; false; }; }; }; then + fail "could not remove all of $INSTALL_ROOT; $SERVICE is left in place" + note "fix the cause above, then run --retire $PROFILE_ID again" + host_report_apply + fi else ok "$INSTALL_ROOT is absent" fi - host_remove_user "$RUNNER_USER" - note "retired. Now delete profiles/$PROFILE_ID.conf and run --all with the other" - note "runners idle: their units drop the mask for $INSTALL_ROOT." + if host_remove_file "$UNIT_DIR/$SERVICE" "retired runner" && [ "$CHECK_ONLY" -eq 0 ]; then + systemctl daemon-reload + fi + if [ "$check_failed" -eq 1 ]; then + note "not fully retired: what is reported above was left behind" + note "fix it, then run --retire $PROFILE_ID again" + else + note "retired. Now delete profiles/$PROFILE_ID.conf and run --all with the other" + note "runners idle: their units drop the mask for $INSTALL_ROOT." + fi fi host_report_apply @@ -501,10 +551,14 @@ else || { echo "runner archive digest mismatch; refusing to unpack" >&2; exit 1; } note "archive digest verified" # Unregistered, so nothing here is the runner's yet: start from an - # empty directory only root can enter, then hand it over. + # empty directory only root can enter, then hand it over. tar would + # otherwise give that directory the archive's own mode for "." (0775 + # in the official archive) while it unpacks; --no-overwrite-dir keeps + # it 0700 until the chmod below. The root above it is root: + # 0750 either way, so no other account could have reached it. rm -rf --one-file-system -- "$FS_RUNNER_DIR" mkdir -m 0700 "$FS_RUNNER_DIR" - tar -xzf "$WORK/release/$RUNNER_ARCHIVE" -C "$FS_RUNNER_DIR" --no-same-owner + tar -xzf "$WORK/release/$RUNNER_ARCHIVE" -C "$FS_RUNNER_DIR" --no-same-owner --no-overwrite-dir chown -R "$RUNNER_USER:$RUNNER_USER" "$FS_RUNNER_DIR" chmod 0750 "$FS_RUNNER_DIR" printf '%s %s\n' "$RUNNER_VERSION" "$RUNNER_SHA" > "$SEED_MARKER" @@ -538,17 +592,29 @@ else registration_required=1 todo "register the runner, as a person, with a token that expires in an hour:" note "" - note " export ACTIONS_RUNNER_INPUT_TOKEN=\"\$(gh api -X POST /repos/$RUNNER_REPO/actions/runners/registration-token --jq .token)\"" - note " sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u $RUNNER_USER -H \\" - note " env HOME=$RUNNER_HOME $RUNNER_DIR/config.sh \\" - note " --url https://github.com/$RUNNER_REPO \\" - note " --name $RUNNER_NAME --labels $LABELS \\" - note " --work $WORK_DIR --unattended --replace" - note " unset ACTIONS_RUNNER_INPUT_TOKEN" + note " (" + note " ACTIONS_RUNNER_INPUT_TOKEN=\"\$(gh api -X POST /repos/$RUNNER_REPO/actions/runners/registration-token --jq .token)\" || exit" + note " export ACTIONS_RUNNER_INPUT_TOKEN" + note " sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u $RUNNER_USER -H \\" + note " env -C $RUNNER_DIR HOME=$RUNNER_HOME ./config.sh \\" + note " --url https://github.com/$RUNNER_REPO \\" + note " --name $RUNNER_NAME --labels $LABELS \\" + note " --work $WORK_DIR --unattended" + note " )" note " sudo chmod 600 $RUNNER_DIR/.runner $RUNNER_DIR/.credentials $RUNNER_DIR/.credentials_rsaparams" note "" note " the token travels in the environment, which only root and the runner account" - note " can read, never on a command line that sudo logs and ps shows" + note " can read, never on a command line that sudo logs and ps shows. The" + note " parentheses keep it in a subshell: it is gone when the subshell ends, even" + note " if config.sh fails or is interrupted, and no gh failure reaches config.sh." + note "" + note " config.sh runs in its own directory (env -C): it checks its libraries with" + note " ./bin paths before changing directory, and from anywhere else that check" + note " silently checks nothing." + note "" + note " config.sh refuses if a runner named $RUNNER_NAME is already registered to the" + note " repository. That may be another host's runner: find out which before adding" + note " --replace, which would take its place." note "" note "then run this again to install and start the service" fi diff --git a/fabric/fabric-check.sh b/fabric/fabric-check.sh index 9e251c7..1747c79 100755 --- a/fabric/fabric-check.sh +++ b/fabric/fabric-check.sh @@ -323,10 +323,12 @@ host_expect_property "$SERVICE" PrivateTmp yes host_expect_property "$SERVICE" ProtectSystem full host_expect_property "$SERVICE" ProtectHome tmpfs host_expect_property "$SERVICE" SupplementaryGroups "" -# systemd prints the namespaces a unit may still create; an empty list is -# RestrictNamespaces=yes. +# systemd 255 prints RestrictNamespaces as yes, no, or the namespace types a +# unit may still create (observed on the ARM64 acceptance VM). baseline is +# yes; userns is no. Anything else is drift. case "$SANDBOX" in - baseline) host_expect_property "$SERVICE" RestrictNamespaces "" ;; + baseline) host_expect_property "$SERVICE" RestrictNamespaces yes ;; + userns) host_expect_property "$SERVICE" RestrictNamespaces no ;; esac # --------------------------------------------------------------------------- diff --git a/fabric/fabric-lib.sh b/fabric/fabric-lib.sh index 7f73d09..0339a71 100755 --- a/fabric/fabric-lib.sh +++ b/fabric/fabric-lib.sh @@ -744,7 +744,7 @@ _fabric_validate_fleet() { # across the whole fleet, pool or not, because a runner name, account, unit # or runtime root shared inside a pool is exactly as much a shared identity # as one shared between tenants -- and a shared runner name is worse, since - # registration runs with --replace and would take over the other runner. + # re-registration with --replace would take over the other runner. local pool prior_pool prior_labels labels for file in "${profiles[@]}"; do repo=$(fabric_value "$file" runner-repo) diff --git a/fabric/host-lib.sh b/fabric/host-lib.sh index c953db7..8bcdecd 100755 --- a/fabric/host-lib.sh +++ b/fabric/host-lib.sh @@ -33,7 +33,11 @@ TRUSTED_OWNER=root # The oldest systemd whose behaviour the units rely on: ProtectHome=tmpfs with # BindPaths, and a slice's IPAddressDeny merged into every unit inside it. +# Accepting a version is not having verified it. Only the versions listed in +# FABRIC_VERIFIED_SYSTEMD have run the acceptance on a real host +# (docs/compatibility.md); any other accepted version is named as unverified. FABRIC_MIN_SYSTEMD=249 +FABRIC_VERIFIED_SYSTEMD="255" # --------------------------------------------------------------------------- # Reporting @@ -153,6 +157,10 @@ host_preflight() { systemd_version=${systemd_version%%[!0-9]*} if [ -n "$systemd_version" ] && [ "$systemd_version" -ge "$FABRIC_MIN_SYSTEMD" ]; then ok "systemd $systemd_version" + case " $FABRIC_VERIFIED_SYSTEMD " in + *" $systemd_version "*) ;; + *) warn "systemd $systemd_version is accepted but unverified; RunnerSlice has been verified on systemd $FABRIC_VERIFIED_SYSTEMD only" ;; + esac else warn "systemd '${systemd_version:-unknown}' is older than $FABRIC_MIN_SYSTEMD or unreadable" failed=1 @@ -279,12 +287,16 @@ host_install_file() { return 0 } -# host_remove_file PATH REASON returns 0 when it removed (or would remove) +# host_remove_file PATH REASON returns 0 when it removed (or would remove), +# 1 when there was nothing to remove, and 2 when the removal failed host_remove_file() { local path=$1 reason=$2 [ -e "$path" ] || { ok "no $path"; return 1; } change "remove $path ($reason)" - [ "$CHECK_ONLY" -eq 0 ] && rm -f "$path" + if [ "$CHECK_ONLY" -eq 0 ] && ! rm -f "$path"; then + fail "could not remove $path" + return 2 + fi return 0 } @@ -366,30 +378,142 @@ host_ensure_user() { return 0 } -# host_remove_user NAME the account, its home and its private group only +# host_remove_user NAME +# +# The account, its home and its own group -- and nothing that is not +# provably those. userdel is not atomic: it can remove the account and fail +# on the home (exit 12), and with USERGROUPS_ENAB no in login.defs it leaves +# the account's group behind. So the outcome is read back afterwards rather +# than inferred from the exit code, and whatever is left is reported by name. +# +# A group left behind is removed only when it is this account's own: the same +# name, the gid the account had as its primary group, no member, and no other +# account using it as its primary group. Anything else is someone's, and is +# left. Once the account is gone that proof is gone with it, so a group found +# on a later run, without the account, is reported and never removed. +# +# Returns 0 when the account is gone (it may still report a leftover home or +# group, as failures), and 1 when the account is still there. A caller stops +# on 1: nothing that depends on the account being gone may be removed first. host_remove_user() { - local name=$1 - if getent passwd "$name" >/dev/null; then + local name=$1 entry gid="" rc=0 group lrc=0 + entry=$(getent passwd "$name") || lrc=$? + if [ "$lrc" -ne 0 ] && [ "$lrc" -ne 2 ]; then + fail "cannot look up account $name (getent exit $lrc); nothing was removed for it" + return 1 + fi + if [ "$lrc" -eq 0 ]; then if ! host_account_is_ours "$name"; then fail "account $name is not a RunnerSlice system account; not removing it" - return 0 + return 1 fi + gid=$(cut -d: -f4 <<<"$entry") change "remove user $name, its home and its own group" - if [ "$CHECK_ONLY" -eq 0 ] && ! userdel --remove "$name"; then - fail "userdel could not remove $name; nothing else was changed for it" - return 0 + [ "$CHECK_ONLY" -eq 0 ] || return 0 + userdel --remove "$name" || rc=$? + if getent passwd "$name" >/dev/null; then + fail "userdel could not remove $name (exit $rc); the account is still there" + return 1 fi + [ "$rc" -eq 0 ] || warn "userdel removed $name but exited $rc; what it left is reported below" else ok "user $name is absent" fi - # userdel removes the account's own group with it. A group of that name - # left behind is not the account's, so it is reported, never deleted. - if [ "$CHECK_ONLY" -eq 0 ] && getent group "$name" >/dev/null; then - note "group $name still exists; it was not created with this account, so it is left" + [ "$CHECK_ONLY" -eq 0 ] || return 0 + + if [ -e "$HOST_ROOT/home/$name" ] || [ -L "$HOST_ROOT/home/$name" ]; then + fail "/home/$name is still there without its account; it is not removed automatically" + note " look at what is in it, then remove it by hand: rm -rf --one-file-system /home/$name" + fi + + group=$(getent group "$name") || return 0 + if [ -n "$gid" ] && ! host_accounts_are_local; then + fail "group $name is left: this host also resolves accounts outside local files (nsswitch), so RunnerSlice cannot prove nothing else uses gid $gid" + note " if nothing uses it, remove it by hand: groupdel $name" + elif [ -n "$gid" ] && host_group_is_own_private "$group" "$gid"; then + change "remove group $name (gid $gid): the account's own group, which userdel left" + if ! groupdel "$name"; then + fail "groupdel could not remove group $name" + note " remove it by hand once nothing uses it: groupdel $name" + fi + elif [ -n "$gid" ]; then + note "group $name is left: it is not $name's own unused group (another account uses it, or it has members)" + else + fail "group $name exists without its account; RunnerSlice can no longer tell whether it was this runner's" + note " if it is the leftover of this runner and nothing else uses it: groupdel $name" fi return 0 } +# host_accounts_are_local +# +# Whether every account this host resolves comes from local files. With a +# directory service (sss, ldap, winbind...) enumerating passwd lists only +# part of the accounts, so "no account uses this gid" cannot be proved. +# No nsswitch.conf means glibc's default, files. An unreadable one is not +# local. +host_accounts_are_local() { + local conf="$HOST_ROOT/etc/nsswitch.conf" src compat=0 + local -a sources=() + if [ -e "$conf" ]; then + [ -r "$conf" ] || return 1 + # glibc accepts "passwd:files" as well as "passwd: files", and a + # comment may follow the sources. + read -ra sources < <(sed 's/#.*//' "$conf" | + awk -F: '$1 ~ /^[ \t]*passwd[ \t]*$/ { sub(/^[^:]*:/, ""); print; exit }') + fi + for src in "${sources[@]}"; do + case "$src" in + files|systemd|\[*) ;; + compat) compat=1 ;; + *) return 1 ;; + esac + done + # compat reads NIS accounts through "+" and "-" lines in /etc/passwd. + if [ "$compat" -eq 1 ] && grep -q '^[+-]' "$HOST_ROOT/etc/passwd" 2>/dev/null; then + return 1 + fi + return 0 +} + +# host_mounts_under PATH +# +# Mount points at or below PATH, from this process's mount table. Prints them +# and returns 1 when there is any, returns 0 when there is none, and returns 2 +# when the table cannot be read. A runtime root with anything mounted in it +# is never removed: the data on that filesystem is not RunnerSlice's. +host_mounts_under() { + local root table="$PROC_ROOT/self/mountinfo" mp found="" lines=0 + # The table lists canonical paths; a root reached through a symlink + # (/var -> /data/var) would otherwise match nothing. + root=$(realpath -m -- "$1") || return 2 + [ -r "$table" ] || return 2 + while read -r _ _ _ _ mp _; do + lines=$((lines + 1)) + # mountinfo escapes space, tab, newline and backslash as \ooo. + mp=$(printf '%b' "${mp//\\/\\0}") + case "$mp" in "$root"|"$root"/*) found+="${found:+ }$mp" ;; esac + done < "$table" + [ "$lines" -gt 0 ] || return 2 + [ -z "$found" ] && return 0 + printf '%s' "$found" + return 1 +} + +# host_group_is_own_private GROUP_ENTRY GID +# +# Whether a group entry (name:x:gid:members) is an account's own private +# group: the given gid, no member, and the primary group of no account that +# still exists. Enumerating passwd is what proves "no one else uses it". +host_group_is_own_private() { + local entry=$1 gid=$2 g_gid members users + g_gid=$(cut -d: -f3 <<<"$entry") + members=$(cut -d: -f4 <<<"$entry") + [[ "$gid" =~ ^[0-9]+$ ]] && [ "$g_gid" = "$gid" ] && [ -z "$members" ] || return 1 + users=$(getent passwd) || return 1 + ! awk -F: -v g="$gid" '$4 == g { found = 1 } END { exit !found }' <<<"$users" +} + # host_owner_mode PATH "user:group mode", as stat reports them host_owner_mode() { stat -c '%U:%G %a' "$1" @@ -473,17 +597,85 @@ host_job_state() { return 1 } -# Python run by root on files the runner account can write. It opens without -# following a symlink, refuses anything but a small regular file, and never -# executes what it reads. +# host_account_processes NAME +# +# The pids of every process that runs as account NAME by any of the uids the +# kernel keeps for it -- real, effective, saved or filesystem -- read from +# /proc//status. A process that dropped its effective uid but kept the +# account's saved uid can take it back, so it still counts. Prints the pids +# and returns 0, returns 1 when there is none, and returns 2 when the process +# table cannot be judged: no status could be read, or one had no Uid line. +# 2 is never "none". This replaces pgrep, whose failure (missing, or exiting +# 2 or 3) looked exactly like "no process" and let retirement go ahead. +host_account_processes() { + local name=$1 entry uid status pid line found="" seen=0 rc=0 + # getent exits 2 for "no such account"; anything else is a failed lookup. + entry=$(getent passwd "$name") || rc=$? + case "$rc" in 0) ;; 2) return 1 ;; *) return 2 ;; esac + uid=$(cut -d: -f3 <<<"$entry") + [[ "$uid" =~ ^[0-9]+$ ]] || return 2 + for status in "$PROC_ROOT"/[0-9]*/status; do + pid=${status%/status}; pid=${pid##*/} + [[ "$pid" =~ ^[0-9]+$ ]] || continue + # A process may exit between the glob and the read; that one is gone, + # not unknown. One that is still there and cannot be read is unknown. + if ! line=$(grep -m1 '^Uid:' "$status" 2>/dev/null); then + [ -e "$status" ] || continue + return 2 + fi + seen=$((seen + 1)) + if awk -v u="$uid" '{ for (i = 2; i <= 5; i++) if ($i == u) found = 1 } END { exit !found }' <<<"$line"; then + found+="$pid"$'\n' + fi + done + [ "$seen" -gt 0 ] || return 2 + [ -n "$found" ] || return 1 + printf '%s' "$found" + return 0 +} + +# Python run by root on files the runner account can write. The runner +# directory itself sits in a root-owned install root, so the runner cannot +# swap it; everything below it is the runner's to rename. safe_load therefore +# opens the directory and walks down from it one component at a time with +# O_NOFOLLOW, so a symlink anywhere below RUNNER_DIR -- not only in the last +# component, which is all O_NOFOLLOW covers on its own -- is refused rather +# than followed. It reads only a small regular file and never executes it. +# Any failure, including one Python raises from inside the parser (deep +# nesting is a RecursionError, not a ValueError), is "cannot tell": exit 2. +# The exception hook is what guarantees that; nothing else catches those. HOST_SAFE_JSON_PY=' import json, os, stat, sys -def safe_load(path, limit=4 << 20): +def _cannot_tell(kind, value, tb): + print(f"cannot read it safely ({kind.__name__})") + sys.stdout.flush() + os._exit(2) +sys.excepthook = _cannot_tell + +def safe_load(base, rel, limit=4 << 20): + parts = rel.split("/") + if any(p in ("", ".", "..") for p in parts): + raise ValueError("bad path") try: - fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC) + fd = os.open(base, os.O_RDONLY | os.O_DIRECTORY | os.O_CLOEXEC) except OSError as e: raise ValueError(f"cannot open ({e.strerror})") + try: + for part in parts[:-1]: + try: + nfd = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC, dir_fd=fd) + except OSError as e: + raise ValueError(f"cannot open {part}/ ({e.strerror})") + os.close(fd) + fd = nfd + try: + ffd = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK | os.O_CLOEXEC, dir_fd=fd) + except OSError as e: + raise ValueError(f"cannot open ({e.strerror})") + finally: + os.close(fd) + fd = ffd try: st = os.fstat(fd) if not stat.S_ISREG(st.st_mode): @@ -517,15 +709,17 @@ host_runner_version() { python3 -I -c "$HOST_SAFE_JSON_PY"' import re try: - data = safe_load(sys.argv[1]) - libs = data.get("libraries", {}) if isinstance(data, dict) else {} - found = [k.split("/", 1)[1] for k in libs if isinstance(k, str) and k.startswith("Runner.Listener/")] + data = safe_load(sys.argv[1], "bin/Runner.Listener.deps.json") except ValueError: sys.exit(2) +libs = data.get("libraries") if isinstance(data, dict) else None +if not isinstance(libs, dict): + sys.exit(2) +found = [k.split("/", 1)[1] for k in libs if isinstance(k, str) and k.startswith("Runner.Listener/")] if len(found) != 1 or not re.fullmatch(r"[0-9]+\.[0-9]+\.[0-9]+", found[0]): sys.exit(2) print(found[0]) -' "$1/bin/Runner.Listener.deps.json" +' "$1" } # host_registration_state RUNNER_DIR REPO NAME WORK_DIR @@ -540,7 +734,7 @@ host_registration_state() { python3 -I -c "$HOST_SAFE_JSON_PY"' path, repo, name, work = sys.argv[1:5] try: - data = safe_load(path, 1 << 20) + data = safe_load(path, ".runner", 1 << 20) except ValueError as e: print(f".runner {e}") sys.exit(2) @@ -561,7 +755,7 @@ if problems: print("; ".join(problems)) sys.exit(1) print("ok") -' "$1/.runner" "$2" "$3" "$4" +' "$1" "$2" "$3" "$4" } # --------------------------------------------------------------------------- diff --git a/scripts/release.sh b/scripts/release.sh index caeb133..5cff408 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -48,7 +48,10 @@ fi name="runner-slice-$version" mkdir -p "$OUT" OUT=$(cd "$OUT" && pwd) -git archive --format=tar --prefix="$name/" HEAD | gzip -n -9 > "$OUT/$name.tar.gz" +# git archive's default tar.umask is 0002, which leaves every file group- +# writable. Root's preflight refuses a tool tree anyone but root could +# change, so an archive unpacked as root must carry 0644/0755 modes. +git -c tar.umask=0022 archive --format=tar --prefix="$name/" HEAD | gzip -n -9 > "$OUT/$name.tar.gz" (cd "$OUT" && sha256sum "$name.tar.gz" > "$name.tar.gz.sha256") printf 'built %s\n' "$OUT/$name.tar.gz" diff --git a/tests/apply-drill.sh b/tests/apply-drill.sh index 88ef777..5b10475 100755 --- a/tests/apply-drill.sh +++ b/tests/apply-drill.sh @@ -47,8 +47,43 @@ expect_out 'the registration command is printed' 'actions/runners/registration-t expect_out 'registration names the runner and its labels' '--name lab-app-01 --labels lab,app' expect_out 'the token travels in the environment' 'sudo --preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u runner-app' expect_no 'the token is never on a command line' '--token' +expect_no 'a first registration never replaces a runner of the same name' '--unattended --replace' +# config.sh checks its libraries with ./bin paths before it changes to its own +# directory; run from elsewhere, that check silently checks nothing. +expect_out 'registration runs config.sh in the runner directory' 'env -C /var/lib/ci-app/runner HOME=/home/runner-app ./config.sh' expect_no 'no token is requested by the tool itself' 'TOKEN=ghs' + +# The printed block is run, with gh and sudo standing in: the token reaches +# config.sh only through sudo's preserved environment, does not outlive the +# block in the operator's shell, and a gh that fails never reaches config.sh. +TOK="$T/tokbin"; mkdir -p "$TOK" +cat > "$TOK/gh" <<'EOF' +#!/usr/bin/env bash +[ -e "$TOK_STATE/gh-fails" ] && { echo "gh: HTTP 401" >&2; exit 1; } +echo fixture-token-value +EOF +cat > "$TOK/sudo" <<'EOF' +#!/usr/bin/env bash +printf 'token=%s args=%s\n' "${ACTIONS_RUNNER_INPUT_TOKEN-unset}" "$*" >> "$TOK_STATE/sudo.log" +EOF +chmod +x "$TOK/gh" "$TOK/sudo" +printed_block() { awk '/^ +\($/ { on = 1 } on { print } /^ +\)$/ { on = 0 }' <<<"$OUT"; } +run_block() { # FILE sources the block, then reports what the shell still holds + TOK_STATE="$T" PATH="$TOK:$PATH" bash -c '. "$1"; echo "after=${ACTIONS_RUNNER_INPUT_TOKEN-unset}"' _ "$1" 2>/dev/null +} +printed_block > "$T/register.sh" +holds 'the registration is printed as one block' [ -s "$T/register.sh" ] +refutes 'and nothing in it replaces a runner of the same name' grep -q -- '--replace' "$T/register.sh" +rm -f "$T/sudo.log" "$T/gh-fails" +result_after=$(run_block "$T/register.sh") +holds 'config.sh receives the token through sudo' grep -q '^token=fixture-token-value args=--preserve-env=ACTIONS_RUNNER_INPUT_TOKEN -u runner-app' "$T/sudo.log" +holds 'and the token is gone from the shell afterwards' [ "$result_after" = after=unset ] +rm -f "$T/sudo.log"; : > "$T/gh-fails" +run_block "$T/register.sh" >/dev/null +refutes 'a gh that fails never reaches config.sh' [ -e "$T/sudo.log" ] +rm -f "$T/gh-fails" holds 'the verified release is unpacked' [ -x "$H/root/var/lib/ci-app/runner/config.sh" ] +holds "the runner directory stayed 0700 through the unpack, not the archive's 0775" grep -qx 700 "$H/fake/tar-dir-mode" expect_lines 'the archive was downloaded once' "$H/fake/curl.log" 1 expect_lines 'the declared package was installed' "$H/fake/apt.log" 1 holds 'the slice unit is the rendering' is_rendered "$H/$UNITS/ci.slice" slice @@ -197,6 +232,31 @@ register app apply --profile app expect_rc 'the profile converges once the names are free' 0 +# A person's account with the runner's name: nothing is created for it -- +# no runtime root, no download, no unit. +new_host humanaccount +only app +: > "$H/fake/human-runner-app" +before=$(snapshot) +apply --profile app +expect_rc "a person's account of the runner's name stops the profile" 1 +expect_out 'it is named' 'account runner-app exists and is not a RunnerSlice system account' +expect_same 'nothing was created, downloaded or installed for it' "$before" + +# Root's temporary files never go where an inherited TMPDIR points. +new_host tmpdir +only app +MT="$T/mktemp-bin"; mkdir -p "$MT" "$T/shared-tmp"; chmod 1777 "$T/shared-tmp" +# shellcheck disable=SC2016 # the fake mktemp expands these when it runs +printf '#!/usr/bin/env bash\necho "${TMPDIR-unset}" >> "%s"\nexec /usr/bin/mktemp "$@"\n' "$T/mktemp.log" > "$MT/mktemp" +chmod +x "$MT/mktemp" +RC=0 +OUT=$(env PATH="$MT:$BIN:$PATH" TMPDIR="$T/shared-tmp" FAKE="$H/fake" FAKE_CG="$H/cg" FAKE_PROC="$H/proc" \ + FAKE_ARCHIVE="$ARCHIVE" bash "$H/fabric/fabric-apply.sh" --config "$H/config" --profile app 2>&1) || RC=$? +expect_rc 'apply runs with an inherited TMPDIR' 1 +holds 'and makes its temporary directory without it' grep -qx unset "$T/mktemp.log" +refutes 'never under the inherited one' grep -q "$T/shared-tmp" "$T/mktemp.log" + # A registration that names another repository, runner or work folder is a # runner the profile does not describe; it is not started. new_host misregistered @@ -312,6 +372,17 @@ idle ci-web-2-runner.service apply --retire web-2 expect_rc 'an idle member stops at deregistration' 10 expect_out 'the deregistration command is printed' 'actions/runners/remove-token' +expect_out 'deregistration runs config.sh in the runner directory' 'env -C /var/lib/ci-web-2/runner HOME=/home/runner-web-2 ./config.sh remove' +# config.sh remove accepts --token, --pat and --local only (actions/runner +# v2.338.0); --unattended is a configure flag and is reported as unrecognised. +expect_no 'deregistration passes no flag remove does not accept' 'config.sh remove --unattended' +printed_block > "$T/deregister.sh" +rm -f "$T/sudo.log" +result_after=$(run_block "$T/deregister.sh") +holds 'deregistration hands config.sh remove the token through sudo' \ + grep -q '^token=fixture-token-value args=.* env -C /var/lib/ci-web-2/runner HOME=/home/runner-web-2 ./config.sh remove$' "$T/sudo.log" +holds 'and leaves no token in the shell' [ "$result_after" = after=unset ] +rm -f "$T/sudo.log" holds 'it is stopped and disabled first' grep -q '^disable ci-web-2-runner.service' "$H/fake/systemctl.log" holds 'its registration is left for the person' [ -e "$H/root/var/lib/ci-web-2/runner/.runner" ] @@ -357,14 +428,99 @@ for id in docs web; do expect_rc "$id stops at deregistration" 10 rm -f "$H/root/var/lib/ci-$id/runner/.runner" "$H/root/var/lib/ci-$id/runner/.credentials" if [ "$id" = docs ]; then + cp "$H/$UNITS/ci-docs-runner.service" "$T/docs.unit" # A job's leftover process blocks the removal; nothing is removed. - echo 4242 > "$H/fake/procs-runner-docs" + mkdir -p "$H/proc/4242" + printf 'Name:\tleftover\nUid:\t999\t999\t999\t999\n' > "$H/proc/4242/status" apply --retire docs expect_rc 'a leftover process of the runner blocks its removal' 1 expect_out 'and is named' 'processes of runner-docs are still running (4242)' holds 'its runtime root is still there' [ -d "$H/root/var/lib/ci-docs" ] holds 'and so is its unit' [ -e "$H/$UNITS/ci-docs-runner.service" ] - rm -f "$H/fake/procs-runner-docs" + # One that dropped to root but kept the account as its saved uid can + # become the account again, so it counts as the account's too. + printf 'Name:\tleftover\nUid:\t0\t0\t999\t0\n' > "$H/proc/4242/status" + apply --retire docs + expect_rc 'a process holding the account only as its saved uid blocks removal' 1 + expect_out 'and is named' 'processes of runner-docs are still running (4242)' + rm -rf "$H/proc/4242" + # A process table that cannot be read is not an empty one -- and a + # failing pgrep (the fixture's exits 3) is not consulted at all. + mv "$H/proc" "$H/proc.away"; mkdir "$H/proc" + before=$(snapshot) + apply --retire docs + expect_rc 'a process table that cannot be read blocks removal' 1 + expect_out 'and says so' 'cannot tell whether any process runs as runner-docs' + expect_same 'and nothing was removed' "$before" + rm -rf "$H/proc"; mv "$H/proc.away" "$H/proc" + # A filesystem mounted below the runtime root -- a bigger disk at + # work/, a shared cache -- is someone's data. rm --one-file-system + # protects it only when the mount is below the argument it was given, + # so retirement refuses outright rather than rely on that. + cp "$H/proc/self/mountinfo" "$T/mountinfo.saved" + printf '97 22 0:55 / %s rw,relatime shared:60 - ext4 /dev/sdb1 rw\n' "$H/root/var/lib/ci-docs/work" >> "$H/proc/self/mountinfo" + : > "$H/root/var/lib/ci-docs/work/cache-data" + before=$(snapshot) + apply --retire docs + expect_rc 'a filesystem mounted below the runtime root blocks removal' 1 + expect_out 'and it is named' "a filesystem is mounted at or below /var/lib/ci-docs: $H/root/var/lib/ci-docs/work" + expect_same 'and nothing was removed, not even on that filesystem' "$before" + cp "$T/mountinfo.saved" "$H/proc/self/mountinfo" + printf '97 22 0:55 / %s rw,relatime shared:60 - ext4 /dev/sdb1 rw\n' "$H/root/var/lib/ci-docs" >> "$H/proc/self/mountinfo" + apply --retire docs + expect_rc 'so does a runtime root that is itself a mount point' 1 + cp "$T/mountinfo.saved" "$H/proc/self/mountinfo" + chmod 0000 "$H/proc/self/mountinfo" + apply --retire docs + chmod 0644 "$H/proc/self/mountinfo" + expect_rc 'a mount table that cannot be read blocks removal' 1 + expect_out 'and says so' 'cannot read the mount table' + rm -f "$H/root/var/lib/ci-docs/work/cache-data" + # userdel failing leaves the account; nothing that depends on it + # being gone is removed, and a second run finishes the job. + : > "$H/fake/userdel-fails" + apply --retire docs + expect_rc 'a failing account removal fails the retirement' 1 + expect_out 'and says what was left' "nothing else was removed; /var/lib/ci-docs and ci-docs-runner.service are left as they were" + holds 'the account is still there' [ -e "$H/fake/user-runner-docs" ] + holds 'its runtime root is still there' [ -f "$H/root/var/lib/ci-docs/.runner-slice-root" ] + holds 'and so is its unit' is_rendered "$H/$UNITS/ci-docs-runner.service" docs + rm -f "$H/fake/userdel-fails" + # The account removed but the root not: the unit is kept, the marker + # still proves the root, and a retry removes it. + mkdir -p "$H/root/var/lib/ci-docs/stuck"; : > "$H/root/var/lib/ci-docs/stuck/file" + chmod 0500 "$H/root/var/lib/ci-docs/stuck" + apply --retire docs + chmod 0700 "$H/root/var/lib/ci-docs/stuck" + expect_rc 'a runtime root that cannot be removed fails the retirement' 1 + expect_out 'and says what was left' 'could not remove all of /var/lib/ci-docs; ci-docs-runner.service is left in place' + refutes 'the account is already gone' [ -e "$H/fake/user-runner-docs" ] + holds 'the unit is kept for the retry' is_rendered "$H/$UNITS/ci-docs-runner.service" docs + holds 'and so is the marker that proves the root' [ -f "$H/root/var/lib/ci-docs/.runner-slice-root" ] + # The root's last step fails: the marker is written back, so the + # retry can still prove the root is this runner's. + : > "$H/fake/rmdir-fails" + apply --retire docs + rm -f "$H/fake/rmdir-fails" + expect_rc 'a runtime root whose last rmdir fails fails the retirement' 1 + holds 'and its marker is written back' grep -qx docs "$H/root/var/lib/ci-docs/.runner-slice-root" + # The unit cannot be removed: reported, and no "retired". + chmod 0555 "$H/$UNITS" + apply --retire docs + chmod 0755 "$H/$UNITS" + expect_rc 'a unit that cannot be removed fails the retirement' 1 + expect_out 'and it is named' 'could not remove' + expect_no 'and the run does not call itself retired' 'retired. Now delete' + holds 'the unit is still there for the retry' [ -e "$H/$UNITS/ci-docs-runner.service" ] + apply --retire docs + expect_rc 'a retry after a partial retirement completes it' 0 + refutes 'the runtime root is gone' [ -e "$H/root/var/lib/ci-docs" ] + refutes 'and the unit' [ -e "$H/$UNITS/ci-docs-runner.service" ] + # Put docs back as it was, for the cases that follow. + : > "$H/fake/user-runner-docs" + mkdir -p "$H/root/var/lib/ci-docs/runner" + echo docs > "$H/root/var/lib/ci-docs/.runner-slice-root" + cp "$T/docs.unit" "$H/$UNITS/ci-docs-runner.service" # A unit someone else put in place of RunnerSlice's is not removed. cp "$H/$UNITS/ci-docs-runner.service" "$T/docs.saved" printf '[Unit]\nDescription=replaced by hand\n' > "$H/$UNITS/ci-docs-runner.service" @@ -391,6 +547,40 @@ refutes 'no runner account remains' compgen -G "$H/fake/user-*" expect_lines 'no service is enabled' "$H/fake/enabled" 0 expect_lines 'no service is active' "$H/fake/active" 0 +# --------------------------------------------------------------------------- +echo +echo '== retirement removes only a root it can prove belongs to this runner ==' +# --------------------------------------------------------------------------- +new_host markers +only app +apply --profile app +apply --retire app +expect_rc 'an unregistered runner is ready for removal' 0 +# Retire leaves nothing to tamper with when it succeeds, so set the runner +# up again, unregistered and stopped, before each case. +reset_app() { + rm -rf "$H/root/var/lib/ci-app" + apply --profile app >/dev/null 2>&1 || true + apply --retire app --check >/dev/null 2>&1 || true +} +for marker in none other link; do + reset_app + m="$H/root/var/lib/ci-app/.runner-slice-root" + case "$marker" in + none) rm -f "$m" ;; + other) echo web > "$m" ;; + link) echo app > "$T/marker-elsewhere"; rm -f "$m"; ln -s "$T/marker-elsewhere" "$m" ;; + esac + # Stop it the way retire does, so only the marker decides. + sed -i '/^ci-app-runner.service$/d' "$H/fake/enabled" "$H/fake/active" + before=$(snapshot) + apply --retire app + expect_rc "a root whose marker is $marker is not removed" 1 + expect_out 'and it is named' 'refusing to remove /var/lib/ci-app: it is not a runtime root RunnerSlice created for this runner' + expect_same 'and nothing else is removed either' "$before" + holds 'not even the account' [ -e "$H/fake/user-runner-app" ] +done + # --------------------------------------------------------------------------- echo echo '== source integrity ==' diff --git a/tests/check-drill.sh b/tests/check-drill.sh index 50c9b92..82cb738 100755 --- a/tests/check-drill.sh +++ b/tests/check-drill.sh @@ -65,6 +65,8 @@ conform() { # ID PID prop "$svc" ProtectSystem full prop "$svc" ProtectHome tmpfs prop "$svc" MainPID "$pid" + # As systemd 255 prints it: yes for baseline, no for userns. + if [ "$id" = docs ]; then prop "$svc" RestrictNamespaces no; else prop "$svc" RestrictNamespaces yes; fi mountinfo "$id" "$pid" } @@ -104,7 +106,7 @@ expect_out 'the declared mask is proven in the namespace' 'masked in namespace: expect_out 'a peer runtime root is proven masked' 'masked in namespace: /var/lib/ci-docs' expect_out 'the home namespace is proven' 'exposes only its own home' expect_out 'the protected path is proven unreadable' 'runner-app cannot read /srv/example-app/.env' -expect_out 'the baseline sandbox is checked' 'RestrictNamespaces=' +expect_out 'the baseline sandbox is checked' 'RestrictNamespaces=yes' check --all expect_pass 'the converged fleet passes' @@ -247,14 +249,20 @@ refutes 'the runner binary was never executed by check' [ -e "$H/fake/listener-e echo echo '== service and namespace ==' # --------------------------------------------------------------------------- -prop ci-app-runner.service RestrictNamespaces 'cgroup ipc mnt net pid user uts' +prop ci-app-runner.service RestrictNamespaces no check --profile app -expect_fail 'a baseline runner allowed namespaces fails' 'RestrictNamespaces is' -rm -f "$H/fake/props/ci-app-runner.service/RestrictNamespaces" +expect_fail 'a baseline runner allowed namespaces fails' "RestrictNamespaces is 'no', expected 'yes'" +prop ci-app-runner.service RestrictNamespaces 'net user' +check --profile app +expect_fail 'a baseline runner allowed some namespaces fails' "RestrictNamespaces is 'net user', expected 'yes'" +prop ci-app-runner.service RestrictNamespaces yes -prop ci-docs-runner.service RestrictNamespaces 'cgroup ipc mnt net pid user uts' check --profile docs expect_pass 'a userns runner may create namespaces' +prop ci-docs-runner.service RestrictNamespaces yes +check --profile docs +expect_fail 'a userns runner that cannot is drift too' "RestrictNamespaces is 'yes', expected 'no'" +prop ci-docs-runner.service RestrictNamespaces no mountinfo app 5001 /srv/example-app check --profile app diff --git a/tests/fixture-host.sh b/tests/fixture-host.sh index 9a25d35..181cbf7 100644 --- a/tests/fixture-host.sh +++ b/tests/fixture-host.sh @@ -25,6 +25,7 @@ start() { listed active "$1" || echo "$1" >> "$FAKE/active" mkdir -p "$FAKE_CG/ci.slice/$1" "$FAKE_PROC/4000" printf '%s\0' /runner/bin/Runner.Listener run > "$FAKE_PROC/4000/cmdline" + printf 'Name:\tRunner.Listener\nUid:\t0\t0\t0\t0\n' > "$FAKE_PROC/4000/status" echo 4000 > "$FAKE_CG/ci.slice/$1/cgroup.procs" } stop() { drop active "$1"; rm -rf "${FAKE_CG:?}/ci.slice/$1"; } @@ -94,10 +95,20 @@ fi exec /usr/bin/id "$@" EOF -# Processes of an account, from a file: one PID per line. +# Processes are read from the fixture's /proc, never from pgrep. This pgrep +# fails the way a broken one does -- nothing printed, exit 3 -- so a script +# that still trusted it would read "no process" and go ahead. cat > "$BIN/pgrep" <<'EOF' #!/usr/bin/env bash -[ "$1" = -u ] && [ -s "$FAKE/procs-$2" ] && cat "$FAKE/procs-$2" +exit 3 +EOF + +# rmdir is the real one, unless the fixture makes it fail the way a racing +# new entry would. +cat > "$BIN/rmdir" <<'EOF' +#!/usr/bin/env bash +[ -e "${FAKE-}/rmdir-fails" ] && { echo "rmdir: failed to remove '${*: -1}': Directory not empty" >&2; exit 1; } +exec /usr/bin/rmdir "$@" EOF cat > "$BIN/runuser" <<'EOF' @@ -106,6 +117,16 @@ while [ "$#" -gt 0 ] && [ "$1" != -- ]; do shift; done shift exec "$@" EOF +# tar is the real one. Right after it extracts, the mode of the directory it +# extracted into is recorded: the window before RunnerSlice's own chmod. +cat > "$BIN/tar" <<'EOF' +#!/usr/bin/env bash +/usr/bin/tar "$@" || exit +dir= +while [ "$#" -gt 0 ]; do [ "$1" = -C ] && dir=$2; shift; done +[ -n "$dir" ] && [ -n "${FAKE-}" ] && stat -c %a "$dir" >> "$FAKE/tar-dir-mode" +exit 0 +EOF chmod +x "$BIN"/* # A release archive with the runner's shape and none of its content. @@ -120,6 +141,8 @@ printf '{"runtimeTarget": {"name": ".NETCoreApp,Version=v8.0/linux-x64"}, "libra > "$PAYLOAD/bin/Runner.Listener.deps.json" printf '#!/bin/sh\nexit 0\n' > "$PAYLOAD/bin/runsvc.sh" chmod +x "$PAYLOAD/config.sh" "$PAYLOAD/bin/"* +# The official archive's "." entry is 0775; so is this one's. +chmod 0775 "$PAYLOAD" ARCHIVE="$T/actions-runner-linux-x64-2.338.0.tar.gz" tar -czf "$ARCHIVE" -C "$PAYLOAD" . @@ -129,7 +152,10 @@ tar -czf "$ARCHIVE" -C "$PAYLOAD" . # new_host NAME builds $T/NAME and points H at it new_host() { H="$T/$1" - mkdir -p "$H/root" "$H/cg" "$H/proc" "$H/fake" + mkdir -p "$H/root" "$H/cg" "$H/proc/1" "$H/proc/self" "$H/fake" + printf 'Name:\tsystemd\nUid:\t0\t0\t0\t0\n' > "$H/proc/1/status" + # The mount table apply reads before removing a runtime root: only "/". + printf '22 1 8:1 / / rw,relatime shared:1 - ext4 /dev/sda1 rw\n' > "$H/proc/self/mountinfo" cp -a "$ROOT/fabric" "$H/fabric" cp -a "$ROOT/examples/basic/." "$H/config/" : > "$H/fake/enabled"; : > "$H/fake/active"; : > "$H/fake/systemctl.log" @@ -155,6 +181,12 @@ new_host() { # --- fixture host: accounts and ownership are recorded, not made --- host_preflight() { step "preflight"; ok "fixture host"; } host_ensure_user() { + # A person's account of that name: the real function refuses it (its + # cases are in the host drill); what matters here is what apply does next. + if [ -e "$H/fake/human-\$1" ]; then + fail "account \$1 exists and is not a RunnerSlice system account; refusing to adopt it" + return 1 + fi if [ -e "$H/fake/user-\$1" ]; then ok "fixture user \$1"; return 0; fi change "fixture user \$1" [ "\$CHECK_ONLY" -eq 0 ] && : > "$H/fake/user-\$1" @@ -163,7 +195,12 @@ host_ensure_user() { host_remove_user() { if [ -e "$H/fake/user-\$1" ]; then change "remove fixture user \$1" - [ "\$CHECK_ONLY" -eq 0 ] && rm -f "$H/fake/user-\$1" + [ "\$CHECK_ONLY" -eq 0 ] || return 0 + if [ -e "$H/fake/userdel-fails" ]; then + fail "userdel could not remove \$1 (exit 8); the account is still there" + return 1 + fi + rm -f "$H/fake/user-\$1" else ok "fixture user \$1 is absent" fi diff --git a/tests/host-drill.sh b/tests/host-drill.sh index 1a5b70e..3912347 100755 --- a/tests/host-drill.sh +++ b/tests/host-drill.sh @@ -339,6 +339,17 @@ pf_reset echo 245 > "$PF_STATE/systemd" result "a systemd older than the units need is refused" \ "1:systemd '245' is older than 249 or unreadable" "$(pf --user)" + +# What the code accepts is wider than what has been verified. A version in +# between passes, and says plainly that it is unverified. +pf_reset; echo 252 > "$PF_STATE/systemd" +result "a systemd the units accept but nobody verified passes, and says so" \ + "0:systemd 252 is accepted but unverified; RunnerSlice has been verified on systemd 255 only" "$(pf --user)" +pf_reset; echo 257 > "$PF_STATE/systemd" +result "so does a newer one" \ + "0:systemd 257 is accepted but unverified; RunnerSlice has been verified on systemd 255 only" "$(pf --user)" +pf_reset +result "the verified version passes without a warning" "0:" "$(pf --user)" pf_reset rm -f "$PF/cg/cgroup.controllers" @@ -398,21 +409,39 @@ echo '== accounts RunnerSlice did not create ==' ACC="$T/acct" mkdir -p "$ACC/root/etc" echo 'UID_MIN 1000' > "$ACC/root/etc/login.defs" +# Exit 2 is "not found", as the real getent; $FAKE_ACCOUNTS.getent-error +# makes every lookup fail the way an unreachable directory service does. cat > "$T/bin/getent" <<'EOF' #!/usr/bin/env bash -grep "^$2:" "$FAKE_ACCOUNTS.$1" 2>/dev/null +[ -e "$FAKE_ACCOUNTS.getent-error" ] && exit 3 +if [ "$#" -eq 1 ]; then cat "$FAKE_ACCOUNTS.$1"; exit; fi +grep "^$2:" "$FAKE_ACCOUNTS.$1" 2>/dev/null || exit 2 EOF -for c in groupadd useradd userdel groupdel; do +for c in groupadd useradd groupdel; do # shellcheck disable=SC2016 # the fake expands these when it runs printf '#!/usr/bin/env bash\necho "%s $*" >> "$FAKE_ACCOUNTS.log"\n' "$c" > "$T/bin/$c" done +# userdel removes the passwd entry, as the real one does, and the group only +# as USERGROUPS_ENAB yes would; $FAKE_ACCOUNTS.userdel can make it fail +# before (8) or after (12) the account is gone, or keep the group (keep). +cat > "$T/bin/userdel" <<'EOF' +#!/usr/bin/env bash +echo "userdel $*" >> "$FAKE_ACCOUNTS.log" +mode=$(cat "$FAKE_ACCOUNTS.userdel" 2>/dev/null) +[ "$mode" = 8 ] && exit 8 +name=${*: -1} +grep -v "^$name:" "$FAKE_ACCOUNTS.passwd" > "$FAKE_ACCOUNTS.new"; mv "$FAKE_ACCOUNTS.new" "$FAKE_ACCOUNTS.passwd" +[ "$mode" = keep ] || { grep -v "^$name:" "$FAKE_ACCOUNTS.group" > "$FAKE_ACCOUNTS.new"; mv "$FAKE_ACCOUNTS.new" "$FAKE_ACCOUNTS.group"; } +[ "$mode" = 12 ] && exit 12 +exit 0 +EOF chmod +x "$T/bin/"* export FAKE_ACCOUNTS="$ACC/db" account() { # NAME UID HOME echo "$1:x:$2:$2::$3:/bin/bash" > "$FAKE_ACCOUNTS.passwd" echo "$1:x:$2:" > "$FAKE_ACCOUNTS.group" - : > "$FAKE_ACCOUNTS.log" + : > "$FAKE_ACCOUNTS.log"; rm -f "$FAKE_ACCOUNTS.userdel" } acct() { # FUNCTION NAME prints rc:first FAIL line local out rc=0 @@ -432,7 +461,7 @@ result "it is not adopted" \ "$(acct host_ensure_user runner-app)" result "and nothing was created for it" 0 "$(grep -c . "$FAKE_ACCOUNTS.log")" result "it is not removed either" \ - "0:account runner-app is not a RunnerSlice system account; not removing it" \ + "1:account runner-app is not a RunnerSlice system account; not removing it" \ "$(acct host_remove_user runner-app)" result "and nothing was deleted" 0 "$(grep -c . "$FAKE_ACCOUNTS.log")" @@ -465,11 +494,89 @@ result "a home of that name without the account is not adopted" \ rmdir "$ACC/root/home/runner-new" account runner-app 998 /home/runner-app -# shellcheck disable=SC2016 # the fake runuser expands these when it runs -printf '#!/usr/bin/env bash\necho "userdel $*" >> "$FAKE_ACCOUNTS.log"\nexit 8\n' > "$T/bin/userdel" -result "a failing userdel is a failure, not hidden" \ - "0:userdel could not remove runner-app; nothing else was changed for it" \ +echo 8 > "$FAKE_ACCOUNTS.userdel" +result "a userdel that leaves the account is a failure the caller stops on" \ + "1:userdel could not remove runner-app (exit 8); the account is still there" \ + "$(acct host_remove_user runner-app)" + +# userdel exits 12 when the account is gone but the home is not. The outcome +# is read back, not inferred from the exit code. +account runner-app 998 /home/runner-app +echo 12 > "$FAKE_ACCOUNTS.userdel"; mkdir -p "$ACC/root/home/runner-app" +result "a home userdel left is reported, not hidden and not deleted" \ + "0:/home/runner-app is still there without its account; it is not removed automatically" \ + "$(acct host_remove_user runner-app)" +result "and the home is still there" yes "$([ -d "$ACC/root/home/runner-app" ] && echo yes || echo no)" +rmdir "$ACC/root/home/runner-app" + +# USERGROUPS_ENAB no: userdel keeps the account's group, which would then +# block reinstalling the runner forever. +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel" +result "the account's own group left by userdel is removed" "0:" "$(acct host_remove_user runner-app)" +result "with groupdel, by name" "userdel --remove runner-app|groupdel runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" + +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel"; echo "runner-app:x:998:alice" > "$FAKE_ACCOUNTS.group" +acct host_remove_user runner-app >/dev/null +result "a group with members is left" "userdel --remove runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" + +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel"; echo "svc:x:997:998::/srv/svc:/bin/false" >> "$FAKE_ACCOUNTS.passwd" +acct host_remove_user runner-app >/dev/null +result "a group another account uses as its primary group is left" "userdel --remove runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" + +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel"; echo "runner-app:x:1234:" > "$FAKE_ACCOUNTS.group" +acct host_remove_user runner-app >/dev/null +result "a group of that name with another gid is left" "userdel --remove runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" + +# Accounts that also come from a directory service cannot all be listed, so +# "no other account uses this gid" cannot be proved, and the group is kept. +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel" +printf 'passwd: files sss\ngroup: files sss\n' > "$ACC/root/etc/nsswitch.conf" +result "with accounts from a directory service, the group is kept and reported" \ + "0:group runner-app is left: this host also resolves accounts outside local files (nsswitch), so RunnerSlice cannot prove nothing else uses gid 998" \ + "$(acct host_remove_user runner-app)" +result "and groupdel was not run" "userdel --remove runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" +printf 'passwd: files systemd\ngroup: files systemd\n' > "$ACC/root/etc/nsswitch.conf" +account runner-app 998 /home/runner-app +echo keep > "$FAKE_ACCOUNTS.userdel" +acct host_remove_user runner-app >/dev/null +result "with local accounts only, the account's own group is removed" "userdel --remove runner-app|groupdel runner-app" "$(paste -sd'|' "$FAKE_ACCOUNTS.log")" +local_accts() { HOST_ROOT="$ACC/root"; host_accounts_are_local && echo local || echo not-local; } +printf 'passwd:files sss\n' > "$ACC/root/etc/nsswitch.conf" +result "no space after the colon still names the directory service" not-local "$(local_accts)" +printf 'passwd: files # sss\n' > "$ACC/root/etc/nsswitch.conf" +result "a comment after the sources is not a source" local "$(local_accts)" +printf 'passwd: files [NOTFOUND=return] ldap\n' > "$ACC/root/etc/nsswitch.conf" +result "an action does not hide the source after it" not-local "$(local_accts)" +printf 'passwd: files [NOTFOUND=return]\n' > "$ACC/root/etc/nsswitch.conf" +: > "$ACC/e"; : > "$ACC/N" +result "a file in the working directory does not change the sources" local "$(cd "$ACC" && local_accts)" +rm -f "$ACC/e" "$ACC/N" +printf 'passwd: compat\n' > "$ACC/root/etc/nsswitch.conf" +printf 'root:x:0:0::/root:/bin/bash\n' > "$ACC/root/etc/passwd" +result "compat with only local lines is local" local "$(local_accts)" +printf '+@staff::::::\n' >> "$ACC/root/etc/passwd" +result "compat pulling in NIS accounts is not" not-local "$(local_accts)" +rm -f "$ACC/root/etc/nsswitch.conf" "$ACC/root/etc/passwd" +result "no nsswitch.conf is glibc's default, files" local "$(local_accts)" + +# A lookup that fails is not "no such account". +account runner-app 998 /home/runner-app; : > "$FAKE_ACCOUNTS.getent-error" +result "a failing account lookup stops removal" \ + "1:cannot look up account runner-app (getent exit 3); nothing was removed for it" \ "$(acct host_remove_user runner-app)" +result "and nothing was deleted" 0 "$(grep -c . "$FAKE_ACCOUNTS.log")" +rm -f "$FAKE_ACCOUNTS.getent-error" + +account runner-app 998 /home/runner-app; : > "$FAKE_ACCOUNTS.passwd" +result "a group found without its account is reported, never removed" \ + "0:group runner-app exists without its account; RunnerSlice can no longer tell whether it was this runner's" \ + "$(acct host_remove_user runner-app)" +result "and nothing was deleted" 0 "$(grep -c . "$FAKE_ACCOUNTS.log")" rm -f "$T/bin/getent" "$T/bin/groupadd" "$T/bin/useradd" "$T/bin/userdel" "$T/bin/groupdel" echo @@ -491,6 +598,103 @@ result "and the boundary check fails rather than passes" yes \ "$(grep -q 'FAIL cannot test whether nobody can read' <<<"$out" && echo yes || echo no)" check_failed=0 +echo +echo '== mounts under a runtime root, as the kernel lists them ==' + +MT="$T/mnt"; mkdir -p "$MT/proc/self" "$MT/real/lib/ci-app" "$MT/real/lib/ci-app2" +ln -s "$MT/real" "$MT/linked" +mounts() { local out rc=0; out=$(PROC_ROOT="$MT/proc"; host_mounts_under "$1") || rc=$?; printf '%s:%s' "$rc" "$out"; } +table() { : > "$MT/proc/self/mountinfo"; printf '22 1 8:1 / / rw - ext4 /dev/sda1 rw\n' >> "$MT/proc/self/mountinfo" + local m; for m in "$@"; do printf '97 22 0:55 / %s rw - tmpfs t rw\n' "$m" >> "$MT/proc/self/mountinfo"; done; } +table +result "nothing mounted under the root" "0:" "$(mounts "$MT/real/lib/ci-app")" +table "$MT/real/lib/ci-app2/work" +result "a sibling sharing the prefix is not under it" "0:" "$(mounts "$MT/real/lib/ci-app")" +table "$MT/real/lib/ci-app/my\\040work" +result "an escaped space in the table is decoded" "1:$MT/real/lib/ci-app/my work" "$(mounts "$MT/real/lib/ci-app")" +table "$MT/real/lib/ci-app/work" +result "a root reached through a symlink is compared as the kernel lists it" "1:$MT/real/lib/ci-app/work" "$(mounts "$MT/linked/lib/ci-app")" +: > "$MT/proc/self/mountinfo" +result "an empty table cannot be judged" "2:" "$(mounts "$MT/real/lib/ci-app")" + +echo +echo '== processes of an account, by every uid the kernel keeps ==' + +# host_account_processes reads /proc//status. A process belongs to the +# account if its real, effective, saved or filesystem uid is the account's. +PR="$T/procscan" +proc_status() { mkdir -p "$PR/$1"; printf 'Name:\t%s\nUid:\t%s\n' "$2" "$3" > "$PR/$1/status"; } +reset_procs() { rm -rf "$PR"; mkdir -p "$PR"; proc_status 1 systemd '0 0 0 0'; } +procs() { local out rc=0; out=$(PROC_ROOT="$PR"; host_account_processes "$1") || rc=$?; printf '%s:%s' "$rc" "$(paste -sd' ' - <<<"$out")"; } +cat > "$T/bin/getent" <<'EOF' +#!/usr/bin/env bash +[ "$1" = passwd ] && [ "$2" = rs-probe ] && echo "rs-probe:x:990:990::/home/rs-probe:/bin/bash" +[ "$1" = passwd ] && [ "$2" = rs-probe ] || exit 2 +EOF +chmod +x "$T/bin/getent" + +reset_procs +result "no process of the account" "1:" "$(procs rs-probe)" +proc_status 200 job '990 990 990 990' +result "a process running as the account is found" "0:200" "$(procs rs-probe)" +reset_procs; proc_status 201 setuid '990 0 990 0' +result "a process with only its real uid is found" "0:201" "$(procs rs-probe)" +reset_procs; proc_status 202 dropped '0 0 990 0' +result "a process with only its saved uid is found" "0:202" "$(procs rs-probe)" +reset_procs; proc_status 203 other '991 991 991 991' +result "another account's process is not counted" "1:" "$(procs rs-probe)" +rm -rf "$PR"; mkdir -p "$PR" +result "a process table with nothing readable cannot be judged" "2:" "$(procs rs-probe)" +reset_procs; mkdir -p "$PR/204"; printf 'Name:\tbroken\n' > "$PR/204/status" +result "a status without a Uid line cannot be judged" "2:" "$(procs rs-probe)" +reset_procs +result "an account that does not exist has no processes to find" "1:" "$(procs rs-nobody)" +proc_status 205 locked '990 990 990 990'; chmod 0000 "$PR/205/status" +result "a status that exists but cannot be read cannot be judged" "2:" "$(procs rs-probe)" +chmod 0644 "$PR/205/status"; reset_procs +printf '#!/usr/bin/env bash\nexit 3\n' > "$T/bin/getent" +result "a failing account lookup cannot be judged" "2:" "$(procs rs-probe)" +rm -f "$T/bin/getent" + +echo +echo '== files the runner writes, read safely or not at all ==' + +RJ="$T/runner-json" +mkdir -p "$RJ/good/bin" "$RJ/bindir" "$RJ/link/bin" "$RJ/fifo/bin" "$RJ/big/bin" "$RJ/deep/bin" "$RJ/type/bin" "$RJ/ctl" +printf '\xef\xbb\xbf{"agentName": "a", "gitHubUrl": "https://github.com/o/r", "workFolder": "/w"}' > "$RJ/good/.runner" +printf '{"libraries": {"Runner.Listener/2.338.0": {}}}' > "$RJ/good/bin/Runner.Listener.deps.json" +ln -s "$RJ/good/.runner" "$RJ/link/.runner" +ln -s "$RJ/good/bin" "$RJ/bindir/bin" +mkfifo "$RJ/fifo/.runner" "$RJ/fifo/bin/Runner.Listener.deps.json" +# Valid, and matching, but past the size limit: only the limit refuses it. +python3 -c 'import json; print(json.dumps({"agentName": "a", "gitHubUrl": "https://github.com/o/r", "workFolder": "/w", "pad": " " * 2000000}))' > "$RJ/big/.runner" +python3 -c 'print("[" * 200000)' > "$RJ/deep/.runner" +python3 -c 'print("[" * 200000)' > "$RJ/deep/bin/Runner.Listener.deps.json" +# A list holding the right name would yield a version if the shape were +# not checked. +printf '{"libraries": ["Runner.Listener/2.338.0"]}' > "$RJ/type/bin/Runner.Listener.deps.json" +printf '[]' > "$RJ/type/.runner" +printf '{"agentName": "a\\u001b[2J", "gitHubUrl": "https://github.com/o/r\\u001b]0;x\\u0007", "workFolder": "/w"}' > "$RJ/ctl/.runner" + +# shellcheck disable=SC2016 # the inner shell expands these +reg() { local out rc=0; out=$(timeout 10 bash -c '. "$1/fabric/fabric-lib.sh"; . "$1/fabric/host-lib.sh"; host_registration_state "$2" o/r a /w' _ "$ROOT" "$1" 2>&1) || rc=$?; printf '%s' "$rc"; } +# shellcheck disable=SC2016 # the inner shell expands these +ver() { local rc=0; timeout 10 bash -c '. "$1/fabric/fabric-lib.sh"; . "$1/fabric/host-lib.sh"; host_runner_version "$2"' _ "$ROOT" "$1" >/dev/null 2>&1 || rc=$?; printf '%s' "$rc"; } +result "a matching registration is ok" 0 "$(reg "$RJ/good")" +result "a .runner that is a symlink is not followed" 2 "$(reg "$RJ/link")" +result "a .runner that is a FIFO is not waited on" 2 "$(reg "$RJ/fifo")" +result "an oversized .runner is refused" 2 "$(reg "$RJ/big")" +result "a deeply nested .runner is 'cannot tell'" 2 "$(reg "$RJ/deep")" +result "a .runner that is not an object is 'cannot tell'" 2 "$(reg "$RJ/type")" +result "a manifest reached through a symlinked bin/ is not followed" 2 "$(ver "$RJ/bindir")" +result "a manifest read safely gives the version" 0 "$(ver "$RJ/good")" +result "a manifest that is a FIFO is not waited on" 2 "$(ver "$RJ/fifo")" +result "a deeply nested manifest is 'cannot tell'" 2 "$(ver "$RJ/deep")" +result "a manifest with the wrong shape is 'cannot tell'" 2 "$(ver "$RJ/type")" +out=$(bash -c '. "$1/fabric/fabric-lib.sh"; . "$1/fabric/host-lib.sh"; host_registration_state "$2" o/r a /w' _ "$ROOT" "$RJ/ctl" 2>&1) +result "what .runner says is printed with control characters escaped" yes \ + "$(printf '%s' "$out" | grep -q "$(printf '\033')" && echo no || echo yes)" + echo echo '== mode comparison ==' diff --git a/tests/release-drill.sh b/tests/release-drill.sh index 63c1af9..b350d70 100755 --- a/tests/release-drill.sh +++ b/tests/release-drill.sh @@ -77,6 +77,14 @@ else fail 'the archive contents differ from the commit' diff "$T/expected.list" "$T/archive.list" | head >&2 fi +# Unpacked as root, the modes stay; root's preflight refuses a tool tree +# anyone but root could change. +writable=$(tar -tvzf "$T/a/$name.tar.gz" | awk 'substr($1, 6, 1) == "w" || substr($1, 9, 1) == "w" { print $NF }') +if [ -z "$writable" ]; then + pass 'no entry in the archive is group- or world-writable' +else + fail "group- or world-writable entries: $(head -3 <<<"$writable" | paste -sd' ' -)" +fi if [ "$(tar -tzf "$T/a/$name.tar.gz" | grep -vc "^$name/")" -eq 0 ]; then pass 'everything is under one top-level directory' else