Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,22 @@ jobs:
export DBUS_SESSION_BUS_ADDRESS="unix:path=$runtime_dir/bus"
TINYBOX_REQUIRE_NATIVE_RESOURCE_TESTS=1 cargo test -j1 -p tinybox-module native_
TINYBOX_REQUIRE_NATIVE_RESOURCE_TESTS=1 cargo test -j1 -p tinybox-module resource_aware_bus_exec_cancellation_reports_events_and_reaps_descendants
# The bubblewrap backend's limits ride on the same user manager. A skip
# is a failure here; the second run removes pasta so the shared-network
# refusal is the path tested.
- name: Prove bubblewrap jails hold their resource limits
shell: bash
run: |
set -euo pipefail
uid="$(id -u)"
export XDG_RUNTIME_DIR="/run/user/${uid}"
export DBUS_SESSION_BUS_ADDRESS="unix:path=$XDG_RUNTIME_DIR/bus"
export TINYBOX_REQUIRE_BWRAP=1 TINYBOX_REQUIRE_BWRAP_LIMITS=1
cargo test --locked -p tinybox-jail --test bwrap_resource_limits -- --nocapture
if dpkg -s passt >/dev/null 2>&1; then sudo apt-get remove --yes passt; fi
TINYBOX_REQUIRE_SHARED_NETWORK=1 cargo test --locked -p tinybox-jail \
--test bwrap_resource_limits a_hard_limit_is_refused_for_a_jail_sharing_the_host_network \
-- --nocapture
# The real-bubblewrap suites. They skip when bwrap cannot run (Ubuntu 24.04
# blocks unprivileged user namespaces by default), so this job makes bwrap
# usable and requires it: a skip here is a failure, not a pass.
Expand Down
2 changes: 1 addition & 1 deletion crates/tinybox-jail/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ spawns, never the core process itself.
| `crates/tinybox-jail/src/jail.rs` | Core types: the `Jail` description struct (builder plus `canonicalize`/`canonicalize_or_log`) and the `JailBackend` trait (`name`/`is_available`/`spawn`). |
| `crates/tinybox-jail/src/detect.rs` | `pick_backend()`: first available OS backend, else an unsupported backend that fails closed. |
| `crates/tinybox-jail/src/noop.rs` | `NoopBackend`: no enforcement, plain `Command::spawn`. Always available. |
| `crates/tinybox-jail/src/bwrap/` | Bubblewrap backend (Linux only): user/pid/ipc/uts (+net) namespaces, grants as bind mounts, `deny_read` masked, `--die-with-parent`. `isolation()` is the minimum over configurations (`Process` with `--disable-userns`, else `None`); `Kernel` only per jail without `allow_net` (`isolation_for` / `is_suitable_for_untrusted_jail`; hosts must use the per-jail form). Never looks `bwrap` up on `PATH`; the probe is a sanity check, not authentication, and `with_program` skips the ownership check. The caller's environment and credentials travel in a private `--args` file, not in argv or any host process environment (`launcher.rs`), and inherited descriptors above 2 are closed by that launcher. No seccomp filter and no resource limits yet. Landlock is the fallback. Landlock is not layered inside because the kernel denies `mount` to a Landlock-restricted thread. |
| `crates/tinybox-jail/src/bwrap/` | Bubblewrap backend (Linux only): user/pid/ipc/uts (+net) namespaces, grants as bind mounts, `deny_read` masked, `--die-with-parent`. `isolation()` is the minimum over configurations (`Process` with `--disable-userns`, else `None`); `Kernel` only per jail without `allow_net` (`isolation_for` / `is_suitable_for_untrusted_jail`; hosts must use the per-jail form). Never looks `bwrap` up on `PATH`; the probe is a sanity check, not authentication, and `with_program` skips the ownership check. The caller's environment and credentials travel in a private `--args` file, not in argv or any host process environment (`launcher.rs`), and inherited descriptors above 2 are closed by that launcher. Jails start under a seccomp deny list. CPU, memory and pid limits run through a rootless systemd scope for the controllers the user manager was delegated, and only those are claimed; a hard limit is refused for a jail that shares the host network or is granted `/run` or `/sys` (`scope.rs`). Landlock is the fallback. Landlock is not layered inside because the kernel denies `mount` to a Landlock-restricted thread. |
| `crates/tinybox-jail/src/linux.rs` | Landlock backend (Linux only, `landlock` feature, on by default). Applies the ruleset to a dedicated spawn thread so no `unsafe` `pre_exec` is needed. |
| `crates/tinybox-jail/src/macos.rs` | Seatbelt backend via `sandbox-exec`. Compiled on every host so the profile renderer is unit-tested everywhere; selected only on macOS. |
| `crates/tinybox-jail/src/windows.rs` | `AppContainerBackend`, compiled on Windows only under the one file-scoped `unsafe_code` allowance. Available through `spawn_owned_captured` / `spawn_owned_captured_with`; the legacy `spawn` returns `Unsupported` because it cannot return a waitable `std::process::Child`. |
Expand Down
25 changes: 25 additions & 0 deletions crates/tinybox-jail/src/bwrap/launcher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,31 @@ pub(super) fn command(
launcher
}

/// What `systemd-run --user` needs to reach the user manager, and nothing
/// else: the launcher otherwise runs with an empty environment.
const SCOPE_ENVIRONMENT: [&str; 2] = ["XDG_RUNTIME_DIR", "DBUS_SESSION_BUS_ADDRESS"];

/// `launcher`, run inside a transient resource scope. `scope` is the
/// `systemd-run … --` prefix (`tinybox_linux::resource_scope_argv`). In scope
/// mode `systemd-run` registers the scope and then execs the launcher in place,
/// so the descriptors mapped onto it reach the launcher unchanged. `None` for
/// an empty prefix.
pub(super) fn in_scope(scope: &[String], launcher: &Command) -> Option<Command> {
let (program, prefix) = scope.split_first()?;
let mut scoped = Command::new(program);
scoped
.env_clear()
.args(prefix)
.arg(launcher.get_program())
.args(launcher.get_args());
for key in SCOPE_ENVIRONMENT {
if let Some(value) = std::env::var_os(key) {
scoped.env(key, value);
}
}
Some(scoped)
}

#[cfg(test)]
#[path = "launcher_tests.rs"]
mod tests;
41 changes: 41 additions & 0 deletions crates/tinybox-jail/src/bwrap/launcher_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -295,3 +295,44 @@ fn entering_namespaces_puts_nsenter_in_front_by_descriptor() -> io::Result<()> {
assert_eq!(argv.get(9).map(String::as_str), Some("4"));
Ok(())
}

#[test]
fn a_scoped_launcher_runs_under_the_prefix_with_only_the_manager_variables() -> io::Result<()> {
let file = ArgsFile::create(&[], &tmp())?;
let launcher = command(
None,
"/bin/bash",
Path::new("/usr/bin/bwrap"),
&file,
0,
"prog".as_ref(),
&["arg".into()],
);
file.discard();
let prefix: Vec<String> = ["/usr/bin/systemd-run", "--user", "--scope", "--"]
.map(String::from)
.into();

let scoped = in_scope(&prefix, &launcher).ok_or_else(|| io::Error::other("no command"))?;

assert_eq!(scoped.get_program(), "/usr/bin/systemd-run");
let args: Vec<&std::ffi::OsStr> = scoped.get_args().collect();
let inner: Vec<&std::ffi::OsStr> = std::iter::once(launcher.get_program())
.chain(launcher.get_args())
.collect();
assert_eq!(
args.get(..3),
Some(&["--user", "--scope", "--"].map(std::ffi::OsStr::new)[..])
);
assert_eq!(args.get(3..), Some(&inner[..]));
for (key, _) in scoped.get_envs() {
assert!(
SCOPE_ENVIRONMENT
.iter()
.any(|allowed| key == std::ffi::OsStr::new(allowed)),
"{key:?}"
);
}
assert!(in_scope(&[], &launcher).is_none());
Ok(())
}
52 changes: 43 additions & 9 deletions crates/tinybox-jail/src/bwrap/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
//! [`JailBackend::is_suitable_for_untrusted_jail`]; the per-backend
//! `is_suitable_for_untrusted_code` is false by construction. `Kernel` here
//! means separate pid, mount, user, ipc and uts (and net) namespaces plus that
//! deny list: it does not imply resource limits.
//! deny list: it does not imply resource limits (see "Resource limits").
//!
//! # Private network
//!
Expand Down Expand Up @@ -79,10 +79,17 @@
//! `bwrap` without `--add-seccomp-fd`, or an architecture other than `x86_64`
//! and `aarch64`, leaves the backend unavailable rather than unfiltered.
//!
//! # Resource limits
//!
//! A jail's `resources` are applied through a rootless systemd scope around
//! the launcher, for each controller the user manager was delegated, and only
//! those are claimed in [`JailBackend::constraint_support`] (see `scope.rs`).
//! A hard limit is refused for a jail that could reach the user manager and
//! leave its scope: one sharing the host network, or one granted `/run` or
//! `/sys`. Disk is never enforced.
//!
//! # Known gaps (not closed in this backend)
//!
//! - No CPU, memory or pid limits (`tinybox-linux` offers them through a
//! `systemd-run` scope; not wired here).
//! - Check-then-use gap on nested grants, **only where `bwrap` lacks
//! `--bind-fd` or the kernel lacks `openat2` (before Linux 5.6)**. Then the
//! symlink check on a grant inside a writable directory
Expand Down Expand Up @@ -135,6 +142,7 @@ use crate::jail::{Jail, JailBackend};
mod args;
mod launcher;
mod pasta;
mod scope;
mod seccomp;

/// Backend name reported by [`BubblewrapBackend`].
Expand All @@ -159,6 +167,9 @@ struct Verified {
/// A working `pasta` and `nsenter`: an `allow_net` jail then gets a
/// private network instead of the host's (see `pasta`).
pasta: Option<pasta::Pasta>,
/// A trusted `systemd-run` whose scope holds CPU, memory and pid limits
/// for the controllers the user manager was delegated (see `scope`).
scope: Option<scope::ResourceScope>,
}

/// Bubblewrap (`bwrap`) namespace jail backend.
Expand Down Expand Up @@ -258,6 +269,7 @@ impl BubblewrapBackend {
bind_fds: false,
seccomp: Vec::new(),
pasta: None,
scope: None,
}),
}
}
Expand Down Expand Up @@ -293,6 +305,7 @@ fn verify(program: &Path) -> Option<Verified> {
bind_fds: supports_fd_binds(program),
seccomp,
pasta: private_network(program),
scope: scope::ResourceScope::detect(),
});
}
}
Expand Down Expand Up @@ -461,6 +474,7 @@ fn prepare_command(
jail: &Jail,
cmd: &Command,
network: Option<&pasta::Network>,
resource_scope: Option<&[String]>,
) -> io::Result<Prepared> {
let args::JailArgs { mut argv, mut fds } = args::jail_args_with_fds(
jail,
Expand Down Expand Up @@ -527,6 +541,11 @@ fn prepare_command(
cmd.get_program(),
&program_args,
);
// Before the descriptors are mapped: they are mapped onto the outermost
// process, which `systemd-run` replaces with the launcher in place.
if let Some(scoped) = resource_scope.and_then(|prefix| launcher::in_scope(prefix, &launcher)) {
launcher = scoped;
}
// Mapped onto 10, 11, … in the child only, after fork: never inheritable
// in this process, where another thread could spawn concurrently.
let mappings = fds
Expand Down Expand Up @@ -582,10 +601,13 @@ impl JailBackend for BubblewrapBackend {
}

fn constraint_support(&self) -> ConstraintSupport {
if self.is_available() {
ConstraintSupport::NONE
.with(Constraint::Filesystem, Enforcement::Enforced)
.with(Constraint::Network, Enforcement::Enforced)
if let Some(verified) = &self.verified {
scope::with_limits(
ConstraintSupport::NONE
.with(Constraint::Filesystem, Enforcement::Enforced)
.with(Constraint::Network, Enforcement::Enforced),
verified.scope.as_ref(),
)
} else {
ConstraintSupport::NONE
}
Expand Down Expand Up @@ -632,7 +654,13 @@ impl BubblewrapBackend {
self.constraint_support()
.require(self.name(), &jail.constraints())
.map_err(|error| io::Error::new(io::ErrorKind::Unsupported, error))?;
log::debug!("[cwd_jail:bwrap] spawning label={}", jail.label);
let resource_scope =
scope::prefix_for(jail, verified.scope.as_ref(), self.networks_privately())?;
log::debug!(
"[cwd_jail:bwrap] spawning label={} limited={}",
jail.label,
resource_scope.is_some()
);
// An `allow_net` jail gets a private network when `pasta` works;
// dropping `network` on any error below kills `pasta` again.
let network = match &verified.pasta {
Expand All @@ -642,7 +670,13 @@ impl BubblewrapBackend {
let Prepared {
launcher: mut wrapper,
args,
} = prepare_command(verified, jail, cmd, network.as_ref())?;
} = prepare_command(
verified,
jail,
cmd,
network.as_ref(),
resource_scope.as_deref(),
)?;
if captured {
wrapper
.stdin(Stdio::piped())
Expand Down
99 changes: 97 additions & 2 deletions crates/tinybox-jail/src/bwrap/mod_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ fn verified(program: &str) -> Verified {
bind_fds: false,
seccomp: Vec::new(),
pasta: None,
scope: None,
}
}

Expand Down Expand Up @@ -84,7 +85,7 @@ fn secrets_stay_out_of_argv_and_the_launcher_environment() -> io::Result<()> {
.env("A", "1")
.env_remove("B");
let jail = Jail::new(root.path(), "t");
let prepared = prepare_command(&verified("/usr/bin/bwrap"), &jail, &cmd, None)?;
let prepared = prepare_command(&verified("/usr/bin/bwrap"), &jail, &cmd, None, None)?;
// The host-side process: empty environment, argv of paths only.
assert_eq!(prepared.launcher.get_envs().count(), 0);
let argv = argv_of(&prepared.launcher);
Expand Down Expand Up @@ -242,7 +243,13 @@ fn a_private_network_jail_enters_the_holder_and_brings_its_own_resolver() -> io:
std::fs::File::open("/proc/self/ns/net")?,
writer.into(),
);
let prepared = prepare_command(&backend_state, &jail, &Command::new("true"), Some(&network))?;
let prepared = prepare_command(
&backend_state,
&jail,
&Command::new("true"),
Some(&network),
None,
)?;
assert_eq!(prepared.launcher.get_program(), "/usr/bin/nsenter");
let argv = argv_of(&prepared.launcher);
assert!(argv.iter().any(|a| a == "--preserve-credentials"));
Expand Down Expand Up @@ -304,3 +311,91 @@ fn a_pasta_that_fails_to_start_refuses_the_jail_instead_of_sharing_the_host_netw
.map(|e| e.kind());
assert_eq!(error, Some(io::ErrorKind::NotFound));
}

#[test]
fn limits_are_claimed_only_for_controllers_the_scope_was_delegated() {
let mut state = verified("/usr/bin/bwrap");
state.scope = Some(scope::ResourceScope::with_probe(
"/usr/bin/systemd-run",
tinybox_linux::ScopeProbe::from_parts(true, Some("memory pids")),
));
let backend = BubblewrapBackend {
verified: Some(state),
};
let support = backend.constraint_support();
assert_eq!(
support.enforcement(Constraint::Filesystem),
Enforcement::Enforced
);
assert_eq!(
support.enforcement(Constraint::Memory),
Enforcement::Enforced
);
assert_eq!(support.enforcement(Constraint::Pids), Enforcement::Enforced);
assert_eq!(
support.enforcement(Constraint::Cpu),
Enforcement::Unsupported
);
assert_eq!(
support.enforcement(Constraint::Disk),
Enforcement::Unsupported
);
// Without a scope, as before: no limit is claimed.
assert_eq!(
available()
.constraint_support()
.enforcement(Constraint::Memory),
Enforcement::Unsupported
);
}

#[test]
fn a_scoped_launch_starts_systemd_run_and_keeps_the_jail_inside_it() -> io::Result<()> {
let root = tempfile::tempdir()?;
let jail = Jail::new(root.path(), "t").deny_net();
let prefix: Vec<String> = ["/usr/bin/systemd-run", "--user", "--scope", "--"]
.map(String::from)
.into();
let prepared = prepare_command(
&verified("/usr/bin/bwrap"),
&jail,
&Command::new("true"),
None,
Some(&prefix),
)?;
assert_eq!(prepared.launcher.get_program(), "/usr/bin/systemd-run");
let argv = argv_of(&prepared.launcher);
// The launcher shell and then bwrap follow the scope's separator.
let separator = argv.iter().position(|a| a == "--");
assert_eq!(separator, Some(2));
assert!(argv.iter().skip(3).any(|a| a == "/usr/bin/bwrap"));
prepared.args.discard();
Ok(())
}

#[test]
fn a_hard_limit_the_scope_cannot_hold_refuses_the_spawn() {
let root = std::env::temp_dir();
let mut jail = Jail::new(&root, "t")
.deny_net()
.with_hard_requirements([Constraint::Cpu]);
jail.resources = Some(tinybox_core::Resources::DEFAULT);
// No scope at all, and a scope without the cpu controller.
for probe in [None, Some("memory pids")] {
let mut state = verified(&fake_bwrap().to_string_lossy());
state.scope = probe.map(|controllers| {
scope::ResourceScope::with_probe(
"/usr/bin/systemd-run",
tinybox_linux::ScopeProbe::from_parts(true, Some(controllers)),
)
});
let backend = BubblewrapBackend {
verified: Some(state),
};
let kind = backend
.spawn(&jail, Command::new("true"))
.err()
.map(|e| e.kind());
assert_eq!(kind, Some(io::ErrorKind::Unsupported), "{probe:?}");
}
}
Loading
Loading