diff --git a/aci_edge_sandboxes/README.md b/aci_edge_sandboxes/README.md index cc9b15595..c1296d51b 100644 --- a/aci_edge_sandboxes/README.md +++ b/aci_edge_sandboxes/README.md @@ -157,7 +157,7 @@ approved, and a `SetupConfig`, which the library applies once per process: }, // How long a library that pulls registry references may take to materialize one. "images": { "references": { "pullTimeoutMs": 1800000 } }, - "diagnostics": { "guestDebug": false, "contentVerification": false } + "diagnostics": { "guestDebug": false } } ``` @@ -170,17 +170,18 @@ runtime ABI. The library checks every file but itself: the caller passes the lib `RuntimeFiles::approved_sha256`. A `SandboxSpec`, passed to `AciEdgeSandbox::provision_with` beside the provision request, sets -one sandbox's image (`ImageSource::Path` of a local GPT disk, `ImageSource::Digest` of a -registered image, or `ImageSource::Reference` of a registry image where the library supports it), -its resources, its guest network, and its forwarded ports. Every guest has one +one sandbox's image (`ImageSource::Path` of a registered local GPT disk, `ImageSource::Digest` of +a registered image, or `ImageSource::Reference` of a registered registry image where the library +supports it), its resources, its guest network, and its forwarded ports. Every guest has one virtual processor, so `resources.vcpus` may only be 1, and `resources.memoryMib` sets its memory. `Capabilities::spec` lists the fields that the library honors; hostnames are not among them yet. A request may still set `microvm.provision.memoryMib`, but not together with `spec.resources.memoryMib`. -A library that honors `spec.image.reference` (`Capabilities::spec.image_reference`) converts a -reference into a local disk image with an external tool the first time a sandbox uses it, and -caches the result by the reference's text. +A library that honors `spec.image.reference` (`Capabilities::spec.image_reference`) registers a +reference with `register_reference`, or as the setup's default image when a backend opens: it +converts the reference into a local disk image with an external tool, once, and keeps the +result by the reference's text. A sandbox boots only a registered reference. The tool comes from the runtime bundle (`bin/direct-images[.exe]`), so a library given explicit runtime files does not materialize references. It does not check a tag again, so a moving tag such as `latest` keeps naming the content that was first pulled: use a version tag or a digest @@ -212,9 +213,10 @@ not search the user's Docker configuration or other credential files. The host selects credentials only for the requested registry, and rereads the file on each actual pull so it can be replaced to rotate credentials. An unrelated registry is pulled anonymously; a malformed or unreadable -selected file fails the pull. Local and cached images do not read it. The -existing state root remains the cache's trust boundary; credentials do not -restrict access to an image already cached there. +selected file fails the pull. Local images and registered references do not +read it. The existing state root remains the trust boundary of registered +images; credentials do not restrict access to an image already registered +there. The native host passes credentials through the image tool's existing password/token command-line options. They are visible to users who can @@ -238,20 +240,27 @@ would expose it. Hosts, guest sessions, and boot-console capture stay in the library for the life of the process; dropping a backend leaves its sandboxes running. - **Images.** The library registers images under the state root by their content ID, - `sha256:` (`ImageId`). Registration hashes an image once; `ImageDigest::Expect` - additionally requires a digest, and `ImageDigest::Trusted` records a digest that the caller's - own policy verified, without reading the file. A later registration, in any process, reuses - one that the file still matches without reading it. Provision records the image ID and the - runtime digests. Start compares each file's seal (volume, file ID, length, and last-write - time, plus the change time on Linux) with the one taken when it was hashed, instead of hashing - again, and fails with `backend_unavailable` if anything changed, so a start costs about the - guest's boot time. A seal detects replacement or modification, not a writer that deliberately - restores timestamps. On Windows, start also keeps writers out of the files until OpenVMM has - opened them. Register a changed image again with `register_image`: unchanged content keeps its - ID, so sandboxes that use it start again. `images` lists the registrations and whether each - file still matches, `verify_image` hashes one again, and `unregister_image` refuses while a - provisioned sandbox uses the image. `Diagnostics::content_verification` hashes the image and - runtime files again before every start, as a diagnostic. + `sha256:` (`ImageId`), the SHA-256 of the registered file as the caller gave it. + Registration is the one operation that reads an image: it hashes the image once, unless + `ImageDigest::Trusted` records a digest that the caller's own policy verified, and + `ImageDigest::Expect` additionally requires a digest. It then stages the image as a read-only + copy of the library's own, in raw form (a fixed VHD loses its footer but keeps its ID), which + `RegisteredImage::path` names and sandboxes attach, so later changes to the registered file do + not reach them. A later registration, in any process, reuses one that the file still matches + without reading the file. Provision takes only registered images: by ID, by the path that was + registered, or by the reference. A path whose file changed since its registration fails until + it is registered again, and provision never pulls, copies, or hashes an image; it records the + image ID and the runtime digests. Start compares the seal of each runtime file and of the + staged copy (volume, file ID, length, and last-write time, plus the change time on Linux) with + the one taken when it was hashed or staged, instead of hashing again, hands OpenVMM the copy + that it checked, and fails with `backend_unavailable` if anything changed, so a start costs + about the guest's boot time. A seal detects replacement or modification, not a writer that + deliberately restores timestamps. On Windows, start also keeps writers out of the files until + OpenVMM has opened them. Register an image again with `register_image` to stage it anew: + unchanged content keeps its ID, so sandboxes that use it start again. `images` lists the + registrations and whether each staged copy still matches, `verify_image` hashes one again, as a + diagnostic, and `unregister_image` refuses while a provisioned sandbox uses the image, and + otherwise removes its staged copy. - **The guest.** The image is attached read-only; the edge guest validates its GPT and p2+ ext4 layers and keeps its writable layer in RAM, so it needs no scratch disk. Start sends OpenVMM a fresh 32-byte capability through standard input, checks the process that serves the control diff --git a/aci_edge_sandboxes/examples/agent_lifecycle.rs b/aci_edge_sandboxes/examples/agent_lifecycle.rs index f3a626749..a4e5d831c 100644 --- a/aci_edge_sandboxes/examples/agent_lifecycle.rs +++ b/aci_edge_sandboxes/examples/agent_lifecycle.rs @@ -222,10 +222,7 @@ fn run() -> Result<(), String> { .with_image(image) .with_image_digest(image_digest), ) - .with_diagnostics(Diagnostics { - guest_debug, - ..Diagnostics::default() - }); + .with_diagnostics(Diagnostics { guest_debug }); if let Some(hypervisor) = hypervisor { setup = setup.with_hypervisor(hypervisor); } diff --git a/aci_edge_sandboxes/model/src/capabilities.rs b/aci_edge_sandboxes/model/src/capabilities.rs index 726da1c54..02d599e37 100644 --- a/aci_edge_sandboxes/model/src/capabilities.rs +++ b/aci_edge_sandboxes/model/src/capabilities.rs @@ -99,7 +99,7 @@ pub struct SpecCapabilities { pub image_path: bool, /// Honors `image.digest`: an image registered with the backend. pub image_digest: bool, - /// Honors `image.reference`: a container image reference. + /// Honors `image.reference`: a container image reference, which the backend also registers. pub image_reference: bool, /// Honors `resources.vcpus`. pub vcpus: bool, diff --git a/aci_edge_sandboxes/model/src/image.rs b/aci_edge_sandboxes/model/src/image.rs index fcd039c60..f550a5d45 100644 --- a/aci_edge_sandboxes/model/src/image.rs +++ b/aci_edge_sandboxes/model/src/image.rs @@ -9,7 +9,8 @@ use serde::{Deserialize, Deserializer, Serialize, Serializer}; use crate::error::{Error, Result}; use crate::hex; -/// Content identity of a registered guest image: the SHA-256 digest of the image file. +/// Content identity of a registered guest image: the SHA-256 digest of the image file as it was +/// registered, such as a fixed VHD with its footer, which a raw copy that a host stages of it keeps. /// /// An ID displays and parses as `sha256:` followed by 64 lowercase hexadecimal digits. #[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] @@ -85,12 +86,12 @@ impl<'de> Deserialize<'de> for ImageId { #[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub enum ImageDigest { - /// Hashes the file. + /// Hashes the registered file. #[default] Compute, - /// Hashes the file and requires this SHA-256 digest. + /// Hashes the registered file and requires this SHA-256 digest. Expect(#[serde(with = "hex::sha256")] [u8; 32]), - /// Records this SHA-256 digest without reading the file. + /// Records this SHA-256 digest of the registered file without hashing it. /// /// Use it only for content that the caller's own policy has verified and protects from /// writers, for example a file that an installer hashed before making it read-only. @@ -109,13 +110,15 @@ impl ImageDigest { #[serde(rename_all = "camelCase", deny_unknown_fields)] #[non_exhaustive] pub struct RegisteredImage { - /// Content identity. + /// Content identity: the SHA-256 digest of the file that was registered. pub id: ImageId, - /// Absolute path that sandboxes attach the image from. + /// Absolute path of the file that sandboxes attach: the host's own staged copy of the image, + /// for a host that stages images. A raw copy lacks the container format of the registered + /// file, such as a fixed VHD's footer, but keeps its ID. pub path: PathBuf, - /// Length in bytes at registration. + /// Length in bytes of the file at [`path`](Self::path). pub length: u64, - /// Whether the caller supplied the digest instead of the host hashing the file. + /// Whether the caller supplied the digest instead of the host hashing the image. pub trusted: bool, /// Whether the file still matches its registration. A sandbox whose image changed does not /// start until the image is registered again. diff --git a/aci_edge_sandboxes/model/src/setup.rs b/aci_edge_sandboxes/model/src/setup.rs index e82c43317..3f7668cca 100644 --- a/aci_edge_sandboxes/model/src/setup.rs +++ b/aci_edge_sandboxes/model/src/setup.rs @@ -461,9 +461,10 @@ impl ImageSettings { /// How a host that pulls registry references materializes them. /// -/// Such a host converts an [`ImageSource::Reference`] into a local disk image with an external -/// tool the first time a sandbox uses it, and caches the result by the reference's text: later -/// sandboxes with the same reference boot the cached image, and the host does not check a tag +/// Such a host registers an [`ImageSource::Reference`] when the caller registers it, or when the +/// host opens with it as the default image: it converts the reference into a local disk image +/// with an external tool, once, and keeps the result by the reference's text. Sandboxes that name +/// the reference boot that image, provisioning never pulls, and the host does not check a tag /// again. A moving tag such as `latest` therefore keeps naming the content that was first pulled; /// use a version tag or a digest (`repository@sha256:…`) to control what a sandbox boots. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] @@ -474,7 +475,8 @@ pub struct ReferenceSettings { #[serde(default = "default_pull_timeout_ms")] pub pull_timeout_ms: u64, /// Host-side Docker-compatible JSON credentials file. `None` keeps anonymous pulls. - /// The host reads it only when it must pull a reference, not for local or cached images. + /// The host reads it only when it must pull a reference, not for local images or registered + /// references. #[serde(default, skip_serializing_if = "Option::is_none")] pub auth_file: Option, } @@ -589,16 +591,15 @@ impl ExecSettings { } /// Diagnostic switches. +/// +/// None of them makes an operation hash an image or a runtime file: `verify_image` of the agent +/// backend hashes an image again, as a diagnostic of its own. #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct Diagnostics { /// Boots guests with verbose kernel and agent logging. #[serde(default)] pub guest_debug: bool, - /// Hashes every runtime file and image again before each start, instead of trusting their - /// recorded identity. - #[serde(default)] - pub content_verification: bool, } #[cfg(test)] diff --git a/aci_edge_sandboxes/model/src/spec.rs b/aci_edge_sandboxes/model/src/spec.rs index f3736480a..21f301f64 100644 --- a/aci_edge_sandboxes/model/src/spec.rs +++ b/aci_edge_sandboxes/model/src/spec.rs @@ -5,7 +5,7 @@ //! guest's resources and network identity, and forwarded ports. Every field is optional; unset //! fields take the backend's defaults, and nothing here is specific to one VMM. -use std::path::PathBuf; +use std::path::{Path, PathBuf}; use serde::{Deserialize, Serialize}; @@ -110,17 +110,22 @@ impl SandboxSpec { /// /// It serializes as `{"reference": "registry/repository:tag"}`, `{"digest": "sha256:…"}`, or /// `{"path": "…"}`. +/// +/// A backend that registers images provisions only images registered with it: a path or a +/// reference that it has not registered fails, and provisioning never pulls, copies, or hashes an +/// image. The setup's default image is registered when the backend opens. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] #[non_exhaustive] pub enum ImageSource { /// A container image reference, such as `registry/repository:tag` or - /// `registry/repository@sha256:…`. A host that pulls references caches each one by its text - /// and does not check a tag again; see [`ReferenceSettings`](crate::ReferenceSettings). + /// `registry/repository@sha256:…`. A host that pulls references registers each one by its + /// text, pulling it once, and does not check a tag again; see + /// [`ReferenceSettings`](crate::ReferenceSettings). Reference(String), /// An image already registered with the backend, by content. Digest(ImageId), - /// A local GPT disk image, by absolute path. + /// A local GPT disk image, by the absolute path from which it was registered. Path(PathBuf), } @@ -130,30 +135,42 @@ impl ImageSource { pub fn validate(&self, field: &str) -> Result<()> { match self { Self::Reference(reference) => { - if reference.is_empty() - || reference - .chars() - .any(|character| character.is_whitespace() || character.is_control()) - { - return Err(Error::malformed_request(format!( - "{field}.reference must be a nonempty image reference without whitespace \ - or control characters" - ))); - } - } - Self::Digest(_) => {} - Self::Path(path) => { - if !path.is_absolute() || path.to_str().is_none_or(|value| value.contains('\0')) { - return Err(Error::malformed_request(format!( - "{field}.path must be an absolute valid UTF-8 path without NUL characters" - ))); - } + validate_reference(reference, &format!("{field}.reference")) } + Self::Digest(_) => Ok(()), + Self::Path(path) => validate_path(path, &format!("{field}.path")), } - Ok(()) } } +/// Checks an image reference, which must be nonempty and hold no whitespace or control +/// characters, reporting [`ErrorCode::MalformedRequest`](crate::ErrorCode::MalformedRequest) for +/// `field`. +pub(crate) fn validate_reference(reference: &str, field: &str) -> Result<()> { + if reference.is_empty() + || reference + .chars() + .any(|character| character.is_whitespace() || character.is_control()) + { + return Err(Error::malformed_request(format!( + "{field} must be a nonempty image reference without whitespace or control characters" + ))); + } + Ok(()) +} + +/// Checks the path of a local image, which must be absolute and valid UTF-8 without NUL +/// characters, reporting [`ErrorCode::MalformedRequest`](crate::ErrorCode::MalformedRequest) for +/// `field`. +pub(crate) fn validate_path(path: &Path, field: &str) -> Result<()> { + if !path.is_absolute() || path.to_str().is_none_or(|value| value.contains('\0')) { + return Err(Error::malformed_request(format!( + "{field} must be an absolute valid UTF-8 path without NUL characters" + ))); + } + Ok(()) +} + /// Guest resources. #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] diff --git a/aci_edge_sandboxes/model/src/wire.rs b/aci_edge_sandboxes/model/src/wire.rs index 44b00a6dc..f5bd75823 100644 --- a/aci_edge_sandboxes/model/src/wire.rs +++ b/aci_edge_sandboxes/model/src/wire.rs @@ -8,15 +8,15 @@ use std::path::PathBuf; use serde::{Deserialize, Serialize}; -use crate::error::ErrorBody; +use crate::error::{Error, ErrorBody, Result}; use crate::image::{ImageDigest, ImageId}; use crate::model::{ExecRequest, Metadata, ProvisionRequest}; use crate::outcome::ExecOutcome; use crate::setup::RuntimeDigests; -use crate::spec::SandboxSpec; +use crate::spec::{SandboxSpec, validate_path, validate_reference}; /// Version of the sandbox ABI. A client requires a library that reports exactly this version. -pub const SANDBOX_ABI_VERSION: u32 = 1; +pub const SANDBOX_ABI_VERSION: u32 = 2; /// Input of the provision call. #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] @@ -83,17 +83,62 @@ pub enum ExecEnd { Error(ErrorBody), } -/// Input of the image registration call. +/// Input of the image registration call: the image, a local disk image or a registry image, and +/// how the host establishes its digest. +/// +/// It serializes as `{"path": "…"}` for a local image or `{"reference": "…"}` for a registry +/// image, which the host pulls first, with a `"digest"` beside either unless it is +/// [`ImageDigest::Compute`]. A registration names exactly one image, as +/// [`validate`](Self::validate) checks. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct ImageRegistration { - /// Absolute path of the image. - pub path: PathBuf, + /// Absolute path of a local GPT disk image. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub path: Option, + /// A container image reference, such as `registry/repository:tag`, which a host that supports + /// references pulls. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub reference: Option, /// How the host establishes the image's digest. #[serde(default, skip_serializing_if = "ImageDigest::is_compute")] pub digest: ImageDigest, } +impl ImageRegistration { + /// Returns the registration of the local image at `path`. + pub fn path(path: impl Into, digest: ImageDigest) -> Self { + Self { + path: Some(path.into()), + reference: None, + digest, + } + } + + /// Returns the registration of the registry image `reference`. + pub fn reference(reference: impl Into, digest: ImageDigest) -> Self { + Self { + path: None, + reference: Some(reference.into()), + digest, + } + } + + /// Checks the registration's structure, reporting + /// [`ErrorCode::MalformedRequest`](crate::ErrorCode::MalformedRequest): it names exactly one + /// image, a `path` that is absolute and valid UTF-8 without NUL characters, or a nonempty + /// `reference` without whitespace or control characters. + pub fn validate(&self) -> Result<()> { + match (&self.path, &self.reference) { + (Some(path), None) => validate_path(path, "path"), + (None, Some(reference)) => validate_reference(reference, "reference"), + _ => Err(Error::malformed_request( + "an image registration names exactly one of path and reference", + )), + } + } +} + /// Facts about an open host. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] @@ -161,6 +206,59 @@ mod tests { ); } + #[test] + fn image_registrations_name_a_local_image_or_a_registry_image() { + // A local image keeps the form that path registrations always had. + let path = std::env::temp_dir().join("image.vhd"); + let local = ImageRegistration::path(&path, ImageDigest::Compute); + assert_eq!( + serde_json::to_value(&local).unwrap(), + serde_json::json!({"path": path.to_str().unwrap()}) + ); + let pulled = + ImageRegistration::reference("registry.example/a:1", ImageDigest::Expect([1; 32])); + assert_eq!( + serde_json::to_value(&pulled).unwrap(), + serde_json::json!({"reference": "registry.example/a:1", "digest": {"expect": "01".repeat(32)}}) + ); + for registration in [&local, &pulled] { + registration.validate().unwrap(); + let json = serde_json::to_string(registration).unwrap(); + assert_eq!( + &serde_json::from_str::(&json).unwrap(), + registration + ); + } + // Exactly one image, of a valid form. + let malformed = |registration: ImageRegistration, message: &str| { + let error = registration.validate().unwrap_err(); + assert_eq!(error.code(), ErrorCode::MalformedRequest, "{message}"); + assert_eq!(error.message(), message); + }; + let one = "an image registration names exactly one of path and reference"; + malformed( + ImageRegistration { + reference: Some("registry.example/a:1".to_owned()), + ..local.clone() + }, + one, + ); + malformed( + serde_json::from_str::(r#"{"digest":"compute"}"#).unwrap(), + one, + ); + malformed( + ImageRegistration::path("image.vhd", ImageDigest::Compute), + "path must be an absolute valid UTF-8 path without NUL characters", + ); + malformed( + ImageRegistration::reference("registry.example/a :1", ImageDigest::Compute), + "reference must be a nonempty image reference without whitespace or control \ + characters", + ); + assert!(serde_json::from_str::(r#"{"image":"a"}"#).is_err()); + } + #[test] fn exec_ends_carry_an_outcome_or_an_error() { let ended = ExecEnd::Outcome(ExecOutcome::Exited(3)); diff --git a/aci_edge_sandboxes/src/agent/mod.rs b/aci_edge_sandboxes/src/agent/mod.rs index a8b2bab42..44b79803e 100644 --- a/aci_edge_sandboxes/src/agent/mod.rs +++ b/aci_edge_sandboxes/src/agent/mod.rs @@ -179,21 +179,46 @@ impl AgentBackend { self.info.runtime_digests } - /// Registers the image at `path`, reusing a registration that the file still matches without - /// reading the file. Otherwise the image is hashed, or recorded with an - /// [`ImageDigest::Trusted`] digest. + /// Registers the local image at `path`. The library hashes it once, unless `digest` is + /// [`ImageDigest::Trusted`], and stages it: sandboxes then attach the library's own read-only + /// copy, which [`RegisteredImage::path`] names, and later changes to the file at `path` do not + /// reach them. A registration that the file still matches is reused without reading the file. pub fn register_image( &self, path: impl Into, digest: ImageDigest, ) -> Result { - let registration = ImageRegistration { - path: std::path::absolute(path.into()).map_err(|error| { - Error::malformed_request("cannot resolve the image path").with_source(error) - })?, - digest, - }; - self.library.image_register(self.host, ®istration) + let path = std::path::absolute(path.into()).map_err(|error| { + Error::malformed_request("cannot resolve the image path").with_source(error) + })?; + self.register(&ImageRegistration::path(path, digest)) + } + + /// Registers the registry image `reference`: the library pulls it with the settings of + /// [`ReferenceSettings`], then hashes and stages it as + /// [`register_image`](Self::register_image) does. A reference that is registered already is + /// reused without pulling it again. A backend that does not honor references, as + /// [`Capabilities::spec`] reports, refuses it with + /// [`ErrorCode::PolicyValidation`](crate::ErrorCode::PolicyValidation). + pub fn register_reference( + &self, + reference: impl Into, + digest: ImageDigest, + ) -> Result { + self.register(&ImageRegistration::reference(reference, digest)) + } + + /// Checks the structure of `registration`, then that this backend honors it, and registers + /// its image. + fn register(&self, registration: &ImageRegistration) -> Result { + registration.validate()?; + if registration.reference.is_some() && !self.capabilities.spec.image_reference { + return Err(Error::policy_validation(format!( + "the {} backend cannot register image references", + self.capabilities.backend + ))); + } + self.library.image_register(self.host, registration) } /// Lists the registered images and whether each file still matches its registration. @@ -201,12 +226,14 @@ impl AgentBackend { self.library.image_list(self.host) } - /// Hashes a registered image and checks that it still has the content that its ID names. + /// Hashes a registered image and checks that it still has the content that its ID names: a + /// diagnostic, since no other operation hashes an image again. pub fn verify_image(&self, id: &ImageId) -> Result<()> { self.library.image_verify(self.host, &id.to_string()) } - /// Removes the registration of an image, returning whether it existed. The file stays. + /// Removes the registration of an image, and the library's staged copy of it, returning + /// whether it existed. The registered file stays. /// /// Fails with [`ErrorCode::PolicyValidation`](crate::ErrorCode::PolicyValidation) while a /// provisioned sandbox boots the image. @@ -805,4 +832,54 @@ mod tests { } ); } + + /// Returns a backend over the fake library, which refuses every image call as one for an + /// unknown handle, that honors image references or not. + fn fake_backend(image_reference: bool) -> AgentBackend { + let runtime = RuntimeSource::Files(RuntimeFiles { + openvmm: PathBuf::from("openvmm"), + kernel: PathBuf::from("vmlinux"), + initrd: PathBuf::from("initramfs"), + approved_sha256: None, + }); + let mut capabilities = Capabilities::new("agent"); + capabilities.spec.image_reference = image_reference; + let digests = RuntimeDigests { + openvmm: [0; 32], + kernel: [0; 32], + initrd: [0; 32], + }; + AgentBackend { + config: AgentConfig::new(SetupConfig::new("state", runtime), "library", [0; 32]), + library: fake::library(), + host: 0, + capabilities, + info: HostInfo::new(None, digests), + } + } + + #[test] + fn only_a_backend_that_honors_references_registers_them() { + // A malformed reference fails first, then one that the backend does not honor, before + // either reaches the library. + let refusing = fake_backend(false); + let error = refusing + .register_reference("registry.example/a :1", ImageDigest::Compute) + .unwrap_err(); + assert_eq!(error.code(), ErrorCode::MalformedRequest, "{error}"); + let error = refusing + .register_reference("registry.example/a:1", ImageDigest::Compute) + .unwrap_err(); + assert_eq!(error.code(), ErrorCode::PolicyValidation, "{error}"); + assert_eq!( + error.message(), + "the agent backend cannot register image references" + ); + + // A backend that honors references passes them to the library. + let error = fake_backend(true) + .register_reference("registry.example/a:1", ImageDigest::Compute) + .unwrap_err(); + assert!(error.message().contains("unknown handle"), "{error}"); + } } diff --git a/aci_edge_sandboxes/tests/agent_guest.rs b/aci_edge_sandboxes/tests/agent_guest.rs index ebe5d169f..43c743c5a 100644 --- a/aci_edge_sandboxes/tests/agent_guest.rs +++ b/aci_edge_sandboxes/tests/agent_guest.rs @@ -7,16 +7,19 @@ //! `EDGE_AGENT_TEST_CPU_PROFILE=host` boots the guests on a host CPU profile, for hosts that no //! built-in profile serves; the default, `auto`, uses the built-in profile. //! The host-path, network, proxy, port-forwarding, and exec-environment tests also need `python3` -//! in the image. Run the tests one at a time so that the check for leftover OpenVMM processes is +//! in the image. `registered_references_are_pulled_once_and_staged` checks a registry image only +//! when `EDGE_AGENT_TEST_IMAGE_REF` names one, with `EDGE_AGENT_TEST_BUNDLE`, a runtime bundle with +//! the image tool, and `EDGE_AGENT_TEST_BUNDLE_SHA256`, the SHA-256 of its `SOURCE-MANIFEST.json`. +//! Run the tests one at a time so that the check for leftover OpenVMM processes is //! exact, and optimized, since creating a backend hashes the runtime files and registering an -//! image hashes the image: +//! image hashes and copies the image: //! `cargo test --release --features agent --test agent_guest -- --ignored --test-threads=1`. #![cfg(feature = "agent")] use std::collections::BTreeSet; use std::collections::hash_map::RandomState; use std::hash::BuildHasher; -use std::io::{BufRead, BufReader, ErrorKind, PipeReader, Read, Write}; +use std::io::{BufRead, BufReader, ErrorKind, PipeReader, Read, Seek, SeekFrom, Write}; use std::net::{IpAddr, Ipv4Addr, TcpListener, UdpSocket}; use std::path::{Path, PathBuf}; use std::process::{Child, Command, Stdio}; @@ -26,8 +29,8 @@ use std::thread; use std::time::{Duration, Instant}; use aci_edge_sandboxes::agent::{ - AgentBackend, AgentConfig, CpuProfile, Hypervisor, ImageDigest, ImageId, RuntimeFiles, - RuntimeSource, SandboxDefaults, SetupConfig, + AgentBackend, AgentConfig, BundleSource, CpuProfile, Hypervisor, ImageDigest, ImageId, + RuntimeFiles, RuntimeSource, SandboxDefaults, SetupConfig, }; use aci_edge_sandboxes::nvx::resolve_guest_path; use aci_edge_sandboxes::{ @@ -47,16 +50,75 @@ fn required(name: &str) -> PathBuf { } fn approved_digest() -> [u8; 32] { - let hex = std::env::var("EDGE_AGENT_TEST_SHA256").expect("EDGE_AGENT_TEST_SHA256 must be set"); - assert_eq!(hex.len(), 64, "the approved digest must contain 64 digits"); + digest_from("EDGE_AGENT_TEST_SHA256") +} + +/// Returns the SHA-256 digest that the environment variable `name` holds as hexadecimal digits. +fn digest_from(name: &str) -> [u8; 32] { + let hex = std::env::var(name).unwrap_or_else(|_| panic!("{name} must be set")); + assert_eq!(hex.len(), 64, "{name} must contain 64 digits"); let mut digest = [0u8; 32]; for (index, value) in digest.iter_mut().enumerate() { *value = u8::from_str_radix(&hex[index * 2..index * 2 + 2], 16) - .expect("the approved digest must be hexadecimal"); + .unwrap_or_else(|_| panic!("{name} must be hexadecimal")); } digest } +/// Returns the SHA-256 digest of the file at `path`. +fn sha256_of(path: &Path) -> [u8; 32] { + use sha2::{Digest, Sha256}; + let mut file = std::fs::File::open(path).unwrap(); + let mut hasher = Sha256::new(); + let mut buffer = vec![0; 1 << 20]; + loop { + match file.read(&mut buffer).unwrap() { + 0 => break hasher.finalize().into(), + read => hasher.update(&buffer[..read]), + } + } +} + +/// Checks that the file at `staged` is the raw disk of the image registered from `registered`: +/// its bytes without the footer of a fixed VHD, comparing their first and last mebibyte. +fn assert_raw_copy(registered: &Path, staged: &Path) { + const FOOTER: u64 = 512; + let length = std::fs::metadata(registered).unwrap().len(); + let mut source = std::fs::File::open(registered).unwrap(); + let mut footer = [0; FOOTER as usize]; + let fixed_vhd = if length >= FOOTER { + source.seek(SeekFrom::End(-(FOOTER as i64))).unwrap(); + source.read_exact(&mut footer).unwrap(); + // A fixed VHD's footer starts with its cookie and gives disk type 2 at offset 60. + footer.starts_with(b"conectix") && footer[60..64] == 2u32.to_be_bytes() + } else { + false + }; + let disk = if fixed_vhd { length - FOOTER } else { length }; + assert_eq!( + std::fs::metadata(staged).unwrap().len(), + disk, + "{}", + staged.display() + ); + let mut copy = std::fs::File::open(staged).unwrap(); + let span = disk.min(1 << 20); + for start in [0, disk - span] { + let read = |file: &mut std::fs::File| { + let mut bytes = vec![0; usize::try_from(span).unwrap()]; + file.seek(SeekFrom::Start(start)).unwrap(); + file.read_exact(&mut bytes).unwrap(); + bytes + }; + assert!( + read(&mut source) == read(&mut copy), + "{} differs from {} at offset {start}", + staged.display(), + registered.display() + ); + } +} + /// Returns the hypervisor that `EDGE_AGENT_TEST_HYPERVISOR` names, which defaults to WHP on Windows. fn hypervisor() -> Hypervisor { match std::env::var("EDGE_AGENT_TEST_HYPERVISOR") { @@ -147,6 +209,21 @@ fn touch(path: &Path) { .unwrap(); } +/// Lets the owner write a read-only file, so that a test can change it. +fn make_writable(path: &Path) { + let mut permissions = std::fs::metadata(path).unwrap().permissions(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + permissions.set_mode(permissions.mode() | 0o200); + } + // Windows records only the read-only attribute, which this clears. + #[cfg(windows)] + #[allow(clippy::permissions_set_readonly_false)] + permissions.set_readonly(false); + std::fs::set_permissions(path, permissions).unwrap(); +} + /// Lists the OpenVMM processes running on this host, failing if they cannot be listed. fn openvmm_processes() -> BTreeSet { let executable = required("EDGE_AGENT_TEST_OPENVMM"); @@ -1364,7 +1441,7 @@ fn executions_end_with_their_caller_and_their_sandbox() { #[test] #[ignore = "requires an approved private library, edge initramfs, GPT image, and a hypervisor host"] -fn registered_images_start_without_hashing_and_fail_closed_after_a_change() { +fn registered_images_are_staged_and_start_without_hashing() { let state = tempfile::tempdir().unwrap(); let before = openvmm_processes(); let image = copied_image(state.path()); @@ -1380,15 +1457,22 @@ fn registered_images_start_without_hashing_and_fail_closed_after_a_change() { assert_eq!(backend.runtime_digests(), first.runtime_digests()); let images = backend.images().unwrap(); assert_eq!(images.len(), 1, "{images:?}"); + let staged = images[0].path.clone(); + assert_ne!(staged, image, "registration stages a copy of the image"); + let metadata = std::fs::metadata(&staged).unwrap(); + assert!(metadata.permissions().readonly(), "{}", staged.display()); assert_eq!( ( images[0].id, - &images[0].path, + images[0].length, images[0].intact, images[0].trusted ), - (id, &image, true, false) + (id, metadata.len(), true, false) ); + // The ID is the SHA-256 of the registered file, whose raw disk the staged copy holds. + assert_eq!(id, ImageId::from_sha256(sha256_of(&image))); + assert_raw_copy(&image, &staged); let client = AciEdgeSandbox::from_shared(backend.clone()); let sandbox = provision(&client, &backend); @@ -1397,8 +1481,31 @@ fn registered_images_start_without_hashing_and_fail_closed_after_a_change() { assert_prints(&client, &sandbox.id, "registered"); assert_graceful(client.stop(&sandbox.id)); - // A changed seal fails closed, without hashing, until the image is registered again. + // Sandboxes boot the staged copy, so a change to the registered file does not reach them, but + // provision refuses the file's path until it is registered again. touch(&image); + client.start(&sandbox.id).unwrap(); + assert_prints(&client, &sandbox.id, "staged"); + assert_graceful(client.stop(&sandbox.id)); + let by_path = SandboxSpec::new().with_image(ImageSource::Path(image.clone())); + assert_unavailable( + client.provision_with(&ProvisionRequest::new(), &by_path), + "changed since it was registered; register it again", + ); + assert_eq!( + backend + .register_image(&image, ImageDigest::Compute) + .unwrap() + .id, + id + ); + let other = provision_spec(&client, &backend, &ProvisionRequest::new(), &by_path); + client.deprovision(&other.release()).unwrap(); + + // A changed staged copy fails closed, without hashing, until registering the image again + // stages it anew. + make_writable(&staged); + touch(&staged); assert_unavailable( client.start(&sandbox.id), "changed since it was registered; register it again", @@ -1413,15 +1520,19 @@ fn registered_images_start_without_hashing_and_fail_closed_after_a_change() { .register_image(&image, ImageDigest::Compute) .unwrap(); assert_eq!((registered.id, registered.intact), (id, true)); + let metadata = std::fs::metadata(®istered.path).unwrap(); + assert!(metadata.permissions().readonly(), "{registered:?}"); client.start(&sandbox.id).unwrap(); assert_prints(&client, &sandbox.id, "reregistered"); assert_graceful(client.stop(&sandbox.id)); backend.verify_image(&id).unwrap(); + // Unregistering removes the staged copy and leaves the registered file. client.deprovision(&sandbox.release()).unwrap(); assert!(backend.unregister_image(&id).unwrap()); assert!(!backend.unregister_image(&id).unwrap()); assert!(backend.images().unwrap().is_empty()); + assert!(!registered.path.exists(), "{registered:?}"); assert_unavailable( client.provision(&ProvisionRequest::new()), "no longer registered", @@ -1432,36 +1543,34 @@ fn registered_images_start_without_hashing_and_fail_closed_after_a_change() { #[test] #[ignore = "requires an approved private library, edge initramfs, GPT image, and a hypervisor host"] -fn trusted_digests_and_content_verification_follow_their_contracts() { +fn trusted_digests_and_approved_runtime_files_follow_their_contracts() { let state = tempfile::tempdir().unwrap(); let before = openvmm_processes(); let image = copied_image(state.path()); - // A trusted digest is recorded without hashing, even one that the content does not have. + // A trusted digest is recorded without hashing, even one that the content does not have, and + // start never hashes the image, so only the diagnostic `verify_image` finds the difference. let claimed = ImageId::from_sha256([7; 32]); - let trusting = AgentBackend::new(adjusted(native_config(state.path(), &image), |setup| { - setup.defaults.image_digest = ImageDigest::Trusted(*claimed.sha256()); - })) - .unwrap(); + let trusting = Arc::new( + AgentBackend::new(adjusted(native_config(state.path(), &image), |setup| { + setup.defaults.image_digest = ImageDigest::Trusted(*claimed.sha256()); + })) + .unwrap(), + ); assert_eq!(trusting.image_id(), Some(claimed)); assert!(trusting.images().unwrap()[0].trusted); assert_unavailable( trusting.verify_image(&claimed), "no longer matches its SHA-256", ); - - // Content verification hashes before every start, so it refuses the misattributed image. - let verified = |setup: &mut SetupConfig| setup.diagnostics.content_verification = true; - let verifying = Arc::new( - AgentBackend::new(adjusted(native_config(state.path(), &image), verified)).unwrap(), - ); - assert_eq!(verifying.image_id(), Some(claimed)); - let client = AciEdgeSandbox::from_shared(verifying.clone()); - let sandbox = provision(&client, &verifying); - assert_unavailable(client.start(&sandbox.id), "no longer matches its SHA-256"); + let client = AciEdgeSandbox::from_shared(trusting.clone()); + let sandbox = provision(&client, &trusting); + client.start(&sandbox.id).unwrap(); + assert_prints(&client, &sandbox.id, "trusted"); + assert_graceful(client.stop(&sandbox.id)); // Runtime files are checked against approved digests and pinned per sandbox. - let digests = verifying.runtime_digests(); + let digests = trusting.runtime_digests(); let approving = |digests| { adjusted(native_config(state.path(), &image), move |setup| { runtime_files(setup).approved_sha256 = Some(digests); @@ -1488,24 +1597,104 @@ fn trusted_digests_and_content_verification_follow_their_contracts() { ); client.deprovision(&sandbox.release()).unwrap(); - // Once the image is registered by its actual digest, a verified start succeeds. - assert!(verifying.unregister_image(&claimed).unwrap()); - let actual = verifying + // Registered by its actual digest, the image verifies, and a backend that opens next reuses + // that registration. + assert!(trusting.unregister_image(&claimed).unwrap()); + let actual = trusting .register_image(&image, ImageDigest::Compute) .unwrap() .id; assert_ne!(actual, claimed); - let verifying = Arc::new( - AgentBackend::new(adjusted(native_config(state.path(), &image), verified)).unwrap(), + trusting.verify_image(&actual).unwrap(); + let reopened = AgentBackend::new(native_config(state.path(), &image)).unwrap(); + assert_eq!(reopened.image_id(), Some(actual)); + assert_no_new_openvmm(&before); +} + +#[test] +#[ignore = "requires an approved private library, edge initramfs, GPT image, and a hypervisor host"] +fn explicit_runtime_files_register_no_references() { + let state = tempfile::tempdir().unwrap(); + let backend = backend(state.path()); + let client = AciEdgeSandbox::from_shared(backend.clone()); + // Without the image tool of a runtime bundle, the library pulls no reference. + assert!(!client.capabilities().spec.image_reference); + let reference = "registry.example/image:1"; + assert_eq!( + failure(backend.register_reference(reference, ImageDigest::Compute)), + ErrorCode::PolicyValidation ); - assert_eq!(verifying.image_id(), Some(actual)); - let client = AciEdgeSandbox::from_shared(verifying.clone()); - let sandbox = provision(&client, &verifying); - let overhead = start_overhead(&client, &sandbox.id); - eprintln!("a start with content verification spent {overhead:?} outside the guest boot"); - assert_prints(&client, &sandbox.id, "verified"); + let spec = SandboxSpec::new().with_image(ImageSource::Reference(reference.to_owned())); + assert_eq!( + failure(client.provision_with(&ProvisionRequest::new(), &spec)), + ErrorCode::PolicyValidation + ); + assert_eq!(backend.images().unwrap().len(), 1); +} + +#[test] +#[ignore = "requires an approved private library, a runtime bundle, a registry, and a hypervisor host"] +fn registered_references_are_pulled_once_and_staged() { + let Ok(reference) = std::env::var("EDGE_AGENT_TEST_IMAGE_REF") else { + eprintln!("EDGE_AGENT_TEST_IMAGE_REF names no reference, so this test checked nothing"); + return; + }; + let state = tempfile::tempdir().unwrap(); + let before = openvmm_processes(); + let config = adjusted( + native_config(state.path(), &required("EDGE_AGENT_TEST_IMAGE")), + |setup| { + setup.runtime = RuntimeSource::Bundle(BundleSource { + path: std::path::absolute(required("EDGE_AGENT_TEST_BUNDLE")).unwrap(), + manifest_sha256: digest_from("EDGE_AGENT_TEST_BUNDLE_SHA256"), + }); + }, + ); + let backend = Arc::new(AgentBackend::new(config).unwrap()); + let client = AciEdgeSandbox::from_shared(backend.clone()); + assert!(client.capabilities().spec.image_reference); + + // Provision takes a reference only once it is registered, which pulls and stages it. + let spec = SandboxSpec::new().with_image(ImageSource::Reference(reference.clone())); + assert_unavailable( + client.provision_with(&ProvisionRequest::new(), &spec), + "register it first", + ); + let pulled = Instant::now(); + let registered = backend + .register_reference(&reference, ImageDigest::Compute) + .unwrap(); + eprintln!("registering {reference} took {:?}", pulled.elapsed()); + assert!(registered.intact && !registered.trusted, "{registered:?}"); + let staged = std::fs::metadata(®istered.path).unwrap(); + assert!(staged.permissions().readonly(), "{registered:?}"); + assert_eq!(registered.length, staged.len()); + + // Registering it again reuses the registration, without pulling or staging it again. + let reused = backend + .register_reference(&reference, ImageDigest::Compute) + .unwrap(); + assert_eq!(reused, registered); + let restaged = std::fs::metadata(®istered.path).unwrap(); + assert_eq!(restaged.modified().unwrap(), staged.modified().unwrap()); + + // Sandboxes boot it by its reference and by its ID. + let sandbox = provision_spec(&client, &backend, &ProvisionRequest::new(), &spec); + client.start(&sandbox.id).unwrap(); + assert_prints(&client, &sandbox.id, "pulled"); assert_graceful(client.stop(&sandbox.id)); + let by_id = SandboxSpec::new().with_image(ImageSource::Digest(registered.id)); + let other = provision_spec(&client, &backend, &ProvisionRequest::new(), &by_id); + client.deprovision(&other.release()).unwrap(); + + // Unregistering removes the staged copy, and the reference must be registered again. client.deprovision(&sandbox.release()).unwrap(); + assert!(backend.unregister_image(®istered.id).unwrap()); + assert!(!registered.path.exists(), "{registered:?}"); + assert_unavailable( + client.provision_with(&ProvisionRequest::new(), &spec), + "register it first", + ); assert_no_new_openvmm(&before); } diff --git a/nvx-rs/README.md b/nvx-rs/README.md index e2d136b90..9829a3332 100644 --- a/nvx-rs/README.md +++ b/nvx-rs/README.md @@ -85,7 +85,7 @@ nvx-rs/ | `nvx::host::supervisor` | The supervisor of a sandbox's VMM, beneath the CLI, `Instance`, and external orchestrators, which keep the configuration, the authorization, and the application protocol. `launch` runs the store's launch steps under the sandbox's lock: it writes the journal's marker, spawns OpenVMM with the claimed log, records its `ProcessRef` right after, and returns a `Starting` launch that the caller commits once the VMM serves its control endpoint, or abandons. A launch that fails never leaves an unrecorded VMM running. `VmmCommand::inherit` lists the descriptors (Linux) or handles (Windows) that the VMM inherits under their numbers, besides its standard streams, and it inherits nothing else of the caller's: on Linux, they must be close-on-exec, a flag that only the VMM's own process clears; on Windows, they join the VMM's explicit handle list. `Vmm::stop` runs one graceful step of the caller's and waits for the VMM's exit event until a deadline, and takes a bounded forced step through the VMM's verified identity only on request. `Vmm::terminate` takes that forced step alone, within the caller's bound, for a VM whose guest the caller must not ask anything, such as one of another runtime identity. `reattach` recovers an interrupted launch, and holds the recorded VMM again after the orchestrator restarts within the same host boot: `Live`, `Mismatched` with another runtime identity, `Exited`, or `Indeterminate` | | `nvx::host::seal` | `FileSeal`: what the host reports about an open file from its metadata alone, without reading it: the platform scheme, the device or volume serial number, the file ID, the kind, the length, the modification time, and on Linux the change time, with a versioned serde form. A seal detects accidental change, not tampering; see [Distro bindings](#distro-bindings) | | `nvx::host::session`, `platform`, `report` | The authenticated control session, which carries NVX's control protocol, `NVXC`, one frame per DATA record of a `HostConnection`; Linux and Windows primitives; and OpenVMM's outcome report | -| `nvx::host::sandbox` | The launch profile of `nvx.py sandbox`, ported from the one that `scripts/nvx_tools/sandbox.py` held: `SandboxLaunch`, with the EROFS distro `SandboxLayer`, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and resource limits, and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` and `--microvm-workload-identity` arguments, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx.py sandbox` and `nvx.py run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx.py run` shares, and the managed exec environment; see [Python parity](#python-parity). `distro`: the `DistroBinding` of a distro layer, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | +| `nvx::host::sandbox` | The launch profile of `nvx.py sandbox`, ported from the one that `scripts/nvx_tools/sandbox.py` held: `SandboxLaunch`, with the distro `SandboxLayer`, an EROFS layer or a GPT disk, the one layer that a sandbox takes, the ext4 scratch disk, the entrypoint, arguments, hostname, workload identity, and resource limits, and the `SandboxPlan` that it makes of them. `plan`: `SandboxPlan`, the typed plan of a cold layered launch, which is persistable in a versioned serde form: one read-only `distro` block device, an EROFS filesystem or a GPT disk with the identity that the guest expects of it, if any, an upper, the ext4 scratch disk or RAM, the launch settings, the `ExportPlan` of its live shares, and optionally a `SandboxNetwork`, the guest network and its typed `NetworkPolicy`, whose proxy's URL as the guest sees it the plan reports. The plan of an adapter that boots its own guest, `SandboxPlan::adapter`, holds no launch settings and no export plan. A plan knows only the `distro` and `scratch` `BlockRole`s, and gives, for every combination, its `--microvm-sandbox-block` arguments, with `--microvm-workload-identity` when it has launch settings, with no scratch block device for a RAM upper, and those of its network, which `nvx::host::network` lowers, and its kernel command line, with the boot tokens of `nvx::protocol::root`, within the 1024-byte budget that OpenVMM leaves it, refusing one whose root tokens the guest's parser refuses. Both deliver the live shares as the managed lifecycle does: the arguments export them at `nvx::host::filesystem::GUEST_EXPORT`, the command line announces the number of their mappings with `nvx_maps=`, and `mapping_table` gives the ordered table that `start` sends once the guest agent is ready; the CLI's one-shot `run` still renders them as boot tokens. An adapter's plan renders the caller's tokens before those of its root, and refuses tokens that its root or OpenVMM's shares own. `mount`: `SandboxMount` requests of live shares of host directories and files, their guest targets and the reserved ones, their access policies and owner, and the checks of their host paths; `plan_mounts`, which plans them through `nvx::host::filesystem` for the container root, and `boot_shares`, which renders a plan as the OpenVMM arguments and boot tokens of one share or of the aggregate of several, as the one-shot lifecycle delivers them. `options`: the checks that `nvx.py sandbox` and `nvx.py run` apply to their options: the attribution of `--mount-deny`, `--mount-allow`, and `--mount-write` to shares, the systemd entrypoint check and the explicit verification of the distro layer, the targets of `nvx.py run` shares, and the managed exec environment; see [Python parity](#python-parity). `distro`: the `DistroBinding` of a distro layer, an EROFS layer or a GPT disk, which explicit verification makes or a staging caller imports, and its check from metadata alone, which returns the layer open; see [Distro bindings](#distro-bindings). `scratch`: the `ScratchBinding` of a scratch image, its identity, kind, and length, and its check from metadata alone, which claims the image for one sandbox at a time and returns it open. `prepared`: `PreparedLaunch`, the launch of a plan with the checked files of its block devices, which renders the plan's OpenVMM arguments and kernel command line, naming the checked files as `/proc/self/fd/N` on Linux and by their final paths on Windows, and lists the descriptors that the VMM inherits through `VmmCommand::inherit`; see [Checked handoff](#checked-handoff) | | `nvx::testing::fake_openvmm` | A fake OpenVMM that serves the control console and runs scripted workloads without a VM, for direct launches and for the managed and one-shot sandboxes of `nvx-rs sandbox` and `nvx.py sandbox` | | `nvx::host::cli` | Feature `cli`: the `nvx-rs` executable; see [The nvx-rs CLI](#the-nvx-rs-cli) | | `nvx::guest` | The programs that run in Linux guests; see [Guest binaries](#guest-binaries) | @@ -187,10 +187,11 @@ verification reads it: A `DistroBinding` ties the `FileSeal` of the layer to the seal of its manifest, or to the manifest's absence, to the revision and result of the distro policy, `DISTRO_POLICY_REVISION`, and to the layer's format and the -identity that the guest expects of it, its EROFS UUID. It comes from explicit +identity that the guest expects of it: the UUID of an EROFS layer, or the disk +GUID of a GPT disk, which `SandboxLayer::gpt` names. It comes from explicit verification, or from `DistroBinding::import`, which a staging caller that verified the layer itself calls to bind it without hashing it again. `check` -fails for a binding of another version, layer, or policy revision, a verdict +fails for a binding of another version, layer, format, or policy revision, a verdict that rejects the layer, a layer whose identity, kind, length, or times changed, and a manifest that appeared, disappeared, or changed. A binding of a layer without a manifest holds only while nothing is at the manifest's path: unlike diff --git a/nvx-rs/src/host/sandbox/distro.rs b/nvx-rs/src/host/sandbox/distro.rs index a5d0f87f6..134dda062 100644 --- a/nvx-rs/src/host/sandbox/distro.rs +++ b/nvx-rs/src/host/sandbox/distro.rs @@ -95,13 +95,16 @@ impl Access<'static> { pub enum DistroFormat { /// An EROFS filesystem image, whose filesystem UUID the guest checks before it mounts it. Erofs, + /// A GPT disk image, whose disk GUID the guest checks before it uses the disk. + Gpt, } impl DistroFormat { - /// Returns the name of the format: `erofs`. + /// Returns the name of the format: `erofs` or `gpt`. pub const fn as_str(self) -> &'static str { match self { Self::Erofs => "erofs", + Self::Gpt => "gpt", } } } @@ -153,7 +156,7 @@ impl PolicyVerdict { /// "image": SEAL, "manifest": SEAL or null, "policy": {"revision": 1, "result": "accepted"}} /// ``` /// -/// where each `SEAL` is the serde form of a [`FileSeal`]. +/// where `format` is `erofs` or `gpt`, and each `SEAL` is the serde form of a [`FileSeal`]. #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase", deny_unknown_fields)] pub struct DistroBinding { @@ -161,7 +164,8 @@ pub struct DistroBinding { version: u32, /// The layer's format. format: DistroFormat, - /// The identity that the guest expects of the layer: the UUID of its EROFS filesystem. + /// The identity that the guest expects of the layer: the UUID of its EROFS filesystem, or the + /// GUID of its GPT disk. identity: String, /// The seal of the layer. image: FileSeal, @@ -290,7 +294,8 @@ impl DistroBinding { self.format } - /// Returns the identity that the guest expects of the layer: the UUID of its filesystem. + /// Returns the identity that the guest expects of the layer: the UUID of its EROFS + /// filesystem, or the GUID of its GPT disk. pub fn identity(&self) -> &str { &self.identity } @@ -319,7 +324,7 @@ impl DistroBinding { ) -> Self { Self { version: BINDING_VERSION, - format: DistroFormat::Erofs, + format: layer.format(), identity: layer.uuid().to_owned(), image, manifest, @@ -343,7 +348,16 @@ impl DistroBinding { self.version ))); } - if (self.format, self.identity.as_str()) != (DistroFormat::Erofs, layer.uuid()) { + if self.format != layer.format() { + return Err(stale(format!( + "names the {} layer with identity {}, not the {} layer with identity {}", + self.format, + self.identity, + layer.format(), + layer.uuid() + ))); + } + if self.identity != layer.uuid() { return Err(stale(format!( "names the {} layer with identity {}, not {}", self.format, @@ -504,9 +518,12 @@ pub(crate) fn import_with( Ok(DistroBinding::new(layer, image, manifest, verdict)) } -/// Returns how messages name the image of `layer`: `distro EROFS layer`. +/// Returns how messages name the image of `layer`: `distro EROFS layer` or `distro GPT image`. fn layer_description(layer: &SandboxLayer) -> String { - format!("{} EROFS layer", layer.role()) + match layer.format() { + DistroFormat::Erofs => format!("{} EROFS layer", layer.role()), + DistroFormat::Gpt => format!("{} GPT image", layer.role()), + } } /// Returns the path of the manifest of the layer at `path`: `LAYER.manifest.json` beside it. diff --git a/nvx-rs/src/host/sandbox/distro/tests.rs b/nvx-rs/src/host/sandbox/distro/tests.rs index a297dd8fa..e4da22e03 100644 --- a/nvx-rs/src/host/sandbox/distro/tests.rs +++ b/nvx-rs/src/host/sandbox/distro/tests.rs @@ -609,6 +609,65 @@ fn imports_bind_a_staged_layer_without_reading_it() { ); } +#[test] +fn gpt_disks_bind_by_their_disk_guid() { + let staged = Staged::new(false); + let image = shown(&staged.image()); + let gpt = SandboxLayer::gpt(LayerRole::Distro, staged.image().to_str().unwrap(), UUID); + + // Verification of a disk without a manifest reads nothing, and binds what an import binds. + let binding = verify_with(&gpt, METADATA_ONLY).unwrap(); + assert_eq!( + (binding.format(), binding.identity()), + (DistroFormat::Gpt, UUID) + ); + assert_eq!( + import_with(&gpt, PolicyVerdict::ACCEPTED, METADATA_ONLY).unwrap(), + binding + ); + let value = serde_json::to_value(&binding).unwrap(); + assert_eq!(value["format"], "gpt"); + assert_eq!( + serde_json::from_value::(value).unwrap(), + binding + ); + + // The check returns the disk, with the format and disk GUID that the binding names. + let checked = binding + .check_with(&gpt, DISTRO_POLICY_REVISION, METADATA_ONLY) + .unwrap(); + assert_eq!(checked.bound(), (DistroFormat::Gpt, UUID)); + assert_eq!(&FileSeal::of(checked.file()).unwrap(), binding.image()); + + // A binding of one format checks no image of the other, even at its path and identity. + let erofs = staged.layer(); + let erofs_binding = DistroBinding::verify(&erofs).unwrap(); + for (bound, launched) in [(&binding, &erofs), (&erofs_binding, &gpt)] { + assert_eq!( + failure(bound.check_with(launched, DISTRO_POLICY_REVISION, METADATA_ONLY)), + ( + ErrorKind::Unavailable, + format!( + "the distro binding of {image} names the {} layer with identity {UUID}, not \ + the {} layer with identity {UUID}", + bound.format(), + launched.format() + ) + ) + ); + } + + // Messages name a missing disk a GPT image. + fs::remove_file(staged.image()).unwrap(); + assert_eq!( + failure(DistroBinding::import(&gpt, PolicyVerdict::ACCEPTED)), + ( + ErrorKind::Unavailable, + format!("distro GPT image not found: {image}") + ) + ); +} + /// Makes `path` a symbolic link to itself, whose lookup fails for another reason than a missing /// file, and returns whether the host allowed it: Windows lets only Developer Mode or a privilege /// create one. diff --git a/nvx-rs/src/host/sandbox/layer.rs b/nvx-rs/src/host/sandbox/layer.rs index a7423ac48..002fd0a6a 100644 --- a/nvx-rs/src/host/sandbox/layer.rs +++ b/nvx-rs/src/host/sandbox/layer.rs @@ -1,8 +1,9 @@ -//! The read-only EROFS layers of a sandbox. +//! The read-only layers of a sandbox: EROFS filesystems and GPT disks. use std::fmt; use std::path::Path; +use super::distro::DistroFormat; use super::refused; use crate::error::Result; use crate::host::python::builtins::repr; @@ -42,20 +43,24 @@ impl fmt::Display for LayerRole { } } -/// A read-only EROFS layer: its role, its host image, and the UUID of its filesystem. +/// A read-only layer: its role, its host image, the image's format, and the identity that the +/// guest expects of it, the UUID of an EROFS filesystem or the GUID of a GPT disk. #[derive(Clone, Debug, PartialEq, Eq)] pub struct SandboxLayer { /// The layer's role. role: LayerRole, /// The image, as `pathlib` normalizes it. path: PurePath, - /// The filesystem UUID, which the guest checks before it mounts the layer. + /// The image's format. + format: DistroFormat, + /// The identity that the guest checks before it uses the image: the UUID of an EROFS + /// filesystem, or the GUID of a GPT disk. uuid: String, } impl SandboxLayer { - /// Returns the layer of `role` at `path` with filesystem `uuid`, which are used as given, - /// except that `path` is normalized as `pathlib.Path` normalizes it. + /// Returns the EROFS layer of `role` at `path` with filesystem `uuid`, which are used as + /// given, except that `path` is normalized as `pathlib.Path` normalizes it. pub fn new(role: LayerRole, path: &str, uuid: impl Into) -> Self { Self::new_in(Flavor::NATIVE, role, path, uuid) } @@ -70,10 +75,30 @@ impl SandboxLayer { Self { role, path: PurePath::new(flavor, path), + format: DistroFormat::Erofs, uuid: uuid.into(), } } + /// Returns the GPT disk of `role` at `path` with the disk GUID `guid`, which are used as + /// given, except that `path` is normalized as `pathlib.Path` normalizes it. + pub fn gpt(role: LayerRole, path: &str, guid: impl Into) -> Self { + Self::gpt_in(Flavor::NATIVE, role, path, guid) + } + + /// Implements [`gpt`](Self::gpt) for paths of `flavor`. + pub(crate) fn gpt_in( + flavor: Flavor, + role: LayerRole, + path: &str, + guid: impl Into, + ) -> Self { + Self { + format: DistroFormat::Gpt, + ..Self::new_in(flavor, role, path, guid) + } + } + /// Parses a `--layer` value, `ROLE,PATH,EROFS_UUID`, whose role is one of the /// [`LAYER_ROLES`]. /// @@ -125,7 +150,13 @@ impl SandboxLayer { &self.path } - /// Returns the UUID of the layer's filesystem. + /// Returns the format of the layer's image. + pub fn format(&self) -> DistroFormat { + self.format + } + + /// Returns the identity that the guest expects of the layer's image: the UUID of an EROFS + /// filesystem, or the GUID of a GPT disk. pub fn uuid(&self) -> &str { &self.uuid } @@ -158,6 +189,37 @@ mod tests { assert_eq!(layer.uuid(), "11111111-1111-1111-1111-111111111111"); } + #[test] + fn layers_keep_the_format_of_their_image() { + let erofs = SandboxLayer::new_in( + Flavor::Posix, + LayerRole::Distro, + "l//d.erofs", + "22222222-2222-2222-2222-222222222222", + ); + assert_eq!(erofs.format(), DistroFormat::Erofs); + let gpt = SandboxLayer::gpt_in( + Flavor::Posix, + LayerRole::Distro, + "l//d.img", + "22222222-2222-2222-2222-222222222222", + ); + assert_eq!(gpt.format(), DistroFormat::Gpt); + assert_eq!(gpt.pure_path().as_str(), "l/d.img"); + assert_eq!(gpt.uuid(), erofs.uuid()); + assert_eq!(gpt.role(), LayerRole::Distro); + // `--layer` names EROFS layers. + assert_eq!( + SandboxLayer::parse_in( + Flavor::Posix, + "distro,l/d.erofs,22222222-2222-2222-2222-222222222222" + ) + .unwrap() + .format(), + DistroFormat::Erofs + ); + } + #[test] fn layer_parser_rejects_invalid_role_and_uuid() { assert_eq!(LayerRole::from_name("distro"), Some(LayerRole::Distro)); diff --git a/nvx-rs/src/host/sandbox/mod.rs b/nvx-rs/src/host/sandbox/mod.rs index 9d4e05a95..f83b76ade 100644 --- a/nvx-rs/src/host/sandbox/mod.rs +++ b/nvx-rs/src/host/sandbox/mod.rs @@ -12,17 +12,18 @@ //! policy may be proxy-only. It renders the OpenVMM arguments, //! [`openvmm_arguments`](SandboxPlan::openvmm_arguments), and the kernel command line, //! [`kernel_command_line`](SandboxPlan::kernel_command_line), whose boot tokens -//! [`protocol::root`](crate::protocol::root) defines. -//! - [`SandboxLayer`]: the EROFS distro layer of `nvx.py sandbox`, `ROLE,PATH,EROFS_UUID`, the -//! one [`LayerRole`] that a sandbox takes. +//! [`protocol::root`](crate::protocol::root) defines. An adapter that runs its own guest plans +//! the same root with [`SandboxPlan::adapter`], without the profile's workload settings. +//! - [`SandboxLayer`]: the distro layer of a sandbox, the one [`LayerRole`] that a sandbox takes: +//! an EROFS filesystem, as `nvx.py sandbox` names it with `ROLE,PATH,EROFS_UUID`, or a GPT disk. //! - [`SandboxMount`]: a requested live share of a host directory or regular file at a guest //! target, with its access mode, owner, and access-policy paths; [`mount`] holds the rules of //! guest targets and the checks of host paths, plans the shares of a sandbox through the planner //! of every NVX caller, [`host::filesystem`](crate::host::filesystem), and renders the plan as //! the guest's shell reads it at boot. //! - [`SandboxLaunch`]: the launch of `nvx.py sandbox`, its layers over its scratch image with -//! its settings and the export plan of its shares, which plans its EROFS distro layer over the -//! scratch image. +//! its settings and the export plan of its shares, which plans its distro layer, in the layer's +//! format, over the scratch image. //! - [`options`]: the checks that `nvx.py sandbox` and `nvx.py run` apply to their options: the //! attribution of access-policy paths to shares, the systemd entrypoint check and the preflight //! that verifies the distro layer, the targets of `nvx.py run` shares, and the managed exec @@ -59,7 +60,7 @@ pub mod scratch; use std::path::Path; -use self::distro::DistroBinding; +use self::distro::{DistroBinding, DistroFormat}; pub use self::layer::{LAYER_ROLES, LayerRole, SandboxLayer}; pub use self::mount::{MountAccess, MountKind, MountOwner, MountPolicy, PolicyKind, SandboxMount}; pub use self::plan::{ @@ -271,8 +272,8 @@ fn validate_settings(settings: &SandboxSettings) -> Result<()> { } /// The launch of a sandbox of `nvx.py sandbox`: its read-only layers, its ext4 scratch image, its -/// settings, the export plan of its shares, and its network, which it plans as an EROFS distro -/// layer over the scratch image. +/// settings, the export plan of its shares, and its network, which it plans as its distro layer, +/// in the layer's format, over the scratch image. /// /// Its layers are keyed by the role of each `--layer` value, of which a launch takes only the /// distro layer. [`plan`](Self::plan) returns its [`SandboxPlan`], which renders its OpenVMM @@ -283,8 +284,8 @@ fn validate_settings(settings: &SandboxSettings) -> Result<()> { pub struct SandboxLaunch { /// The read-only layers, in the order given: the distro layer alone. layers: Vec, - /// The plan: the distro layer, an EROFS image with its UUID, over the scratch image, with the - /// settings and the export plan. + /// The plan: the distro layer, in its format and with its identity, over the scratch image, + /// with the settings and the export plan. plan: SandboxPlan, } @@ -339,8 +340,12 @@ impl SandboxLaunch { repr(distro.uuid()) ))); }; + let image = match distro.format() { + DistroFormat::Erofs => RootImage::erofs(Some(uuid)), + DistroFormat::Gpt => RootImage::gpt(Some(uuid)), + }; let plan = SandboxPlan::new( - SandboxDistro::at(distro.pure_path().clone(), RootImage::erofs(Some(uuid))), + SandboxDistro::at(distro.pure_path().clone(), image), SandboxUpper::scratch_in(flavor, scratch), settings, )?; @@ -367,7 +372,9 @@ impl SandboxLaunch { /// Returns the settings besides the layers and the scratch image. pub fn settings(&self) -> &SandboxSettings { - self.plan.settings() + self.plan + .settings() + .expect("a launch plans the sandbox profile, with its settings") } /// Returns the export plan of the shares, if any. @@ -401,9 +408,9 @@ impl SandboxLaunch { self.plan.network() } - /// Returns the plan of the launch: its distro layer, an EROFS image with the UUID of its - /// `--layer` value, over its scratch image, with its settings, the export plan of its shares, - /// and its network. + /// Returns the plan of the launch: its distro layer, in the format and with the identity of + /// its layer, as a `--layer` value names an EROFS layer and its UUID, over its scratch image, + /// with its settings, the export plan of its shares, and its network. pub fn plan(&self) -> &SandboxPlan { &self.plan } diff --git a/nvx-rs/src/host/sandbox/plan.rs b/nvx-rs/src/host/sandbox/plan.rs index 27754229c..4bc26e1f9 100644 --- a/nvx-rs/src/host/sandbox/plan.rs +++ b/nvx-rs/src/host/sandbox/plan.rs @@ -9,11 +9,13 @@ //! [`RootImage`] names them; //! - an upper, a [`SandboxUpper`]: a whole-device ext4 scratch image on the `scratch` block //! device, or RAM, with no scratch block device; -//! - the launch settings, [`SandboxSettings`]: the entrypoint and its arguments, the hostname, the -//! workload identity, and the resource limits; +//! - the launch settings of NVX's sandbox profile, [`SandboxSettings`]: the entrypoint and its +//! arguments, the hostname, the workload identity, and the resource limits. An adapter's plan, +//! [`SandboxPlan::adapter`], has none: the adapter's own guest runs its workloads; //! - the export plan of the live shares, an [`ExportPlan`] that [`filesystem::plan`](crate::host::filesystem::plan) made for the //! container root, if the sandbox has shares: the children of OpenVMM's aggregate export, the -//! ordered binds of the guest, and the identities that each start checks again; +//! ordered binds of the guest, and the identities that each start checks again. An adapter's +//! plan has none: the adapter attaches its shares itself; //! - optionally, a network, [`SandboxNetwork`]: the guest's address and the [`NetworkPolicy`] //! that OpenVMM enforces there, which one lowering, [`network::openvmm_arguments`], renders for //! every caller. A proxy-only policy reports the URL of its proxy as the guest sees it, @@ -31,13 +33,17 @@ //! sends the plan's [`mapping_table`](SandboxPlan::mapping_table) in `MAPS` requests once the //! guest agent is ready, which maps each in order beneath the container's root. The CLI's //! one-shot lifecycle still renders them as boot tokens, from which the guest's shell binds them, -//! as [`mount::boot_shares`] does. The plan's arguments name the block devices by their paths; a +//! as [`mount::boot_shares`] does. An adapter's plan renders its root and devices alone, and lets +//! the adapter's own boot tokens through, as +//! [`kernel_command_line`](SandboxPlan::kernel_command_line) says. The plan's arguments name the +//! block devices by their paths; a //! launch renders them through a [`PreparedLaunch`](super::PreparedLaunch), which names the files //! that the checks of the images opened instead. //! //! A plan is persistable. Its serde form, of version [`PLAN_VERSION`], requires every field and //! refuses unknown ones and other versions, and reading it checks the plan as -//! [`SandboxPlan::new`], [`SandboxPlan::with_export`], and [`SandboxNetwork::new`] do: +//! [`SandboxPlan::new`] or [`SandboxPlan::adapter`], [`SandboxPlan::with_export`], and +//! [`SandboxNetwork::new`] do: //! //! ```json //! {"version": 3, @@ -53,7 +59,8 @@ //! //! The `format` of the image is `erofs` or `gpt`, its `identity` a UUID in its canonical text or //! `null`, and a RAM upper is `{"kind": "ram"}`. The limits are decimal strings or `null`, since a -//! memory limit may exceed the integers that a JSON number holds exactly. `EXPORT` is `null` +//! memory limit may exceed the integers that a JSON number holds exactly. The `settings` of an +//! adapter's plan are `null`, as is its `EXPORT`. `EXPORT` is `null` //! without shares, and otherwise the serde form of an [`ExportPlan`]: //! //! ```json @@ -115,7 +122,8 @@ use crate::host::python::builtins::{repr, split, strip}; use crate::host::python::path::{Flavor, PurePath}; use crate::protocol::HostMap; use crate::protocol::root::{ - DISTRO_MMIO_BASE, ImageFormat, RootImage, RootLayout, RootUpper, SCRATCH_MMIO_BASE, + DISTRO_MMIO_BASE, ImageFormat, LAYER_TOKEN, RAM_UPPER_TOKEN, RootImage, RootLayout, RootUpper, + SCRATCH_MMIO_BASE, SCRATCH_TOKEN, }; use crate::uuid::Uuid; @@ -351,6 +359,10 @@ pub(crate) fn guest_gateway(guest_network: &str) -> Result { /// The plan of a cold layered launch: everything that its OpenVMM arguments and kernel command /// line depend on. /// +/// A plan launches either NVX's sandbox profile, whose guest runs one workload with the plan's +/// [`SandboxSettings`], [`new`](Self::new), or an adapter's own guest, which runs its workloads +/// itself, [`adapter`](Self::adapter). +/// /// Its serde form is the persisted plan of the [module](self). #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] #[serde(into = "form::PlanForm", try_from = "form::PlanForm")] @@ -359,9 +371,9 @@ pub struct SandboxPlan { distro: SandboxDistro, /// The upper. upper: SandboxUpper, - /// The launch settings. - settings: SandboxSettings, - /// The export plan of the live shares, if any. + /// The launch settings of the sandbox profile, or `None` for an adapter's plan. + settings: Option, + /// The export plan of the live shares, if any, which an adapter's plan never has. export: Option, /// The network, or `None` for no network device. network: Option, @@ -385,20 +397,44 @@ impl SandboxPlan { Ok(Self { distro, upper, - settings, + settings: Some(settings), export: None, network: None, }) } + /// Returns the plan of `distro` over `upper` for an adapter's own guest, with no network + /// device, which [`with_network`](Self::with_network) adds. + /// + /// An adapter's plan renders the root and its devices alone. Its guest chooses the identity, + /// the program, and the limits of its workloads, so the plan names none of them to the VMM or + /// the guest, and the adapter attaches its live shares itself, with the arguments that + /// [`ExportPlan::openvmm_arguments`] renders, so the plan holds no export plan. Its kernel + /// command line takes the adapter's own `nvx_*` tokens beside those of the root, as + /// [`kernel_command_line`](Self::kernel_command_line) says. + pub fn adapter(distro: SandboxDistro, upper: SandboxUpper) -> Self { + Self { + distro, + upper, + settings: None, + export: None, + network: None, + } + } + /// Returns the plan with `export`, the export plan of its live shares, if any, which /// [`filesystem::plan`](crate::host::filesystem::plan) made for the container root. /// /// Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported) for an export plan /// that [`ExportPlan::check`] refuses for the container root, - /// [`ReservedTargets::CONTAINER_ROOT`]. + /// [`ReservedTargets::CONTAINER_ROOT`], and for any export plan of an adapter's plan. pub fn with_export(self, export: Option) -> Result { if let Some(export) = &export { + if self.settings.is_none() { + return Err(refused( + "an adapter's plan holds no export plan: the adapter attaches its shares itself", + )); + } export.check(ReservedTargets::CONTAINER_ROOT)?; } Ok(Self { export, ..self }) @@ -421,9 +457,9 @@ impl SandboxPlan { &self.upper } - /// Returns the launch settings. - pub fn settings(&self) -> &SandboxSettings { - &self.settings + /// Returns the launch settings of the sandbox profile, or `None` for an adapter's plan. + pub fn settings(&self) -> Option<&SandboxSettings> { + self.settings.as_ref() } /// Returns the export plan of the live shares, if any. @@ -539,11 +575,13 @@ impl SandboxPlan { /// Returns the OpenVMM arguments of the plan as the managed lifecycle launches it: the /// microVM machine, a sandbox block device for each block device of the plan in role order, - /// `ROLE:file:PATH` with `,ro` for the read-only image, the workload identity, the arguments - /// that export the shares at [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), which + /// `ROLE:file:PATH` with `,ro` for the read-only image, the workload identity of the sandbox + /// profile, the arguments that export the shares at + /// [`GUEST_EXPORT`](crate::host::filesystem::GUEST_EXPORT), which /// [`ExportPlan::openvmm_arguments`] renders, and from which the guest agent maps them as /// [`mapping_table`](Self::mapping_table) orders, and those of the network, - /// [`SandboxNetwork::openvmm_arguments`]. + /// [`SandboxNetwork::openvmm_arguments`]. An adapter's plan names no workload identity and + /// exports no shares. /// /// Fails as [`SandboxNetwork::openvmm_arguments`] does. pub fn openvmm_arguments(&self) -> Result> { @@ -574,11 +612,13 @@ impl SandboxPlan { arguments.push("--microvm-sandbox-block".to_owned()); arguments.push(format!("{role}:file:{}{access}", block(role, path))); } - let (uid, gid) = self.settings.workload_identity; - arguments.extend([ - "--microvm-workload-identity".to_owned(), - format!("{uid}:{gid}"), - ]); + if let Some(settings) = &self.settings { + let (uid, gid) = settings.workload_identity; + arguments.extend([ + "--microvm-workload-identity".to_owned(), + format!("{uid}:{gid}"), + ]); + } arguments.extend(self.shares(delivery)?.arguments); if let Some(network) = &self.network { arguments.extend(network.openvmm_arguments()?); @@ -609,6 +649,14 @@ impl SandboxPlan { /// then read the plan's root from the command line: one that [`RootLayout::parse`] refuses /// fails too, as when the user's command line adds a virtio-mmio device at the scratch base /// beside a RAM upper. Fails with [`ErrorKind::Unsupported`](crate::ErrorKind::Unsupported). + /// + /// For an adapter's plan, the command line is `user_command_line`, stripped, then the boot + /// tokens of the root alone. The adapter's command line may hold its own `nvx_*` tokens, but + /// no NUL, none of the root's tokens, `nvx_layer=`, `nvx_scratch=`, and `nvx_overlay_upper=`, + /// and no `virtfs_*` token, which OpenVMM appends for the shares that the adapter attaches. + /// The command line and a NUL must fit in [`SANDBOX_COMMAND_LINE_MAX_SIZE`] bytes, which + /// leaves the bootstrap tokens of the adapter's shares to the VMM's own check of the whole + /// command line; the guest must read the plan's root from it, as for the sandbox profile. pub fn kernel_command_line(&self, user_command_line: &str) -> Result { self.command_line(user_command_line, Delivery::MappingTable) } @@ -633,6 +681,9 @@ impl SandboxPlan { if user_command_line.contains('\0') { return Err(refused("kernel command line contains an embedded NUL")); } + let Some(settings) = &self.settings else { + return self.adapter_command_line(user_command_line); + }; for token in split(user_command_line) { let name = token.split('=').next().unwrap_or(token); if token.starts_with("nvx_") { @@ -644,7 +695,6 @@ impl SandboxPlan { ))); } } - let settings = &self.settings; let mut tokens = vec![ strip(user_command_line).to_owned(), "nvx_sandbox=1".to_owned(), @@ -697,6 +747,44 @@ impl SandboxPlan { debug_assert_eq!(layout, Some(self.layout())); Ok(command_line) } + + /// Implements [`kernel_command_line`](Self::kernel_command_line) for an adapter's plan. + fn adapter_command_line(&self, user_command_line: &str) -> Result { + for token in split(user_command_line) { + let name = token.split('=').next().unwrap_or(token); + if is_root_parameter(name) { + return Err(refused(format!("{name} is owned by the plan's root"))); + } + if token.starts_with("virtfs_") { + return Err(refused(format!("{name} is owned by OpenVMM's shares"))); + } + } + let mut tokens = vec![strip(user_command_line).to_owned()]; + tokens.extend(self.layout().tokens()); + tokens.retain(|token| !token.is_empty()); + let command_line = tokens.join(" "); + if command_line.len() + 1 > SANDBOX_COMMAND_LINE_MAX_SIZE { + return Err(refused( + "the kernel command line of an adapter's plan exceeds its 1024-byte x86 budget", + )); + } + let layout = RootLayout::parse(command_line.as_bytes()).map_err(|error| { + refused(format!( + "the kernel command line of an adapter's plan conflicts with the boot tokens of \ + its root: {error}" + )) + })?; + debug_assert_eq!(layout, Some(self.layout())); + Ok(command_line) + } +} + +/// Returns whether `name` is that of a kernel parameter of the boot tokens of a plan's root, which +/// [`RootLayout::tokens`] renders: `nvx_layer`, `nvx_scratch`, or `nvx_overlay_upper`. +fn is_root_parameter(name: &str) -> bool { + [LAYER_TOKEN, SCRATCH_TOKEN, RAM_UPPER_TOKEN] + .iter() + .any(|token| token.split('=').next() == Some(name)) } /// Returns the image that a distro binding records: the staged format, and the identity that the @@ -719,6 +807,7 @@ pub(crate) fn image_of(format: DistroFormat, identity: &str) -> Result RootImage::erofs(Some(identity)), + DistroFormat::Gpt => RootImage::gpt(Some(identity)), }) } diff --git a/nvx-rs/src/host/sandbox/plan/form.rs b/nvx-rs/src/host/sandbox/plan/form.rs index b032400f2..48da4c9b9 100644 --- a/nvx-rs/src/host/sandbox/plan/form.rs +++ b/nvx-rs/src/host/sandbox/plan/form.rs @@ -24,8 +24,10 @@ pub(super) struct PlanForm { distro: DistroForm, /// The upper. upper: UpperForm, - /// The launch settings. - settings: SettingsForm, + /// The launch settings of the sandbox profile, or `None` for an adapter's plan, which the form + /// requires as `null`. + #[serde(deserialize_with = "Option::deserialize")] + settings: Option, /// The export plan of the live shares, or `None`, which the form requires as `null`. #[serde(deserialize_with = "Option::deserialize")] export: Option, @@ -264,7 +266,7 @@ impl From for PlanForm { }, None => UpperForm::Ram {}, }, - settings: SettingsForm::from(settings), + settings: settings.map(SettingsForm::from), export, network: network.map(NetworkForm::from), } @@ -275,8 +277,9 @@ impl TryFrom for SandboxPlan { type Error = String; /// Reads a plan, which must be of [`PLAN_VERSION`], hold an identity in its canonical text and - /// limits in decimal text, and pass the checks of [`SandboxPlan::new`], - /// [`SandboxPlan::with_export`], and [`SandboxNetwork::new`]. + /// limits in decimal text, and pass the checks of [`SandboxPlan::new`], or be an adapter's + /// plan, [`SandboxPlan::adapter`], and then pass those of [`SandboxPlan::with_export`] and + /// [`SandboxNetwork::new`]. fn try_from(form: PlanForm) -> Result { if form.version != PLAN_VERSION { return Err(format!( @@ -309,10 +312,15 @@ impl TryFrom for SandboxPlan { UpperForm::Scratch { path } => SandboxUpper::scratch(&path), UpperForm::Ram {} => SandboxUpper::RAM, }; - let settings = SandboxSettings::try_from(form.settings)?; - let plan = Self::new(distro, upper, settings) - .and_then(|plan| plan.with_export(form.export)) - .map_err(|error| error.to_string())?; + let plan = match form.settings { + Some(settings) => { + let settings = SandboxSettings::try_from(settings)?; + Self::new(distro, upper, settings) + } + None => Ok(Self::adapter(distro, upper)), + } + .and_then(|plan| plan.with_export(form.export)) + .map_err(|error| error.to_string())?; let network = form.network.map(SandboxNetwork::try_from).transpose()?; Ok(plan.with_network(network)) } diff --git a/nvx-rs/src/host/sandbox/plan/tests.rs b/nvx-rs/src/host/sandbox/plan/tests.rs index a0db5fa83..a9ef7435a 100644 --- a/nvx-rs/src/host/sandbox/plan/tests.rs +++ b/nvx-rs/src/host/sandbox/plan/tests.rs @@ -974,7 +974,7 @@ fn launches_plan_their_distro_layer_over_their_scratch_image() { assert_eq!(plan.upper().kind(), RootUpper::Scratch); assert_eq!(plan.upper().scratch_path(), Some(Path::new("scratch.ext4"))); assert_eq!(launch.scratch(), Path::new("scratch.ext4")); - assert_eq!(plan.settings(), &SandboxSettings::default()); + assert_eq!(plan.settings(), Some(&SandboxSettings::default())); // The guest takes the UUID in its canonical text, which `--layer` writes, and nothing else. for invalid in ["", "not-a-uuid", "11111111-1111-1111-1111-11111111111A"] { let error = SandboxLaunch::new( @@ -1240,6 +1240,163 @@ fn plan_with(image: RootImage, upper: SandboxUpper) -> SandboxPlan { plan(image, upper, SandboxSettings::default()) } +/// Returns the plan of an adapter's own guest of `image` over `upper`. +fn adapter(image: RootImage, upper: SandboxUpper) -> SandboxPlan { + SandboxPlan::adapter(SandboxDistro::new("images/distro.img", image), upper) +} + +#[test] +fn adapter_plans_render_their_root_and_devices_alone() { + // The adapter's own tokens, `nvx_*` ones among them, come first, then the root's. + let user = "console=hvc0 nvx_map=c291cmNl,dGFyZ2V0,ro nvx_control_tty=hvc2 nvx_sandbox=1"; + for image in images() { + for upper in uppers() { + let plan = adapter(image, upper.clone()); + assert_eq!(plan.settings(), None); + assert_eq!(plan.export(), None); + let [layer, upper_token] = plan.layout().tokens(); + assert_eq!( + plan.kernel_command_line(user).unwrap(), + format!("{user} {layer} {upper_token}") + ); + assert_eq!( + plan.kernel_command_line(" \t").unwrap(), + format!("{layer} {upper_token}") + ); + // No workload identity and no shares: the block devices alone. + let mut expected = vec![ + "--machine".to_owned(), + "microvm".to_owned(), + "--microvm-sandbox-block".to_owned(), + format!( + "distro:file:{},ro", + PurePath::native("images/distro.img").as_str() + ), + ]; + if let Some(scratch) = upper.scratch_path() { + expected.extend([ + "--microvm-sandbox-block".to_owned(), + format!("scratch:file:{}", scratch.display()), + ]); + } + assert_eq!(plan.openvmm_arguments().unwrap(), expected); + // Then the network's, as for the sandbox profile. + let network = proxy_network(None); + expected.extend(network.openvmm_arguments().unwrap()); + let plan = plan.with_network(Some(network)); + assert_eq!(plan.openvmm_arguments().unwrap(), expected); + assert_eq!( + plan.proxy_url().as_deref(), + Some("10.0.0.1:3128"), + "{plan:?}" + ); + } + } +} + +#[test] +fn adapter_plans_refuse_what_their_root_and_openvmm_own() { + let plan = adapter(RootImage::gpt(Some(uuid(GPT_GUID))), SandboxUpper::RAM); + for (user, name, owner) in [ + ( + "nvx_layer=distro,0xd0003000,gpt", + "nvx_layer", + "the plan's root", + ), + ("nvx_layer", "nvx_layer", "the plan's root"), + ( + "quiet nvx_overlay_upper=ramfs", + "nvx_overlay_upper", + "the plan's root", + ), + ( + "nvx_scratch=0xd0006000,ext4", + "nvx_scratch", + "the plan's root", + ), + ("virtfs_tag=microvm", "virtfs_tag", "OpenVMM's shares"), + ] { + let error = plan.kernel_command_line(user).unwrap_err(); + assert_eq!(error.kind(), ErrorKind::Unsupported, "{user}"); + assert_eq!( + error.to_string(), + format!("{name} is owned by {owner}"), + "{user}" + ); + } + assert_eq!( + message(plan.kernel_command_line("quiet\0")), + "kernel command line contains an embedded NUL" + ); + // The guest must read the plan's root from the command line. + assert_eq!( + message(plan.kernel_command_line("virtio_mmio.device=0x1000@0xd0006000:11")), + "the kernel command line of an adapter's plan conflicts with the boot tokens of its root: \ + a RAM upper takes no scratch block device, but the command line adds a virtio-mmio \ + device at 0xd0006000" + ); + // The adapter's command line, a space, the root's tokens, and a NUL fill the budget exactly. + for image in images() { + for upper in uppers() { + let plan = adapter(image, upper); + let base = plan.kernel_command_line("").unwrap(); + let fitting = "x".repeat(SANDBOX_COMMAND_LINE_MAX_SIZE - base.len() - 2); + let command_line = plan.kernel_command_line(&fitting).unwrap(); + assert_eq!(command_line.len() + 1, SANDBOX_COMMAND_LINE_MAX_SIZE); + assert_eq!( + message(plan.kernel_command_line(&format!("{fitting}x"))), + "the kernel command line of an adapter's plan exceeds its 1024-byte x86 budget" + ); + } + } + // The adapter attaches its shares itself. + assert_eq!( + message(plan.clone().with_export(Some(shares(true)))), + "an adapter's plan holds no export plan: the adapter attaches its shares itself" + ); + assert_eq!(plan.clone().with_export(None).unwrap(), plan); +} + +#[test] +fn adapter_plans_persist_without_settings_or_an_export() { + let plan = SandboxPlan::adapter( + SandboxDistro::new("/images/distro.img", RootImage::gpt(Some(uuid(GPT_GUID)))), + SandboxUpper::RAM, + ) + .with_network(Some(proxy_network(Some(ProxyScheme::Http)))); + let form = serde_json::to_value(&plan).unwrap(); + assert_eq!(form["version"], json!(PLAN_VERSION)); + assert_eq!(form["settings"], Value::Null); + assert_eq!(form["export"], Value::Null); + let read: SandboxPlan = serde_json::from_value(form.clone()).unwrap(); + assert_eq!(read, plan); + assert_eq!( + read.kernel_command_line("quiet").unwrap(), + plan.kernel_command_line("quiet").unwrap() + ); + assert_eq!( + read.openvmm_arguments().unwrap(), + plan.openvmm_arguments().unwrap() + ); + // An adapter's plan never holds an export plan, and its settings are required as `null`. + let (_, sandbox) = persisted(); + let mut exporting = form.clone(); + exporting["export"] = sandbox["export"].clone(); + let error = serde_json::from_value::(exporting) + .unwrap_err() + .to_string(); + assert!( + error.contains("an adapter's plan holds no export plan"), + "{error}" + ); + let mut missing = form; + missing.as_object_mut().unwrap().remove("settings"); + let error = serde_json::from_value::(missing) + .unwrap_err() + .to_string(); + assert!(error.contains("missing field `settings`"), "{error}"); +} + #[test] fn persisted_plans_that_break_the_form_or_the_checks_are_refused() { let (_, form) = persisted(); diff --git a/nvx-rs/src/host/sandbox/prepared.rs b/nvx-rs/src/host/sandbox/prepared.rs index 866502a19..78de97043 100644 --- a/nvx-rs/src/host/sandbox/prepared.rs +++ b/nvx-rs/src/host/sandbox/prepared.rs @@ -46,7 +46,8 @@ //! images, and the claim of the scratch image, which it inherited. //! //! A plan never names a root identity for the workload: [`SandboxPlan::new`] refuses a zero UID -//! or GID, so the VMM's `--microvm-workload-identity` never encodes root. +//! or GID, so the VMM's `--microvm-workload-identity` never encodes root, and an adapter's plan, +//! [`SandboxPlan::adapter`], names no workload identity, which the adapter's own guest chooses. use std::fs::File; use std::path::Path; diff --git a/nvx-rs/src/host/sandbox/prepared/tests.rs b/nvx-rs/src/host/sandbox/prepared/tests.rs index b3b062822..b80840adb 100644 --- a/nvx-rs/src/host/sandbox/prepared/tests.rs +++ b/nvx-rs/src/host/sandbox/prepared/tests.rs @@ -336,6 +336,63 @@ fn an_upper_in_ram_takes_no_scratch_image() { ); } +#[test] +fn gpt_disks_are_prepared_from_their_bindings() { + let staged = Staged::new(); + fs::write(staged.path("distro.img"), b"gpt disk\n").unwrap(); + let path = staged.text("distro.img"); + let layer = SandboxLayer::gpt(LayerRole::Distro, &path, UUID); + let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); + let distro = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); + #[cfg(target_os = "linux")] + let distro_number = number(distro.file()); + let plan = SandboxPlan::new( + SandboxDistro::new(&path, RootImage::gpt(Uuid::parse(UUID))), + SandboxUpper::RAM, + SandboxSettings::default(), + ) + .unwrap(); + let prepared = PreparedLaunch::new(plan.clone(), distro, None, "").unwrap(); + let token = format!("nvx_layer=distro,0xd0003000,gpt:{UUID}"); + assert!( + prepared + .kernel_command_line() + .split(' ') + .any(|t| t == token), + "{}", + prepared.kernel_command_line() + ); + let blocks = values(prepared.openvmm_arguments(), "--microvm-sandbox-block"); + #[cfg(target_os = "linux")] + assert_eq!( + blocks, + [format!("distro:file:/proc/self/fd/{distro_number},ro")] + ); + #[cfg(windows)] + assert_eq!( + blocks, + [format!( + "distro:file:{},ro", + final_path(&staged, "distro.img") + )] + ); + drop(prepared); + + // The disk's binding prepares no plan of an EROFS image at its path. + let distro = binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(); + let erofs = plan.with_image(RootImage::erofs(Uuid::parse(UUID))); + assert_eq!( + failure(PreparedLaunch::new(erofs, distro, None, "")), + ( + ErrorKind::Unsupported, + format!( + "the plan's image is not the gpt layer with identity {UUID} that the distro \ + binding of {path} names" + ) + ) + ); +} + #[test] fn prepared_launches_take_only_the_checked_files_of_their_plan() { let staged = Staged::new(); @@ -597,4 +654,27 @@ fn no_plan_encodes_root_as_the_workload_identity() { values(prepared.openvmm_arguments(), "--microvm-workload-identity"), ["65534:65534"] ); + + // An adapter's plan names no workload identity: its own guest chooses one, root's too. + fs::write(staged.path("distro.img"), b"gpt disk\n").unwrap(); + let path = staged.text("distro.img"); + let layer = SandboxLayer::gpt(LayerRole::Distro, &path, UUID); + let binding = DistroBinding::import(&layer, PolicyVerdict::ACCEPTED).unwrap(); + let plan = SandboxPlan::adapter( + SandboxDistro::new(&path, RootImage::gpt(Uuid::parse(UUID))), + SandboxUpper::RAM, + ); + let user = "console=hvc0 nvx_control_tty=hvc2"; + let prepared = PreparedLaunch::new( + plan.clone(), + binding.check(&layer, DISTRO_POLICY_REVISION).unwrap(), + None, + user, + ) + .unwrap(); + assert!(values(prepared.openvmm_arguments(), "--microvm-workload-identity").is_empty()); + assert_eq!( + prepared.kernel_command_line(), + plan.kernel_command_line(user).unwrap() + ); }