From 4227cd022d7bab9baf7ed1d7b31381735a4cd227 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Wed, 21 Oct 2026 11:49:02 +0200 Subject: [PATCH 1/2] fix(scan): Robot names only inside case sections; ignore files above the project root are not read; walk-policy tests made potent An independent review of v0.41.1 found these gaps in the marker scanner. Robot attribution. find_enclosing_robot_case treated any non-indented line as a test name, so a marker under Settings was named after "Library Collections", one under Variables after a variable line, and a marker before a Settings line took that line through the forward scan. The scan now finds the nearest section header above the marker and names a case only inside Test Cases, Test Case, Tasks, Task, Keywords or Keyword. Headers are matched case-insensitively with any number of leading and optional trailing stars. Elsewhere the marker falls back to file:line. The backward scan stops at a header. Tests cover Settings, Variables, a Settings section between two case sections, and nine header spellings. Negative controls: disabling the section guard failed five of those tests by assertion, and letting the backward scan cross a header failed the new backward-scan test by assertion. Walk policy. New tests cover .ignore and .git/info/exclude (with .git itself still skipped); disabling either flag failed its test. The global ignore file is proven off in a child process of the test binary with its own XDG_CONFIG_HOME, so no environment is shared between parallel tests; turning git_global on failed it. walk_terminates_on_symlink_loop now builds sibling links that only a link-following walk traverses and asserts the exact marker paths; follow_links(true) failed it and the sibling-link test. Scope of ignore files. The ignore crate read .gitignore and .ignore in directories above the scan root, which with require_git(false) can reach a home directory. The walk now starts at the project root, uses parents(false), and is pruned to the path leading to the scan root, so ignore files from the project root down apply with their own anchoring and nothing above it is read. scan_source_files_in_project(project, paths, patterns) carries the root; scan_source_files keeps its signature and treats each scanned directory as its own root. Every call site in rivet-cli passes cli.project. A scan path outside the project, or a symlinked one, is its own root. Tests cover a stray ignore file above the project, a project-root ignore and an anchored pattern applying to a sub-root scan, and an outside scan root. Negative controls: parents(true) failed the first and scanning from the scan root failed the second. The links.rs comment for REQ-406 is now a real marker. REQ-405 and REQ-407 descriptions record the new bounds; the changelog gains a bullet and an upgrade note. Fixes: REQ-405, REQ-407 Verifies: REQ-405, REQ-406, REQ-407 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015HMQUV3u86jN2hmCtXNTc9 --- CHANGELOG.md | 17 ++ artifacts/requirements.yaml | 4 +- rivet-cli/src/main.rs | 19 +- rivet-core/src/links.rs | 2 +- rivet-core/src/test_scanner.rs | 403 +++++++++++++++++++++++++++++++-- 5 files changed, 413 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f44c88d5..3ee6de1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,13 @@ A patch for one regression and two reports from downstream projects. - **The HTML export is reproducible** and no longer embeds the builder's workspace path (#1104). - **Robot Framework suites count as evidence** (#1105). +- **Marker attribution and ignore-file scope tightened** (REQ-405, REQ-407). + +### Upgrade note + +Markers inside gitignored or symlinked directories no longer count as +evidence. List such paths in a `.rivetignore` negation (`!path`) or move them, +and run `rivet coverage --tests` before and after upgrading to compare. ### Security - **wasmtime 48.0.6 → 48.0.7** (RUSTSEC-2026-0378, RUSTSEC-2026-0379) — a @@ -46,6 +53,16 @@ A patch for one regression and two reports from downstream projects. Artifact pages also wrote `data-source-file` as an absolute path of the building machine; a static export now writes it relative to the project root, while `rivet serve` and the VS Code webview keep the real path they open. +- **Marker attribution and ignore-file scope tightened** (REQ-405, REQ-407) — + a Robot marker is named after a test case only inside `Test Cases`, `Tasks` + or `Keywords` sections (any case, singular or plural, any number of `*`); + under `Settings` or `Variables` it falls back to `file:line` instead of + taking the neighbouring setting as its name. The marker walk reads ignore + files from the project root down to each scanned directory and no longer + reads any above the project root, so a stray `.gitignore` in a home + directory cannot hide evidence. `.ignore`, `.git/info/exclude`, the + global-ignore opt-out and the symlink-loop guard now have tests that fail + when the behaviour is reverted. ### Added diff --git a/artifacts/requirements.yaml b/artifacts/requirements.yaml index b65a4d05..62d2185b 100644 --- a/artifacts/requirements.yaml +++ b/artifacts/requirements.yaml @@ -10283,7 +10283,7 @@ artifacts: title: marker scanning walks the whole tree, follows symlinks and ignores .gitignore, so validate takes hours on a repo with symlinks to large external trees (#1097) status: verified release: v0.41.1 - description: "Since v0.41.0 every command that computes coverage or verification (validate, coverage, context, MCP reload) scans source files for rivet: verifies markers (REQ-389 A1). test_scanner::scan_directory used Path::is_dir, which follows symlinks, consulted no ignore file and had no cycle guard; on WSL2 a project with symlinks into the Windows drive (376k files) did not finish validate in minutes to hours. ACCEPTANCE: the marker walk does not enter symlinked directories (no cycles), honours .gitignore, .ignore, .git/info/exclude and a project .rivetignore, keeps the existing skips, and finds exactly the same markers on a tree without symlinks; tests in the module cover each case with a negative control." + description: "Since v0.41.0 every command that computes coverage or verification (validate, coverage, context, MCP reload) scans source files for rivet: verifies markers (REQ-389 A1). test_scanner::scan_directory used Path::is_dir, which follows symlinks, consulted no ignore file and had no cycle guard; on WSL2 a project with symlinks into the Windows drive (376k files) did not finish validate in minutes to hours. ACCEPTANCE: the marker walk does not enter symlinked directories (no cycles), honours .gitignore, .ignore, .git/info/exclude and a project .rivetignore, keeps the existing skips, and finds exactly the same markers on a tree without symlinks; tests in the module cover each case with a negative control. Ignore files above the project root are not read." tags: [scanning, performance, regression] fields: category: functional @@ -10313,7 +10313,7 @@ artifacts: title: "rivet: verifies markers in Robot Framework files (.robot, .resource) are not scanned (#1105)" status: verified release: v0.41.1 - description: "The marker scanner mapped only Rust, Python, C/C++, JS/TS, Go and shell files to comment styles, so a project whose tests are Robot Framework suites got no verification credit and a higher warning count. ACCEPTANCE: .robot and .resource files are scanned with # comments; a marker before or inside a Robot test case or keyword is attributed to that case; a module test with Robot fixtures fails when the extension mapping is removed." + description: "The marker scanner mapped only Rust, Python, C/C++, JS/TS, Go and shell files to comment styles, so a project whose tests are Robot Framework suites got no verification credit and a higher warning count. ACCEPTANCE: .robot and .resource files are scanned with # comments; a marker before or inside a Robot test case or keyword is attributed to that case; a module test with Robot fixtures fails when the extension mapping is removed. Markers outside Test Cases/Tasks/Keywords sections fall back to file:line." tags: [scanning, testing] fields: category: functional diff --git a/rivet-cli/src/main.rs b/rivet-cli/src/main.rs index f0eb3136..4fcaede6 100644 --- a/rivet-cli/src/main.rs +++ b/rivet-cli/src/main.rs @@ -8361,7 +8361,8 @@ impl ReadinessCtx { // Source markers are `verifies` evidence by another route (REQ-329); // without them a marker-verified requirement would read as missing // its verification rule. - let markers = rivet_core::test_scanner::scan_source_files( + let markers = rivet_core::test_scanner::scan_source_files_in_project( + &cli.project, &default_marker_scan_paths(&cli.project), &rivet_core::test_scanner::default_patterns(), ); @@ -9882,7 +9883,8 @@ fn cmd_stats_qualification(cli: &Cli) -> Result { /// Scanning takes seconds on a large tree, so only commands that compute /// coverage or verification call this, not every project load. fn marker_verified_ids(project: &std::path::Path) -> std::collections::BTreeSet { - rivet_core::test_scanner::scan_source_files( + rivet_core::test_scanner::scan_source_files_in_project( + project, &default_marker_scan_paths(project), &rivet_core::test_scanner::default_patterns(), ) @@ -10281,7 +10283,11 @@ fn cmd_verify(cli: &Cli, id: &str, scan: &[std::path::PathBuf], dry_run: bool) - } else { scan.to_vec() }; - let markers = test_scanner::scan_source_files(&paths, &test_scanner::default_patterns()); + let markers = test_scanner::scan_source_files_in_project( + &cli.project, + &paths, + &test_scanner::default_patterns(), + ); let marker_hits: Vec<&test_scanner::TestMarker> = markers .iter() .filter(|m| m.target_id == id && m.link_type == "verifies") @@ -10719,7 +10725,8 @@ fn cmd_coverage( .collect(); if !uncovered.is_empty() { let paths = default_marker_scan_paths(&cli.project); - let markers = rivet_core::test_scanner::scan_source_files( + let markers = rivet_core::test_scanner::scan_source_files_in_project( + &cli.project, &paths, &rivet_core::test_scanner::default_patterns(), ); @@ -11327,7 +11334,7 @@ fn cmd_coverage_tests(cli: &Cli, format: &str, scan_paths: &[PathBuf]) -> Result }; let patterns = test_scanner::default_patterns(); - let markers = test_scanner::scan_source_files(&paths, &patterns); + let markers = test_scanner::scan_source_files_in_project(&cli.project, &paths, &patterns); let coverage = test_scanner::compute_test_coverage(&markers, &store, Some(&schema)); if format == "json" { @@ -19006,7 +19013,7 @@ fn cmd_import_results_junit( defaults }; let patterns = test_scanner::default_patterns(); - let markers = test_scanner::scan_source_files(&scan_paths, &patterns); + let markers = test_scanner::scan_source_files_in_project(&cli.project, &scan_paths, &patterns); let runs = parse_junit_xml_with_markers(&xml, &markers) .with_context(|| format!("failed to parse JUnit XML from {}", file.display()))?; diff --git a/rivet-core/src/links.rs b/rivet-core/src/links.rs index 84815e8f..207ee0b3 100644 --- a/rivet-core/src/links.rs +++ b/rivet-core/src/links.rs @@ -559,7 +559,7 @@ mod tests { assert!(from_c.is_empty(), "leaf has no reachables, got {from_c:?}"); } - // Verifies: REQ-406 + // rivet: verifies REQ-406 // Backlinks were collected in store (HashMap) order, so the incoming-link // table differed from run to run (#1104). They are now ordered by link // type, then source id, whatever order the store yields. diff --git a/rivet-core/src/test_scanner.rs b/rivet-core/src/test_scanner.rs index 70de240f..e242b38d 100644 --- a/rivet-core/src/test_scanner.rs +++ b/rivet-core/src/test_scanner.rs @@ -377,14 +377,40 @@ fn find_enclosing_function(lines: &[&str], marker_line: usize, language: &str) - /// Robot Framework: a test case (or keyword/task) is a NON-INDENTED line /// under a `*** Test Cases ***` style section header; its body is indented. /// A marker on the comment lines just before a case names that case; a -/// marker inside a body names the case it sits in. Section headers -/// (`*** ... ***`) are never names, and a marker with no case around it -/// falls back to `file:line` (None). +/// marker inside a body names the case it sits in. Only the case-bearing +/// sections (Test Cases, Tasks, Keywords) hold names: a non-indented line +/// under `*** Settings ***` or `*** Variables ***` is a setting or a +/// variable, not a test, and a marker there falls back to `file:line` +/// (None), as does a marker with no case around it. Neither scan crosses a +/// section header. fn find_enclosing_robot_case(lines: &[&str], marker_line: usize) -> Option { + // The section header on `l`, if any: Some(true) when it is a case-bearing + // section. Robot accepts any number of leading `*`, optional trailing + // `*`, any case, and singular or plural names. + let header = |l: &str| -> Option { + if !l.starts_with('*') { + return None; + } + let name = l.trim_matches(|c: char| c == '*' || c.is_whitespace()); + let name = name.to_ascii_lowercase(); + Some(matches!( + name.as_str(), + "test cases" | "test case" | "tasks" | "task" | "keywords" | "keyword" + )) + }; let is_name = |l: &str| { let starts_col0 = l.chars().next().is_some_and(|c| !c.is_whitespace()); - starts_col0 && !l.starts_with('#') && !l.starts_with("***") + starts_col0 && !l.starts_with('#') && header(l).is_none() }; + // Is the marker inside a case-bearing section? Nearest header above it. + let in_case_section = lines[..marker_line.min(lines.len())] + .iter() + .rev() + .find_map(|l| header(l)) + .unwrap_or(false); + if !in_case_section { + return None; + } // Forward over blank and comment lines to the case the marker precedes. for line in lines.iter().skip(marker_line) { let t = line.trim(); @@ -397,12 +423,12 @@ fn find_enclosing_robot_case(lines: &[&str], marker_line: usize) -> Option String { /// /// Recursively walks directories. For each file, detects the language from /// its extension and applies the matching patterns. +/// +/// With no project root known, each scanned directory is its own project +/// root: ignore files inside it apply, none above it is read. Callers that +/// scan sub-roots of a project should use [`scan_source_files_in_project`]. pub fn scan_source_files(paths: &[PathBuf], patterns: &[MarkerPattern]) -> Vec { + scan_with_root(None, paths, patterns) +} + +/// Like [`scan_source_files`], for scan paths that live inside `project`. +/// +/// Ignore files from the project root down to each scan root, and below, +/// apply (a `.gitignore` at the project root still governs `/src`); +/// ignore files in directories ABOVE the project root are never read, so a +/// stray `.gitignore` in the user's home directory cannot hide evidence. A +/// scan path that is not inside the project is treated as its own root. +pub fn scan_source_files_in_project( + project: &Path, + paths: &[PathBuf], + patterns: &[MarkerPattern], +) -> Vec { + scan_with_root(Some(project), paths, patterns) +} + +fn scan_with_root( + project: Option<&Path>, + paths: &[PathBuf], + patterns: &[MarkerPattern], +) -> Vec { let mut markers = Vec::new(); for path in paths { if path.is_dir() { - scan_directory(path, patterns, &mut markers); + scan_directory(project.unwrap_or(path), path, patterns, &mut markers); } else if path.is_file() { scan_file(path, patterns, &mut markers); } @@ -439,6 +492,28 @@ pub fn scan_source_files(paths: &[PathBuf], patterns: &[MarkerPattern]) -> Vec (PathBuf, PathBuf) { + let own = || (dir.to_path_buf(), dir.to_path_buf()); + if std::fs::symlink_metadata(dir).is_ok_and(|m| m.file_type().is_symlink()) { + return own(); + } + let (Ok(cp), Ok(cd)) = (project.canonicalize(), dir.canonicalize()) else { + return own(); + }; + match cd.strip_prefix(&cp) { + Ok(rel) => (project.to_path_buf(), project.join(rel)), + Err(_) => own(), + } +} + /// Recursively walk a directory and scan each source file. /// /// The walk never follows symlinks: a symlinked directory is not entered, so @@ -446,21 +521,50 @@ pub const RIVETIGNORE_FILE: &str = ".rivetignore"; /// stall validation. Symlinked files are skipped too, because a file reached /// through a link is either also reachable by its real path or lives outside /// the scanned tree. Ignore rules from `.gitignore`, `.ignore`, -/// `.git/info/exclude` and `.rivetignore` apply whether or not the tree is a -/// checkout; the user's global ignore file is not read, so results do not +/// `.git/info/exclude` and `.rivetignore` apply from the project root down +/// whether or not the tree is a checkout; the user's global ignore file and +/// ignore files above the project root are not read, so results do not /// depend on the machine. -fn scan_directory(dir: &Path, patterns: &[MarkerPattern], markers: &mut Vec) { - let walker = ignore::WalkBuilder::new(dir) +fn scan_directory( + project: &Path, + dir: &Path, + patterns: &[MarkerPattern], + markers: &mut Vec, +) { + let (root, target) = walk_root_and_target(project, dir); + let walker = build_walker(&root, target.clone()); + + for entry in walker.flatten() { + if entry.file_type().is_some_and(|t| t.is_file()) && entry.path().starts_with(&target) { + scan_file(entry.path(), patterns, markers); + } + } +} + +/// The walk policy, in one place so tests can pin each setting. +fn build_walker(root: &Path, target: PathBuf) -> ignore::Walk { + ignore::WalkBuilder::new(root) .follow_links(false) .hidden(false) + .parents(false) .git_global(false) .require_git(false) .add_custom_ignore_filename(RIVETIGNORE_FILE) .sort_by_file_path(|a, b| a.cmp(b)) - .filter_entry(|entry| { + .filter_entry(move |entry| { if entry.depth() == 0 || !entry.file_type().is_some_and(|t| t.is_dir()) { return true; } + let path = entry.path(); + // The scan root and the directories leading to it from the + // project root are never name-skipped; siblings off that path + // are not entered at all. + if target.starts_with(path) { + return true; + } + if !path.starts_with(&target) { + return false; + } // Skip hidden directories and common non-source dirs. let name = entry.file_name().to_string_lossy(); // `.github` is a real source directory — workflows are where CI @@ -470,13 +574,7 @@ fn scan_directory(dir: &Path, patterns: &[MarkerPattern], markers: &mut Vec = + scan_source_files(&[tmp.path().to_path_buf()], &default_patterns()) + .into_iter() + .map(|m| m.file.strip_prefix(tmp.path()).unwrap().to_path_buf()) + .collect(); + files.sort(); + assert_eq!( + files, + vec![PathBuf::from("a/x.rs"), PathBuf::from("b/y.rs")] + ); } // rivet: verifies REQ-405 @@ -901,6 +1016,248 @@ fn test_something() { assert_eq!(scanned_ids(tmp.path()), vec!["REQ-951"]); } + // rivet: verifies REQ-405 + #[test] + fn walk_honours_dot_ignore() { + let tmp = TempDir::new().unwrap(); + write_file(tmp.path(), ".ignore", "scratch/\n"); + write_file(tmp.path(), "src/keep.rs", &marked("REQ-961")); + write_file(tmp.path(), "scratch/s.rs", &marked("REQ-962")); + assert_eq!(scanned_ids(tmp.path()), vec!["REQ-961"]); + } + + // rivet: verifies REQ-405 + #[test] + fn walk_honours_info_exclude() { + let tmp = TempDir::new().unwrap(); + write_file(tmp.path(), ".git/info/exclude", "private/\n"); + // `.git` itself stays skipped, so a marker inside it never counts. + write_file(tmp.path(), ".git/hooks/h.rs", &marked("REQ-973")); + write_file(tmp.path(), "src/keep.rs", &marked("REQ-971")); + write_file(tmp.path(), "private/p.rs", &marked("REQ-972")); + assert_eq!(scanned_ids(tmp.path()), vec!["REQ-971"]); + } + + /// The user's global ignore file must not change what is scanned. The + /// ignore crate reads `$XDG_CONFIG_HOME/git/ignore`; that is process + /// state and tests share a process under `cargo test`, so the check runs + /// in a child process of this very test binary with its own environment + /// and nothing is set in the parent. + // rivet: verifies REQ-405 + #[test] + fn walk_ignores_the_global_ignore_file() { + const CHILD: &str = "RIVET_TEST_GLOBAL_IGNORE_PROJECT"; + if let Some(project) = std::env::var_os(CHILD) { + // Child: the global file (set up by the parent) hides `vendored/`. + let ids = scanned_ids(Path::new(&project)); + assert_eq!(ids, vec!["REQ-981", "REQ-982"]); + return; + } + let tmp = TempDir::new().unwrap(); + let project = tmp.path().join("project"); + write_file(&project, "src/keep.rs", &marked("REQ-981")); + write_file(&project, "vendored/v.rs", &marked("REQ-982")); + let cfg = tmp.path().join("cfg"); + write_file(&cfg, "git/ignore", "vendored/\n"); + let out = std::process::Command::new(std::env::current_exe().unwrap()) + .args([ + "walk_ignores_the_global_ignore_file", + "--test-threads=1", + "--nocapture", + ]) + .env(CHILD, &project) + .env("XDG_CONFIG_HOME", &cfg) + .env("HOME", tmp.path()) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&out.stdout); + // Guard against a child that ran zero tests and "passed". + assert!(stdout.contains("1 passed"), "child did not run: {stdout}"); + assert!( + out.status.success(), + "global ignore file leaked into the scan: {stdout}{}", + String::from_utf8_lossy(&out.stderr) + ); + } + + // rivet: verifies REQ-405 + #[test] + fn ignore_files_above_the_project_root_are_not_read() { + let tmp = TempDir::new().unwrap(); + // Stray ignore files in a parent of the project (think `~/.gitignore`). + write_file(tmp.path(), ".gitignore", "*.rs\n"); + write_file(tmp.path(), ".ignore", "*.rs\n"); + let project = tmp.path().join("proj"); + write_file(&project, "src/x.rs", &marked("REQ-991")); + let found = + scan_source_files_in_project(&project, &[project.join("src")], &default_patterns()); + assert_eq!(found.len(), 1, "markers hidden by a file above the project"); + let found = scan_source_files_in_project(&project, &[project.clone()], &default_patterns()); + assert_eq!(found.len(), 1); + } + + // rivet: verifies REQ-405 + #[test] + fn project_root_ignore_files_apply_to_a_sub_root_scan() { + let tmp = TempDir::new().unwrap(); + let project = tmp.path().join("proj"); + write_file(&project, ".gitignore", "vendored/\n/src/gen.rs\n"); + write_file(&project, RIVETIGNORE_FILE, "fixtures/\n"); + write_file(&project, "src/keep.rs", &marked("REQ-1001")); + write_file(&project, "src/vendored/v.rs", &marked("REQ-1002")); + write_file(&project, "src/fixtures/f.rs", &marked("REQ-1003")); + // Anchored to the project root: hides src/gen.rs, not src/deep/gen.rs. + write_file(&project, "src/gen.rs", &marked("REQ-1004")); + write_file(&project, "src/deep/gen.rs", &marked("REQ-1005")); + // A sibling of the scan root is not part of the scan. + write_file(&project, "other/o.rs", &marked("REQ-1006")); + let mut ids: Vec = + scan_source_files_in_project(&project, &[project.join("src")], &default_patterns()) + .into_iter() + .map(|m| m.target_id) + .collect(); + ids.sort(); + assert_eq!(ids, vec!["REQ-1001", "REQ-1005"]); + } + + // rivet: verifies REQ-405 + #[test] + fn a_scan_root_outside_the_project_is_its_own_root() { + let tmp = TempDir::new().unwrap(); + let project = tmp.path().join("proj"); + let elsewhere = tmp.path().join("elsewhere"); + write_file(&project, ".gitignore", "*.rs\n"); + write_file(&elsewhere, "e.rs", &marked("REQ-1011")); + let found = + scan_source_files_in_project(&project, &[elsewhere.clone()], &default_patterns()); + assert_eq!(found.len(), 1); + } + + // ── Robot attribution ──────────────────────────────────────────────── + + fn robot_names(content: &str) -> Vec<(String, String)> { + let dir = TempDir::new().unwrap(); + write_file(dir.path(), "s.robot", content); + let mut v: Vec<(String, String)> = + scan_source_files(&[dir.path().to_path_buf()], &default_patterns()) + .into_iter() + .map(|m| (m.target_id, m.test_name)) + .collect(); + v.sort(); + v + } + + /// A Robot marker comment line. Built with `concat!` so this test module + /// does not itself contain a scannable marker line. + fn robot_marker(id: &str) -> String { + format!("{}{id}\n", concat!("# riv", "et: verifies ")) + } + + // rivet: verifies REQ-407 + #[test] + fn robot_marker_in_settings_is_not_attributed_to_a_setting() { + let src = format!( + "*** Settings ***\nLibrary Collections\n{}Resource x.resource\n\n\ + *** Test Cases ***\nA Case\n Log hi\n", + robot_marker("REQ-1101") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert_eq!(got[0].0, "REQ-1101"); + assert!(got[0].1.contains("s.robot:"), "{got:?}"); + } + + // rivet: verifies REQ-407 + #[test] + fn robot_marker_in_variables_falls_back_to_file_line() { + let src = format!( + "*** Variables ***\n${{X}} 1\n{}${{Y}} 2\n", + robot_marker("REQ-1102") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert!(got[0].1.contains("s.robot:"), "{got:?}"); + } + + // rivet: verifies REQ-407 + #[test] + fn robot_marker_before_settings_header_does_not_see_a_later_case() { + // A Settings section sits between two case sections. The marker in + // it must reach neither `First` (backward) nor `Later` (forward). + let src = format!( + "*** Test Cases ***\nFirst\n Log x\n\n\ + *** Settings ***\n{}\ + Resource r.resource\n\n\ + *** Test Cases ***\nLater\n Log y\n", + robot_marker("REQ-1103") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert!(got[0].1.contains("s.robot:"), "{got:?}"); + } + + // rivet: verifies REQ-407 + #[test] + fn robot_marker_before_a_settings_line_in_a_settings_section_is_unnamed() { + let src = format!( + "*** Settings ***\n{}Library Collections\n\n\ + *** Test Cases ***\nLater\n Log y\n", + robot_marker("REQ-1106") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert!(got[0].1.contains("s.robot:"), "{got:?}"); + } + + // rivet: verifies REQ-407 + #[test] + fn robot_backward_scan_stops_at_a_section_header() { + // `First` is in an earlier section; the marker's own (empty) Keywords + // section must not borrow it. + let src = format!( + "*** Test Cases ***\nFirst\n Log x\n\n*** Keywords ***\n{}", + robot_marker("REQ-1107") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert!(got[0].1.contains("s.robot:"), "{got:?}"); + } + + // rivet: verifies REQ-407 + #[test] + fn robot_section_headers_are_recognised_in_every_spelling() { + for header in [ + "*** Test Cases ***", + "*** test cases ***", + "*** Test Case ***", + "*** Tasks ***", + "*** Task ***", + "*** Keywords ***", + "*** Keyword ***", + "* Test Cases", + "** Keywords **", + ] { + let src = format!( + "{header}\n{}My Name\n Log x\n", + robot_marker("REQ-1104") + ); + assert_eq!( + robot_names(&src), + vec![("REQ-1104".to_string(), "My Name".to_string())], + "header {header:?}" + ); + } + for header in ["*** Settings ***", "*** Variables ***", "*** Comments ***"] { + let src = format!( + "{header}\n{}My Name\n Log x\n", + robot_marker("REQ-1105") + ); + let got = robot_names(&src); + assert_eq!(got.len(), 1); + assert!(got[0].1.contains("s.robot:"), "header {header:?}: {got:?}"); + } + } + // ── Test: Rust attribute marker ────────────────────────────────────── // rivet: verifies REQ-026 From 74f9932a53a338d800924f19079145c0e44717d4 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Wed, 21 Oct 2026 20:56:01 +0200 Subject: [PATCH 2/2] test(scan): a file in a directory on the way to the scan root is not scanned PR-diff mutation found that relaxing the scan-root check in scan_directory from and to or survived: the walk from the project root yields files in the directories it passes through, and only that check keeps them out of a sub-root scan. The sub-root test now places a marker at the project root and expects it to be ignored; with the mutant applied the test fails by assertion, restored it passes. Verifies: REQ-405 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015HMQUV3u86jN2hmCtXNTc9 --- rivet-core/src/test_scanner.rs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rivet-core/src/test_scanner.rs b/rivet-core/src/test_scanner.rs index e242b38d..58fc2503 100644 --- a/rivet-core/src/test_scanner.rs +++ b/rivet-core/src/test_scanner.rs @@ -1111,6 +1111,10 @@ fn test_something() { write_file(&project, "src/deep/gen.rs", &marked("REQ-1005")); // A sibling of the scan root is not part of the scan. write_file(&project, "other/o.rs", &marked("REQ-1006")); + // Nor is a file in a directory the walk passes through on its way to + // the scan root: the walker yields it, and only the scan-root check + // keeps it out (a mutant that relaxed that check went unnoticed). + write_file(&project, "top.rs", &marked("REQ-1007")); let mut ids: Vec = scan_source_files_in_project(&project, &[project.join("src")], &default_patterns()) .into_iter()