From e7d7f7cc7ddd5a05c81d2599029b4fcafec43af6 Mon Sep 17 00:00:00 2001 From: Matt Cossins Date: Wed, 19 Aug 2026 19:00:32 +0100 Subject: [PATCH 1/4] Add Metis security review learning path --- .../1-understand-metis.md | 79 +++++++ .../2-install-metis.md | 167 +++++++++++++++ .../3-review-vulnerable-c.md | 145 +++++++++++++ .../4-prepare-libpng.md | 150 ++++++++++++++ .../5-triage-findings.md | 125 +++++++++++ .../review-c-code-with-metis/_index.md | 82 ++++++++ .../review-c-code-with-metis/_next-steps.md | 9 + .../files/prepare_magma_libpng.py | 195 ++++++++++++++++++ 8 files changed, 952 insertions(+) create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/_index.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md create mode 100644 content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md new file mode 100644 index 0000000000..db4b7a4a38 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md @@ -0,0 +1,79 @@ +--- +title: Understand the Metis review workflow +description: Learn how Metis gathers evidence, runs security reviews, and presents findings for human triage. +weight: 2 +layout: "learningpathall" +--- + +## What is Metis? + +Arm Metis is an open-source, agentic security code-review framework. It combines language-aware source tooling with a large language model (LLM) to investigate security-relevant behavior and return structured review candidates. Metis supports multiple programming and hardware description languages through language plugins. This Learning Path uses C examples, but the same concepts can be applied to the other supported languages. + +You first run Metis on a small C program with known vulnerabilities. This gives you a quick look at the command flow, JSON output, SARIF output, and reported findings. You then prepare a libpng source tree from the [Magma](https://hexhive.epfl.ch/magma/) benchmark and triage a captured Metis finding against the source diff. + +The exercises show how to use Metis during a security review. They do not benchmark Metis accuracy. Model choice, provider behavior, Metis version, and source changes can all affect a run. Use published benchmark results for comparative performance data. + +{{% notice Note %}} +Model-backed review can consume substantial API tokens. The initial exercise is less expensive than the optional live libpng run. A pre-captured libpng report candidate is included so you can complete the triage exercise without running the larger review. +{{% /notice %}} + +## How Metis reviews code + +A Metis review starts with source files and project context. Metis collects language-aware evidence, asks your configured model to investigate possible vulnerabilities, and returns structured findings for a human reviewer. + +For C programs, reviewers often care about memory safety, integer arithmetic, resource management, parser state, and input validation. Tree-sitter is a parser that turns source files into syntax trees, so tools can identify functions, variables, calls, and other language constructs without treating code as plain text. Metis uses Tree-sitter to build a code graph of symbols and relationships. Its reachability review follows paths between possible attacker-controlled inputs and security-sensitive operations. The model receives source paths and targeted checks, rather than only a large prompt containing raw files. + +## Providing Metis with context + +Metis has two main ways to give the model source-code context: + +- **Source navigation:** this is the default for review commands. Metis reads files from the codebase and gives the model targeted source snippets, paths, and checks. For C code, this includes Tree-sitter-based reachability analysis. Use this mode when you want to review a file, directory, patch, or small codebase without building an index first. +- **Vector indexing:** this is optional. The `index` command builds an embedding index for code and documentation, and `update` refreshes that index from a patch. Use indexing when you want to ask broad questions with `ask`, search across a larger repository, or add indexed context to later review and triage commands. + +The exercises in this Learning Path use source navigation and C reachability analysis. You do not build an index because the review targets are small and the tasks are focused. + +A typical Metis workflow looks like this: + +1. Pick the review question, such as "review this parser file" or "review this patch." +2. Choose a command: `review_code`, `review_dir`, `review_file`, `review_patch`, `ask`, or `triage`. +3. Add project-specific security context in `.metis.md` when the source needs extra assumptions. +4. Run Metis and save the result as JSON, SARIF, or both. +5. Check each finding against the source code, input boundaries, build configuration, and tests. + +Metis supplies the review workflow, source navigation, prompts, validation stages, and output handling. The model provider supplies the reasoning model. Results can vary between models and providers even when the source code is unchanged. + +## Choose the review scope + +Use the narrowest scope that answers your review question: + +- `review_code` reviews files selected from the codebase and is useful for a broad first pass. +- `review_dir` focuses on one directory. +- `review_file` focuses on one source file while retaining codebase context. +- `review_patch` focuses on changed code and is suited to pull-request or pre-merge review. + +This Learning Path demonstrates use of `review_code` and `review_file`. + +## Interpret Metis findings + +A Metis finding normally includes the issue, source location, reasoning, suggested mitigation, Common Weakness Enumeration (CWE) identifier, severity, and confidence. These fields provide a starting point for you to then investigate the findings. + +{{% notice Note %}} +CWE identifiers, such as `CWE-190` for integer overflow, provide standard names for common classes of software weakness. +{{% /notice %}} + +The main benefit is that Metis can reduce the amount of source navigation you do before a review becomes in-depth. You still decide whether a candidate is reachable, exploitable, already mitigated, or worth fixing. + +For each finding, ask: + +- Can untrusted data reach the reported value or operation? +- Does the build configuration include the reported code? +- Are the arithmetic limits and platform assumptions correct? +- Does an existing check or caller contract invalidate the report? +- Can you reproduce the behavior with a focused test or sanitizer? +- Does the proposed mitigation preserve intended behavior? + +You will apply these questions to the libpng report later in the Learning Path. + +## What you've learned and what's next + +You have seen how Metis gathers source evidence, when indexing is useful, and why every result needs human triage. Next, you will install the tested release and add your provider API key. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md new file mode 100644 index 0000000000..56c4ff3b5f --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md @@ -0,0 +1,167 @@ +--- +title: Install and configure Metis +description: Install a tested Metis release and configure a model provider with an API key. +weight: 3 +layout: "learningpathall" +--- + +## Install system tools + +The commands in this Learning Path use a Unix-like shell. Use Terminal on macOS, a Linux terminal, or an existing Ubuntu/Debian terminal running under WSL2 on Windows. Native Windows PowerShell and Command Prompt are not covered. + +Select your platform and install the compiler, Git, and command-line utilities used to fetch code, apply patches, and inspect reports: + +{{< tabpane code=true >}} + {{< tab header="macOS" language="bash">}} +xcode-select --install + {{< /tab >}} + {{< tab header="Ubuntu Linux or WSL2" language="bash">}} +sudo apt-get update +sudo apt-get install -y build-essential ca-certificates curl git grep patch sed + {{< /tab >}} +{{< /tabpane >}} + +If the Xcode Command Line Tools are already installed, macOS reports that no installation is needed. On WSL2, use the Linux filesystem for this exercise and avoid creating the workspace under `/mnt/c`. + +## Install uv and Python + +Metis is a Python project. Use `uv` to install Python 3.12 and create an isolated Metis environment: + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Load the shell environment created by the installer and confirm that `uv` is available: + +```bash +source "$HOME/.local/bin/env" +uv --version +``` + +Install Python 3.12 using `uv`: + +```bash +uv python install 3.12 +uv python find 3.12 +``` + +The second command prints the path to the managed Python 3.12 executable. + +Check that all command-line tools used by the exercises are available: + +```bash +for tool in curl git grep patch sed uv; do + command -v "$tool" >/dev/null || echo "Missing: $tool" +done +``` + +The command prints nothing when all required tools are available. Install any tools that are missing. + +## Create the exercise workspace + +Create one workspace in your home directory for Metis, the review targets, and generated reports. On WSL2, `$HOME` is inside the Linux filesystem: + +```bash +mkdir -p "$HOME/metis-security-review/results" +cd "$HOME/metis-security-review" +``` + +## Clone and install Metis + +Clone the Metis repository at the 1.5.0 release: + +```bash +git clone --branch metis-v1.5.0 --depth 1 \ + https://github.com/arm/metis.git \ + "$HOME/metis-security-review/metis" + +cd "$HOME/metis-security-review/metis" + +uv sync --no-dev +source "$HOME/metis-security-review/metis/.venv/bin/activate" + +command -v metis +metis --version + +cd "$HOME/metis-security-review" +``` + +`uv sync --no-dev` installs only the runtime dependencies needed to run the Metis CLI. It selects a Python version that satisfies the Metis project requirements, creates `metis/.venv`, and installs Metis there. Activating the environment makes the `metis` command and the matching Python interpreter available in the current terminal. + +The version command should report: + +```output +Metis 1.5.0 +``` + +If you open a new terminal, return to the exercise workspace and reactivate the Metis environment before continuing: + +```bash +cd "$HOME/metis-security-review" +source "$HOME/metis-security-review/metis/.venv/bin/activate" +command -v metis +``` + +The path printed by `command -v` should end with `metis-security-review/metis/.venv/bin/metis`. If it points elsewhere, open a new terminal or run `deactivate`, then run the activation command again. + +## Configure the model provider + +Metis supports several hosted and local model providers. The following commands use OpenAI as an example. Export your API key in the same terminal that will run Metis: + +```bash +export OPENAI_API_KEY="your-api-key" +# Optional: set this if your organization uses an OpenAI-compatible proxy. +# export OPENAI_BASE_URL="https://your-openai-compatible-endpoint/v1" +``` + +Do not add secrets to `.metis.md`, shell history, source control, or report files. + +If your organization provides an OpenAI-compatible endpoint, export `OPENAI_BASE_URL` before running Metis. Additionally, remember to install any certificates required by your organization so Python can verify the endpoint. Otherwise, the examples use the public OpenAI API endpoint. + +For other model providers, follow the [Metis provider instructions](https://github.com/arm/metis#2-set-up-llm-provider). + +Check that Python inside the active Metis environment can reach the configured OpenAI endpoint: + +```bash +python - <<'PY' +import os +import httpx + +base_url = ( + os.environ.get("OPENAI_BASE_URL") + or os.environ.get("OPENAI_API_BASE") + or "https://api.openai.com/v1" +) +response = httpx.get( + base_url.rstrip("/") + "/models", + headers={"Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]}, + timeout=20, +) +print(f"status: {response.status_code}") +response.raise_for_status() +PY +``` + +The expected output is: + +```output +status: 200 +``` + +If this check reports `401` or `403`, check the API key and model access. If it reports `404`, check the base URL. If it reports a certificate verification error or timeout, check your network, proxy, and Python certificate configuration before running a review. + +The endpoint you use can affect runtime, rate limits, retry behavior, response formatting, and the set of reported findings. + +## Confirm that the command starts + +Display the command help without starting a model-backed review. This verifies the CLI before you spend tokens on a review: + +```bash +metis --help +``` + +The help output should include options such as `--codebase-path`, `--command`, `--non-interactive`, and `--output-file`. + +## What you've accomplished and what's next + +You installed Metis v1.5.0 and set it up with your model provider. Next, you will review a small C program and compare the report with defects that are intentionally present in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md new file mode 100644 index 0000000000..9136ef8c8b --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md @@ -0,0 +1,145 @@ +--- +title: Compare a Metis report with vulnerable C code +description: Run a Metis review, inspect JSON and SARIF output, and compare the report with known defects in a small C program. +weight: 4 +layout: "learningpathall" +--- + +## Prepare the example program + +The [Damn Vulnerable C Program](https://github.com/hardik05/Damn_Vulnerable_C_Program) contains vulnerabilities intentionally placed and labeled in one C file. Use it first to learn the Metis review process: prepare a small target, run `review_code`, save JSON and SARIF, summarize findings, and compare source locations. + +In your terminal, clone the repository and copy `dvcp.c` into a separate review directory. The copy keeps the review target small and avoids sending the rest of the repository to Metis: + +```bash +git clone --depth 1 https://github.com/hardik05/Damn_Vulnerable_C_Program.git +mkdir -p dvcp-review +cp Damn_Vulnerable_C_Program/dvcp.c dvcp-review/ +``` + +Add a short threat-model note for the review. Metis reads `.metis.md` as project context: + +```bash +cat > dvcp-review/.metis.md <<'EOF' +# Project security context + +This command-line C program parses an image file supplied by an untrusted user. +Treat file contents, dimensions, offsets, and derived allocation sizes as +attacker-controlled. Review memory safety, integer arithmetic, resource +exhaustion, input validation, and cleanup on error paths. +EOF +``` + +The note tells Metis how to treat inputs and security boundaries. The source file still contains teaching comments, which is acceptable here because this step teaches the review workflow. You will use libpng for a more realistic triage example. + +## Run the review + +Change into the review directory and run `review_code`. Save JSON for detailed inspection and SARIF for code-scanning tools: + +```bash +cd "$HOME/metis-security-review/dvcp-review" +metis \ + --codebase-path . \ + --non-interactive \ + --command "review_code" \ + --output-file ../results/dvcp.json \ + --output-file ../results/dvcp.sarif +cd "$HOME/metis-security-review" +``` + +The run sends source-derived evidence to your configured model provider. Runtime, token use, and findings depend on the model, provider, and Metis version. + +It can take a couple of minutes to complete. + +## Summarize the JSON report + +Use the Python interpreter from the active Metis environment to print the issue type and source location for each finding: + +```bash +python - <<'PY' +import json +from pathlib import Path + +data = json.loads(Path("results/dvcp.json").read_text()) +findings = [ + finding + for review_group in data["reviews"] + for finding in review_group["reviews"] +] + +print(f"Finding count: {len(findings)}") +for finding in findings: + anchor = finding.get("anchor") or {} + file_path = ( + anchor.get("file_path") + or finding.get("primary_file") + or finding.get("file") + ) + line = anchor.get("start_line") or finding.get("line_number") + print( + f"- {finding['cwe']} {finding['issue']} " + f"({file_path}:{line})" + ) +PY +``` + +The output is similar to: + +```output +Finding count: 9 +- CWE-190 Image width and height read from the file are added in a signed int without validation, allowing overflow before the result is used as an allocation size. (dvcp.c:54) +- CWE-120 The code copies the full img.data array into buff1 without ensuring that the allocated size1 buffer is at least sizeof(img.data). (dvcp.c:58) +- CWE-415 buff1 is freed unconditionally and then freed again when size1 is even. (dvcp.c:62) +- CWE-416 After buff1 is freed, an alternate branch can write to buff1 when size1 is divisible by 3. (dvcp.c:67) +- CWE-191 Image dimensions read from the file are subtracted and offset in a signed int without validation, allowing underflow or negative values before allocation. (dvcp.c:74) +- CWE-120 The code copies the full img.data array into buff2 without ensuring that the allocated size2 buffer is at least sizeof(img.data). (dvcp.c:79) +- CWE-787 The code copies the full img.data array into buff4 even though buff4 was allocated with attacker-controlled size3 and may be smaller than sizeof(img.data). (dvcp.c:87) +- CWE-125 The computed size3 value is used as an index into a 10-byte stack buffer without bounds checking. (dvcp.c:90) +- CWE-674 stack_operation recursively calls itself in an infinite loop with a large stack buffer, causing stack exhaustion. (dvcp.c:25) +``` + +Your exact count and wording can differ. A valid report can contain zero findings. An empty report means that this run did not report any findings; it does not mean the program is safe. If you expected findings, check the provider reachability test and any Metis log output before judging the result. + +## Compare the report with known defects + +Display four source regions that the program identifies as deliberately vulnerable: + +```bash +sed -n '20,28p' dvcp-review/dvcp.c +sed -n '51,68p' dvcp-review/dvcp.c +sed -n '72,83p' dvcp-review/dvcp.c +sed -n '89,108p' dvcp-review/dvcp.c +``` + +Check whether your report covers these concrete cases: + +- Signed image dimensions are combined and passed to `malloc()` without overflow or range checks. +- `buff1` is freed, then freed again when `size1` is even or written after free when it is divisible by three. +- `img.height` is used as a divisor without a zero check. +- `size3` is used as an index into fixed-size and heap buffers without bounds checks. +- Attacker-controlled dimensions can select recursive stack exhaustion or repeated heap allocation. + +During validation for this Learning Path, Metis v1.5.0 reported nine findings. This table shows how that run compared with the known defects in the teaching program: + +| Source issue | Tested-run outcome | +| --- | --- | +| `size1` signed overflow before `malloc()` and `memcpy()` | Reported as `CWE-190` at `dvcp.c:54` | +| `memcpy()` into `buff1` without checking `size1` against `sizeof(img.data)` | Reported as `CWE-120` at `dvcp.c:58` | +| `buff1` double free | Reported as `CWE-415` at `dvcp.c:62` | +| `buff1` use-after-free write | Reported as `CWE-416` at `dvcp.c:67` | +| `size2` signed underflow or negative allocation size before `memcpy()` | Reported as `CWE-191` at `dvcp.c:74` | +| `memcpy()` into `buff2` without checking `size2` against `sizeof(img.data)` | Reported as `CWE-120` at `dvcp.c:79` | +| `buff4` allocation smaller than `memcpy()` length | Reported as `CWE-787` at `dvcp.c:87` | +| `size3` out-of-bounds stack-buffer read | Reported as `CWE-125` at `dvcp.c:90` | +| Recursive `stack_operation()` with a large stack buffer | Reported as `CWE-674` at `dvcp.c:25` | +| Divide by zero, indexed heap out-of-bounds read/write, indexed stack out-of-bounds write, memory leak, and heap exhaustion | Not reported in that run | + +For each reported finding, verify the source location, attacker control, and operation. Also record known source defects that the run omitted. The goal is to practice reviewing Metis output, not to turn the finding count into a coverage score. + +The same review is also available in `results/dvcp.sarif`. SARIF allows code-hosting services and security tools to associate findings with source locations. Keep JSON when you want the complete Metis reasoning and SARIF when you want to integrate results into an existing code-scanning workflow. + +This shows that Metis can report several issues, but it is not guaranteed to find everything. Runs are non-deterministic and also depend on the model used. + +## What you've accomplished and what's next + +You ran a complete Metis review, exported two report formats, and compared the reported findings with known defects. Next, you will prepare a realistic libpng target without leaving benchmark canaries or fixes in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md new file mode 100644 index 0000000000..a17b638f9c --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md @@ -0,0 +1,150 @@ +--- +title: Prepare a realistic Magma libpng target +description: Build a reviewable libpng source tree from Magma while removing benchmark canaries and fix branches. +weight: 5 +layout: "learningpathall" +--- + +## Why prepare a Magma libpng target? + +The previous section used a small labeled program to teach the Metis review process. This section prepares a larger C codebase so you can see how a Metis finding is triaged against source that looks closer to real project code. + +[Magma](https://hexhive.epfl.ch/magma/) gives you a repeatable way to prepare that target. Magma is a benchmark for evaluating fuzzers against real programs with documented vulnerabilities. A fuzzer is a testing tool that repeatedly runs a program with generated or mutated inputs to trigger crashes, hangs, memory errors, or other unexpected behavior. + +You use Magma here only to create a prepared libpng source tree for review. You are not using Magma to benchmark Metis. You also do not give Metis the benchmark metadata, bug names, fuzzing harnesses, canaries, or fix branches, because those details would give a review model clues that a normal codebase would not contain. + +The preparation script creates the review target in four steps: + +1. Checks out the libpng revision selected by Magma. +2. Applies Magma patches that introduce known vulnerable code. +3. Removes benchmark controls so the source looks closer to a normal libpng tree. +4. Writes neutral Metis context for the review. + +The result is a deliberately vulnerable example target. Do not use it outside this exercise. + +## Download the preparation script + +Download [the Magma libpng preparation script](https://github.com/ArmDeveloperEcosystem/arm-learning-paths/blob/main/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py) into the exercise workspace: + +```bash +curl -L \ + https://raw.githubusercontent.com/ArmDeveloperEcosystem/arm-learning-paths/main/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py \ + -o prepare_magma_libpng.py +``` + +Clone Magma, then check out the commit tested for this Learning Path: + +```bash +git clone https://github.com/HexHive/magma.git +git -C magma checkout 75d1ae7b180443a778b8830c79176ca5f93642ac +``` + +Run the script to create `libpng-review`: + +```bash +python prepare_magma_libpng.py \ + --magma-dir magma \ + --output-dir libpng-review +``` + +The output is similar to: + +```output +Prepared libpng commit a37d4836519517bdce6cb9d956092321eca3e73b +Applied 7 Magma vulnerability-introducing patches +Output: /Users/you/metis-security-review/libpng-review +``` + +## Inspect the prepared source + +The preparation script removes Magma benchmark controls from the prepared source tree. This keeps the review target closer to a normal libpng checkout instead of a benchmark harness. + +Confirm that the prepared tree has local changes: + +```bash +git -C libpng-review status --short +git -C libpng-review diff --stat +``` + +The diff summary shows that the script applied patches that introduce known vulnerable code to libpng. You will inspect the relevant `pngrutil.c` changes during triage. Do not send patch names, benchmark descriptions, or known vulnerability lists to Metis. + +## Choose a libpng report option + +You need a Metis report before you can triage a finding. Choose one of these options: + +- Run a live Metis review if you have enough API quota and want to compare your model configuration. +- Use the pre-captured report if you are interested to try Metis, but want to minimize your API usage. + +{{< tabpane-normal >}} + {{< tab header="Run live review" >}} +This option asks Metis to review only `pngrutil.c`, because the triage exercise focuses on a finding in that file. + +The live libpng review can run for many minutes, and consume substantial model tokens. Use the pre-captured report option if a live run is too costly or does not produce a finding you can triage. + +```bash +cd "$HOME/metis-security-review/libpng-review" +metis \ + --codebase-path . \ + --non-interactive \ + --command "review_file pngrutil.c" \ + --output-file ../results/libpng-live.json +cd "$HOME/metis-security-review" +``` + +This writes `results/libpng-live.json`. + {{< /tab >}} + {{< tab header="Use captured report" >}} +This option creates `results/libpng-review-candidate.json` from a captured Metis report. Use it when you want the next triage section to match the source region and explanation shown in this Learning Path. + +```bash +cat > results/libpng-review-candidate.json <<'EOF' +{ + "_provenance": { + "artifact_type": "Captured Metis report finding", + "release_under_test": "metis-v1.5.0", + "capture_note": "Use this artifact to study report structure and triage; do not treat it as guaranteed output for every run." + }, + "reviews": [ + { + "reviews": [ + { + "issue": "Externally controlled image dimensions can trigger division by zero in IDAT chunk length validation", + "code_snippet": "size_t row_factor_l =\n (size_t)png_ptr->width\n * (size_t)png_ptr->channels\n * (png_ptr->bit_depth > 8? 2: 1)\n + 1\n + (png_ptr->interlaced? 6: 0);\n\n\n size_t row_factor = (png_uint_32)row_factor_l;\n if (png_ptr->height > PNG_UINT_32_MAX/row_factor)", + "start_line": 3163, + "end_line": 3172, + "reasoning": "For IDAT chunks, row_factor_l is derived from PNG IHDR-controlled fields stored in png_ptr (width, channels/color type, bit depth, and interlace). On platforms where size_t is wider than png_uint_32, row_factor_l can exceed 32 bits for very large but 31-bit-valid PNG dimensions. It is then truncated by the cast to png_uint_32 and stored back in size_t. Certain valid arithmetic combinations can truncate to 0 (for example, an RGB 8-bit non-interlaced row factor of width*3+1 can equal 2^32). The next line divides by row_factor without checking for zero, so a crafted PNG can crash the decoder during chunk header validation, causing denial of service.", + "mitigation": "Do not truncate row_factor_l before using it as a divisor. Keep the computation in png_alloc_size_t/size_t, explicitly check row_factor_l != 0 and row_factor_l <= PNG_UINT_32_MAX where required, or clamp oversized row factors to a safe maximum before division. Prefer overflow-checked helpers for all dimension-derived arithmetic.", + "confidence": 0.86, + "cwe": "CWE-369", + "severity": "Medium", + "anchor": { + "file_path": "pngrutil.c", + "start_line": 3163, + "end_line": 3172, + "start_col": 0, + "end_col": 55, + "start_byte": 97432, + "end_byte": 97734, + "symbol": "pngrutil.c::png_check_chunk_length", + "kind": "range", + "content_hash": "d4fcc275896c8895", + "confidence": "exact" + }, + "line_number": 3163 + } + ], + "file": "pngrutil.c", + "file_path": "pngrutil.c" + } + ] +} +EOF +``` + +The captured report demonstrates the Metis report structure and provides a stable triage exercise. It is not expected output for every run. + {{< /tab >}} +{{< /tabpane-normal >}} + +## What you've accomplished and what's next + +You prepared production-like vulnerable libpng source without benchmark hints and selected a report for triage. Next, you will check a reported finding against the benchmark diff and decide how to handle additional live findings. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md new file mode 100644 index 0000000000..d3f5bdd106 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md @@ -0,0 +1,125 @@ +--- +title: Triage a libpng finding +description: Check a Metis report finding against source code, attacker control, and the Magma ground-truth diff. +weight: 6 +layout: "learningpathall" +--- + +## List the reported findings + +Metis reports findings, reasoning, source locations, and mitigations. This page adds a manual triage workflow: you compare a finding with source code, attacker control, build assumptions, and the Magma diff before treating it as confirmed. + +Select the report you want to summarize: + +{{< tabpane-normal >}} + {{< tab header="Live report" >}} +Use this option if you ran the live review and want to inspect its results first: + +```bash +REPORT=results/libpng-live.json +``` + {{< /tab >}} + {{< tab header="Captured report" >}} +Use this option for the guided row-factor triage, or if your live review returned no findings: + +```bash +REPORT=results/libpng-review-candidate.json +``` + {{< /tab >}} +{{< /tabpane-normal >}} + +Print a compact summary: + +```bash +python - "$REPORT" <<'PY' +import json +import sys +from pathlib import Path + +data = json.loads(Path(sys.argv[1]).read_text()) +for review_group in data["reviews"]: + for finding in review_group["reviews"]: + anchor = finding.get("anchor") or {} + file_path = ( + anchor.get("file_path") + or finding.get("primary_file") + or finding.get("file") + or review_group.get("file") + ) + line = anchor.get("start_line") or finding.get("line_number") + print( + f"{finding['severity']:>6} {finding['cwe']}: " + f"{finding['issue']} " + f"({file_path}:{line})" + ) +PY +``` + +If you selected the captured report, the expected output is: + +```output +Medium CWE-369: Externally controlled image dimensions can trigger division by zero in IDAT chunk length validation (pngrutil.c:3163) +``` + +The pre-provided report contains one medium-severity finding in `pngrutil.c`: a possible division by zero after a row-size value is narrowed. If the summary from your live report includes a similar row-factor or division-by-zero finding in `pngrutil.c`, you can continue with the live report. Otherwise, set `REPORT=results/libpng-review-candidate.json` and use the captured report for the guided triage. + +## Triage the row-factor finding + +DVCP taught the mechanics of running Metis and reading a report. The prepared libpng tree is the more realistic example: you need to inspect the reported source region, check whether values parsed from the PNG file can influence the operation (attacker-controlled values), and compare the region with the prepared-source diff. + +For this finding, you are checking four things: + +1. Does the reported source location contain the operation Metis described? +2. Can PNG-controlled fields influence the value? +3. Can the narrowing conversion produce a zero divisor? +4. Does the source region overlap a known vulnerability introduced by the Magma preparation? + +First, display the reported source region: + +```bash +sed -n '3155,3180p' libpng-review/pngrutil.c +``` + +Next, inspect the prepared-source diff around the same expression: + +```bash +git -C libpng-review diff -U8 -- pngrutil.c | sed -n '/row_factor_l/,+24p' +``` + +Use the report, source region, and diff to decide whether this is a credible finding: + +1. `png_ptr->width`, channel count, bit depth, and interlace state originate from the parsed PNG. +2. The arithmetic is performed in `size_t`, then narrowed to `png_uint_32`. +3. A sufficiently large row factor can wrap to zero during the narrowing conversion. +4. The narrowed value is used as a divisor before a zero check. +5. The changed lines overlap a Magma-introduced vulnerability in the prepared-source diff. + +This is a strong match to the known vulnerable change in the prepared libpng target. Metis identified the relevant operation, explained attacker control, and proposed avoiding the narrowing conversion or checking the value before division. The triage decision is therefore **confirmed for this exercise target**. + +## Triage additional live findings + +Your live run might return no findings, the row-factor finding, or different findings. An empty report does not prove the target is safe. A finding outside the injected changes is not automatically false, but it is not evidence by itself that Metis discovered a new vulnerability. + +Classify each additional finding as **unconfirmed** until you complete more work: + +- Check the relevant file-format specification and API contracts. +- Trace callers and helper functions to find earlier validation or cleanup behavior. +- Check whether build configuration makes the code reachable. +- Create a focused reproducer and observe behavior under AddressSanitizer and UndefinedBehaviorSanitizer. +- Search upstream issues and security advisories before reporting a new vulnerability. + +## Record a triage decision + +For each finding, record at least: + +- Status: confirmed, rejected, duplicate, accepted risk, or unconfirmed. +- Attacker-controlled input and the path to the operation. +- Relevant build and platform assumptions. +- Reproduction evidence or the missing evidence needed next. +- Proposed fix owner and test coverage. + +Do not publish unverified findings against active open-source projects. Follow the project's security policy and coordinated-disclosure process if further investigation confirms an issue. + +## What you've accomplished and what's next + +You installed Metis, compared a review with known C defects, prepared a realistic benchmark target, and checked a Metis finding against source evidence. You can apply the same workflow to a bounded directory, file, or patch in your own C project, or for other languages. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md new file mode 100644 index 0000000000..bfdbd80814 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md @@ -0,0 +1,82 @@ +--- +title: Review C code for security vulnerabilities with Arm's Metis (AI-powered Security Code Review) + +description: Install Arm Metis, review vulnerable C code, and triage findings from a prepared libpng target. + +minutes_to_complete: 45 + +who_is_this_for: This Learning Path is for developers who want to add AI-assisted security review to an existing code-review workflow. The examples in this learning path use C, but Metis is applicable to other languages as well. + +learning_objectives: + - Explain how Metis uses language-aware source analysis and model reasoning + - Install Metis and connect it to a supported model provider + - Review vulnerable C code and inspect JSON and SARIF output + - Triage Metis findings against source code and a Magma-derived libpng diff + +prerequisites: + - A macOS computer, Linux computer, or Windows computer with WSL2 + - An API key and sufficient quota for a [Metis-supported model provider](https://github.com/arm/metis#2-set-up-llm-provider) + - Basic familiarity with C, Git, command-line tools, and security code-review procedures + +author: + - Matt Cossins + - Michalis Spyrou + - Michael Koslov + +generate_summary_faq: false +rerun_summary: false +rerun_faqs: false + +### Tags +skilllevels: Introductory +subjects: Security +armips: + - Neoverse + - Cortex-A + - Cortex-M +operatingsystems: + - Linux + - macOS + - Windows +tools_software_languages: + - Arm Metis + - C + - Python + - Git + - JSON + - SARIF + +shared_path: true +shared_between: + - embedded-and-microcontrollers + - laptops-and-desktops + - servers-and-cloud-computing + +further_reading: + - resource: + title: Arm Metis GitHub repository + link: https://github.com/arm/metis + type: website + - resource: + title: Empowering engineers with AI-enabled security code review + link: https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/empowering-engineers-with-ai-enabled-security-code-review + type: blog + - resource: + title: Arm Metis brings agentic AI to software security + link: https://newsroom.arm.com/blog/arm-metis-agentic-ai-security + type: blog + - resource: + title: Magma ground-truth fuzzing benchmark + link: https://hexhive.epfl.ch/magma/ + type: website + - resource: + title: Damn Vulnerable C Program repository + link: https://github.com/hardik05/Damn_Vulnerable_C_Program + type: website + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 +layout: "learningpathall" +learning_path_main_page: "yes" +--- diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md new file mode 100644 index 0000000000..9196d79577 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md @@ -0,0 +1,9 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 +title: "Next Steps" +layout: "learningpathall" +--- + diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py b/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py new file mode 100644 index 0000000000..22b1b9e152 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py @@ -0,0 +1,195 @@ +#!/usr/bin/env python3 + +import argparse +import re +import shutil +import subprocess +from pathlib import Path + + +LIBPNG_REPOSITORY = "https://github.com/pnggroup/libpng.git" +LIBPNG_COMMIT = "a37d4836519517bdce6cb9d956092321eca3e73b" +TARGET_MACROS = { + "ENABLE_MAGMA_FIXES": False, + "MAGMA_ENABLE_CANARIES": False, + "MAGMA_ENABLE_FIXES": False, +} + +IFDEF = re.compile(r"^\s*#\s*(ifdef|ifndef)\s+(\w+)\s*$") +IF = re.compile(r"^\s*#\s*if\s+defined\s*\(\s*(\w+)\s*\)\s*$") +ELSE = re.compile(r"^\s*#\s*else\b") +ELIF = re.compile(r"^\s*#\s*elif\b") +ENDIF = re.compile(r"^\s*#\s*endif\b") +ANY_IF = re.compile(r"^\s*#\s*(if|ifdef|ifndef)\b") + + +def run(*command: str, cwd: Path | None = None, input_text: str | None = None) -> None: + subprocess.run( + command, + cwd=cwd, + input=input_text, + text=True, + check=True, + ) + + +def sanitize_file(path: Path) -> None: + lines = path.read_text(errors="surrogateescape").splitlines(keepends=True) + output: list[str] = [] + stack: list[dict[str, object]] = [] + emitting = True + + for line_number, line in enumerate(lines, start=1): + match = IFDEF.match(line) + if match and match.group(2) in TARGET_MACROS: + directive, macro = match.groups() + condition = TARGET_MACROS[macro] + if directive == "ifndef": + condition = not condition + stack.append( + { + "targeted": True, + "parent_emitting": emitting, + "condition": condition, + "else_seen": False, + } + ) + emitting = emitting and condition + continue + + match = IF.match(line) + if match and match.group(1) in TARGET_MACROS: + condition = TARGET_MACROS[match.group(1)] + stack.append( + { + "targeted": True, + "parent_emitting": emitting, + "condition": condition, + "else_seen": False, + } + ) + emitting = emitting and condition + continue + + if ANY_IF.match(line): + stack.append({"targeted": False}) + if emitting: + output.append(line) + continue + + if ELSE.match(line): + if not stack: + raise RuntimeError(f"unmatched #else in {path}:{line_number}") + frame = stack[-1] + if frame["targeted"]: + if frame["else_seen"]: + raise RuntimeError(f"duplicate #else in {path}:{line_number}") + frame["else_seen"] = True + emitting = bool(frame["parent_emitting"]) and not bool( + frame["condition"] + ) + elif emitting: + output.append(line) + continue + + if ELIF.match(line): + if stack and stack[-1]["targeted"]: + raise RuntimeError( + f"targeted #elif is unsupported in {path}:{line_number}" + ) + if emitting: + output.append(line) + continue + + if ENDIF.match(line): + if not stack: + raise RuntimeError(f"unmatched #endif in {path}:{line_number}") + frame = stack.pop() + if frame["targeted"]: + emitting = bool(frame["parent_emitting"]) + elif emitting: + output.append(line) + continue + + if emitting: + output.append(line) + + if stack: + raise RuntimeError(f"unterminated conditional in {path}") + + path.write_text("".join(output), errors="surrogateescape") + + +def parse_arguments() -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="Prepare the sanitized Magma libpng target for Metis review." + ) + parser.add_argument( + "--magma-dir", + type=Path, + required=True, + help="Path to a cloned HexHive/magma repository.", + ) + parser.add_argument( + "--output-dir", + type=Path, + required=True, + help="Destination for the prepared libpng source tree.", + ) + return parser.parse_args() + + +def main() -> None: + arguments = parse_arguments() + magma_dir = arguments.magma_dir.resolve() + output_dir = arguments.output_dir.resolve() + patch_dir = magma_dir / "targets" / "libpng" / "patches" / "bugs" + + if not patch_dir.is_dir(): + raise SystemExit(f"Magma libpng patches not found: {patch_dir}") + if output_dir.exists(): + raise SystemExit(f"Output directory already exists: {output_dir}") + + run("git", "clone", LIBPNG_REPOSITORY, str(output_dir)) + run("git", "checkout", LIBPNG_COMMIT, cwd=output_dir) + + patch_files = sorted(patch_dir.glob("*.patch")) + for patch_file in patch_files: + patch_text = patch_file.read_text().replace( + "%MAGMA_BUG%", patch_file.stem + ) + run("patch", "-p1", cwd=output_dir, input_text=patch_text) + + for path in output_dir.rglob("*"): + if path.is_file() and path.suffix.lower() in { + ".c", + ".cc", + ".cpp", + ".cxx", + ".h", + ".hpp", + }: + sanitize_file(path) + + for pattern in ("*.orig", "*.rej"): + for path in output_dir.rglob(pattern): + path.unlink() + + (output_dir / ".metis.md").write_text( + "# Project security context\n\n" + "This library parses PNG files supplied by untrusted users. Treat image " + "dimensions, chunk lengths, offsets, and compressed data as " + "attacker-controlled. Review memory safety, integer arithmetic, parser " + "state, resource exhaustion, and cleanup on error paths.\n" + ) + (output_dir / ".metisignore").write_text( + ".git/\ncontrib/\ndocs/\nexamples/\nscripts/\ntests/\n" + ) + + print(f"Prepared libpng commit {LIBPNG_COMMIT}") + print(f"Applied {len(patch_files)} Magma vulnerability-introducing patches") + print(f"Output: {output_dir}") + + +if __name__ == "__main__": + main() From b9ff4d70a200029771c532eefdab397c8d370186 Mon Sep 17 00:00:00 2001 From: Michael Kozlov Date: Tue, 13 Oct 2026 10:22:49 +0100 Subject: [PATCH 2/4] docs: update Metis CLI examples and captured output --- .../1-understand-metis.md | 37 ++++-- .../2-install-metis.md | 66 ++++------ .../3-review-vulnerable-c.md | 117 ++++++++++-------- .../4-prepare-libpng.md | 76 ++++++++---- .../5-triage-findings.md | 93 ++++++++++---- 5 files changed, 237 insertions(+), 152 deletions(-) diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md index db4b7a4a38..df77c0d298 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md @@ -21,24 +21,23 @@ Model-backed review can consume substantial API tokens. The initial exercise is A Metis review starts with source files and project context. Metis collects language-aware evidence, asks your configured model to investigate possible vulnerabilities, and returns structured findings for a human reviewer. -For C programs, reviewers often care about memory safety, integer arithmetic, resource management, parser state, and input validation. Tree-sitter is a parser that turns source files into syntax trees, so tools can identify functions, variables, calls, and other language constructs without treating code as plain text. Metis uses Tree-sitter to build a code graph of symbols and relationships. Its reachability review follows paths between possible attacker-controlled inputs and security-sensitive operations. The model receives source paths and targeted checks, rather than only a large prompt containing raw files. +Metis offers two review approaches: -## Providing Metis with context +- **Simple LLM review (default):** the model examines source code or a patch using language-specific security prompts and project guidance. It returns structured findings without requiring a code graph. This is also the review path for languages without C/C++ reachability support. +- **Optional reachability review:** for C/C++, Tree-sitter builds a CodeGraph of functions, calls, and source facts. When enabled, `reachability` reviews selected functions with this evidence and surrounding caller/callee context. Graph evidence does not by itself prove attacker control, build reachability, or exploitability. -Metis has two main ways to give the model source-code context: +In Metis 2.0.0, the packaged Review stage runs `simple_llm_review`, `finding_dedup`, and `result`. Reachability is opt-in, not part of this default review. Initialize still builds the CodeGraph for Triage. The exercises use the shipped graph through `review_code` and `review_file` without enabling reachability. -- **Source navigation:** this is the default for review commands. Metis reads files from the codebase and gives the model targeted source snippets, paths, and checks. For C code, this includes Tree-sitter-based reachability analysis. Use this mode when you want to review a file, directory, patch, or small codebase without building an index first. -- **Vector indexing:** this is optional. The `index` command builds an embedding index for code and documentation, and `update` refreshes that index from a patch. Use indexing when you want to ask broad questions with `ask`, search across a larger repository, or add indexed context to later review and triage commands. - -The exercises in this Learning Path use source navigation and C reachability analysis. You do not build an index because the review targets are small and the tasks are focused. +## Follow the review workflow A typical Metis workflow looks like this: 1. Pick the review question, such as "review this parser file" or "review this patch." -2. Choose a command: `review_code`, `review_dir`, `review_file`, `review_patch`, `ask`, or `triage`. +2. Choose a review command: `review_code`, `review_dir`, `review_file`, or `review_patch`. 3. Add project-specific security context in `.metis.md` when the source needs extra assumptions. 4. Run Metis and save the result as JSON, SARIF, or both. -5. Check each finding against the source code, input boundaries, build configuration, and tests. +5. Optionally run Metis triage to assess the candidates against additional source evidence. +6. Check the remaining findings against the source code, input boundaries, build configuration, and tests. Metis supplies the review workflow, source navigation, prompts, validation stages, and output handling. The model provider supplies the reasoning model. Results can vary between models and providers even when the source code is unchanged. @@ -51,7 +50,23 @@ Use the narrowest scope that answers your review question: - `review_file` focuses on one source file while retaining codebase context. - `review_patch` focuses on changed code and is suited to pull-request or pre-merge review. -This Learning Path demonstrates use of `review_code` and `review_file`. +This Learning Path demonstrates `review_code` and `review_file` at the prompt started with `--interactive`. Direct review commands run their Initialize prerequisites automatically, so the exercises do not need a separate `init` command. + +## Strengthen findings with automated triage + +Review discovers candidate issues; triage assesses existing findings. Metis starts with a source window around the reported location. For C/C++, available CodeGraph context adds the enclosing function, callers, and callees. The model can use navigation tools to inspect missing definitions, macros, or guards, then decide whether the evidence supports the finding (`valid`), refutes it (`invalid`), or cannot settle it (`inconclusive`). Keyword rules do not override these verdicts. + +In the built-in workflow, both `valid` and `invalid` require citations and a resolution chain. Metis checks that cited lines were displayed to the model for that finding, remain inside the codebase, and match current source. Unusable citations produce `inconclusive`. These checks validate citation provenance, not the semantic truth of the model's reasoning; even `valid` is not proof of a working exploit. + +Without `--interactive`, Metis 2.0.0 runs the full configured graph. The packaged graph includes Initialize, Review, and Triage, so it triages findings automatically. In interactive mode, `--triage` optionally chains Triage after `review_code`, `review_dir`, `review_file`, or `review_patch`; the `triage` command can instead assess an existing Metis review JSON or SARIF report. + +Automated triage can consume additional model tokens. These exercises use `--interactive` without `--triage` so you can inspect review-only findings and practice manual triage. See the [Metis 2.0.0 triage flow](https://github.com/arm/metis/blob/metis-v2.0.0/docs/triage-flow.md) for decisions, citation validation, and report annotations. + +## Configure the workflow with graph YAML + +The execution graph describes the workflow, not the source code's call graph. In `metis.yaml`, `metis_engine.execution` selects stages and their nodes. The packaged graph runs `initialize` to prepare context and build the CodeGraph, `review` to run simple LLM review and deduplicate findings, and `triage` to assess Review's SARIF findings. Input bindings pass data and establish dependencies; `depends_on` adds ordering without passing a value. + +The exercises configure only the model provider and inherit this graph. An explicit `execution` section replaces the whole packaged graph rather than merging individual nodes. To customize it or enable reachability, start from the [Metis 2.0.0 execution-graph configuration](https://github.com/arm/metis/blob/metis-v2.0.0/docs/execution-graph.md#default-graph) and retain the required stages, outputs, and dependencies. ## Interpret Metis findings @@ -76,4 +91,4 @@ You will apply these questions to the libpng report later in the Learning Path. ## What you've learned and what's next -You have seen how Metis gathers source evidence, when indexing is useful, and why every result needs human triage. Next, you will install the tested release and add your provider API key. +You have seen how default simple LLM review and optional reachability review find candidates, how automated triage assesses the evidence, and why human validation is still needed. Next, you will install the tested release and add your provider API key. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md index 56c4ff3b5f..36324589ce 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md @@ -25,7 +25,7 @@ If the Xcode Command Line Tools are already installed, macOS reports that no ins ## Install uv and Python -Metis is a Python project. Use `uv` to install Python 3.12 and create an isolated Metis environment: +Metis requires Python 3.12 or later. This Learning Path uses `uv` to install Python 3.12 and create an isolated Metis environment: ```bash curl -LsSf https://astral.sh/uv/install.sh | sh @@ -68,32 +68,35 @@ cd "$HOME/metis-security-review" ## Clone and install Metis -Clone the Metis repository at the 1.5.0 release: +Clone the Metis repository at the tested [2.0.0 release](https://github.com/arm/metis/releases/tag/metis-v2.0.0): ```bash -git clone --branch metis-v1.5.0 --depth 1 \ +git clone --branch metis-v2.0.0 --depth 1 \ https://github.com/arm/metis.git \ "$HOME/metis-security-review/metis" cd "$HOME/metis-security-review/metis" -uv sync --no-dev +uv sync --no-dev --python 3.12 source "$HOME/metis-security-review/metis/.venv/bin/activate" command -v metis metis --version +git rev-parse HEAD cd "$HOME/metis-security-review" ``` -`uv sync --no-dev` installs only the runtime dependencies needed to run the Metis CLI. It selects a Python version that satisfies the Metis project requirements, creates `metis/.venv`, and installs Metis there. Activating the environment makes the `metis` command and the matching Python interpreter available in the current terminal. +`uv sync --no-dev --python 3.12` installs only the runtime dependencies needed to run the Metis CLI, using Python 3.12. It creates `metis/.venv` and installs Metis there. Activating the environment makes the `metis` command and the matching Python interpreter available in the current terminal. -The version command should report: +The expected output from `metis --version` for this tag is: ```output -Metis 1.5.0 +Metis 2.0.0 ``` +The `metis-v2.0.0` tag points to commit [`742b20d685b0aeb3216b36ddfc44bc5e05ace82c`](https://github.com/arm/metis/commit/742b20d685b0aeb3216b36ddfc44bc5e05ace82c). The output of `git rev-parse HEAD` should match this commit. Record the version and commit when comparing runs. + If you open a new terminal, return to the exercise workspace and reactivate the Metis environment before continuing: ```bash @@ -106,51 +109,32 @@ The path printed by `command -v` should end with `metis-security-review/metis/.v ## Configure the model provider -Metis supports several hosted and local model providers. The following commands use OpenAI as an example. Export your API key in the same terminal that will run Metis: +Metis supports several hosted and local model providers. These examples use an OpenAI-compatible endpoint that serves the model ID `gpt-6.1-sol`. Export your API key in the same terminal that will run Metis: ```bash export OPENAI_API_KEY="your-api-key" -# Optional: set this if your organization uses an OpenAI-compatible proxy. -# export OPENAI_BASE_URL="https://your-openai-compatible-endpoint/v1" ``` -Do not add secrets to `.metis.md`, shell history, source control, or report files. - -If your organization provides an OpenAI-compatible endpoint, export `OPENAI_BASE_URL` before running Metis. Additionally, remember to install any certificates required by your organization so Python can verify the endpoint. Otherwise, the examples use the public OpenAI API endpoint. - -For other model providers, follow the [Metis provider instructions](https://github.com/arm/metis#2-set-up-llm-provider). +Do not add secrets to `.metis.md`, `metis-review.yaml`, shell history, source control, or report files. -Check that Python inside the active Metis environment can reach the configured OpenAI endpoint: +Create a provider configuration outside the review targets. Replace `https://your-provider.example/v1` below with the trusted base URL of the provider that serves `gpt-6.1-sol`, including any required API prefix. The example address is a placeholder, not a service endpoint: ```bash -python - <<'PY' -import os -import httpx - -base_url = ( - os.environ.get("OPENAI_BASE_URL") - or os.environ.get("OPENAI_API_BASE") - or "https://api.openai.com/v1" -) -response = httpx.get( - base_url.rstrip("/") + "/models", - headers={"Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]}, - timeout=20, -) -print(f"status: {response.status_code}") -response.raise_for_status() -PY +cat > "$HOME/metis-security-review/metis-review.yaml" <<'EOF' +llm_provider: + name: openai + model: gpt-6.1-sol + base_url: https://your-provider.example/v1 +EOF ``` -The expected output is: +The review commands select this file explicitly with `--config`. The captured reviews use `gpt-6.1-sol`; choosing another model can change the findings. Metis reads the API key from `OPENAI_API_KEY`. This provider-only configuration inherits the packaged execution graph; adding an `execution` section replaces that graph as a whole. -```output -status: 200 -``` +Model IDs and availability depend on the endpoint. Use a provider that offers `gpt-6.1-sol` and supports the OpenAI Responses API; do not assume that a model ID from one provider is available at another provider's URL. -If this check reports `401` or `403`, check the API key and model access. If it reports `404`, check the base URL. If it reports a certificate verification error or timeout, check your network, proxy, and Python certificate configuration before running a review. +For custom CA certificates, follow the [HTTPX `SSL_CERT_FILE` guidance](https://www.python-httpx.org/environment_variables/#ssl_cert_file). Keep certificate verification enabled. -The endpoint you use can affect runtime, rate limits, retry behavior, response formatting, and the set of reported findings. +For other providers, follow the [Metis 2.0.0 provider setup instructions](https://github.com/arm/metis/blob/metis-v2.0.0/README.md#2-set-up-llm-provider). ## Confirm that the command starts @@ -160,8 +144,8 @@ Display the command help without starting a model-backed review. This verifies t metis --help ``` -The help output should include options such as `--codebase-path`, `--command`, `--non-interactive`, and `--output-file`. +The help output should include `--codebase-path`, `--config`, `--interactive`, and `--output-file`. Repeat `--output-file` to save more than one format. Metis 2.0.0 runs the configured graph by default; the packaged graph includes Initialize, Review, and Triage. Use `--interactive` to open the prompt for the exercises. Their direct review commands run Initialize prerequisites automatically, and omitting `--triage` keeps the results review-only so you can assess the findings manually afterward. ## What you've accomplished and what's next -You installed Metis v1.5.0 and set it up with your model provider. Next, you will review a small C program and compare the report with defects that are intentionally present in its source. +You installed the tested Metis 2.0.0 release and set it up with your model provider. Next, you will review a small C program and compare the report with defects that are intentionally present in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md index 9136ef8c8b..cf768ec8d3 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md @@ -34,16 +34,37 @@ The note tells Metis how to treat inputs and security boundaries. The source fil ## Run the review -Change into the review directory and run `review_code`. Save JSON for detailed inspection and SARIF for code-scanning tools: +Change into the review directory and start the Metis interactive prompt with the provider configuration you created. Save JSON for detailed inspection and SARIF for code-scanning tools: ```bash cd "$HOME/metis-security-review/dvcp-review" metis \ + --interactive \ --codebase-path . \ - --non-interactive \ - --command "review_code" \ + --config "$HOME/metis-security-review/metis-review.yaml" \ --output-file ../results/dvcp.json \ --output-file ../results/dvcp.sarif +``` + +At the Metis prompt, run the review, then exit after it completes. The command runs its Initialize prerequisites automatically, then performs the packaged simple LLM review and deduplication: + +```console +review_code +exit +``` + +Metis prints review progress and detailed findings under `File:`, with an `Identified issue` heading. Finding fields include `Snippet`, `Line number`, `CWE`, `Severity`, `Why`, `Mitigation`, and `Confidence`. After the findings, the export and token-usage output is similar to: + +```output +Results saved to ../results/dvcp.json +SARIF report saved to ../results/dvcp.sarif +Token usage (review_code) +Current (input: 8,420 · output: 3,349 · total: 11,769) +``` + +Back in your shell, return to the exercise workspace: + +```bash cd "$HOME/metis-security-review" ``` @@ -53,7 +74,7 @@ It can take a couple of minutes to complete. ## Summarize the JSON report -Use the Python interpreter from the active Metis environment to print the issue type and source location for each finding: +The JSON report stores findings in `reviews[].reviews`. Source locations can appear in an `anchor`, in `primary_file` and `line_number`, or in the enclosing review group's `file` field. Use the Python interpreter from the active Metis environment to count the findings and print their issue types and source locations: ```bash python - <<'PY' @@ -61,54 +82,55 @@ import json from pathlib import Path data = json.loads(Path("results/dvcp.json").read_text()) -findings = [ - finding - for review_group in data["reviews"] - for finding in review_group["reviews"] -] - -print(f"Finding count: {len(findings)}") -for finding in findings: - anchor = finding.get("anchor") or {} - file_path = ( - anchor.get("file_path") - or finding.get("primary_file") - or finding.get("file") - ) - line = anchor.get("start_line") or finding.get("line_number") - print( - f"- {finding['cwe']} {finding['issue']} " - f"({file_path}:{line})" - ) +review_groups = data["reviews"] +count = sum(len(group["reviews"]) for group in review_groups) +print(f"Finding count: {count}") +for review_group in review_groups: + for finding in review_group["reviews"]: + anchor = finding.get("anchor") or {} + file_path = ( + anchor.get("file_path") + or finding.get("primary_file") + or finding.get("file") + or review_group.get("file") + ) + line = anchor.get("start_line") or finding.get("line_number") + print( + f"- {finding['cwe']} {finding['issue']} " + f"({file_path}:{line})" + ) PY ``` +This example was captured on November 2, 2026 using the `metis-v2.0.0` tag ([`742b20d`](https://github.com/arm/metis/commit/742b20d685b0aeb3216b36ddfc44bc5e05ace82c)), with `gpt-6.1-sol` through an OpenAI-compatible endpoint and DVCP commit [`8dcbd20`](https://github.com/hardik05/Damn_Vulnerable_C_Program/commit/8dcbd202887f587783952a320a2301de701be529). The provider reported the model ID as `gpt-6.1-sol`. The run used the packaged simple LLM review and deduplication nodes; reachability review and automated triage were not enabled. + The output is similar to: ```output -Finding count: 9 -- CWE-190 Image width and height read from the file are added in a signed int without validation, allowing overflow before the result is used as an allocation size. (dvcp.c:54) -- CWE-120 The code copies the full img.data array into buff1 without ensuring that the allocated size1 buffer is at least sizeof(img.data). (dvcp.c:58) -- CWE-415 buff1 is freed unconditionally and then freed again when size1 is even. (dvcp.c:62) -- CWE-416 After buff1 is freed, an alternate branch can write to buff1 when size1 is divisible by 3. (dvcp.c:67) -- CWE-191 Image dimensions read from the file are subtracted and offset in a signed int without validation, allowing underflow or negative values before allocation. (dvcp.c:74) -- CWE-120 The code copies the full img.data array into buff2 without ensuring that the allocated size2 buffer is at least sizeof(img.data). (dvcp.c:79) -- CWE-787 The code copies the full img.data array into buff4 even though buff4 was allocated with attacker-controlled size3 and may be smaller than sizeof(img.data). (dvcp.c:87) -- CWE-125 The computed size3 value is used as an index into a 10-byte stack buffer without bounds checking. (dvcp.c:90) -- CWE-674 stack_operation recursively calls itself in an infinite loop with a large stack buffer, causing stack exhaustion. (dvcp.c:25) +Finding count: 10 +- CWE-125 Unterminated image fields cause out-of-bounds string reads (dvcp.c:48) +- CWE-122 Unchecked dimension-derived allocations permit heap overflows and NULL dereferences (dvcp.c:54) +- CWE-415 An even dimension sum causes a double free (dvcp.c:59) +- CWE-416 An odd dimension sum divisible by three causes a use-after-free write (dvcp.c:64) +- CWE-369 Unvalidated image dimensions cause invalid integer division (dvcp.c:82) +- CWE-787 File-controlled quotient indexes outside a stack buffer (dvcp.c:90) +- CWE-787 The heap buffer is accessed one byte past its allocation (dvcp.c:91) +- CWE-674 Recursive operation has no termination condition (dvcp.c:22) +- CWE-400 Allocation loop intentionally retains memory until exhaustion (dvcp.c:109) +- CWE-401 Large-quotient branch discards an allocated buffer without freeing it (dvcp.c:97) ``` -Your exact count and wording can differ. A valid report can contain zero findings. An empty report means that this run did not report any findings; it does not mean the program is safe. If you expected findings, check the provider reachability test and any Metis log output before judging the result. +Your exact count and wording can differ. A valid report can contain zero findings. An empty report means that this run did not report any findings; it does not mean the program is safe. If you expected findings, check your provider connection and any Metis log output before judging the result. ## Compare the report with known defects Display four source regions that the program identifies as deliberately vulnerable: ```bash -sed -n '20,28p' dvcp-review/dvcp.c -sed -n '51,68p' dvcp-review/dvcp.c -sed -n '72,83p' dvcp-review/dvcp.c -sed -n '89,108p' dvcp-review/dvcp.c +sed -n '14,28p' dvcp-review/dvcp.c +sed -n '42,69p' dvcp-review/dvcp.c +sed -n '72,87p' dvcp-review/dvcp.c +sed -n '89,115p' dvcp-review/dvcp.c ``` Check whether your report covers these concrete cases: @@ -119,20 +141,17 @@ Check whether your report covers these concrete cases: - `size3` is used as an index into fixed-size and heap buffers without bounds checks. - Attacker-controlled dimensions can select recursive stack exhaustion or repeated heap allocation. -During validation for this Learning Path, Metis v1.5.0 reported nine findings. This table shows how that run compared with the known defects in the teaching program: +The captured Metis 2.0.0 JSON and SARIF reports contain ten findings. Source anchors can span multiple lines, and one finding can discuss additional sites in its reasoning. The table maps those reported regions and explanations to the known source defects: | Source issue | Tested-run outcome | | --- | --- | -| `size1` signed overflow before `malloc()` and `memcpy()` | Reported as `CWE-190` at `dvcp.c:54` | -| `memcpy()` into `buff1` without checking `size1` against `sizeof(img.data)` | Reported as `CWE-120` at `dvcp.c:58` | -| `buff1` double free | Reported as `CWE-415` at `dvcp.c:62` | -| `buff1` use-after-free write | Reported as `CWE-416` at `dvcp.c:67` | -| `size2` signed underflow or negative allocation size before `memcpy()` | Reported as `CWE-191` at `dvcp.c:74` | -| `memcpy()` into `buff2` without checking `size2` against `sizeof(img.data)` | Reported as `CWE-120` at `dvcp.c:79` | -| `buff4` allocation smaller than `memcpy()` length | Reported as `CWE-787` at `dvcp.c:87` | -| `size3` out-of-bounds stack-buffer read | Reported as `CWE-125` at `dvcp.c:90` | -| Recursive `stack_operation()` with a large stack buffer | Reported as `CWE-674` at `dvcp.c:25` | -| Divide by zero, indexed heap out-of-bounds read/write, indexed stack out-of-bounds write, memory leak, and heap exhaustion | Not reported in that run | +| Non-NUL-terminated image fields passed to `%s` | `CWE-125` at `dvcp.c:48` | +| Dimension-derived allocations followed by fixed-size copies | `CWE-122` anchored at `dvcp.c:54–58`; its reasoning also covers `dvcp.c:74–79` and `dvcp.c:82–87` | +| `buff1` double free and use-after-free | `CWE-415` at `dvcp.c:59–63`; `CWE-416` at `dvcp.c:64–69` | +| Integer division using unvalidated image dimensions | `CWE-369` at `dvcp.c:82` | +| Indexed stack and heap accesses | Separate `CWE-787` findings at `dvcp.c:90–94` and `dvcp.c:91–95` | +| `buff4` allocation lost without being freed | `CWE-401` at `dvcp.c:97–100` | +| Unbounded recursion and repeated heap allocation | `CWE-674` at `dvcp.c:22–27`; `CWE-400` at `dvcp.c:109–115` | For each reported finding, verify the source location, attacker control, and operation. Also record known source defects that the run omitted. The goal is to practice reviewing Metis output, not to turn the finding count into a coverage score. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md index a17b638f9c..c09711bfa4 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md @@ -66,7 +66,7 @@ git -C libpng-review status --short git -C libpng-review diff --stat ``` -The diff summary shows that the script applied patches that introduce known vulnerable code to libpng. You will inspect the relevant `pngrutil.c` changes during triage. Do not send patch names, benchmark descriptions, or known vulnerability lists to Metis. +The diff summary shows that the script applied patches that introduce known vulnerable code to libpng. During triage, you will inspect a `png.c` guard and the related `pngrutil.c` allocation. Do not send patch names, benchmark descriptions, or known vulnerability lists to Metis. ## Choose a libpng report option @@ -79,58 +79,80 @@ You need a Metis report before you can triage a finding. Choose one of these opt {{< tab header="Run live review" >}} This option asks Metis to review only `pngrutil.c`, because the triage exercise focuses on a finding in that file. -The live libpng review can run for many minutes, and consume substantial model tokens. Use the pre-captured report option if a live run is too costly or does not produce a finding you can triage. +The captured Metis 2.0.0 review with `gpt-6.1-sol` used 353,579 model tokens (351,293 input and 2,286 output). Usage varies with the run and model configuration. A live run can take several minutes and consume substantial API quota. Use the captured report option to avoid those model calls or if your run does not produce a finding you can triage. ```bash cd "$HOME/metis-security-review/libpng-review" metis \ + --interactive \ --codebase-path . \ - --non-interactive \ - --command "review_file pngrutil.c" \ + --config "$HOME/metis-security-review/metis-review.yaml" \ --output-file ../results/libpng-live.json +``` + +`--interactive` opens the command prompt instead of running the full configured graph. At the prompt, `review_file` runs the required Initialize prerequisites automatically, then the shipped simple LLM review and finding deduplication. This command omits `--triage`, so Metis does not run automated triage; you will validate a finding manually in the next section. Review the file, then exit after the review completes: + +```console +review_file pngrutil.c +exit +``` + +Back in your shell, return to the exercise workspace: + +```bash cd "$HOME/metis-security-review" ``` -This writes `results/libpng-live.json`. +This writes `results/libpng-live.json` and `results/libpng-live.sarif`. Use the JSON report in the next section. {{< /tab >}} {{< tab header="Use captured report" >}} -This option creates `results/libpng-review-candidate.json` from a captured Metis report. Use it when you want the next triage section to match the source region and explanation shown in this Learning Path. +This option creates `results/libpng-review-candidate.json` from the Metis 2.0.0 review captured with `gpt-6.1-sol` on November 2, 2026. The full run returned three findings; this example selects the row-buffer size finding for the guided triage. The finding is unchanged from the live report; only the review group's `file_path` is made relative for portability, and `_provenance` records the capture details. ```bash cat > results/libpng-review-candidate.json <<'EOF' { "_provenance": { - "artifact_type": "Captured Metis report finding", - "release_under_test": "metis-v1.5.0", - "capture_note": "Use this artifact to study report structure and triage; do not treat it as guaranteed output for every run." + "artifact_type": "Single finding extracted from a Metis review", + "release_under_test": "metis-v2.0.0", + "metis_commit": "742b20d685b0aeb3216b36ddfc44bc5e05ace82c", + "capture_date": "2026-11-02", + "configured_model": "gpt-6.1-sol", + "reported_model": "gpt-6.1-sol", + "magma_commit": "75d1ae7b180443a778b8830c79176ca5f93642ac", + "libpng_commit": "a37d4836519517bdce6cb9d956092321eca3e73b", + "prepared_source_sha256": "f628738930426bea4dccd718e5d90daaf012d5cef365880fa0f6adc4f936a78d", + "command": "review_file pngrutil.c", + "automatic_triage": false, + "capture_note": "One of three findings from the captured run; only the review group file_path is made relative for portability." }, "reviews": [ { "reviews": [ { - "issue": "Externally controlled image dimensions can trigger division by zero in IDAT chunk length validation", - "code_snippet": "size_t row_factor_l =\n (size_t)png_ptr->width\n * (size_t)png_ptr->channels\n * (png_ptr->bit_depth > 8? 2: 1)\n + 1\n + (png_ptr->interlaced? 6: 0);\n\n\n size_t row_factor = (png_uint_32)row_factor_l;\n if (png_ptr->height > PNG_UINT_32_MAX/row_factor)", - "start_line": 3163, - "end_line": 3172, - "reasoning": "For IDAT chunks, row_factor_l is derived from PNG IHDR-controlled fields stored in png_ptr (width, channels/color type, bit depth, and interlace). On platforms where size_t is wider than png_uint_32, row_factor_l can exceed 32 bits for very large but 31-bit-valid PNG dimensions. It is then truncated by the cast to png_uint_32 and stored back in size_t. Certain valid arithmetic combinations can truncate to 0 (for example, an RGB 8-bit non-interlaced row factor of width*3+1 can equal 2^32). The next line divides by row_factor without checking for zero, so a crafted PNG can crash the decoder during chunk header validation, causing denial of service.", - "mitigation": "Do not truncate row_factor_l before using it as a divisor. Keep the computation in png_alloc_size_t/size_t, explicitly check row_factor_l != 0 and row_factor_l <= PNG_UINT_32_MAX where required, or clamp oversized row factors to a safe maximum before division. Prefer overflow-checked helpers for all dimension-derived arithmetic.", - "confidence": 0.86, - "cwe": "CWE-369", - "severity": "Medium", + "id": "6867943ebb524f57a0132a08a96d3115", + "issue": "Unchecked row-size arithmetic can cause a heap buffer overflow on 32-bit builds", + "code_snippet": "row_bytes = PNG_ROWBYTES(max_pixel_depth, row_bytes) +\n 1 + ((max_pixel_depth + 7) >> 3U);", + "start_line": 4591, + "end_line": 4592, + "reasoning": "PNG_ROWBYTES multiplies using size_t without checking overflow. On a build with 32-bit size_t, an 8-bit RGBA image of width 536870912, with png_set_expand_16 enabled, produces max_pixel_depth 64. The multiplication by eight wraps to zero, leaving row_bytes equal to nine. The subsequent row allocations are consequently only 57 bytes each, but line 4648 clears png_ptr->rowbytes + 1 bytes, where the original RGBA rowbytes is 2147483648, causing a massive heap overwrite before image decompression. This requires the application to raise the default width limit sufficiently, or a build configured with a larger width limit. The inspected png_check_IHDR architecture-size check in png.c has an empty body, so it does not reject this image; the later rowbytes check also accepts the original row size. The default width limit prevents this example, and 64-bit size_t does not exhibit this particular overflow.", + "mitigation": "Check the aligned width against the maximum representable allocation size divided by the maximum pixel byte size before multiplication, reserving all padding and alignment overhead. Restore effective architecture-size rejection in png_check_IHDR and retain checked arithmetic at this allocation site.", + "confidence": 0.96, + "cwe": "CWE-190", + "severity": "High", "anchor": { "file_path": "pngrutil.c", - "start_line": 3163, - "end_line": 3172, + "start_line": 4591, + "end_line": 4592, "start_col": 0, - "end_col": 55, - "start_byte": 97432, - "end_byte": 97734, - "symbol": "pngrutil.c::png_check_chunk_length", + "end_col": 41, + "start_byte": 144001, + "end_byte": 144100, + "symbol": "pngrutil.c::png_read_start_row", "kind": "range", - "content_hash": "d4fcc275896c8895", + "content_hash": "21587413f66cc92e", "confidence": "exact" }, - "line_number": 3163 + "line_number": 4591 } ], "file": "pngrutil.c", @@ -141,7 +163,7 @@ cat > results/libpng-review-candidate.json <<'EOF' EOF ``` -The captured report demonstrates the Metis report structure and provides a stable triage exercise. It is not expected output for every run. +This is a captured finding from the same release used in the exercises, not guaranteed output for every run. Its reasoning depends on the reviewed application using a sufficiently narrow `size_t` and image-width limits that permit very large inputs. The next section checks those conditions against the source; this capture is not a native exploit reproduction. {{< /tab >}} {{< /tabpane-normal >}} diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md index d3f5bdd106..446202364c 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md @@ -7,7 +7,9 @@ layout: "learningpathall" ## List the reported findings -Metis reports findings, reasoning, source locations, and mitigations. This page adds a manual triage workflow: you compare a finding with source code, attacker control, build assumptions, and the Magma diff before treating it as confirmed. +Both report options provide Metis 2.0.0 review findings without automated triage verdicts. This page provides the manual validation step: compare the selected finding with source code, attacker control, build and runtime configuration, and the Magma diff before deciding its status. + +Optional automated Triage accepts Metis review JSON and SARIF. In 2.0.0, decisive verdicts require citations to current, in-codebase source that was displayed to the model. Those checks validate citation provenance, not the truth of the model's reasoning, so they do not replace the source and runtime checks below. Select the report you want to summarize: @@ -20,7 +22,7 @@ REPORT=results/libpng-live.json ``` {{< /tab >}} {{< tab header="Captured report" >}} -Use this option for the guided row-factor triage, or if your live review returned no findings: +Use this option for the guided row-buffer triage, or if your live review returned no findings: ```bash REPORT=results/libpng-review-candidate.json @@ -28,7 +30,7 @@ REPORT=results/libpng-review-candidate.json {{< /tab >}} {{< /tabpane-normal >}} -Print a compact summary: +The JSON reports store findings inside `reviews` groups. The summary below uses the finding's source `anchor` when present, then falls back to its primary file or review-group file and `line_number`. Print a compact summary: ```bash python - "$REPORT" <<'PY' @@ -58,47 +60,90 @@ PY If you selected the captured report, the expected output is: ```output -Medium CWE-369: Externally controlled image dimensions can trigger division by zero in IDAT chunk length validation (pngrutil.c:3163) + High CWE-190: Unchecked row-size arithmetic can cause a heap buffer overflow on 32-bit builds (pngrutil.c:4591) ``` -The pre-provided report contains one medium-severity finding in `pngrutil.c`: a possible division by zero after a row-size value is narrowed. If the summary from your live report includes a similar row-factor or division-by-zero finding in `pngrutil.c`, you can continue with the live report. Otherwise, set `REPORT=results/libpng-review-candidate.json` and use the captured report for the guided triage. +The captured example selects one high-severity finding from the three-finding `gpt-6.1-sol` run: possible row-buffer size overflow in `png_read_start_row`, anchored at `pngrutil.c:4591–4592`. Its reasoning connects that calculation to the allocations and clearing operation later in the function. If your live report includes the same issue, you can continue with it. Otherwise, set `REPORT=results/libpng-review-candidate.json` and use the captured example. The severity and confidence are model assessments, not runtime confirmation. + +## Triage the row-buffer finding + +This finding depends on the target application's ABI, image limits, and selected transforms. Running Metis on an Arm64 machine does not establish whether a reviewed application uses 32-bit or 64-bit `size_t`. + +First, inspect the reported allocation and clearing operations, the row-size macro, and the original row-size calculation: + +```bash +sed -n '4584,4648p' libpng-review/pngrutil.c +sed -n '738,741p' libpng-review/pngpriv.h +sed -n '888,904p' libpng-review/pngrutil.c +``` -## Triage the row-factor finding +Check the expansion API named in the finding and the order of row initialization: -DVCP taught the mechanics of running Metis and reading a report. The prepared libpng tree is the more realistic example: you need to inspect the reported source region, check whether values parsed from the PNG file can influence the operation (attacker-controlled values), and compare the region with the prepared-source diff. +```bash +sed -n '941,950p' libpng-review/pngrtran.c +sed -n '4484,4500p' libpng-review/pngrutil.c +sed -n '267,279p' libpng-review/pngread.c +``` -For this finding, you are checking four things: +Trace these relationships in the source: -1. Does the reported source location contain the operation Metis described? -2. Can PNG-controlled fields influence the value? -3. Can the narrowing conversion produce a zero divisor? -4. Does the source region overlap a known vulnerability introduced by the Magma preparation? +1. PNG width and pixel depth influence the row size. An RGBA8 input has 32 bits per pixel; enabling `png_set_expand_16()` makes the maximum transformed depth 64 bits. `PNG_ROWBYTES` multiplies in `size_t`. +2. `png_read_start_row` rounds the width up to an eight-pixel boundary, calculates `row_bytes`, then adds padding before allocation. +3. The later `memset` uses the original `png_ptr->rowbytes + 1`, not the allocation's `row_bytes`. `png_read_update_info()` initializes the row buffers before updating the transformed image information. Check whether the allocation is large enough for that write. +4. On a 32-bit `size_t` build, a sufficiently large width can wrap the aligned row calculation. The same arithmetic does not wrap at that width with a 64-bit `size_t`. -First, display the reported source region: +Next, inspect the earlier architecture-width guard and the prepared-source diff: ```bash -sed -n '3155,3180p' libpng-review/pngrutil.c +sed -n '2561,2595p' libpng-review/png.c +git -C libpng-review diff -U8 -- png.c ``` -Next, inspect the prepared-source diff around the same expression: +The architecture-width test is still present, but its body no longer warns or sets `error = 1`. The diff shows that Magma preparation removed those statements. This connects the finding to the injected vulnerability: the missing defense is in `png.c`, even though the reported allocation is in `pngrutil.c`. + +Now check the default limits and how an application can change them: ```bash -git -C libpng-review diff -U8 -- pngrutil.c | sed -n '/row_factor_l/,+24p' +grep -n 'PNG_USER_WIDTH_MAX' libpng-review/scripts/pnglibconf.h.prebuilt +sed -n '654,668p' libpng-review/libpng-manual.txt ``` -Use the report, source region, and diff to decide whether this is a credible finding: +The default width limit is `1000000`. It rejects the very large input used below during IHDR validation, before row initialization. An application can raise the limit with `png_set_user_limits()` before `png_read_info()`; build-time settings can also change it. Check the deployed ABI, active limits, pixel format, and transforms rather than assuming every PNG reader is affected. -1. `png_ptr->width`, channel count, bit depth, and interlace state originate from the parsed PNG. -2. The arithmetic is performed in `size_t`, then narrowed to `png_uint_32`. -3. A sufficiently large row factor can wrap to zero during the narrowing conversion. -4. The narrowed value is used as a divisor before a zero check. -5. The changed lines overlap a Magma-introduced vulnerability in the prepared-source diff. +Also inspect `PNG_MAX_MALLOC_64K`: if enabled, its later `png_ptr->rowbytes` check rejects this example before `memset`. The arithmetic alone does not account for every build-time defense. + +To isolate the arithmetic in the captured finding, model a 32-bit `size_t` with an RGBA8 input and `png_set_expand_16()` enabled. The original row size uses 32 bits per pixel, while the allocation calculation uses a maximum transformed depth of 64 bits: + +```bash +python - <<'PY' +width = 1 << 29 +input_pixel_depth = 32 +max_pixel_depth = 64 +mask = (1 << 32) - 1 +aligned_width = (width + 7) & ~7 +row_bytes = ( + ((aligned_width * (max_pixel_depth >> 3)) & mask) + + 1 + ((max_pixel_depth + 7) >> 3) +) +allocation = row_bytes + 48 +original_rowbytes = (width * (input_pixel_depth >> 3)) & mask +print(f"Width: {width}; aligned width: {aligned_width}") +print(f"Allocation bytes: {allocation}; clear bytes: {original_rowbytes + 1}") +PY +``` + +The expected output is: + +```output +Width: 536870912; aligned width: 536870912 +Allocation bytes: 57; clear bytes: 2147483649 +``` -This is a strong match to the known vulnerable change in the prepared libpng target. Metis identified the relevant operation, explained attacker control, and proposed avoiding the narrowing conversion or checking the value before division. The triage decision is therefore **confirmed for this exercise target**. +This calculation models unsigned wraparound; it does not run libpng or prove that a decoder reaches the write. The source supports an **architecture- and configuration-dependent finding**, but this exercise does not include a native 32-bit reproducer. Record it as **unconfirmed at runtime** until you reproduce the path with the intended build and application settings. Neither the model's heap-overflow claim nor its potential code-execution impact is established by the calculation alone. ## Triage additional live findings -Your live run might return no findings, the row-factor finding, or different findings. An empty report does not prove the target is safe. A finding outside the injected changes is not automatically false, but it is not evidence by itself that Metis discovered a new vulnerability. +Your live run might return no findings, the row-buffer finding, or different findings. An empty report does not prove the target is safe. A finding outside the injected changes is not automatically false, but it is not evidence by itself that Metis discovered a new vulnerability. Classify each additional finding as **unconfirmed** until you complete more work: From a3d3d489547f1f91aba6d8de9030f6d90079d905 Mon Sep 17 00:00:00 2001 From: Matt Cossins Date: Wed, 18 Nov 2026 13:04:14 +0000 Subject: [PATCH 3/4] Final pass --- assets/contributors.csv | 4 +- .../1-understand-metis.md | 64 ++++++------------- .../2-install-metis.md | 12 +--- .../3-review-vulnerable-c.md | 9 ++- .../4-prepare-libpng.md | 10 ++- .../5-triage-findings.md | 29 ++++----- .../review-c-code-with-metis/_index.md | 2 - .../{files => }/prepare_magma_libpng.py | 0 8 files changed, 43 insertions(+), 87 deletions(-) rename content/learning-paths/cross-platform/review-c-code-with-metis/{files => }/prepare_magma_libpng.py (100%) diff --git a/assets/contributors.csv b/assets/contributors.csv index d47ed1439a..c5c2f6a6e6 100644 --- a/assets/contributors.csv +++ b/assets/contributors.csv @@ -133,4 +133,6 @@ Steve Jordahl,Rafay,,,, Nii Osae Osae Dade,Mindbeam,StarkOsae,,, Tony Morri,Mindbeam,tonymindbeamai,,, Sayandip Pal,Mindbeam,sayandip-pal,,, -Dave Neary,Ampere,,,, \ No newline at end of file +Dave Neary,Ampere,,,, +Michalis Spyrou,Arm,,,, +Michael Koslov,Arm,,,, diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md index df77c0d298..7bb726bc25 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md @@ -26,69 +26,41 @@ Metis offers two review approaches: - **Simple LLM review (default):** the model examines source code or a patch using language-specific security prompts and project guidance. It returns structured findings without requiring a code graph. This is also the review path for languages without C/C++ reachability support. - **Optional reachability review:** for C/C++, Tree-sitter builds a CodeGraph of functions, calls, and source facts. When enabled, `reachability` reviews selected functions with this evidence and surrounding caller/callee context. Graph evidence does not by itself prove attacker control, build reachability, or exploitability. -In Metis 2.0.0, the packaged Review stage runs `simple_llm_review`, `finding_dedup`, and `result`. Reachability is opt-in, not part of this default review. Initialize still builds the CodeGraph for Triage. The exercises use the shipped graph through `review_code` and `review_file` without enabling reachability. +In Metis 2.0.0, the packaged Review stage runs `simple_llm_review`, `finding_dedup`, and `result`. `review_code` and `review_file` use simple LLM review by default. Metis builds a CodeGraph for triage, but runs reachability review only when you enable it. -## Follow the review workflow +## Follow a review workflow with Metis -A typical Metis workflow looks like this: +A security review with Metis typically follows these steps: -1. Pick the review question, such as "review this parser file" or "review this patch." -2. Choose a review command: `review_code`, `review_dir`, `review_file`, or `review_patch`. -3. Add project-specific security context in `.metis.md` when the source needs extra assumptions. -4. Run Metis and save the result as JSON, SARIF, or both. -5. Optionally run Metis triage to assess the candidates against additional source evidence. -6. Check the remaining findings against the source code, input boundaries, build configuration, and tests. +1. Choose a codebase, directory, file, or patch to review. +2. Add project-specific security context in `.metis.md` when needed. +3. Run a review and save the findings as JSON or SARIF. +4. Optionally use automated triage to check the findings against more source code. +5. Verify the findings against the source code, build configuration, and tests. -Metis supplies the review workflow, source navigation, prompts, validation stages, and output handling. The model provider supplies the reasoning model. Results can vary between models and providers even when the source code is unchanged. +## Assess findings with automated triage -## Choose the review scope +Automated triage checks a reported finding against more source code and marks it `valid`, `invalid`, or `inconclusive`. Metis checks the source lines cited for a `valid` or `invalid` verdict, but the verdict does not prove that the finding is correct or that an exploit works. -Use the narrowest scope that answers your review question: +## How this Learning Path uses Metis -- `review_code` reviews files selected from the codebase and is useful for a broad first pass. -- `review_dir` focuses on one directory. -- `review_file` focuses on one source file while retaining codebase context. -- `review_patch` focuses on changed code and is suited to pull-request or pre-merge review. +You will use two review commands on C code: -This Learning Path demonstrates `review_code` and `review_file` at the prompt started with `--interactive`. Direct review commands run their Initialize prerequisites automatically, so the exercises do not need a separate `init` command. +- In the first exercise, you use `review_code` to review the small C codebase. +- In the second exercise, you use `review_file` to focus on `pngrutil.c` with context from the rest of the libpng codebase. -## Strengthen findings with automated triage - -Review discovers candidate issues; triage assesses existing findings. Metis starts with a source window around the reported location. For C/C++, available CodeGraph context adds the enclosing function, callers, and callees. The model can use navigation tools to inspect missing definitions, macros, or guards, then decide whether the evidence supports the finding (`valid`), refutes it (`invalid`), or cannot settle it (`inconclusive`). Keyword rules do not override these verdicts. - -In the built-in workflow, both `valid` and `invalid` require citations and a resolution chain. Metis checks that cited lines were displayed to the model for that finding, remain inside the codebase, and match current source. Unusable citations produce `inconclusive`. These checks validate citation provenance, not the semantic truth of the model's reasoning; even `valid` is not proof of a working exploit. - -Without `--interactive`, Metis 2.0.0 runs the full configured graph. The packaged graph includes Initialize, Review, and Triage, so it triages findings automatically. In interactive mode, `--triage` optionally chains Triage after `review_code`, `review_dir`, `review_file`, or `review_patch`; the `triage` command can instead assess an existing Metis review JSON or SARIF report. - -Automated triage can consume additional model tokens. These exercises use `--interactive` without `--triage` so you can inspect review-only findings and practice manual triage. See the [Metis 2.0.0 triage flow](https://github.com/arm/metis/blob/metis-v2.0.0/docs/triage-flow.md) for decisions, citation validation, and report annotations. - -## Configure the workflow with graph YAML - -The execution graph describes the workflow, not the source code's call graph. In `metis.yaml`, `metis_engine.execution` selects stages and their nodes. The packaged graph runs `initialize` to prepare context and build the CodeGraph, `review` to run simple LLM review and deduplicate findings, and `triage` to assess Review's SARIF findings. Input bindings pass data and establish dependencies; `depends_on` adds ordering without passing a value. - -The exercises configure only the model provider and inherit this graph. An explicit `execution` section replaces the whole packaged graph rather than merging individual nodes. To customize it or enable reachability, start from the [Metis 2.0.0 execution-graph configuration](https://github.com/arm/metis/blob/metis-v2.0.0/docs/execution-graph.md#default-graph) and retain the required stages, outputs, and dependencies. +The exercises start Metis with `--interactive` and run these commands at its prompt. Each command initializes the target automatically, so you do not need to run `init` first. You will inspect the findings yourself without enabling automated triage. ## Interpret Metis findings -A Metis finding normally includes the issue, source location, reasoning, suggested mitigation, Common Weakness Enumeration (CWE) identifier, severity, and confidence. These fields provide a starting point for you to then investigate the findings. +A Metis finding normally includes the issue, source location, reasoning, suggested mitigation, Common Weakness Enumeration (CWE) identifier, severity, and confidence. {{% notice Note %}} CWE identifiers, such as `CWE-190` for integer overflow, provide standard names for common classes of software weakness. {{% /notice %}} -The main benefit is that Metis can reduce the amount of source navigation you do before a review becomes in-depth. You still decide whether a candidate is reachable, exploitable, already mitigated, or worth fixing. - -For each finding, ask: - -- Can untrusted data reach the reported value or operation? -- Does the build configuration include the reported code? -- Are the arithmetic limits and platform assumptions correct? -- Does an existing check or caller contract invalidate the report? -- Can you reproduce the behavior with a focused test or sanitizer? -- Does the proposed mitigation preserve intended behavior? - -You will apply these questions to the libpng report later in the Learning Path. +You still decide whether a candidate is reachable, exploitable, already mitigated, or worth fixing. ## What you've learned and what's next -You have seen how default simple LLM review and optional reachability review find candidates, how automated triage assesses the evidence, and why human validation is still needed. Next, you will install the tested release and add your provider API key. +You have seen how Metis reports possible issues and why each finding needs a source check. Next, you will install the tested release and add your provider API key. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md index 36324589ce..93055edc11 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md @@ -128,23 +128,17 @@ llm_provider: EOF ``` -The review commands select this file explicitly with `--config`. The captured reviews use `gpt-6.1-sol`; choosing another model can change the findings. Metis reads the API key from `OPENAI_API_KEY`. This provider-only configuration inherits the packaged execution graph; adding an `execution` section replaces that graph as a whole. - -Model IDs and availability depend on the endpoint. Use a provider that offers `gpt-6.1-sol` and supports the OpenAI Responses API; do not assume that a model ID from one provider is available at another provider's URL. - -For custom CA certificates, follow the [HTTPX `SSL_CERT_FILE` guidance](https://www.python-httpx.org/environment_variables/#ssl_cert_file). Keep certificate verification enabled. - -For other providers, follow the [Metis 2.0.0 provider setup instructions](https://github.com/arm/metis/blob/metis-v2.0.0/README.md#2-set-up-llm-provider). +The review commands load this file with `--config` and read the API key from `OPENAI_API_KEY`. Your endpoint must support the OpenAI Responses API and the model named in the file. If you choose a different model from the captured `gpt-6.1-sol` runs, your findings might differ. For other providers, see the [Metis 2.0.0 provider setup instructions](https://github.com/arm/metis/blob/metis-v2.0.0/README.md#2-set-up-llm-provider). ## Confirm that the command starts -Display the command help without starting a model-backed review. This verifies the CLI before you spend tokens on a review: +Display the command help before starting a model-backed review. Metis 2.0.0 runs its full workflow by default, including automated triage. The exercises use `--interactive` to run review-only commands; Metis initializes each target automatically. ```bash metis --help ``` -The help output should include `--codebase-path`, `--config`, `--interactive`, and `--output-file`. Repeat `--output-file` to save more than one format. Metis 2.0.0 runs the configured graph by default; the packaged graph includes Initialize, Review, and Triage. Use `--interactive` to open the prompt for the exercises. Their direct review commands run Initialize prerequisites automatically, and omitting `--triage` keeps the results review-only so you can assess the findings manually afterward. +The help output should include `--codebase-path`, `--config`, `--interactive`, and `--output-file`. Repeat `--output-file` to request more than one format. ## What you've accomplished and what's next diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md index cf768ec8d3..f7745302eb 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md @@ -9,10 +9,11 @@ layout: "learningpathall" The [Damn Vulnerable C Program](https://github.com/hardik05/Damn_Vulnerable_C_Program) contains vulnerabilities intentionally placed and labeled in one C file. Use it first to learn the Metis review process: prepare a small target, run `review_code`, save JSON and SARIF, summarize findings, and compare source locations. -In your terminal, clone the repository and copy `dvcp.c` into a separate review directory. The copy keeps the review target small and avoids sending the rest of the repository to Metis: +In your terminal, clone the repository at the tested commit and copy `dvcp.c` into a separate review directory. The copy keeps the review target small and avoids sending the rest of the repository to Metis: ```bash -git clone --depth 1 https://github.com/hardik05/Damn_Vulnerable_C_Program.git +git clone https://github.com/hardik05/Damn_Vulnerable_C_Program.git +git -C Damn_Vulnerable_C_Program checkout 8dcbd202887f587783952a320a2301de701be529 mkdir -p dvcp-review cp Damn_Vulnerable_C_Program/dvcp.c dvcp-review/ ``` @@ -46,7 +47,7 @@ metis \ --output-file ../results/dvcp.sarif ``` -At the Metis prompt, run the review, then exit after it completes. The command runs its Initialize prerequisites automatically, then performs the packaged simple LLM review and deduplication: +At the Metis prompt, run the review, then exit after it completes. Metis initializes the project automatically, then runs the default simple LLM review and deduplicates the findings: ```console review_code @@ -157,8 +158,6 @@ For each reported finding, verify the source location, attacker control, and ope The same review is also available in `results/dvcp.sarif`. SARIF allows code-hosting services and security tools to associate findings with source locations. Keep JSON when you want the complete Metis reasoning and SARIF when you want to integrate results into an existing code-scanning workflow. -This shows that Metis can report several issues, but it is not guaranteed to find everything. Runs are non-deterministic and also depend on the model used. - ## What you've accomplished and what's next You ran a complete Metis review, exported two report formats, and compared the reported findings with known defects. Next, you will prepare a realistic libpng target without leaving benchmark canaries or fixes in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md index c09711bfa4..ee86e8b562 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md @@ -24,11 +24,11 @@ The result is a deliberately vulnerable example target. Do not use it outside th ## Download the preparation script -Download [the Magma libpng preparation script](https://github.com/ArmDeveloperEcosystem/arm-learning-paths/blob/main/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py) into the exercise workspace: +Download [the Magma libpng preparation script](https://github.com/ArmDeveloperEcosystem/arm-learning-paths/blob/main/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py) into the exercise workspace: ```bash curl -L \ - https://raw.githubusercontent.com/ArmDeveloperEcosystem/arm-learning-paths/main/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py \ + https://raw.githubusercontent.com/ArmDeveloperEcosystem/arm-learning-paths/main/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py \ -o prepare_magma_libpng.py ``` @@ -57,8 +57,6 @@ Output: /Users/you/metis-security-review/libpng-review ## Inspect the prepared source -The preparation script removes Magma benchmark controls from the prepared source tree. This keeps the review target closer to a normal libpng checkout instead of a benchmark harness. - Confirm that the prepared tree has local changes: ```bash @@ -90,7 +88,7 @@ metis \ --output-file ../results/libpng-live.json ``` -`--interactive` opens the command prompt instead of running the full configured graph. At the prompt, `review_file` runs the required Initialize prerequisites automatically, then the shipped simple LLM review and finding deduplication. This command omits `--triage`, so Metis does not run automated triage; you will validate a finding manually in the next section. Review the file, then exit after the review completes: +At the Metis prompt, review the file, then exit. The results are review-only so you can validate a finding manually in the next section: ```console review_file pngrutil.c @@ -103,7 +101,7 @@ Back in your shell, return to the exercise workspace: cd "$HOME/metis-security-review" ``` -This writes `results/libpng-live.json` and `results/libpng-live.sarif`. Use the JSON report in the next section. +Metis writes `results/libpng-live.json` and a companion `results/libpng-live.sarif`. Use the JSON report in the next section. {{< /tab >}} {{< tab header="Use captured report" >}} This option creates `results/libpng-review-candidate.json` from the Metis 2.0.0 review captured with `gpt-6.1-sol` on November 2, 2026. The full run returned three findings; this example selects the row-buffer size finding for the guided triage. The finding is unchanged from the live report; only the review group's `file_path` is made relative for portability, and `_provenance` records the capture details. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md index 446202364c..acbfe09517 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md @@ -7,9 +7,9 @@ layout: "learningpathall" ## List the reported findings -Both report options provide Metis 2.0.0 review findings without automated triage verdicts. This page provides the manual validation step: compare the selected finding with source code, attacker control, build and runtime configuration, and the Magma diff before deciding its status. +Both report options provide Metis 2.0.0 review findings without automated triage verdicts. You will check one finding against the source code, build conditions, and Magma diff before deciding its status. -Optional automated Triage accepts Metis review JSON and SARIF. In 2.0.0, decisive verdicts require citations to current, in-codebase source that was displayed to the model. Those checks validate citation provenance, not the truth of the model's reasoning, so they do not replace the source and runtime checks below. +For optional automated triage of Metis JSON or SARIF reports, see the [Metis 2.0.0 triage flow](https://github.com/arm/metis/blob/metis-v2.0.0/docs/triage-flow.md). Select the report you want to summarize: @@ -63,7 +63,7 @@ If you selected the captured report, the expected output is: High CWE-190: Unchecked row-size arithmetic can cause a heap buffer overflow on 32-bit builds (pngrutil.c:4591) ``` -The captured example selects one high-severity finding from the three-finding `gpt-6.1-sol` run: possible row-buffer size overflow in `png_read_start_row`, anchored at `pngrutil.c:4591–4592`. Its reasoning connects that calculation to the allocations and clearing operation later in the function. If your live report includes the same issue, you can continue with it. Otherwise, set `REPORT=results/libpng-review-candidate.json` and use the captured example. The severity and confidence are model assessments, not runtime confirmation. +If your live report includes the row-buffer finding, use it below. Otherwise, set `REPORT=results/libpng-review-candidate.json` to follow the captured example. The severity and confidence are model assessments, not runtime confirmation. ## Triage the row-buffer finding @@ -141,27 +141,20 @@ Allocation bytes: 57; clear bytes: 2147483649 This calculation models unsigned wraparound; it does not run libpng or prove that a decoder reaches the write. The source supports an **architecture- and configuration-dependent finding**, but this exercise does not include a native 32-bit reproducer. Record it as **unconfirmed at runtime** until you reproduce the path with the intended build and application settings. Neither the model's heap-overflow claim nor its potential code-execution impact is established by the calculation alone. -## Triage additional live findings +## Check and record other findings Your live run might return no findings, the row-buffer finding, or different findings. An empty report does not prove the target is safe. A finding outside the injected changes is not automatically false, but it is not evidence by itself that Metis discovered a new vulnerability. -Classify each additional finding as **unconfirmed** until you complete more work: +For each additional finding, check: -- Check the relevant file-format specification and API contracts. -- Trace callers and helper functions to find earlier validation or cleanup behavior. -- Check whether build configuration makes the code reachable. -- Create a focused reproducer and observe behavior under AddressSanitizer and UndefinedBehaviorSanitizer. -- Search upstream issues and security advisories before reporting a new vulnerability. +- Whether untrusted input reaches the reported operation, including through callers or helper functions. +- Whether the build includes that code and the arithmetic and platform assumptions hold. +- Whether an existing guard, API contract, or file-format rule invalidates the finding. +- Whether a focused test or sanitizer reproduces the behavior and the proposed fix preserves intended behavior. -## Record a triage decision +Search upstream issues and security advisories before reporting a new vulnerability. Treat the finding as **unconfirmed** until you have enough evidence to decide otherwise. -For each finding, record at least: - -- Status: confirmed, rejected, duplicate, accepted risk, or unconfirmed. -- Attacker-controlled input and the path to the operation. -- Relevant build and platform assumptions. -- Reproduction evidence or the missing evidence needed next. -- Proposed fix owner and test coverage. +Record a status of **confirmed**, **rejected**, **duplicate**, **accepted risk**, or **unconfirmed**. Include the input path, build assumptions, reproduction evidence or remaining checks, proposed fix owner, and test coverage. Do not publish unverified findings against active open-source projects. Follow the project's security policy and coordinated-disclosure process if further investigation confirms an issue. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md index bfdbd80814..0e66b5961c 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md @@ -33,7 +33,6 @@ subjects: Security armips: - Neoverse - Cortex-A - - Cortex-M operatingsystems: - Linux - macOS @@ -48,7 +47,6 @@ tools_software_languages: shared_path: true shared_between: - - embedded-and-microcontrollers - laptops-and-desktops - servers-and-cloud-computing diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py b/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py similarity index 100% rename from content/learning-paths/cross-platform/review-c-code-with-metis/files/prepare_magma_libpng.py rename to content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py From ae402ca2f726436bb5091cbd319e549f994c0031 Mon Sep 17 00:00:00 2001 From: pareenaverma Date: Wed, 18 Nov 2026 11:00:08 -0500 Subject: [PATCH 4/4] Update _index.md --- .../cross-platform/review-c-code-with-metis/_index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md index 0e66b5961c..60bf42d175 100644 --- a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md @@ -1,6 +1,10 @@ --- title: Review C code for security vulnerabilities with Arm's Metis (AI-powered Security Code Review) +draft: true +cascade: + draft: true + description: Install Arm Metis, review vulnerable C code, and triage findings from a prepared libpng target. minutes_to_complete: 45