diff --git a/assets/contributors.csv b/assets/contributors.csv index 0f33a5f390..bfffdc9881 100644 --- a/assets/contributors.csv +++ b/assets/contributors.csv @@ -186,6 +186,8 @@ Karthik Gopalakrishnan,Tinkerblox,,,, David Torres,ThirdAI,,,, Sammy Sahnine,,,,, Matthew Sloyan,Arm,matthewsloyanARM,,, -Austin Hill,,,,, -Andreas Mullen,Arm,,,, -Liam Randall,Cosmonic,,hectaman,,https://cosmonic.com/ +Austin Hill,,,,, +Andreas Mullen,Arm,,,, +Michalis Spyrou,Arm,,,, +Michael Koslov,Arm,,,, +Liam Randall,Cosmonic,,hectaman,,https://cosmonic.com/ diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md new file mode 100644 index 0000000000..7bb726bc25 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/1-understand-metis.md @@ -0,0 +1,66 @@ +--- +title: Understand the Metis review workflow +description: Learn how Metis gathers evidence, runs security reviews, and presents findings for human triage. +weight: 2 +layout: "learningpathall" +--- + +## What is Metis? + +Arm Metis is an open-source, agentic security code-review framework. It combines language-aware source tooling with a large language model (LLM) to investigate security-relevant behavior and return structured review candidates. Metis supports multiple programming and hardware description languages through language plugins. This Learning Path uses C examples, but the same concepts can be applied to the other supported languages. + +You first run Metis on a small C program with known vulnerabilities. This gives you a quick look at the command flow, JSON output, SARIF output, and reported findings. You then prepare a libpng source tree from the [Magma](https://hexhive.epfl.ch/magma/) benchmark and triage a captured Metis finding against the source diff. + +The exercises show how to use Metis during a security review. They do not benchmark Metis accuracy. Model choice, provider behavior, Metis version, and source changes can all affect a run. Use published benchmark results for comparative performance data. + +{{% notice Note %}} +Model-backed review can consume substantial API tokens. The initial exercise is less expensive than the optional live libpng run. A pre-captured libpng report candidate is included so you can complete the triage exercise without running the larger review. +{{% /notice %}} + +## How Metis reviews code + +A Metis review starts with source files and project context. Metis collects language-aware evidence, asks your configured model to investigate possible vulnerabilities, and returns structured findings for a human reviewer. + +Metis offers two review approaches: + +- **Simple LLM review (default):** the model examines source code or a patch using language-specific security prompts and project guidance. It returns structured findings without requiring a code graph. This is also the review path for languages without C/C++ reachability support. +- **Optional reachability review:** for C/C++, Tree-sitter builds a CodeGraph of functions, calls, and source facts. When enabled, `reachability` reviews selected functions with this evidence and surrounding caller/callee context. Graph evidence does not by itself prove attacker control, build reachability, or exploitability. + +In Metis 2.0.0, the packaged Review stage runs `simple_llm_review`, `finding_dedup`, and `result`. `review_code` and `review_file` use simple LLM review by default. Metis builds a CodeGraph for triage, but runs reachability review only when you enable it. + +## Follow a review workflow with Metis + +A security review with Metis typically follows these steps: + +1. Choose a codebase, directory, file, or patch to review. +2. Add project-specific security context in `.metis.md` when needed. +3. Run a review and save the findings as JSON or SARIF. +4. Optionally use automated triage to check the findings against more source code. +5. Verify the findings against the source code, build configuration, and tests. + +## Assess findings with automated triage + +Automated triage checks a reported finding against more source code and marks it `valid`, `invalid`, or `inconclusive`. Metis checks the source lines cited for a `valid` or `invalid` verdict, but the verdict does not prove that the finding is correct or that an exploit works. + +## How this Learning Path uses Metis + +You will use two review commands on C code: + +- In the first exercise, you use `review_code` to review the small C codebase. +- In the second exercise, you use `review_file` to focus on `pngrutil.c` with context from the rest of the libpng codebase. + +The exercises start Metis with `--interactive` and run these commands at its prompt. Each command initializes the target automatically, so you do not need to run `init` first. You will inspect the findings yourself without enabling automated triage. + +## Interpret Metis findings + +A Metis finding normally includes the issue, source location, reasoning, suggested mitigation, Common Weakness Enumeration (CWE) identifier, severity, and confidence. + +{{% notice Note %}} +CWE identifiers, such as `CWE-190` for integer overflow, provide standard names for common classes of software weakness. +{{% /notice %}} + +You still decide whether a candidate is reachable, exploitable, already mitigated, or worth fixing. + +## What you've learned and what's next + +You have seen how Metis reports possible issues and why each finding needs a source check. Next, you will install the tested release and add your provider API key. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md new file mode 100644 index 0000000000..93055edc11 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/2-install-metis.md @@ -0,0 +1,145 @@ +--- +title: Install and configure Metis +description: Install a tested Metis release and configure a model provider with an API key. +weight: 3 +layout: "learningpathall" +--- + +## Install system tools + +The commands in this Learning Path use a Unix-like shell. Use Terminal on macOS, a Linux terminal, or an existing Ubuntu/Debian terminal running under WSL2 on Windows. Native Windows PowerShell and Command Prompt are not covered. + +Select your platform and install the compiler, Git, and command-line utilities used to fetch code, apply patches, and inspect reports: + +{{< tabpane code=true >}} + {{< tab header="macOS" language="bash">}} +xcode-select --install + {{< /tab >}} + {{< tab header="Ubuntu Linux or WSL2" language="bash">}} +sudo apt-get update +sudo apt-get install -y build-essential ca-certificates curl git grep patch sed + {{< /tab >}} +{{< /tabpane >}} + +If the Xcode Command Line Tools are already installed, macOS reports that no installation is needed. On WSL2, use the Linux filesystem for this exercise and avoid creating the workspace under `/mnt/c`. + +## Install uv and Python + +Metis requires Python 3.12 or later. This Learning Path uses `uv` to install Python 3.12 and create an isolated Metis environment: + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Load the shell environment created by the installer and confirm that `uv` is available: + +```bash +source "$HOME/.local/bin/env" +uv --version +``` + +Install Python 3.12 using `uv`: + +```bash +uv python install 3.12 +uv python find 3.12 +``` + +The second command prints the path to the managed Python 3.12 executable. + +Check that all command-line tools used by the exercises are available: + +```bash +for tool in curl git grep patch sed uv; do + command -v "$tool" >/dev/null || echo "Missing: $tool" +done +``` + +The command prints nothing when all required tools are available. Install any tools that are missing. + +## Create the exercise workspace + +Create one workspace in your home directory for Metis, the review targets, and generated reports. On WSL2, `$HOME` is inside the Linux filesystem: + +```bash +mkdir -p "$HOME/metis-security-review/results" +cd "$HOME/metis-security-review" +``` + +## Clone and install Metis + +Clone the Metis repository at the tested [2.0.0 release](https://github.com/arm/metis/releases/tag/metis-v2.0.0): + +```bash +git clone --branch metis-v2.0.0 --depth 1 \ + https://github.com/arm/metis.git \ + "$HOME/metis-security-review/metis" + +cd "$HOME/metis-security-review/metis" + +uv sync --no-dev --python 3.12 +source "$HOME/metis-security-review/metis/.venv/bin/activate" + +command -v metis +metis --version +git rev-parse HEAD + +cd "$HOME/metis-security-review" +``` + +`uv sync --no-dev --python 3.12` installs only the runtime dependencies needed to run the Metis CLI, using Python 3.12. It creates `metis/.venv` and installs Metis there. Activating the environment makes the `metis` command and the matching Python interpreter available in the current terminal. + +The expected output from `metis --version` for this tag is: + +```output +Metis 2.0.0 +``` + +The `metis-v2.0.0` tag points to commit [`742b20d685b0aeb3216b36ddfc44bc5e05ace82c`](https://github.com/arm/metis/commit/742b20d685b0aeb3216b36ddfc44bc5e05ace82c). The output of `git rev-parse HEAD` should match this commit. Record the version and commit when comparing runs. + +If you open a new terminal, return to the exercise workspace and reactivate the Metis environment before continuing: + +```bash +cd "$HOME/metis-security-review" +source "$HOME/metis-security-review/metis/.venv/bin/activate" +command -v metis +``` + +The path printed by `command -v` should end with `metis-security-review/metis/.venv/bin/metis`. If it points elsewhere, open a new terminal or run `deactivate`, then run the activation command again. + +## Configure the model provider + +Metis supports several hosted and local model providers. These examples use an OpenAI-compatible endpoint that serves the model ID `gpt-6.1-sol`. Export your API key in the same terminal that will run Metis: + +```bash +export OPENAI_API_KEY="your-api-key" +``` + +Do not add secrets to `.metis.md`, `metis-review.yaml`, shell history, source control, or report files. + +Create a provider configuration outside the review targets. Replace `https://your-provider.example/v1` below with the trusted base URL of the provider that serves `gpt-6.1-sol`, including any required API prefix. The example address is a placeholder, not a service endpoint: + +```bash +cat > "$HOME/metis-security-review/metis-review.yaml" <<'EOF' +llm_provider: + name: openai + model: gpt-6.1-sol + base_url: https://your-provider.example/v1 +EOF +``` + +The review commands load this file with `--config` and read the API key from `OPENAI_API_KEY`. Your endpoint must support the OpenAI Responses API and the model named in the file. If you choose a different model from the captured `gpt-6.1-sol` runs, your findings might differ. For other providers, see the [Metis 2.0.0 provider setup instructions](https://github.com/arm/metis/blob/metis-v2.0.0/README.md#2-set-up-llm-provider). + +## Confirm that the command starts + +Display the command help before starting a model-backed review. Metis 2.0.0 runs its full workflow by default, including automated triage. The exercises use `--interactive` to run review-only commands; Metis initializes each target automatically. + +```bash +metis --help +``` + +The help output should include `--codebase-path`, `--config`, `--interactive`, and `--output-file`. Repeat `--output-file` to request more than one format. + +## What you've accomplished and what's next + +You installed the tested Metis 2.0.0 release and set it up with your model provider. Next, you will review a small C program and compare the report with defects that are intentionally present in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md new file mode 100644 index 0000000000..f7745302eb --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/3-review-vulnerable-c.md @@ -0,0 +1,163 @@ +--- +title: Compare a Metis report with vulnerable C code +description: Run a Metis review, inspect JSON and SARIF output, and compare the report with known defects in a small C program. +weight: 4 +layout: "learningpathall" +--- + +## Prepare the example program + +The [Damn Vulnerable C Program](https://github.com/hardik05/Damn_Vulnerable_C_Program) contains vulnerabilities intentionally placed and labeled in one C file. Use it first to learn the Metis review process: prepare a small target, run `review_code`, save JSON and SARIF, summarize findings, and compare source locations. + +In your terminal, clone the repository at the tested commit and copy `dvcp.c` into a separate review directory. The copy keeps the review target small and avoids sending the rest of the repository to Metis: + +```bash +git clone https://github.com/hardik05/Damn_Vulnerable_C_Program.git +git -C Damn_Vulnerable_C_Program checkout 8dcbd202887f587783952a320a2301de701be529 +mkdir -p dvcp-review +cp Damn_Vulnerable_C_Program/dvcp.c dvcp-review/ +``` + +Add a short threat-model note for the review. Metis reads `.metis.md` as project context: + +```bash +cat > dvcp-review/.metis.md <<'EOF' +# Project security context + +This command-line C program parses an image file supplied by an untrusted user. +Treat file contents, dimensions, offsets, and derived allocation sizes as +attacker-controlled. Review memory safety, integer arithmetic, resource +exhaustion, input validation, and cleanup on error paths. +EOF +``` + +The note tells Metis how to treat inputs and security boundaries. The source file still contains teaching comments, which is acceptable here because this step teaches the review workflow. You will use libpng for a more realistic triage example. + +## Run the review + +Change into the review directory and start the Metis interactive prompt with the provider configuration you created. Save JSON for detailed inspection and SARIF for code-scanning tools: + +```bash +cd "$HOME/metis-security-review/dvcp-review" +metis \ + --interactive \ + --codebase-path . \ + --config "$HOME/metis-security-review/metis-review.yaml" \ + --output-file ../results/dvcp.json \ + --output-file ../results/dvcp.sarif +``` + +At the Metis prompt, run the review, then exit after it completes. Metis initializes the project automatically, then runs the default simple LLM review and deduplicates the findings: + +```console +review_code +exit +``` + +Metis prints review progress and detailed findings under `File:`, with an `Identified issue` heading. Finding fields include `Snippet`, `Line number`, `CWE`, `Severity`, `Why`, `Mitigation`, and `Confidence`. After the findings, the export and token-usage output is similar to: + +```output +Results saved to ../results/dvcp.json +SARIF report saved to ../results/dvcp.sarif +Token usage (review_code) +Current (input: 8,420 · output: 3,349 · total: 11,769) +``` + +Back in your shell, return to the exercise workspace: + +```bash +cd "$HOME/metis-security-review" +``` + +The run sends source-derived evidence to your configured model provider. Runtime, token use, and findings depend on the model, provider, and Metis version. + +It can take a couple of minutes to complete. + +## Summarize the JSON report + +The JSON report stores findings in `reviews[].reviews`. Source locations can appear in an `anchor`, in `primary_file` and `line_number`, or in the enclosing review group's `file` field. Use the Python interpreter from the active Metis environment to count the findings and print their issue types and source locations: + +```bash +python - <<'PY' +import json +from pathlib import Path + +data = json.loads(Path("results/dvcp.json").read_text()) +review_groups = data["reviews"] +count = sum(len(group["reviews"]) for group in review_groups) +print(f"Finding count: {count}") +for review_group in review_groups: + for finding in review_group["reviews"]: + anchor = finding.get("anchor") or {} + file_path = ( + anchor.get("file_path") + or finding.get("primary_file") + or finding.get("file") + or review_group.get("file") + ) + line = anchor.get("start_line") or finding.get("line_number") + print( + f"- {finding['cwe']} {finding['issue']} " + f"({file_path}:{line})" + ) +PY +``` + +This example was captured on November 2, 2026 using the `metis-v2.0.0` tag ([`742b20d`](https://github.com/arm/metis/commit/742b20d685b0aeb3216b36ddfc44bc5e05ace82c)), with `gpt-6.1-sol` through an OpenAI-compatible endpoint and DVCP commit [`8dcbd20`](https://github.com/hardik05/Damn_Vulnerable_C_Program/commit/8dcbd202887f587783952a320a2301de701be529). The provider reported the model ID as `gpt-6.1-sol`. The run used the packaged simple LLM review and deduplication nodes; reachability review and automated triage were not enabled. + +The output is similar to: + +```output +Finding count: 10 +- CWE-125 Unterminated image fields cause out-of-bounds string reads (dvcp.c:48) +- CWE-122 Unchecked dimension-derived allocations permit heap overflows and NULL dereferences (dvcp.c:54) +- CWE-415 An even dimension sum causes a double free (dvcp.c:59) +- CWE-416 An odd dimension sum divisible by three causes a use-after-free write (dvcp.c:64) +- CWE-369 Unvalidated image dimensions cause invalid integer division (dvcp.c:82) +- CWE-787 File-controlled quotient indexes outside a stack buffer (dvcp.c:90) +- CWE-787 The heap buffer is accessed one byte past its allocation (dvcp.c:91) +- CWE-674 Recursive operation has no termination condition (dvcp.c:22) +- CWE-400 Allocation loop intentionally retains memory until exhaustion (dvcp.c:109) +- CWE-401 Large-quotient branch discards an allocated buffer without freeing it (dvcp.c:97) +``` + +Your exact count and wording can differ. A valid report can contain zero findings. An empty report means that this run did not report any findings; it does not mean the program is safe. If you expected findings, check your provider connection and any Metis log output before judging the result. + +## Compare the report with known defects + +Display four source regions that the program identifies as deliberately vulnerable: + +```bash +sed -n '14,28p' dvcp-review/dvcp.c +sed -n '42,69p' dvcp-review/dvcp.c +sed -n '72,87p' dvcp-review/dvcp.c +sed -n '89,115p' dvcp-review/dvcp.c +``` + +Check whether your report covers these concrete cases: + +- Signed image dimensions are combined and passed to `malloc()` without overflow or range checks. +- `buff1` is freed, then freed again when `size1` is even or written after free when it is divisible by three. +- `img.height` is used as a divisor without a zero check. +- `size3` is used as an index into fixed-size and heap buffers without bounds checks. +- Attacker-controlled dimensions can select recursive stack exhaustion or repeated heap allocation. + +The captured Metis 2.0.0 JSON and SARIF reports contain ten findings. Source anchors can span multiple lines, and one finding can discuss additional sites in its reasoning. The table maps those reported regions and explanations to the known source defects: + +| Source issue | Tested-run outcome | +| --- | --- | +| Non-NUL-terminated image fields passed to `%s` | `CWE-125` at `dvcp.c:48` | +| Dimension-derived allocations followed by fixed-size copies | `CWE-122` anchored at `dvcp.c:54–58`; its reasoning also covers `dvcp.c:74–79` and `dvcp.c:82–87` | +| `buff1` double free and use-after-free | `CWE-415` at `dvcp.c:59–63`; `CWE-416` at `dvcp.c:64–69` | +| Integer division using unvalidated image dimensions | `CWE-369` at `dvcp.c:82` | +| Indexed stack and heap accesses | Separate `CWE-787` findings at `dvcp.c:90–94` and `dvcp.c:91–95` | +| `buff4` allocation lost without being freed | `CWE-401` at `dvcp.c:97–100` | +| Unbounded recursion and repeated heap allocation | `CWE-674` at `dvcp.c:22–27`; `CWE-400` at `dvcp.c:109–115` | + +For each reported finding, verify the source location, attacker control, and operation. Also record known source defects that the run omitted. The goal is to practice reviewing Metis output, not to turn the finding count into a coverage score. + +The same review is also available in `results/dvcp.sarif`. SARIF allows code-hosting services and security tools to associate findings with source locations. Keep JSON when you want the complete Metis reasoning and SARIF when you want to integrate results into an existing code-scanning workflow. + +## What you've accomplished and what's next + +You ran a complete Metis review, exported two report formats, and compared the reported findings with known defects. Next, you will prepare a realistic libpng target without leaving benchmark canaries or fixes in its source. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md new file mode 100644 index 0000000000..ee86e8b562 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/4-prepare-libpng.md @@ -0,0 +1,170 @@ +--- +title: Prepare a realistic Magma libpng target +description: Build a reviewable libpng source tree from Magma while removing benchmark canaries and fix branches. +weight: 5 +layout: "learningpathall" +--- + +## Why prepare a Magma libpng target? + +The previous section used a small labeled program to teach the Metis review process. This section prepares a larger C codebase so you can see how a Metis finding is triaged against source that looks closer to real project code. + +[Magma](https://hexhive.epfl.ch/magma/) gives you a repeatable way to prepare that target. Magma is a benchmark for evaluating fuzzers against real programs with documented vulnerabilities. A fuzzer is a testing tool that repeatedly runs a program with generated or mutated inputs to trigger crashes, hangs, memory errors, or other unexpected behavior. + +You use Magma here only to create a prepared libpng source tree for review. You are not using Magma to benchmark Metis. You also do not give Metis the benchmark metadata, bug names, fuzzing harnesses, canaries, or fix branches, because those details would give a review model clues that a normal codebase would not contain. + +The preparation script creates the review target in four steps: + +1. Checks out the libpng revision selected by Magma. +2. Applies Magma patches that introduce known vulnerable code. +3. Removes benchmark controls so the source looks closer to a normal libpng tree. +4. Writes neutral Metis context for the review. + +The result is a deliberately vulnerable example target. Do not use it outside this exercise. + +## Download the preparation script + +Download [the Magma libpng preparation script](https://github.com/ArmDeveloperEcosystem/arm-learning-paths/blob/main/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py) into the exercise workspace: + +```bash +curl -L \ + https://raw.githubusercontent.com/ArmDeveloperEcosystem/arm-learning-paths/main/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py \ + -o prepare_magma_libpng.py +``` + +Clone Magma, then check out the commit tested for this Learning Path: + +```bash +git clone https://github.com/HexHive/magma.git +git -C magma checkout 75d1ae7b180443a778b8830c79176ca5f93642ac +``` + +Run the script to create `libpng-review`: + +```bash +python prepare_magma_libpng.py \ + --magma-dir magma \ + --output-dir libpng-review +``` + +The output is similar to: + +```output +Prepared libpng commit a37d4836519517bdce6cb9d956092321eca3e73b +Applied 7 Magma vulnerability-introducing patches +Output: /Users/you/metis-security-review/libpng-review +``` + +## Inspect the prepared source + +Confirm that the prepared tree has local changes: + +```bash +git -C libpng-review status --short +git -C libpng-review diff --stat +``` + +The diff summary shows that the script applied patches that introduce known vulnerable code to libpng. During triage, you will inspect a `png.c` guard and the related `pngrutil.c` allocation. Do not send patch names, benchmark descriptions, or known vulnerability lists to Metis. + +## Choose a libpng report option + +You need a Metis report before you can triage a finding. Choose one of these options: + +- Run a live Metis review if you have enough API quota and want to compare your model configuration. +- Use the pre-captured report if you are interested to try Metis, but want to minimize your API usage. + +{{< tabpane-normal >}} + {{< tab header="Run live review" >}} +This option asks Metis to review only `pngrutil.c`, because the triage exercise focuses on a finding in that file. + +The captured Metis 2.0.0 review with `gpt-6.1-sol` used 353,579 model tokens (351,293 input and 2,286 output). Usage varies with the run and model configuration. A live run can take several minutes and consume substantial API quota. Use the captured report option to avoid those model calls or if your run does not produce a finding you can triage. + +```bash +cd "$HOME/metis-security-review/libpng-review" +metis \ + --interactive \ + --codebase-path . \ + --config "$HOME/metis-security-review/metis-review.yaml" \ + --output-file ../results/libpng-live.json +``` + +At the Metis prompt, review the file, then exit. The results are review-only so you can validate a finding manually in the next section: + +```console +review_file pngrutil.c +exit +``` + +Back in your shell, return to the exercise workspace: + +```bash +cd "$HOME/metis-security-review" +``` + +Metis writes `results/libpng-live.json` and a companion `results/libpng-live.sarif`. Use the JSON report in the next section. + {{< /tab >}} + {{< tab header="Use captured report" >}} +This option creates `results/libpng-review-candidate.json` from the Metis 2.0.0 review captured with `gpt-6.1-sol` on November 2, 2026. The full run returned three findings; this example selects the row-buffer size finding for the guided triage. The finding is unchanged from the live report; only the review group's `file_path` is made relative for portability, and `_provenance` records the capture details. + +```bash +cat > results/libpng-review-candidate.json <<'EOF' +{ + "_provenance": { + "artifact_type": "Single finding extracted from a Metis review", + "release_under_test": "metis-v2.0.0", + "metis_commit": "742b20d685b0aeb3216b36ddfc44bc5e05ace82c", + "capture_date": "2026-11-02", + "configured_model": "gpt-6.1-sol", + "reported_model": "gpt-6.1-sol", + "magma_commit": "75d1ae7b180443a778b8830c79176ca5f93642ac", + "libpng_commit": "a37d4836519517bdce6cb9d956092321eca3e73b", + "prepared_source_sha256": "f628738930426bea4dccd718e5d90daaf012d5cef365880fa0f6adc4f936a78d", + "command": "review_file pngrutil.c", + "automatic_triage": false, + "capture_note": "One of three findings from the captured run; only the review group file_path is made relative for portability." + }, + "reviews": [ + { + "reviews": [ + { + "id": "6867943ebb524f57a0132a08a96d3115", + "issue": "Unchecked row-size arithmetic can cause a heap buffer overflow on 32-bit builds", + "code_snippet": "row_bytes = PNG_ROWBYTES(max_pixel_depth, row_bytes) +\n 1 + ((max_pixel_depth + 7) >> 3U);", + "start_line": 4591, + "end_line": 4592, + "reasoning": "PNG_ROWBYTES multiplies using size_t without checking overflow. On a build with 32-bit size_t, an 8-bit RGBA image of width 536870912, with png_set_expand_16 enabled, produces max_pixel_depth 64. The multiplication by eight wraps to zero, leaving row_bytes equal to nine. The subsequent row allocations are consequently only 57 bytes each, but line 4648 clears png_ptr->rowbytes + 1 bytes, where the original RGBA rowbytes is 2147483648, causing a massive heap overwrite before image decompression. This requires the application to raise the default width limit sufficiently, or a build configured with a larger width limit. The inspected png_check_IHDR architecture-size check in png.c has an empty body, so it does not reject this image; the later rowbytes check also accepts the original row size. The default width limit prevents this example, and 64-bit size_t does not exhibit this particular overflow.", + "mitigation": "Check the aligned width against the maximum representable allocation size divided by the maximum pixel byte size before multiplication, reserving all padding and alignment overhead. Restore effective architecture-size rejection in png_check_IHDR and retain checked arithmetic at this allocation site.", + "confidence": 0.96, + "cwe": "CWE-190", + "severity": "High", + "anchor": { + "file_path": "pngrutil.c", + "start_line": 4591, + "end_line": 4592, + "start_col": 0, + "end_col": 41, + "start_byte": 144001, + "end_byte": 144100, + "symbol": "pngrutil.c::png_read_start_row", + "kind": "range", + "content_hash": "21587413f66cc92e", + "confidence": "exact" + }, + "line_number": 4591 + } + ], + "file": "pngrutil.c", + "file_path": "pngrutil.c" + } + ] +} +EOF +``` + +This is a captured finding from the same release used in the exercises, not guaranteed output for every run. Its reasoning depends on the reviewed application using a sufficiently narrow `size_t` and image-width limits that permit very large inputs. The next section checks those conditions against the source; this capture is not a native exploit reproduction. + {{< /tab >}} +{{< /tabpane-normal >}} + +## What you've accomplished and what's next + +You prepared production-like vulnerable libpng source without benchmark hints and selected a report for triage. Next, you will check a reported finding against the benchmark diff and decide how to handle additional live findings. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md new file mode 100644 index 0000000000..acbfe09517 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/5-triage-findings.md @@ -0,0 +1,163 @@ +--- +title: Triage a libpng finding +description: Check a Metis report finding against source code, attacker control, and the Magma ground-truth diff. +weight: 6 +layout: "learningpathall" +--- + +## List the reported findings + +Both report options provide Metis 2.0.0 review findings without automated triage verdicts. You will check one finding against the source code, build conditions, and Magma diff before deciding its status. + +For optional automated triage of Metis JSON or SARIF reports, see the [Metis 2.0.0 triage flow](https://github.com/arm/metis/blob/metis-v2.0.0/docs/triage-flow.md). + +Select the report you want to summarize: + +{{< tabpane-normal >}} + {{< tab header="Live report" >}} +Use this option if you ran the live review and want to inspect its results first: + +```bash +REPORT=results/libpng-live.json +``` + {{< /tab >}} + {{< tab header="Captured report" >}} +Use this option for the guided row-buffer triage, or if your live review returned no findings: + +```bash +REPORT=results/libpng-review-candidate.json +``` + {{< /tab >}} +{{< /tabpane-normal >}} + +The JSON reports store findings inside `reviews` groups. The summary below uses the finding's source `anchor` when present, then falls back to its primary file or review-group file and `line_number`. Print a compact summary: + +```bash +python - "$REPORT" <<'PY' +import json +import sys +from pathlib import Path + +data = json.loads(Path(sys.argv[1]).read_text()) +for review_group in data["reviews"]: + for finding in review_group["reviews"]: + anchor = finding.get("anchor") or {} + file_path = ( + anchor.get("file_path") + or finding.get("primary_file") + or finding.get("file") + or review_group.get("file") + ) + line = anchor.get("start_line") or finding.get("line_number") + print( + f"{finding['severity']:>6} {finding['cwe']}: " + f"{finding['issue']} " + f"({file_path}:{line})" + ) +PY +``` + +If you selected the captured report, the expected output is: + +```output + High CWE-190: Unchecked row-size arithmetic can cause a heap buffer overflow on 32-bit builds (pngrutil.c:4591) +``` + +If your live report includes the row-buffer finding, use it below. Otherwise, set `REPORT=results/libpng-review-candidate.json` to follow the captured example. The severity and confidence are model assessments, not runtime confirmation. + +## Triage the row-buffer finding + +This finding depends on the target application's ABI, image limits, and selected transforms. Running Metis on an Arm64 machine does not establish whether a reviewed application uses 32-bit or 64-bit `size_t`. + +First, inspect the reported allocation and clearing operations, the row-size macro, and the original row-size calculation: + +```bash +sed -n '4584,4648p' libpng-review/pngrutil.c +sed -n '738,741p' libpng-review/pngpriv.h +sed -n '888,904p' libpng-review/pngrutil.c +``` + +Check the expansion API named in the finding and the order of row initialization: + +```bash +sed -n '941,950p' libpng-review/pngrtran.c +sed -n '4484,4500p' libpng-review/pngrutil.c +sed -n '267,279p' libpng-review/pngread.c +``` + +Trace these relationships in the source: + +1. PNG width and pixel depth influence the row size. An RGBA8 input has 32 bits per pixel; enabling `png_set_expand_16()` makes the maximum transformed depth 64 bits. `PNG_ROWBYTES` multiplies in `size_t`. +2. `png_read_start_row` rounds the width up to an eight-pixel boundary, calculates `row_bytes`, then adds padding before allocation. +3. The later `memset` uses the original `png_ptr->rowbytes + 1`, not the allocation's `row_bytes`. `png_read_update_info()` initializes the row buffers before updating the transformed image information. Check whether the allocation is large enough for that write. +4. On a 32-bit `size_t` build, a sufficiently large width can wrap the aligned row calculation. The same arithmetic does not wrap at that width with a 64-bit `size_t`. + +Next, inspect the earlier architecture-width guard and the prepared-source diff: + +```bash +sed -n '2561,2595p' libpng-review/png.c +git -C libpng-review diff -U8 -- png.c +``` + +The architecture-width test is still present, but its body no longer warns or sets `error = 1`. The diff shows that Magma preparation removed those statements. This connects the finding to the injected vulnerability: the missing defense is in `png.c`, even though the reported allocation is in `pngrutil.c`. + +Now check the default limits and how an application can change them: + +```bash +grep -n 'PNG_USER_WIDTH_MAX' libpng-review/scripts/pnglibconf.h.prebuilt +sed -n '654,668p' libpng-review/libpng-manual.txt +``` + +The default width limit is `1000000`. It rejects the very large input used below during IHDR validation, before row initialization. An application can raise the limit with `png_set_user_limits()` before `png_read_info()`; build-time settings can also change it. Check the deployed ABI, active limits, pixel format, and transforms rather than assuming every PNG reader is affected. + +Also inspect `PNG_MAX_MALLOC_64K`: if enabled, its later `png_ptr->rowbytes` check rejects this example before `memset`. The arithmetic alone does not account for every build-time defense. + +To isolate the arithmetic in the captured finding, model a 32-bit `size_t` with an RGBA8 input and `png_set_expand_16()` enabled. The original row size uses 32 bits per pixel, while the allocation calculation uses a maximum transformed depth of 64 bits: + +```bash +python - <<'PY' +width = 1 << 29 +input_pixel_depth = 32 +max_pixel_depth = 64 +mask = (1 << 32) - 1 +aligned_width = (width + 7) & ~7 +row_bytes = ( + ((aligned_width * (max_pixel_depth >> 3)) & mask) + + 1 + ((max_pixel_depth + 7) >> 3) +) +allocation = row_bytes + 48 +original_rowbytes = (width * (input_pixel_depth >> 3)) & mask +print(f"Width: {width}; aligned width: {aligned_width}") +print(f"Allocation bytes: {allocation}; clear bytes: {original_rowbytes + 1}") +PY +``` + +The expected output is: + +```output +Width: 536870912; aligned width: 536870912 +Allocation bytes: 57; clear bytes: 2147483649 +``` + +This calculation models unsigned wraparound; it does not run libpng or prove that a decoder reaches the write. The source supports an **architecture- and configuration-dependent finding**, but this exercise does not include a native 32-bit reproducer. Record it as **unconfirmed at runtime** until you reproduce the path with the intended build and application settings. Neither the model's heap-overflow claim nor its potential code-execution impact is established by the calculation alone. + +## Check and record other findings + +Your live run might return no findings, the row-buffer finding, or different findings. An empty report does not prove the target is safe. A finding outside the injected changes is not automatically false, but it is not evidence by itself that Metis discovered a new vulnerability. + +For each additional finding, check: + +- Whether untrusted input reaches the reported operation, including through callers or helper functions. +- Whether the build includes that code and the arithmetic and platform assumptions hold. +- Whether an existing guard, API contract, or file-format rule invalidates the finding. +- Whether a focused test or sanitizer reproduces the behavior and the proposed fix preserves intended behavior. + +Search upstream issues and security advisories before reporting a new vulnerability. Treat the finding as **unconfirmed** until you have enough evidence to decide otherwise. + +Record a status of **confirmed**, **rejected**, **duplicate**, **accepted risk**, or **unconfirmed**. Include the input path, build assumptions, reproduction evidence or remaining checks, proposed fix owner, and test coverage. + +Do not publish unverified findings against active open-source projects. Follow the project's security policy and coordinated-disclosure process if further investigation confirms an issue. + +## What you've accomplished and what's next + +You installed Metis, compared a review with known C defects, prepared a realistic benchmark target, and checked a Metis finding against source evidence. You can apply the same workflow to a bounded directory, file, or patch in your own C project, or for other languages. diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md new file mode 100644 index 0000000000..60bf42d175 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_index.md @@ -0,0 +1,84 @@ +--- +title: Review C code for security vulnerabilities with Arm's Metis (AI-powered Security Code Review) + +draft: true +cascade: + draft: true + +description: Install Arm Metis, review vulnerable C code, and triage findings from a prepared libpng target. + +minutes_to_complete: 45 + +who_is_this_for: This Learning Path is for developers who want to add AI-assisted security review to an existing code-review workflow. The examples in this learning path use C, but Metis is applicable to other languages as well. + +learning_objectives: + - Explain how Metis uses language-aware source analysis and model reasoning + - Install Metis and connect it to a supported model provider + - Review vulnerable C code and inspect JSON and SARIF output + - Triage Metis findings against source code and a Magma-derived libpng diff + +prerequisites: + - A macOS computer, Linux computer, or Windows computer with WSL2 + - An API key and sufficient quota for a [Metis-supported model provider](https://github.com/arm/metis#2-set-up-llm-provider) + - Basic familiarity with C, Git, command-line tools, and security code-review procedures + +author: + - Matt Cossins + - Michalis Spyrou + - Michael Koslov + +generate_summary_faq: false +rerun_summary: false +rerun_faqs: false + +### Tags +skilllevels: Introductory +subjects: Security +armips: + - Neoverse + - Cortex-A +operatingsystems: + - Linux + - macOS + - Windows +tools_software_languages: + - Arm Metis + - C + - Python + - Git + - JSON + - SARIF + +shared_path: true +shared_between: + - laptops-and-desktops + - servers-and-cloud-computing + +further_reading: + - resource: + title: Arm Metis GitHub repository + link: https://github.com/arm/metis + type: website + - resource: + title: Empowering engineers with AI-enabled security code review + link: https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/empowering-engineers-with-ai-enabled-security-code-review + type: blog + - resource: + title: Arm Metis brings agentic AI to software security + link: https://newsroom.arm.com/blog/arm-metis-agentic-ai-security + type: blog + - resource: + title: Magma ground-truth fuzzing benchmark + link: https://hexhive.epfl.ch/magma/ + type: website + - resource: + title: Damn Vulnerable C Program repository + link: https://github.com/hardik05/Damn_Vulnerable_C_Program + type: website + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 +layout: "learningpathall" +learning_path_main_page: "yes" +--- diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md b/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md new file mode 100644 index 0000000000..9196d79577 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/_next-steps.md @@ -0,0 +1,9 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 +title: "Next Steps" +layout: "learningpathall" +--- + diff --git a/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py b/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py new file mode 100644 index 0000000000..22b1b9e152 --- /dev/null +++ b/content/learning-paths/cross-platform/review-c-code-with-metis/prepare_magma_libpng.py @@ -0,0 +1,195 @@ +#!/usr/bin/env python3 + +import argparse +import re +import shutil +import subprocess +from pathlib import Path + + +LIBPNG_REPOSITORY = "https://github.com/pnggroup/libpng.git" +LIBPNG_COMMIT = "a37d4836519517bdce6cb9d956092321eca3e73b" +TARGET_MACROS = { + "ENABLE_MAGMA_FIXES": False, + "MAGMA_ENABLE_CANARIES": False, + "MAGMA_ENABLE_FIXES": False, +} + +IFDEF = re.compile(r"^\s*#\s*(ifdef|ifndef)\s+(\w+)\s*$") +IF = re.compile(r"^\s*#\s*if\s+defined\s*\(\s*(\w+)\s*\)\s*$") +ELSE = re.compile(r"^\s*#\s*else\b") +ELIF = re.compile(r"^\s*#\s*elif\b") +ENDIF = re.compile(r"^\s*#\s*endif\b") +ANY_IF = re.compile(r"^\s*#\s*(if|ifdef|ifndef)\b") + + +def run(*command: str, cwd: Path | None = None, input_text: str | None = None) -> None: + subprocess.run( + command, + cwd=cwd, + input=input_text, + text=True, + check=True, + ) + + +def sanitize_file(path: Path) -> None: + lines = path.read_text(errors="surrogateescape").splitlines(keepends=True) + output: list[str] = [] + stack: list[dict[str, object]] = [] + emitting = True + + for line_number, line in enumerate(lines, start=1): + match = IFDEF.match(line) + if match and match.group(2) in TARGET_MACROS: + directive, macro = match.groups() + condition = TARGET_MACROS[macro] + if directive == "ifndef": + condition = not condition + stack.append( + { + "targeted": True, + "parent_emitting": emitting, + "condition": condition, + "else_seen": False, + } + ) + emitting = emitting and condition + continue + + match = IF.match(line) + if match and match.group(1) in TARGET_MACROS: + condition = TARGET_MACROS[match.group(1)] + stack.append( + { + "targeted": True, + "parent_emitting": emitting, + "condition": condition, + "else_seen": False, + } + ) + emitting = emitting and condition + continue + + if ANY_IF.match(line): + stack.append({"targeted": False}) + if emitting: + output.append(line) + continue + + if ELSE.match(line): + if not stack: + raise RuntimeError(f"unmatched #else in {path}:{line_number}") + frame = stack[-1] + if frame["targeted"]: + if frame["else_seen"]: + raise RuntimeError(f"duplicate #else in {path}:{line_number}") + frame["else_seen"] = True + emitting = bool(frame["parent_emitting"]) and not bool( + frame["condition"] + ) + elif emitting: + output.append(line) + continue + + if ELIF.match(line): + if stack and stack[-1]["targeted"]: + raise RuntimeError( + f"targeted #elif is unsupported in {path}:{line_number}" + ) + if emitting: + output.append(line) + continue + + if ENDIF.match(line): + if not stack: + raise RuntimeError(f"unmatched #endif in {path}:{line_number}") + frame = stack.pop() + if frame["targeted"]: + emitting = bool(frame["parent_emitting"]) + elif emitting: + output.append(line) + continue + + if emitting: + output.append(line) + + if stack: + raise RuntimeError(f"unterminated conditional in {path}") + + path.write_text("".join(output), errors="surrogateescape") + + +def parse_arguments() -> argparse.Namespace: + parser = argparse.ArgumentParser( + description="Prepare the sanitized Magma libpng target for Metis review." + ) + parser.add_argument( + "--magma-dir", + type=Path, + required=True, + help="Path to a cloned HexHive/magma repository.", + ) + parser.add_argument( + "--output-dir", + type=Path, + required=True, + help="Destination for the prepared libpng source tree.", + ) + return parser.parse_args() + + +def main() -> None: + arguments = parse_arguments() + magma_dir = arguments.magma_dir.resolve() + output_dir = arguments.output_dir.resolve() + patch_dir = magma_dir / "targets" / "libpng" / "patches" / "bugs" + + if not patch_dir.is_dir(): + raise SystemExit(f"Magma libpng patches not found: {patch_dir}") + if output_dir.exists(): + raise SystemExit(f"Output directory already exists: {output_dir}") + + run("git", "clone", LIBPNG_REPOSITORY, str(output_dir)) + run("git", "checkout", LIBPNG_COMMIT, cwd=output_dir) + + patch_files = sorted(patch_dir.glob("*.patch")) + for patch_file in patch_files: + patch_text = patch_file.read_text().replace( + "%MAGMA_BUG%", patch_file.stem + ) + run("patch", "-p1", cwd=output_dir, input_text=patch_text) + + for path in output_dir.rglob("*"): + if path.is_file() and path.suffix.lower() in { + ".c", + ".cc", + ".cpp", + ".cxx", + ".h", + ".hpp", + }: + sanitize_file(path) + + for pattern in ("*.orig", "*.rej"): + for path in output_dir.rglob(pattern): + path.unlink() + + (output_dir / ".metis.md").write_text( + "# Project security context\n\n" + "This library parses PNG files supplied by untrusted users. Treat image " + "dimensions, chunk lengths, offsets, and compressed data as " + "attacker-controlled. Review memory safety, integer arithmetic, parser " + "state, resource exhaustion, and cleanup on error paths.\n" + ) + (output_dir / ".metisignore").write_text( + ".git/\ncontrib/\ndocs/\nexamples/\nscripts/\ntests/\n" + ) + + print(f"Prepared libpng commit {LIBPNG_COMMIT}") + print(f"Applied {len(patch_files)} Magma vulnerability-introducing patches") + print(f"Output: {output_dir}") + + +if __name__ == "__main__": + main()