Skip to content

clazel-webp: WebP container and lossless decoding (item 6.8) - #199

Draft
atiernan wants to merge 7 commits into
phase6/6.3-shapefrom
phase6/6.8-webp-lossless
Draft

atiernan wants to merge 7 commits into
phase6/6.3-shapefrom
phase6/6.8-webp-lossless

Conversation

@atiernan

Copy link
Copy Markdown
Member

Item 6.8 of Phase 6: clazel-webp, a WebP decoder on clazel-image-shape's Decoder interface (tier 0, target tier 1; the tier is not raised).

Built

  • RFC 9649: RIFF, VP8X, ANIM and ANMF with every frame composited (disposal, blending, Webp::frame()); ICCP/ALPH/EXIF/XMP /unknown chunks skipped; VP8 recognised and refused as Error::Unsupported (lossy is 6.9).
  • VP8L: predictor (14 modes), colour, subtract green, colour indexing with pixel bundling; colour cache; meta prefix codes; simple and normal code length codes; LZ77 backward references with the distance map. Twelve listed choices in lib.rs.
  • Vendored Wuffs' WebP files (commit b36a81d) with a converter in xtask. libwebp's own vectors have no licence, so Wuffs' lossless files stand in; image-webp 0.2.4 is the reference (new test dependencies: image-webp, byteorder-lite, quick-error, rows added to DEPENDENCIES.md).

Tested (native)

  • 22 unit tests and 15 integration tests pass.
  • All 11 Wuffs lossless files are exact against image-webp. The animation is exact on its first frame and within 1 per channel on blended frames, because image-webp rounds blending differently from libwebp's formula (a listed difference). 2 lossy files are refused.
  • Every file at chunk sizes 1 to 4096 equals the whole-file decode. 1,440 byte mutants never panic.
  • 531 images made by a test-side VP8L encoder decode exactly and agree with image-webp; animation against a test-side compositor; 12 malformed inputs.
  • cargo build, cargo clippy --workspace --all-targets -D warnings, cargo fmt --check and cargo xtask check are clean.

Not done

  • 16 Kani harnesses are written but have not been run. Neither Kani, Miri nor cargo xtask verify was run locally. CI's approved proof jobs have to run them, and bounds may need adjusting. The whole step is not a harness, as for GIF; it is covered in parts (README).
  • Miri strides are sized (about 0.002 s native with the Miri branch forced) but not run.
  • Lossy VP8 and ALPH (6.9), loop count and timing (6.19), more than 1,024 prefix code groups.
  • The tier stays 0 until a full verification is green.

🤖 Generated with Claude Code

atiernan and others added 5 commits November 20, 2026 00:01
…m 6.8)

14 files of commit b36a81d (11 lossless, the animation, 2 small lossy ones for the refusal test); libwebp's own vectors have no licence.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
RFC 9649: RIFF, VP8X, ANIM and ANMF with composited frames; VP8L with every transform, the colour cache, meta prefix codes and backward references; VP8 refused as unsupported. Sixteen Kani harnesses written, not yet run. Adds image-webp, byteorder-lite and quick-error as test dependencies.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ts (item 6.8)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e Miri and changed-crate CI)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@atiernan
atiernan temporarily deployed to full-verification November 20, 2026 01:09 — with GitHub Actions Inactive
…riant gaps (item 6.8)

CI's Kani job for clazel-webp lost its runner after two hours. Run one at a
time locally (Kani 0.68.0, 4 GB cap), the step harnesses with a symbolic
phase each ran out of memory or past 15 minutes, while each phase alone
takes seconds. So:

- `any_vp8l` and `any_webp` take one phase; `act_harness!`,
  `finish_harness!` and `container_harness!` make one harness per phase or
  role (57 harnesses in all, from 16).
- `act_build_code` bounds the alphabet to 8 symbols; `huffman_build_any`
  covers `build` itself over any lengths.
- The pixel, finish and container harnesses use MiniSat: CaDiCaL and kissat
  each ran out of 4 GB on `act_red`, which MiniSat proves in 30 s.
- `predictor_row_any`'s unwind goes from 6 to 13 (a 12-byte comparison).

Running them found three places where any state satisfying `inv` could
break it in one step, though real input cannot:

- `NormalCl`: `inv` now says `cl_i < cl_n` in that phase.
- `Lengths`: a code length symbol over 18 is an error (a built code length
  code has 19 symbols).
- `LenExtra`/`DistExtra`: `inv` now pairs `extra_n` with
  `extra_bits(dprefix)` and bounds `dprefix`; `DistSym` rejects a distance
  code over 39 (a built distance code has 40 symbols).

Locally green: the 11 part harnesses and all 25 header and code phases,
`act_len_extra`, four of five finish roles. Still running locally when
pushed: the other pixel phases, `finish_main` and the container phases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@atiernan
atiernan temporarily deployed to full-verification November 20, 2026 15:42 — with GitHub Actions Inactive
…variant gaps (item 6.8)

All 69 harnesses pass locally, each run on its own under Kani 0.68.0 with a
4 GB cap: 56 minutes in all, the slowest 280 s (`act_dist_sym_main`).

Harnesses:
- The distance phases get one harness per role, and the main image's finish
  one per transform kind (with no transform, or one of each: `invert` is a
  loop whose body depends only on that transform and the pixels).
- `finish_main_predictor` is at exactly 2 by 2 pixels: with the size
  symbolic every loop unrolled along every path and no solver fitted.
- The finish and container harnesses unwind 6, not 40 or 20.
- Where the colour table is written or indexed at a symbolic position, the
  harness gives it 4 colours, the bound on `ncolors`.
- The container harness sets its phase after unwrapping `any_webp`'s
  `Option`, which otherwise lost the constant and explored every phase.

Gaps closed (each unreachable from real input):
- `emit` refuses a pixel once `pos` has reached `total`.
- `state_inv` checks the open chunk and its padding byte with `checked_add`:
  a saturated sum at `u32::MAX` broke after one byte.
- `state_inv`: between chunks (`ChunkHdr`, `Pad`) no chunk is open.
- The end of the RIFF header clears `chunk_left`, `padded` and `in_anmf`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch is waiting to be deployed

1 waiting deployment
full-verification — 1bff51e7 Waiting Nov 20, 2026 by atiernan via Approve the proofs #612
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant