Skip to content

refactor(codegen): expand Python flattened fields from the shared layout - #211

Merged
hardbyte merged 1 commit into
mainfrom
brian/cool-maxwell-8p8zj1
Oct 29, 2026
Merged

hardbyte merged 1 commit into
mainfrom
brian/cool-maxwell-8p8zj1

Conversation

@hardbyte

Copy link
Copy Markdown
Contributor

Follows #204, another step towards #96 and #129. #204 moved OpenAPI and Python's optional-group validator onto a shared flattened layout, and left one walk for later:

collect_flattened_fields, which renders the expanded Python fields, keeps its own walk for now. It renders field types and reports flatten errors (a flattened primitive, an unresolved type) that the layout deliberately treats as Unknown.

This PR moves that walk onto the layout. It was the last flatten walk with its own pass over the raw schema, its own Option unwrapping and its own depth limit.

The layout

codegen/schema/layout.rs now keeps two things Python needs and used to drop:

  • The definition of a flattened enum. InternallyTagged and AdjacentlyTagged carry enm.
  • Why a type's keys are unknown. Layout::Unknown(Opaque) says which:
    • Enum: an externally tagged or untagged enum
    • Primitive: (), which has no keys, or a type such as u32, which can't be flattened
    • Unresolved: the name isn't in the typespace
    • TooDeep: past the depth limit
    • Unsupported: a tuple struct, a transparent struct without exactly one field, or an Option without an argument

OpenAPI only calls keys() and has_required_key(), so its output is unchanged.

Python

  • collect_flattened_fields takes the flattened field, builds its FlattenedGroup, and expands it with one exhaustive match over the layout.
  • The four call sites pass the field itself. The group optionality they each worked out (flattened_group_required, depth 0, field name) now comes from the group.
  • make_flattened_field and collect_flattened_enum_fields are unchanged.

Output

Every generated file is byte-identical to main, except in two cases.

A flattened #[serde(transparent)] struct with a named field now contributes the keys of the struct it wraps, as serde writes them:

#[derive(Serialize, Deserialize, reflectapi::Input, reflectapi::Output)]
#[serde(transparent)]
struct Trace { context: TraceContext }  // TraceContext { trace_id: String }

struct WireRequest { /* ... */ #[serde(flatten)] trace: Trace }
// serde: {"value": ..., "trace_id": "trace-0"}
main         context: TraceContext   (required, so serde's output fails validation: "context: Field required")
this branch  trace_id: str

#204 said derived schemas weren't affected by transparent wrappers. That's true for a tuple wrapper like struct W(Inner), which reflection replaces with the wrapped type. A wrapper with a named field keeps its own definition unless the builder calls fold_transparent_types(). OpenAPI has followed such wrappers since #204, and Python now does too.

Flattening a tuple struct is a codegen error. Main generated a field named 0, which isn't valid Python, so ruff format failed on it. Serde flattens a newtype struct transparently, but the layout can't describe one yet, and following it would change OpenAPI output. That belongs in its own PR.

Both are in the CHANGELOG under Unreleased.

Testing

  • Generated output. I generated the demo client (Python, TypeScript, Rust) and the codegen coverage, enum compatibility and field presence fixtures on main and on this branch, then diffed all 59 files. Without the new fixture they're identical. With it, exactly one line differs: the context: TraceContext → trace_id: str change above.
  • Wire test. The codegen coverage wire fixture flattens a transparent struct, so test_wire.py replays serde's own bytes through the generated client. It fails 12/12 on main and passes 12/12 here.
  • Old behaviour pinned first. I wrote tests against main before changing the walk: the depth limit (ten nested flattens followed, an eleventh replaced by the placeholder), a flattened (), and the errors for a flattened primitive and an unresolved type. They pass unchanged.
  • New unit tests.
    • Layout: each Opaque reason, tagged enums keeping their definition, transparent structs.
    • Python: a flattened transparent struct, and a flattened tuple struct as an error.
  • Snapshots. No snapshot changed. I tried a snapshot test for the transparent case and dropped it, because the snapshot helpers call fold_transparent_types(), so it would pass on main too.
  • Passing locally:
    • cargo fmt --check, clippy -D warnings --all-targets, and cargo test --workspace --all-features: 364 passed, main's 355 plus 9 new
    • every Python codegen smoke step, on 3.14 and on 3.12 with pydantic 2.10.0: no demo client drift, client imports, wire replay, pyright, demo pytest, enum compatibility and field presence
  • Open PRs. Trial merges with fix(python): type omittable collections without None #205 and fix(python): describe the wire shape in JSON schemas of tagged and flattened enums #210, including fix(python): describe the wire shape in JSON schemas of tagged and flattened enums #210's latest commit, are clean, and both compile and pass the codegen and demo snapshot tests.

Not changed

These Python paths still read the raw schema for flatten and are next:

  • materialize_nested_policy_flatten
  • the flattened-enum detection in render_struct_with_flatten
  • variant_flattened_groups
  • the flattened enum lookup in the optional-group rendering

Flattened fields still lose their field descriptions, because the template drops any description starting with (flattened. That predates this change.


Generated by Claude Code

Python's flattened-field expansion was the last flatten walk with its own
traversal of the raw schema, its own Option unwrapping and its own depth
limit. It now reads its structure from codegen/schema/layout.rs, as OpenAPI
and Python's optional-group validator already do since #204.

Layout records what Python needs that it used to drop: the definition of a
flattened enum, and why a type's keys are unknown (an untagged or externally
tagged enum, a primitive, an unresolved name, too deep, or unsupported).
OpenAPI only reads keys() and has_required_key(), so its output is unchanged.

The four call sites now pass the flattened field itself, so group
optionality comes from the shared presence rule instead of each caller.

Generated output is unchanged except in two cases:

- A flattened #[serde(transparent)] struct with a named field contributes the
  keys of the struct it wraps, as serde writes them. Python used to render
  the wrapper's own field, so the model rejected serde's output. Derived
  schemas keep such wrappers unless fold_transparent_types() is called.
- Flattening a tuple struct is a codegen error. It used to generate a field
  named `0`, which isn't valid Python.

Tests pin the depth limit, a flattened unit type and the errors for a
flattened primitive or unresolved type. The codegen coverage wire fixture
flattens a transparent struct, so test_wire.py replays serde's bytes through
the generated client; it fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VjkNsfHSd5osRevK95McpQ
@hardbyte
hardbyte requested a review from antonzherdev October 28, 2026 19:14

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T19:17:08.540040Z af02a4e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

📖 Documentation Preview: https://reflectapi-docs-preview-pr-211.partly.workers.dev

Updated automatically from commit 64d2820

@hardbyte
hardbyte merged commit cb316f9 into main Oct 29, 2026
11 checks passed
@hardbyte
hardbyte deleted the brian/cool-maxwell-8p8zj1 branch October 29, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants