Skip to content

feat(python): generate the EQL type markers from the catalog (#1385) - #1415

Draft
yujiyokoo wants to merge 2 commits into
feat/python-sdk-gate-factsfrom
feat/python-sdk-eql-markers
Draft

yujiyokoo wants to merge 2 commits into
feat/python-sdk-gate-factsfrom
feat/python-sdk-eql-markers

Conversation

@yujiyokoo

Copy link
Copy Markdown
Contributor

Summary

A Python developer will declare each encrypted field with a marker named after its EQL type, for example Annotated[str, TextEq]. EQL (Encrypt Query Language) is the set of PostgreSQL types that store and query encrypted values. This PR adds those markers, generated from the EQL catalog by the same generator that writes the Go package. So Python and Go cannot disagree about a type.

Nothing reads the markers yet. The model decorator that reads them comes next, in #1386.

Changes

  • The generator: eql-codegen has a new py-eql command. It writes languages/python/python/stackencrypt/_eql.py from the same catalog rows as the Go encrypt/eql package.
  • The generated module: one marker class per EQL type, 51 today. Each class has the ten catalog attributes the spec lists, such as sql_domain (public.eql_v4_text_eq) and query_sql_domain. A TYPES tuple lists them all.
  • The hand-written markers: _markers.py has the Marker base class, Passthrough (stored unencrypted), and Ignore (not a column). Marker declares the ten attributes with types, for mypy and pyright.
  • The package exports: from stackencrypt import TextEq works for the 50 types the engine produces today. The one it cannot produce, Json, gives ImportError.
  • The wiring: mise run types:generate runs py-eql, and types:check diffs _eql.py. A failed types:generate run restores _eql.py and the Go file, as it already did for the bindings.
  • Shared wording: the Go and Python emitters take their doc text from Row::summary and Row::status in targets.rs. The Go output is unchanged.
  • The output-root override: go-eql and py-eql write outside packages/eql/, so they now refuse EQL_CODEGEN_OUT_ROOT instead of writing outside the throwaway tree.
  • Docs: the eql-bindings README and AGENTS.md say that _eql.py is generated.

Verification

  • cargo test -p eql-codegen: all pass. The new py_eql_parity.rs checks the committed file against the catalog and every marker against its Go Types row. I planted an error for each check, and each one failed with a clear message.
  • cargo fmt --check and cargo clippy -D warnings for eql-codegen: both pass.
  • mise run test in languages/python: 61 passed and 2 skipped. The skipped tests are the probe tests from test(python): verify the two engine facts the spec depends on (#1384) #1414. They run only in a special build.
  • mypy --strict and pyright: both accept m.query_sql_domain on a type[Marker]. Both reject from stackencrypt import Json.
  • mise run types:generate: exits 0 and changes no other generated file. With its last step forced to fail, it restored both cross-tree files.
  • pnpm run test:scripts on Node 22: 83 of 83 test files pass.

Not tested:

  • CI has not run on this branch yet.
  • mypy and pyright ran locally only. CI does not run either one.

Related

Closes #1385. Refs #1341 (the spec). Stacked on #1414.

Review notes

Start with packages/eql/crates/eql-codegen/src/py_eql.rs, then the generated _eql.py.

The EQL workflow path filters do not list _eql.py, the same as the Go file eql_gen.go. A path outside packages/eql/ would need a new exception in the guard eql-suite-ci.test.mjs. Instead, tests-python.yml runs the parity tests, so a hand edit to _eql.py fails the Python workflow.

Passthrough and Ignore inherit the attribute declarations from Marker, but set no values. A type checker accepts Passthrough.sql_domain, but it raises AttributeError at run time.

#1385 says 11 catalog types are unproducible. On the current base, only Json is. The tests compare the Python and Go lists and assume no count.

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: d777757

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

yujiyokoo and others added 2 commits October 29, 2026 16:03
A model declares each field with a marker named after its EQL type,
such as TextEq. The spec (#1341) wants the markers generated from the
EQL catalog, beside the Go package, so Python and Go cannot disagree
about a type.

eql-codegen gains a py-eql emitter over the same rows as go-eql. It
writes stackencrypt/_eql.py: one Marker subclass per row with the ten
catalog attributes, a TYPES tuple, and an __all__ of the producible
markers that the package re-exports. types:generate runs it and
types:check diffs it. Two parity tests hold it: the committed file must
match the render, and every marker must match its Go Types row.

The file is not added to the EQL workflow path filters, the same as
eql_gen.go. A path outside packages/eql/ would need a new exception in
the filter guard, and the catalog changes that matter already start
the EQL workflow.

Refs #1385

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`go-eql` and `py-eql` write outside the EQL subtree, so an
`EQL_CODEGEN_OUT_ROOT` override sent them outside the throwaway tree.
Both now refuse the override with exit 2, and a CLI test covers it.

`types:generate` now also snapshots `eql_gen.go` and `_eql.py`. A failed
run restores them with the bindings, so the tree is never left mixed.

`Marker` declares the ten attributes with types, so mypy and pyright
accept them on a `type[Marker]`.

The Go and Python emitters now take their shared wording from two
methods on `Row` in `targets.rs`: `summary` and `status`. Both
generated files are unchanged.

The parity test skips each docstring before it reads attributes, so a
catalog reason containing ` = ` cannot be read as one.

The eql-bindings README and AGENTS.md now say that `_eql.py` is
generated.

Refs #1385

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yujiyokoo
yujiyokoo force-pushed the feat/python-sdk-eql-markers branch from 0ebe128 to d777757 Compare October 29, 2026 07:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant