Skip to content

Batch 2026-10-01: lazy imports, ORM ORDER_DETAIL generation, Dependabot cap ignores (BACKLOG #2514, #2497, #2505) - #1883

Merged
wshallwshall merged 18 commits into
mainfrom
batch/deps2-1001
Oct 1, 2026
Merged

wshallwshall merged 18 commits into
mainfrom
batch/deps2-1001

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Three items, one merge commit each, on base ab8029f. Head 5fa55dd. No conflicts.

#2514: lazy_modules (PEP 810)

Source: b2514-lazy-modules @ 230cf26.

  • On 3.15, CLI startup drops from 273 to 186 modules and the tray's first process from 204 to 120. Inert on 3.14.
  • The main win is in config/init.py, whose eager re-export of config.models pulls in pydantic.
  • Kept eager: logging_setup, the tray log-scrub chain (PHI filter), console_streams, odbc_env and tls_policy.
  • tests/test_startup_import_budget.py compares module sets, not times. It is advisory below 3.15, and its eager control arm runs everywhere.
  • QA: Level: xhigh. Tag: none returned. Rounds: 2. Findings: 3 confirmed and fixed, 0 rejected, 2 open (out of scope).
  • Follow-up: vault #2545 (get the savings on 3.14 now).

#2497: ORM^O01 ORDER_DETAIL

Source: b2499-2497-choice-groups @ 602673a.

  • generators/_core.py _emit picks one alternative of a choice group, with OBR preferred.
  • ORM^O01 gets ORC then OBR, and OBR-2/3 repeat the ORC's order numbers.
  • validate._is_choice is now the public is_choice_group.
  • Over 3,720 generated messages, all pass strict validation.
  • QA: Level: xhigh. Tag: none returned. Rounds: 2. Findings: 7 confirmed and fixed, 4 rejected, 0 open.
  • KNOWN DEFECT, fix to follow: tests/test_generators_choice.py::test_handing_over_order_numbers_moves_no_other_value raises KeyError: 'OML' when run alone, because the module imports only orm and OML registers only once generators.all_types is imported. It passed in the full suite, but it would go red on an xdist worker that gets it first. The fix is a one-line import of all_types in the test.
  • #2499 is NOT in this PR: its prescribed fix was measured as a no-op, and the item is amended in the vault.

#2505: Dependabot cap ignore entries

Source: b2505-cap-ignores @ ad67091.

  • uv ignore entries are added for pynetdicom >=4.0.0, pydicom >=3.1.0 and webauthn >=4.0.0.
  • tests/test_dependabot_cap_ignores.py holds every pyproject upper bound, [build-system] included, to an exact ignore range or a named exemption: hvac, hatchling, and the exact group pins except sigstore.
  • It also fails on a stale entry, on an extras-spelled name, and on an entry for a named exemption (hvac must NOT have one).
  • The design went through class-rule and per-package versions. Per-package won, because python-deps-major and python-security are single batches and dependabot.yml documents the accepted security-track trade-off.
  • Tension recorded, not resolved: hvac's pyproject opt-out reasoning would apply equally to the other two major caps.
  • pynetdicom<4 has no stated reason anywhere in the tree; history begins at the 2026-07-06 snapshot. The entry mirrors the cap as-is.
  • QA: Level: xhigh. Tag: none returned. Rounds: 4. Findings: 8 confirmed and fixed this round (26 across rounds), 0 open in code.

Checks on the combined tree

  • ruff and format clean; mypy clean on messagefoundry (304) and tests (1054); ledger_check --ci passes; 15 changelog fragments well-formed.
  • Full pytest in 6 chunks over all 1011 test files plus the webconsole tests: all green except one sandbox flake that is load-dependent and passes when rerun alone.

Landing order: none.

wshallwshall added 16 commits September 30, 2026 22:16
…re ranges (BACKLOG #2505)

Dependabot widens a pyproject cap unless dependabot.yml restates it as an
ignore range. Three caps had no entry: pynetdicom<4, pydicom<3.1 and
webauthn<4. hvac<3 stays without one, as pyproject.toml says beside it.

tests/test_dependabot_cap_ignores.py holds every upper bound in the three
dependency tables to a covering ignore range or a stated exemption, and
every ignore entry to a cap that still exists. Work in progress: the
/simplify pass and code review follow in later commits.

(cherry picked from commit b008f221e047224b08eeed906272a664b8b9413c)
…s an order detail (BACKLOG #2497)

_emit walked a choice group like a sequence and emitted every required
child, so ORM^O01 left ORDER_DETAIL out: its OBR/RQD/RQ1/RXO/ODS/ODT group
is a choice. _emit now picks one alternative, through the strict
validator's own _is_choice, so the groups hl7apy mislabels as choices
still get every part. MessageSpec.preferred_alternative, OBR by default,
wins without drawing from the seeded rng; otherwise a seeded pick among
buildable alternatives. ORM adds _ORDER_DETAIL to its group suffixes.

Tests: tests/test_generators_choice.py. Generated ORM^O01 (v2.5.1, the
only version the generator writes) passes strict validation across
seeds; five mutation arms each go red.
… restate (BACKLOG #2505)

Applies the simplify pass and the Manager's decisions to the cap test:
_load is cached, _cap_of returns one Version, the pin's allowed version
is _just_below like every other cap, one _EXEMPT dict, one _uv_entry
helper, and one two-document mutation callable. The group-pin exemption
is now EXACT_GROUP_PINS from test_ci_venv_pinning.py minus sigstore.

The mutation arms break a made-up cap and entry pair, never a real one,
behind a control that the unbroken fixture passes. The census test is
gone: the stale-entry direction already reds on every real entry if the
parser finds no caps.

dependabot.yml: the three new entries point at the pyproject reason
rather than restating it, and the uvicorn note no longer says pyproject
declares uvicorn[standard]. Comment changes only.
…ACKLOG #2505)

Code review round 1 found that two sample versions could not see a
hole in an ignore range or a range that blocks an allowed version. An
entry for a capped package must now be exactly ">=" the first version
the cap excludes, the one range that blocks all it excludes and nothing
it allows. _just_below and _covers go with the sampling.

The census now reads [build-system].requires too, so "every upper
bound" is true; hatchling's pin is exempt with a pointer to the
pyproject comment that expects Dependabot to bump it. Entry names drop
extras before matching, as dependabot-core does. Added arms for a hole,
a blocked allowed range, a non-specifier range, and the three
exemption-table violations, each over the made-up pair only.

dependabot.yml: the "the one cap with NO entry" claim becomes an "at
least" list, the reason map no longer enumerates two of seven, and the
pynetdicom pointer says what the [dicom] comment actually holds.
…#2514)

PEP 810 lists in four modules. Inert on 3.14; on 3.15 the named
imports bind lazily. No `lazy` keyword (a SyntaxError on 3.14).

- config/__init__.py: config.models. Every CLI command reached
  pydantic through logging_setup -> config.tls_policy -> this
  package's re-export. 3.15.0b3: 273 -> 186 modules for --help.
- __main__.py: sqlite3, tomllib.
- tray/config.py: service_status, tomllib.
- tray/__main__.py: tray.instance. Tray first process 204 -> 120.

tests/test_startup_import_budget.py: module-set budget, skipped on
3.14, asserted on 3.15, with an eager control arm.
…e made public (BACKLOG #2497)

Code review round 1 on the previous commit:
- OBR-2/OBR-3 drew fresh numbers instead of repeating ORC-2/ORC-3, which
  the new ORM order detail made visible. The ORC builder hands its numbers
  to the next OBR through Ctx, consumed once. The OBR's own draws are kept,
  so only those two fields move: ORM, OML, MDM, ORU, 3720 messages measured,
  all still strictly valid.
- The generator imported the private _is_choice from the hl7apy issue-151
  shim, whose notes say to delete it. Renamed is_choice_group, and both
  deletion notes now say to keep it and _SEQUENCES_LABELLED_CHOICE.
- _pick_alternative no longer counts a group alternative as usable; none
  ships in hl7apy, and it was never checked for buildability.
- The ORM test runs under two seeds and checks OBR-2/3 against ORC-2/3.
…re ranges, and test every pyproject cap against one (BACKLOG #2505)

Ledger banner: BUILT 2026-09-30 on branch b2505-cap-ignores -- three
uv ignore entries (pynetdicom >=4.0.0, pydicom >=3.1.0, webauthn
>=4.0.0) and tests/test_dependabot_cap_ignores.py, which holds every
pyproject upper bound to an exact ignore range or a stated exemption
(hvac, hatchling, the exact group pins) and every entry to a live cap.

Code review round 2 changes. Each exemption now names the table it
holds in, so a new cap on hvac or hatchling elsewhere is not exempt,
and both halves of the group-pin rule have their own arm. An entry name
with extras is now a violation rather than matched leniently. The
suggested range keeps pre-release and epoch parts, a non-string range
reds instead of crashing, the pin test lives in one helper, and the
docstring points at dependabot.yml rather than restating its history.
dependabot.yml says pynetdicom's cap has no reason of its own in
pyproject.toml.
…y pyproject cap against the class rule (BACKLOG #2505)

Manager revision: a cap at minor level or lower needs an exact uv
ignore entry, because python-deps would widen it in the routine batch.
A major-level cap (<N) is exempt by class, since a major goes to its
own python-deps-major PR and an entry would only hide the security
track; pyproject's hvac comment holds that reasoning. An entry for a
major cap stays a legal opt-in and is held to the exact range.

dependabot.yml drops the pynetdicom and webauthn entries, keeps
pydicom >=3.1.0, and states the class rule once. The test replaces the
hvac exemption with the rule and adds arms: a made-up major cap with no
entry is green, an opt-in exact entry for it is green, and a non-exact
one is red. pyproject.toml is unchanged: pynetdicom's <4 first appears
in the history root 5464262, whose message does not state it.

Ledger banner: BUILT 2026-09-30 on branch b2505-cap-ignores -- pydicom
>=3.1.0 uv ignore entry, the minor-or-lower class rule in
dependabot.yml, and tests/test_dependabot_cap_ignores.py holding it.
…y pyproject cap against the class rule (BACKLOG #2505)

Review round on the class-rule delta. One ignore entry serves a
package, so only its tightest cap is matched against it; a looser major
cap beside a minor one no longer reds. A post-release or epoch bound is
not major-level. Comments that still said "every upper bound" now say
"below major level", and the dependabot.yml header and class rule say
which specs count as major-level.

Ledger banner: BUILT 2026-09-30 on branch b2505-cap-ignores -- pydicom
>=3.1.0 uv ignore entry, the minor-or-lower class rule in
dependabot.yml, and tests/test_dependabot_cap_ignores.py holding it.
… PYTHON_LAZY_IMPORTS (BACKLOG #2514)

Code-review round 1 findings 1-3:
- the CLI probe swallowed SystemExit, so a refused argument passed;
- the tray first process also imports tray.branding before re-exec;
- an inherited PYTHON_LAZY_IMPORTS would override both arms.

Proposed PR title: Defer CLI and tray startup imports with
__lazy_modules__ (PEP 810), plus an advisory startup-budget test
(BACKLOG #2514)

Proposed ledger banner: BUILT -- __lazy_modules__ in config/__init__,
__main__, tray/__main__, tray/config; inert on 3.14, on 3.15.0b3 CLI
--help 273 -> 186 modules, tray first process 204 -> 120; budget test
skips on 3.14 and asserts on 3.15.
…t (BACKLOG #2497)

Code review round 2: no correctness defect. Its test gaps are closed
here. One test pins that an OBR takes its ORC's numbers once. Another
pins that the hand-over moves only OBR-2/3 and no later random value.
is_choice_group joins validate.__all__. The OBR numbering fix moves
to its own fixed fragment.

Proposed PR title:
Generator emits one alternative of a choice group, so ORM^O01 carries an
order detail; OBR repeats its ORC numbers (BACKLOG #2497; #2499 measured)

Proposed ledger banners:
#2497 SHIPPED on this branch: _emit picks one choice alternative, OBR by
default; ORM^O01 ends ORC then OBR and passes strict validation at
v2.5.1, the only version the generator writes.
#2499 OPEN, prescription measured as a no-op: hl7apy's parser places a
segment only by exact name and drops one it cannot place, and
ANYHL7SEGMENT never matches a real segment. A table alternative changes
no verdict: 0 of 120 differ, against a control that differs in 4. The
real reject, ORC then a non-OBR detail then NTE/VAR/OBX, needs a
parser-side decision on what "etc." names. Owner or spec question.
…re ranges, and test every pyproject cap against one (BACKLOG #2505)

Manager revision: the class rule is withdrawn. python-deps-major is one
batch on "*", so a widened major cap holds that batch hostage too, and
python-security is not split by update type. Every cap now needs an
exact entry unless it is a named exemption: hvac (the [vault] comment),
hatchling ([build-system]) and the exact group pins. A named exemption
is also a must-not: an entry for it reds, with an arm over hvac and
hatchling.

dependabot.yml restores the webauthn and pynetdicom >=4.0.0 entries
beside pydicom >=3.1.0, states the rule and the named exemptions once,
and says pyproject records no reason for pynetdicom<4; its history
stops at root 5464262. Kept: tightest cap per package, table-scoped
exemptions, the exact range, the extras violation and the
[build-system] census. _is_major_level and the class-rule arms are gone.

Ledger banner: BUILT 2026-09-30 on branch b2505-cap-ignores -- uv
ignore entries for pydicom >=3.1.0, webauthn >=4.0.0 and pynetdicom
>=4.0.0, and tests/test_dependabot_cap_ignores.py holding every
pyproject cap to an exact entry or a named exemption (hvac, hatchling,
the exact group pins), and every entry to a live cap.
…re ranges, and test every pyproject cap against one (BACKLOG #2505)

Review round on the per-package revert. An exemption's table now
matches exactly or at a "." boundary, so an extra named vault-legacy is
not covered by hvac's [vault] exemption; an arm holds it. A major cap
with no entry has its own red arm. The must-not arm names hvac and
hatchling explicitly, so dropping either reds instead of skipping.

dependabot.yml: the header admits named exemptions; the rule says how
the range is derived for a pin, points at _EXEMPT for the list rather
than closing it, and excepts sigstore. The pynetdicom note is corrected:
the clone is shallow at 5464262, and the cap is already in
5fa6db9, the 2026-07-06 history-reset snapshot. The [dicom] comment
gives pynetdicom's pydicom requirement but no reason for its own <4.

Ledger banner: BUILT 2026-09-30 on branch b2505-cap-ignores -- uv
ignore entries for pydicom >=3.1.0, webauthn >=4.0.0 and pynetdicom
>=4.0.0, and tests/test_dependabot_cap_ignores.py holding every
pyproject cap to an exact entry or a named exemption (hvac, hatchling,
the exact group pins), and every entry to a live cap.
@wshallwshall wshallwshall added the qa Builder QA record posted; not a merge gate label Oct 1, 2026
wshallwshall added 2 commits October 1, 2026 02:52
… OML alone

test_handing_over_order_numbers_moves_no_other_value raised KeyError 'OML'
when run by itself: the file imported only generators.orm, so OML was
registered only when another test module happened to import all_types
first. Import generators.all_types instead, as the other generator test
files do; it imports orm too, so ORM still registers.

Proved by running the one test alone with -p no:randomly.

Also merges origin/main in.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

QA: code-review subagent xhigh, head 80bd731, verdict PASS; all_types import registers OML and ORM, all 13 tests pass run alone, control arm with only orm reproduces KeyError 'OML'.

@wshallwshall
wshallwshall enabled auto-merge October 1, 2026 08:04
@wshallwshall
wshallwshall added this pull request to the merge queue Oct 1, 2026
@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

Security failed in pip-audit (dependency vulnerabilities) / Check the lockfile is in sync with pyproject (DEP-1) while this pull request was in the merge queue, so the queue ejected it.

Its own head can still be green: the queue revalidates the merge, and the path gates that skip on a pull request run there. Read the run before retrying.

https://github.com/MEFORORG/MessageFoundry/actions/runs/36836473201

Merged via the queue into main with commit 490ee0c Oct 1, 2026
46 of 49 checks passed
@wshallwshall
wshallwshall deleted the batch/deps2-1001 branch October 1, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying. qa Builder QA record posted; not a merge gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant