[typotter]

Motivation

Rules-based corpus consumers need portable per-platform exclusions for known SDK gaps and directory-level isolation for feature-specific UFC syntax. Filtering must happen before runner setup and configuration side effects.

Changes

• add the canonical rules-based platform identifiers directly to ci/validate-fixtures.py
• validate _skip and _include annotations for mutual exclusion, shape, duplicates, reasons, and canonical platforms
• define the root corpus as core and validate self-contained evaluation-suites/<suite>/ directories
• document consumer ordering: select suites, discover/filter cases, avoid UFC loading when no cases apply, then initialize and execute
• annotate the observed PHP gaps for disabled flags, empty flags, and missing targeting keys
• annotate the observed Python gaps for object and array opt-out attributes
• extend fixture and downstream workflow path coverage for feature suites

Decisions

• _skip and _include are mutually exclusive case-level fields
• unannotated cases apply to every canonical platform
• feature suites are explicitly activated and never auto-enabled
• the README defines the consumer semantics and points to the validator for the canonical platform list
• no separate specification schema or validation-vector file is introduced
• precomputed-assignment fixtures, schemas, validators, documentation, and workflows are unchanged

This PR is stacked on #45, which introduces the self-contained dependent-flags feature suite. Its diff remains limited to the shared suite/applicability contract, validator, core annotations, and workflow coverage. The final simplification received a clean local re-review.

Checklist

• JSON files are valid (no syntax errors)
• New test cases reference existing flags in config/ufc-config.json (N/A: no new corpus cases)
• New flags have corresponding test cases in evaluation-cases/ (N/A: no new flags)
• Expected results match the flag configuration logic (expectations are unchanged; applicability annotations were added)

Validation

• python3 ci/validate-fixtures.py — 2 suites, 376 cases across 48 files
• python3 -B -m unittest discover -s ci -p "test_*.py" — 17 tests passed
• git diff --check

[Copilot]

🟡 Changes recommended

Contract validation accepts incorrect JSON types for version and filtering controls, weakening cross-platform consistency.

1 open issue

• Reject boolean-integer type substitutions in JSON validation · New

What changed in this PR

Adds platform applicability controls and optional feature-suite isolation to the rules-based evaluation corpus, so consumers can filter cases before initialization or configuration side effects.

Changes:

• Defines canonical platforms, case filtering, and explicit suite selection.
• Adds validation and regression tests, plus PHP and Python exclusions for known SDK gaps.
• Documents consumer ordering and expands workflow triggers for the new contract and suites.

File	Description
README.md	Documents filtering, suite selection, and runner ordering.
evaluation-platforms.json	Defines the platform registry and applicability contract.
evaluation-cases/​test-case-experimentation-opt-out.json	Adds PHP and Python exclusions for known gaps.
evaluation-cases/​test-case-empty-flag.json	Excludes PHP empty-flag cases.
evaluation-cases/​test-case-disabled-flag.json	Excludes PHP disabled-flag cases.
CONTRIBUTING.md	Adds applicability and feature-suite guidance.
ci/​validate-fixtures.py	Validates platform annotations and self-contained suites.
ci/​test_validate_fixtures.py	Adds filter and suite validation tests.
.github/​workflows/​validate-fixtures.yml	Triggers validation for contract and suite changes.
.github/​workflows/​downstream-compatibility.yml	Triggers downstream checks for contract and suite changes.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

The scoped changes consistently implement the documented applicability contract, with regression coverage and no identified blocking issues.

No open issues 🎉

1 resolved since last review

• Reject boolean-integer type substitutions in JSON validation

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

The validation logic, documented contract, fixture annotations, tests, and workflow changes are consistent, with no unresolved blocking issues.

No open issues 🎉

_{Review effort: Balanced}