Repository navigation
task: support shared YAML task input for the InferRSI CLI and agents #365
Description
Goal and acceptance
Make it easier to start and repeat an InferRSI task. A user should be able to write a task once, or ask an Agent to generate it, then use the same YAML file from the CLI or hand it directly to an Agent.
- The CLI accepts a YAML path for validation and preview, and can use a complete task file with existing plan and run operations. Invalid or incomplete input identifies the fields to fix before execution.
- An Agent can generate or read the same file using the published schema and template. It validates the file through the shared contract, reviews the resulting task with the user, and uses it with existing InferRSI operations.
- The same file yields the same normalized task fields through either entry point. Validation and preview leave the workspace and experiment state untouched.
Scope and constraints
Use one versioned YAML schema with three sections.
| Section | Inputs |
|---|---|
| Optimization target (优化对象) | Repository, model and framework, workload, objective, and SIAS change plane |
| Test environment (测试环境) | Backend and device, prepared runtime, benchmark and quality scripts, iteration and time budgets |
| Acceptance criteria (验收标准) | Acceptance issue or plan reference, baseline revision, agreed metrics, thresholds and constraints |
Partial v1 example for validation and preview:
schema_version: 1
optimization:
repository: .
model: "<MODEL>"
objective: "<OPTIMIZATION_GOAL>"
plane: harness
environment:
backend: cuda
acceptance:
reference: "<ACCEPTANCE_PLAN>"
constraints: "<AGREED_CRITERIA>"Proposed contract for triage
| Operation | Required inputs and result |
|---|---|
inferrsi task validate <task.yaml> and inferrsi task preview <task.yaml> --json |
Require integer schema_version: 1. Each of the three sections may be omitted in a draft. A present section must be a mapping and all supplied values must satisfy the schema. Null field placeholders count as missing. Validation reports format and value errors by field. Preview also returns the effective normalized values and a field-level list of blockers for a full plan/run. A valid draft may still be incomplete. |
inferrsi plan --task <task.yaml> and inferrsi run --task <task.yaml> |
Require all three sections and an execution-complete task. Require repository, model, framework/repo type, workload, objective and SIAS plane under optimization. Require backend, baseline/candidate benchmark commands and iteration/time budgets under environment, plus device/runtime selectors and a quality command where the selected operation or accepted plan requires them. Require the acceptance issue or plan reference, baseline revision, metric key, direction and unit, the agreed threshold, and any plan-specific constraints under acceptance. Report missing or invalid inputs by field before operation-specific probes, commands or writes. |
- Read-only boundary. Task validation and preview read the YAML and write results to stdout/stderr. They start no subprocesses, run no probes or task scripts, install nothing, and create no workspace, plan, run or evidence files. Preview reports input completeness only. A complete file feeds the existing RunSpec and operation checks.
planpreviews that specification andrunrequires explicit invocation under the existing acceptance, frozen gate and evidence rules. An Agent uses the same installed CLI and machine-readable preview, reviews the effective task with the user, then invokes the authorized operation. Preview and execution review - Merge order. For flags mapped to YAML fields, use explicit CLI values first, then YAML, then built-in defaults for optional fields. Track whether each flag was supplied before applying defaults. Accept the same mapped overrides in preview, plan and run. An omitted flag cannot replace a YAML value. Reject invalid YAML fields even when overridden. Acceptance fields, including the explicit threshold, come from the YAML in task-file mode. Reject conflicting acceptance flags and require the effective task to agree with the accepted frozen gates before execution. Keep existing flag-only invocations compatible. CLI defaults review
- Types and paths. Use the existing CLI backend values, such as
cudaandascend. Benchmark repetitions are positive integers and time budgets are positive finite numbers in minutes within the existing operation bounds. DSH optimization rounds are a separate budget. Resolve YAML filesystem paths relative to the YAML file's directory and explicit CLI paths relative to the caller's working directory. Treatacceptance.referenceas an HTTP(S) issue URL or a local plan path, resolving only the local form. Commands use the existing argv and repository working-directory contract. Reject unknown and duplicate keys, unsupported versions and type errors with field paths. Credentials use the existing runtime channels. - Core install. Declare PyYAML as a core dependency, use a safe loader with explicit duplicate-key rejection, and ship the versioned schema and template in the ordinary wheel. Add a regression check in the existing Ready lane that installs the wheel with its declared dependencies and no extras, runs outside the checkout, and verifies validation, preview and packaged resources. Core installation review
- DSH boundary. Core CLI and direct Agent use are the consumers for task: support shared YAML task input for the InferRSI CLI and agents #365. Track DSH adoption under [RFC] Agent plugins: a consistent InferRSI workflow in Codex, DSH and openJiuwen #152, with its implementation owner responsible for migrating the draft format or explicitly identifying it as a legacy format. Before DSH advertises core v1 support, it must consume the core contract and match its shared fixtures for complete and partial tasks, relative paths, budget types, acceptance references and invalid fields. Validate existing DSH draft files against the core schema and migrate incompatible fields explicitly. DSH compatibility review · DSH tracking Issue · DSH draft PR
Ownership and authorization
Proposed usability task. The requester asked for a standalone core Issue. Implementation and acceptance owners are to be assigned during triage.
Inputs and deliverables
- A documented YAML schema, an Agent-friendly template and an example workflow for both entry points.
- Shared validation and normalization, integrated with the existing CLI operations and documented for direct Agent use.
- Use one complete fixture with non-default backend, repetitions, and threshold through both direct CLI and Agent workflows and compare the normalized task fields. Verify that omitted flags preserve YAML and an allowed explicit override appears in the final preview. Cover a partial draft, invalid keys and values, and rejection of a conflicting acceptance flag. Show that the complete fixture produces the expected RunSpec through the existing plan path. Assert that validation, preview and invalid plan/run inputs neither launch task commands nor create workspace or experiment artifacts. Reuse the same complete fixture in the no-extras wheel check.
- Existing operation boundaries are documented in the InferRSI operation guide.
Completion and closure
Review the schema and CLI interface during triage. Close after the acceptance owner can reproduce the two workflows from the same YAML fixture and review the validation evidence.
Activity
Dong1017 commentedon Nov 6, 2026
两处现有接口的接入风险，供 schema/CLI 定稿时核对（静态检查 main
1246348f，不是已实现缺陷）：CLI 默认值不能冒充用户显式覆盖。 当前
arguments.py会填入backend=cpu；实验参数 也默认填入repetitions=3、min_improvement_pct=0.0等。若把解析后的 Namespace 整体覆盖到 YAML，上述默认值会悄悄替换文件中的设备、重复次数或门槛。建议先区分“未提供的 flag”和“显式 flag”，按约定优先级合并，最后才补默认值并统一校验。用一份包含非默认值的 YAML 分别经过 Agent 与 CLI，断言规范化结果一致；再覆盖显式 flag 冲突/优先级的案例，预览显示最终有效值。“普通 CLI 安装可用”需要覆盖无 extras 的安装路径。 当前
pyproject.toml的核心依赖为空，PyYAML仅属于skillsextra。复用现有可选 YAML 路径时，开发环境能通过不代表普通 wheel 安装可用。接口方案应明确解析器如何随核心交付，并在仅安装普通 wheel 的干净环境中验证同一 fixture 的 validation/preview，不隐式安装依赖或运行任务脚本。这些检查对应本 Issue 已有的共享规范化、普通 CLI 和只读预览要求；未运行测试、接手实施或接受范围。
Dong1017 commentedon Nov 6, 2026
增量设计评审｜P2：需要明确已有 DSH YAML v1 与本次核心 YAML v1 的兼容边界（静态核对，非实施授权）。
目前 Draft PR #361 已有一份
schema_version: 1、同名optimization/environment/acceptance三段式任务模板；其form.cjs自行解析 YAML，并将预算字段转为表单字符串、对未填本机仓库使用当前工作目录默认值，导入时允许部分字段缺失而提交时才进行完整校验（解析和提交校验）。它属于 DSH 插件草案，而不是当前 main 的核心 CLI 契约；不能把该实现当作本 Issue 已交付。可触发的风险： 如果本 Issue 又独立定义一个名字相同的 YAML v1，但对缺字段、默认工作目录、预算类型或
acceptance.reference的规范化语义不同，同一文件可能被 DSH 接受却被核心 CLI 拒绝，或两个入口给出不同的有效任务，直接违背这里“同一文件、同一规范化结果”的验收目标。建议 owner 在冻结 schema 前作一个范围决定： DSH 是否属于本期消费者？若是，明确核心契约为唯一规范，并用一个覆盖完整/部分输入、相对路径、预算类型和非法字段的跨入口固定 fixture，校验规范化结果/拒绝原因一致；若否，请明确 DSH 原型不构成本期验收，但记录它的
schema_version: 1兼容/迁移责任，避免静默同名分叉。可以由各语言使用不同解析器实现，只要共享版本化语义和符合性用例；无需本轮改动插件或运行实验。依据：核心任务要求 #365 · DSH 集成范围 #152 · 在途 Draft #361。此意见不同于已提出的 CLI 默认 flag / core wheel 依赖问题。
amy-why-3459 commentedon Nov 6, 2026
Thanks for opening this as a standalone core usability task — a single versioned YAML that both the CLI and Agents can generate, validate, and reuse is the right shape for repeatable InferRSI work.
A few triage points that look worth freezing before implementation starts:
Preview vs execution contract. Please spell out which of the three sections (
optimization/environment/acceptance) are required for validation+preview versus a full plan/run. Preview must stay read-only (no workspace or experiment mutation), and incomplete files should return field-level errors rather than partial side effects.Merge order for YAML + CLI flags. Define a documented precedence (e.g. explicit CLI flags override YAML; YAML overrides built-in defaults; unset flags must not silently overwrite YAML). This matches the shared-normalization acceptance goal and avoids default values impersonating user intent.
Core install surface. Since ordinary CLI installs should work without extra Agent/plugin extras, the schema parser and template need to ship with the core package (or document an explicit core dependency), with a clean-wheel regression that runs validation/preview on one shared fixture.
Compatibility with in-flight DSH draft work. There is already a same-shaped
schema_version: 1task template on the DSH path (see discussion on #361). During triage, decide whether DSH is an in-scope consumer of this core contract. If yes, make the core schema the single source of truth and add a cross-entry fixture; if not, record the migration/compatibility ownership so we do not silently fork two v1 schemas.Happy to help assign implementation and acceptance owners once the above are decided. Existing operation boundaries in
docs/operations.mdshould remain the execution gate — YAML only feeds the same plan/run path.zengchuang-hw commentedon Nov 6, 2026
Thanks @Dong1017 and @amy-why-3459 for the design review. I updated the proposed contract in the issue body to cover the following.
The contract is ready for triage. Implementation and acceptance checks are still pending.
Dong1017 commentedon Nov 7, 2026
Thanks @zengchuang-hw — posting as @Dong1017's on-call proxy.
The revised contract works for triage from this side: preview vs execution, explicit-flag merge order, core PyYAML/wheel install, and DSH ownership under #152 all match the earlier design notes. No further contract blockers here; what's left for triage is assigning implementation and acceptance owners (checks still pending, as you noted).