Skip to content

nemotron_asr: move options to the model spec - #685

Merged
0xShug0 merged 2 commits into
0xShug0:mainfrom
LysanderdeJong:feature/nemotron-asr-spec
Sep 25, 2026
Merged

0xShug0 merged 2 commits into
0xShug0:mainfrom
LysanderdeJong:feature/nemotron-asr-spec

Conversation

@LysanderdeJong

@LysanderdeJong LysanderdeJong commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • declare every existing nemotron_asr option in model_specs/nemotron_asr.json (schema_version: 1)
  • validate session and request options with the spec-backed helpers; unknown request keys now fail
  • generate the CLI help from the spec

First of three stacked PRs split from #681: this PR, then the diarizer safetensors output, then speaker-tagged ASR.

Why

The family validated session options with a hard-coded whitelist, read request options ad hoc, and hard-coded its CLI help. Adding an option needed three edits.

Changes

  • model_specs/nemotron_asr.json: request options language, lookahead_tokens, max_tokens, keep_language_tags, return_timestamps, streaming; the seven existing session options.
  • Sessions get the contract from the loader and call validate_spec_backed_session_options / validate_spec_backed_request_options in prepare(), run() and start_stream().
  • inspect() returns the contract's CLI help. The loader stays custom, so metadata and the prompt-dictionary language list are unchanged.

Maintainer notes

  • An undeclared request key, such as a typo, used to be ignored and now fails. Session keys were already checked by the old whitelist.
  • The published Q8_0 GGUF embeds a legacy (pre-v1) spec. It keeps working through the existing fallback to the workspace v1 contract, which prints the usual legacy-spec warning. Repackaging it would embed the new options.
  • Without any v1 contract (legacy embedded spec, non-deployment build, no model_specs/ reachable), nemotron_asr keeps main's behaviour: the old session whitelist, unvalidated request options and the hardcoded help. model_spec::find_model_contract() is the non-throwing lookup that makes this possible.

Validation

Windows 11, MSVC (VS 2026), CUDA 13.3.

  • offline and streaming text and --words-out JSON are byte-identical before and after
  • every existing option still works
  • --request-option bogus=1 fails with unknown Nemotron ASR request option: bogus; --session-option nemotron_asr.bogus=1 fails with unknown Nemotron ASR session option: nemotron_asr.bogus
  • tools/check_loader_catalog_sync.py passes

Declare Nemotron ASR request and session options in
model_specs/nemotron_asr.json (schema_version 1) and validate them
through the spec-backed helpers. Unknown request options are now
rejected, and the CLI help comes from the spec instead of a hardcoded
list in the loader.
@0xShug0

0xShug0 commented Sep 25, 2026

Copy link
Copy Markdown
Owner

@LysanderdeJong The migration is generally in the right direction, but the PR breaks legacy GGUFs in non-deployment builds when no external v1 spec is available embeds a legacy model spec, but no current schema-v1 model contract was found. There are two approaches, and I'm fine with either. (1) Defer the migration for now and migrate all models together in a dedicated pass later, rather than breaking compatibility one model at a time. (2) Add a compataible layer: With a contract: use spec-backed validation. Without one: retain main’s existing whitelist. This way, downstream consumers still have a clean, single source of truth, while the compatibility mess stays inside the code.

GGUFs that embed a pre-v1 spec failed to load in non-deployment builds
when no schema-v1 contract was available. Add a non-throwing
model_spec::find_model_contract() and, when it finds none, keep the
pre-migration session whitelist, unvalidated request options and help.
@LysanderdeJong

Copy link
Copy Markdown
Contributor Author

Thanks, went with option 2 in 8e59ca1:

  • General (framework): model_spec::find_model_contract() / find_contract_spec_path() return nullopt instead of throwing when the GGUF embeds a legacy spec and no v1 contract exists. default_contract_spec_path() keeps its current behaviour and error.
  • Model-specific (nemotron_asr): with a contract, spec-backed validation and help; without one, main's session whitelist, unvalidated request options and hardcoded help, unchanged.

Tested with the published Q8_0 GGUF from a working directory outside the repo (no model_specs/ reachable, non-deployment build): offline and streaming transcribe, unknown session keys still fail, help shows the legacy list. Inside the repo, v1 validation still rejects bogus=1. #686 and #681 are rebased on top.

@0xShug0
0xShug0 merged commit 7c79c9a into 0xShug0:main Sep 25, 2026
6 checks passed
@LysanderdeJong
LysanderdeJong deleted the feature/nemotron-asr-spec branch September 26, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants