nemotron_asr: move options to the model spec - #685
Conversation
Declare Nemotron ASR request and session options in model_specs/nemotron_asr.json (schema_version 1) and validate them through the spec-backed helpers. Unknown request options are now rejected, and the CLI help comes from the spec instead of a hardcoded list in the loader.
|
@LysanderdeJong The migration is generally in the right direction, but the PR breaks legacy GGUFs in non-deployment builds when no external v1 spec is available |
GGUFs that embed a pre-v1 spec failed to load in non-deployment builds when no schema-v1 contract was available. Add a non-throwing model_spec::find_model_contract() and, when it finds none, keep the pre-migration session whitelist, unvalidated request options and help.
|
Thanks, went with option 2 in 8e59ca1:
Tested with the published Q8_0 GGUF from a working directory outside the repo (no |
Summary
nemotron_asroption inmodel_specs/nemotron_asr.json(schema_version: 1)First of three stacked PRs split from #681: this PR, then the diarizer safetensors output, then speaker-tagged ASR.
Why
The family validated session options with a hard-coded whitelist, read request options ad hoc, and hard-coded its CLI help. Adding an option needed three edits.
Changes
model_specs/nemotron_asr.json: request optionslanguage,lookahead_tokens,max_tokens,keep_language_tags,return_timestamps,streaming; the seven existing session options.validate_spec_backed_session_options/validate_spec_backed_request_optionsinprepare(),run()andstart_stream().inspect()returns the contract's CLI help. The loader stays custom, so metadata and the prompt-dictionary language list are unchanged.Maintainer notes
model_specs/reachable),nemotron_asrkeeps main's behaviour: the old session whitelist, unvalidated request options and the hardcoded help.model_spec::find_model_contract()is the non-throwing lookup that makes this possible.Validation
Windows 11, MSVC (VS 2026), CUDA 13.3.
--words-outJSON are byte-identical before and after--request-option bogus=1fails withunknown Nemotron ASR request option: bogus;--session-option nemotron_asr.bogus=1fails withunknown Nemotron ASR session option: nemotron_asr.bogustools/check_loader_catalog_sync.pypasses