This map says where each concept lives. Each row names a concept, the module that owns it, the rule that holds across the seam (in one sentence), and what enforces that rule. The detail is in the owning module's docstring. Read the docstring before editing; the map only tells you which one to open.
-
Owner: repo-relative paths. Function, class or constant names follow the path after a colon (
dw/runs.py:run_versions). -
Enforced by: one of four kinds.
- A test, written as a file, or as
file::Class::test_name. - A ratchet, named by its key in
docs/stabilization/baseline.json. - The CodeQL pack.
- The validation registry.
A
—means nothing mechanical holds the rule, so a change there is caught only in review. - A test, written as a file, or as
-
tests/test_architecture_map.pychecks three things: every path named here exists, every name after a path is defined in that file, and every test named here is defined in its class.
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Workflow | dw/workflow.py: Workflow, workflow_from_file |
A workflow is loaded and schema-checked here (the hand-off of each run phase to dw/workflow_run.py: —). |
tests/test_workflow.py::test_workflow_validation_invalid |
| One execution, in phases | dw/workflow_run.py: prepare_definition, cache_lookup, cache_hits |
The cache probe shares prepare_definition and cache_lookup with the run, so the probe answers for the run that would happen. |
tests/test_workflow_step_cache.py::TestCacheHits::test_after_a_run_the_probe_names_what_the_next_run_reuses |
| Step | dw/step.py: Step, dw/workflow.py: Workflow.create_step_action |
A step is a pipeline, a task or a sub-workflow, create_step_action builds its action, and dw/workflow_run.py runs it. |
— |
| Pipeline loading | dw/pipeline_processors/pipeline.py: Pipeline, dw/pipeline_processors/components.py: load_component |
A quantization config is built from a config_type that the _type key convention loads by name, so a new backend needs no code. |
tests/test_config_objects.py::TestQuantizationConfiguration::test_the_config_type_is_constructed_with_its_arguments |
| Device translation | dw/pipeline_processors/placement.py: place_component |
A device is translated by resolve_device() before anything reads its backend, so the MPS accommodations also fire for a translated device. |
tests/test_device_portability.py::TestPlacement::test_a_translated_device_gets_its_backends_offload_downgrade |
| On-demand wrappers | dw/pipeline_processors/placement.py: apply_on_demand_placement |
The on-demand wrappers keep the wrapped signature, because H3's denoiser reads signature(transformer.forward). |
tests/test_modular_pipeline.py::TestOnDemandResidency::test_the_wrapped_signature_survives |
| On-demand and offload | dw/pipeline_processors/placement.py: place_component, has_component_group_offload |
On-demand residency and group_offload cannot both be set on one component, and either one stops the whole pipeline from being moved to the device. |
tests/test_modular_pipeline.py::TestOnDemandResidency::test_group_offload_and_on_demand_together_are_rejected, tests/test_modular_pipeline.py::TestOffloadDevice::test_component_group_offload_is_not_moved_to_device |
| Quantized component cache | dw/quantized_cache.py: load_components, from_pretrained, cache_key |
A key that can't pin the weights isn't cached, the cache's own failures never fail a job, and the save comes before pruning, so an entry is the whole component. Every load_components call, cache on or off, goes through _load_checked: a component OOM that diffusers logs and swallows fails the job naming the component, rather than surfacing later as a missing-adapter error. |
tests/test_quantized_cache.py |
| LoRAs | dw/pipeline_processors/adapters.py: active_loras, load_loras |
A loras entry whose model_name is null is switched off. |
tests/test_lora_disable.py::TestLoadLoras::test_an_all_null_entry_loads_nothing |
| Step progress | dw/pipeline_processors/progress.py: reported_progress_bars, reported_blocks |
A pipeline with no step callback reports its denoise steps through its progress bars. reported_blocks names each top-level block of a sequential modular graph in a log event as it begins; a dw block that declares idle_without = <input> is left unnamed when that input is absent from the state, and is still run - whether a block acts is the block's own decision, never the narrator's. |
tests/test_modular_progress.py::test_every_advance_of_the_bar_is_reported, tests/test_modular_progress.py::test_an_idle_block_is_unnamed_but_still_run_without_its_input, tests/test_modular_progress.py::test_an_idle_block_is_named_and_run_when_its_input_is_given |
| Step timing | dw/pipeline_processors/timing.py: StepTimer, step_timer, timed_step, mark |
Only a pipeline step that ran is timed, on the worker's monotonic clock: step_timer opens a StepTimer for a step with a pipeline or pipeline_reference and None for any other, which also hides an outer step's timer from a composed child. Load, call, denoise start, each pass and save reach the open timer through mark, and nowhere else. Denoise start is never taken inside a compiled region: a whole-compiled denoiser (_compiled_call_impl set) has that callable wrapped for the call instead of a forward pre-hook, which fullgraph: true cannot trace. A span with a missing mark is null, never guessed. record_step puts the reading on the step's manifest entry and step_end, and a reused step gets no timing. A chain sums its pipeline calls into one timing with segments: N. ManifestEntry.timing declares the shape (docs/WORKSPACES.md Step timing). |
tests/test_step_timing.py::test_a_missing_denoise_start_nulls_encode_and_first_pass, tests/test_step_timing.py::test_a_reused_step_carries_no_timing_even_when_one_is_passed, tests/test_step_timing.py::test_a_chain_sums_its_segments_and_takes_one_median_over_both, tests/test_step_timing.py::test_a_task_step_run_through_the_step_loop_has_no_timing, tests/test_step_timing.py::test_the_denoise_hook_marks_the_first_forward_once_and_removes_itself, tests/test_step_timing.py::test_a_fullgraph_compiled_denoiser_runs_and_still_gets_the_start_mark |
| Step-cache skip counts | dw/pipeline_processors/timing.py: CacheCounter, counting_forwards, report_cache_stats, StepTimer.cache_counted; dw/pipeline_processors/components.py: stateful_cache_context; dw/pipeline_processors/mag_calibration.py: get_cache_transformer, captured_mag_calibration |
For first_block and mag, a forward hook on the cache transformer counts computed or skipped once per forward, reading the leader hook's should_compute inside the cond/uncond cache_context the pipeline opened, so classifier-free guidance counts both forwards. stateful_cache_context registers the hook for one pipeline call; its finally emits cache_stats before _reset_stateful_cache() and removes the hook after the event. Private diffusers state that has moved gives null counts and one cache_stats_unavailable warning, and never fails the run; taylorseer, faster and text_kv report type and config only, with no warning. The denoiser a cache attaches to (get_cache_transformer) and the MagCache calibration capture (captured_mag_calibration, the end-of-run log record turned into cache_stats-side job events, #891/#927) live in mag_calibration.py, which components.py imports and re-exports for pipeline.py, never the reverse. The counts reach the step's timing.cache only through StepTimer.cache_counted; a chain sums its calls' counts, and a null in any call nulls the sum. A step with no cache has no cache key and no event (docs/ACCELERATION.md Reading the skip counts). |
tests/test_cache_stats.py::test_cfg_counts_two_forwards_per_step_and_they_add_up, tests/test_cache_stats.py::test_the_event_names_the_step_and_segment_and_goes_out_before_the_reset, tests/test_cache_stats.py::test_moved_hook_state_nulls_the_counts_warns_once_and_the_run_finishes, tests/test_cache_stats.py::test_the_private_diffusers_names_are_where_the_counter_reads_them, tests/test_cache_stats.py::test_the_timer_sums_counts_and_nulls_on_any_null, tests/test_cache_stats.py::test_types_without_a_decision_report_type_only_and_no_warning |
| Vendored model ports | dw/vendored/ (audio8_tts_model.py, mmaudio_model.py, mmaudio_sync_model.py, mmaudio_clip.py) |
Upstream model code dw carries is one module per port, pinned to the commit or Hub revision its docstring names, so it can be diffed against its source and re-pinned as a unit; a port reaches dw only for the validated media loaders and is imported only by the task that runs it. Like dw/community_pipelines/, the package is outside the architecture ratchet (EXCLUDED in scripts/arch_metrics.py): a port keeps upstream's shape, which the module and function ceilings are not written for. |
tests/test_arch_metrics.py::test_vendored_model_ports_are_not_counted, tests/test_mmaudio_model.py::test_modules_are_imported_only_by_the_sound_effects_task, tests/test_mmaudio_sync_model.py::test_module_is_imported_only_by_the_sound_effects_task |
| Community pipelines | dw/community_pipelines/ (e.g. pipeline_ltx2_refine.py: LTX2RefinePipeline), dw/introspection.py: community_pipelines, load_allowed_class |
A pipeline dw ships is named by its dotted path (dw.community_pipelines.<module>.<Class>), loads untrusted because it is under dw, and is listed by list_pipelines and resolved by get_pipeline_signature - only the modules in that directory, never a caller-chosen import. LTX2RefinePipeline VAE-encodes video at width x height and passes it unnormalized as latents, because LTX2Pipeline.prepare_latents normalizes 5-D latents itself. |
tests/test_ltx2_refine_pipeline.py::TestIntrospection::test_only_the_shipped_modules_resolve, tests/test_ltx2_refine_pipeline.py::TestEncode::test_the_latents_are_left_unnormalized_for_the_parent |
| Adding a task | dw/tasks/registry.py: register_command, RegistryTable |
A task is a function registered with @register_command, and the signature of its implementation is its argument schema. Its validate-time rules are declared on the same decorator - domains, choices, static_check, whole_numbers and media_arguments - and TASK_ARGUMENT_DOMAINS, TASK_ARGUMENT_CHOICES, TASK_STATIC_CHECKS, TASK_WHOLE_NUMBER_ARGUMENTS and TASK_MEDIA_ARGUMENTS are read-only views derived from them, so there is no second table to forget (#692). The registry lives in dw/tasks/registry.py; dw/tasks/task.py re-exports it, and a task module may register its own handler, as dw/tasks/beats.py does, provided dw/tasks/task.py imports it (docs/TASKS.md Adding a task). The built-in handlers are grouped by family the same way - dw/tasks/audio_handlers.py, dw/tasks/video_handlers.py, dw/tasks/finish_handlers.py and dw/tasks/model_handlers.py (#790) - and dw/tasks/task.py keeps the general-purpose commands, the processor dispatch and Task. |
tests/test_task_discovery.py::TestDescribeTask::test_signature_becomes_the_schema, tests/test_task_registry_rules.py::test_every_derived_table_is_the_registrys_view, tests/test_task_registry_rules.py::test_every_static_check_is_registered_to_exactly_one_command, tests/test_task_registry_rules.py::test_every_media_argument_is_a_parameter_of_its_command |
| Task argument domains | dw/task_domains.py: check_arguments, task_argument_errors; dw/task_problems.py: face_track_errors, beats_errors, cuts_errors, lut_errors |
A numeric domain that a signature cannot express is declared on the command's @register_command(domains=...), and it is checked at validation and again at run time. A numeric argument is read by whole_number or real_number, the only numeric coercion in dw/tasks/, and validation applies the same rule (number_problem), so a value is refused at both or at neither: "3.0" is a whole number, "3.5" for one of whole_numbers is not, and true is no number (#774). Task.run coerces every declared argument through coerce_arguments before dispatch, so a handler sees a number, and an argument that has no range besides being a number declares finite. A command's cross-argument rules are its registered static_check, which task_argument_errors runs at validation (attribute_voices' voices_argument_errors, kept in dw/tasks/voice_attribution.py, among them). A literal enum choice (choice_errors) is refused at validation by the same module. The per-family rules past a single domain live in dw/task_problems.py, which reads task_domains' helpers and is never imported by it (task_argument_errors reaches them only through the registered static_check, so import_cycles stays 0; #790): the ingredients_grid image count against max_images (ingredients_grid_errors), and crop_face_track's cross-argument rules (face_track_problems, face_track_errors: crop size a multiple of the step's multiple (32 by default), remainder below modulus, gate_zero above gate_full, max_frames at or above min_frames, padding and confidence limits), which the task checks again at run time, as analyze_beats' are too (beats_problems, beats_errors: min_bpm below max_bpm, and anchors that are one kind per list, ascending, with whole beat indexes; tests/test_analyze_beats.py), and plan_cuts' (cuts_problems, cuts_errors: a bare-string transcript (transcript_problem, which plan_cuts also calls at run time), min_scene_s above max_scene_s, segment_by: "beat" without beats), and apply_lut's (lut_source_problem, palette_problem, lut_errors: exactly one of lut or palette, and a palette of 2..16 #rrggbb colours, re-checked at run time by check_lut_source), and check_face_detector_source vets its Hub repo and .onnx file before any download. select's index stays in task_domains with the other media-timing rules (select_index_problem, select_rule_problems: a whole number, at least 0, and below the candidate count when the count is known, i.e. at run time or when candidates is a written list). |
tests/test_task_domains.py::TestTheStaticPass::test_a_negative_frame_count_is_refused_at_its_path, tests/test_numeric_coercion.py::TestGradeExposure::test_a_string_number_grades_like_the_number, tests/test_task_domains.py::TestAtRunTime::test_slice_audio_refuses_a_negative_count, tests/test_ingredients_grid.py::TestStaticValidation::test_bad_layout_and_fit, tests/test_face_track.py::TestStaticValidation::test_bad_literal_is_an_error_at_its_argument, tests/test_face_track.py::TestCropFaceTrack::test_bad_arguments_are_refused_at_run_time, tests/test_face_track.py::TestStaticValidation::test_a_declared_grid_is_accepted, tests/test_face_track.py::TestDetectorSource::test_traversing_repo_is_refused, tests/test_select_validation.py::TestSelectErrors::test_an_out_of_range_literal_index_names_both_ends, tests/test_lut.py::TestPaletteAtValidate::test_both_refuse_naming_the_two, tests/test_lut.py::TestPalette::test_a_bad_palette_refuses_naming_the_entry |
| Variables | dw/variables.py: set_variables, argument_errors, resolve_variable_values; dw/validation.py: workflow_errors, ValidationContext.argument_errors |
argument_errors folds a caller's arguments in exactly as set_variables does when a run starts, so a bad name or value is refused before anything is queued. validation_errors(arguments=...) reports them itself, so dw.validate, /api/validate and a submit refuse one argument with one message; admit() only tells them apart from the definition's errors. |
tests/test_variables.py::test_argument_errors_reports_a_dict_passed_for_a_string_variable, tests/test_validation.py::test_an_argument_submit_refuses_fails_validation_with_its_message |
| List variables from the CLI | dw/run.py, dw/variables.py: get_value |
A name=value string given for a list variable is split on commas, so a list of objects can only be passed as JSON over the API or MCP. |
tests/test_variables.py::test_set_variables_list_default_splits_on_comma |
for_each expansion |
dw/for_each.py: expand_for_each, group_of |
Each entry becomes an ordinary step named <step>@<entry>, after substitution and before the reference check. |
tests/test_for_each.py::TestNaming::test_member_name_joins_with_at |
| Optional entry field | dw/for_each.py: _optional_item, _item_fields |
{"item": "field", "default": X} reads an entry field with a fallback; without default it is the plain required read, and list_fields / entry_field_warnings count the field as one entries take. The editor's string scan needs no twin code, only the shared case file. |
tests/test_for_each.py::TestOptionalItem::test_a_present_field_wins_and_an_absent_one_takes_the_default, tests/test_ui_twins.py::test_the_engine_decides_each_shared_reference_case |
| Previous results | dw/previous_results.py: get_iterations, previous_result_reference_errors |
Several previous_result: references in one step multiply into a cartesian product. |
tests/test_previous_results.py::TestGetIterations::test_multiple_references_create_cartesian_product |
| Previous-result check | dw/previous_results.py: previous_result_reference_errors, cross_entry_reference_errors; the sibling rule is dw/for_each.py: cross_entry_key (beside group_of, which it calls), which the cross-entry check, the earlier-step check and dw/video_extensions.py: video_into_image_errors all ask |
A literal reference to a step that is not earlier in the workflow is a validation error. A for_each member reading another entry of its own list (shot@a inside shot@b, either spelling, any .property) is a validation error too, cross_entry_reference_errors (#1101); same-key pairing, other groups and gather: stay legal. |
tests/test_previous_results.py::TestStaticReferenceChecking::test_a_step_cannot_reference_itself_or_a_later_one, tests/test_previous_results.py::TestCrossEntryReferences::test_the_prefix_spelled_in_an_entry_is_refused |
| Type conversion | dw/arguments.py: realize_args, dw/type_helpers.py: get_type |
A key ending _type or _dtype loads a Python object at realize time. The other spellings, and {} escaping, are in docs/WORKFLOW_GUIDE.md. |
tests/test_type_helpers.py::TestGetType::test_get_type_from_diffusers, tests/test_arguments.py::TestRealizeArgs::test_realize_escaped_type_reference |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Reference prefixes | dw/references.py |
Every reference prefix is spelled only in this module, and every other module goes through it. | docs/stabilization/baseline.json: prefix_literals, prefix_handling; tests/test_references.py::test_references_imports_nothing_from_dw |
| Explicit references resolve first | dw/arguments.py: _realize_explicit_reference |
asset:, output:, constant: and prompt: resolve in realize_args before any key-name convention. One exception, because only the chain resume may resolve it: an output: value under a key named resume_from (chain_rules.CHAIN_RESUME_FROM) is left as the reference, at any depth; and the variables pass (dw/workflow_run.py prepare_definition) leaves out of realize_args every variable a resume_from key in the authored steps names (chain_rules.resume_variables), whatever the variable is called, since the variables are realized before they are substituted into the chain block. |
tests/test_chain_resume.py::test_realize_args_leaves_a_resume_from_output_reference_for_the_resume, tests/test_chain_resume.py::test_any_variable_a_chain_resumes_from_keeps_its_output_reference |
| Reference confinement | dw/assets.py: resolve_asset_reference, dw/runs.py: resolve_output_reference, dw/library.py: LibraryPath, dw/security.py: validate_asset_reference, validate_output_reference |
dw/security.py validators confine an asset: path to its library and an output: path to the output root. |
tests/test_assets.py::TestReferences::test_a_symlink_out_of_the_library_is_refused, tests/test_security_symlinks.py::TestOutputs::test_an_output_reference_does_not_follow_a_linked_run_directory |
| Reference name shape | dw/reference_names.py: reference_name_errors |
The shape of every asset:, prompt: and output: name is checked before the queue, and existence is checked later. |
tests/test_reference_names.py::TestTheValidationPass::test_a_malformed_reference_is_refused_before_the_queue |
| Library reads | dw/library.py: LibraryPath |
Reads go through the roots front to back, so an earlier name shadows a later one. | tests/test_library_path.py::TestResolution::test_the_front_of_the_path_shadows_the_rest |
| Library writes | dw/library.py: LibraryPath |
Writes go only to the front root, so saving something opened from a read-only root writes a copy. | tests/test_library_path.py::TestConstruction::test_the_writable_root_comes_first, tests/test_library_sources.py::TestServer::test_saving_an_example_prompt_writes_a_copy |
| LoRA catalog matching | dw/lora_catalog.py: matches, workflow_bases |
An entry fits a base only by exact repo id (and H3 partition); no alias or family match. | tests/test_lora_catalog.py::TestMatching::test_the_base_must_match_exactly |
| Hub search | dw/lora_hub.py: search_hub |
The Hub is searched only by GET /api/loras/recommend, never raises, and offers no pickle-only repo. |
tests/test_lora_hub.py::TestCandidates::test_a_pickle_only_repo_is_dropped, tests/test_lora_hub.py::TestFailure::test_a_hub_error_is_returned_not_raised |
| Packaged builtins | dw/library.py: builtin_root, resolve_sub_workflow_reference |
builtin: names the packaged dw/workflows/, not the top-level workflows/ examples. |
tests/test_sub_workflow_resolver.py::TestTheResolver::test_a_builtin_resolves_in_the_packaged_root |
| Sub-workflow paths | dw/library.py: resolve_sub_workflow_reference, resolve_sub_workflow |
A sub-workflow path is confined to the root it resolves in. The search order is in the docstring of resolve_sub_workflow. |
tests/test_sub_workflow_resolver.py::TestTheResolver::test_a_path_escaping_its_root_is_refused |
| Workspace | dw/workspace.py: resolve_workspace, set_workspace |
--workspace beats DW_WORKSPACE, which beats the setting, which beats a working directory that looks like a workspace, which beats ~/diffusers-workspace. |
tests/test_workspace.py::TestResolution::test_a_flag_wins_over_everything, tests/test_workspace.py::TestResolution::test_a_bare_working_directory_falls_back_to_the_home_workspace |
| Workspace copy | dw/workspace_copy.py: copy_between |
Copies runs and asset-library entries between workspaces under one root: media hard-linked, sidecars always copied, never a symlink; an asset keeps its library-relative path and its note and kept-shots entry are merged into the destination folder's sidecars; all or nothing, a differing asset refusing runs too; vN renumbered by runs.copy_runs_in under the run lock; entries checked by security.validate_output_prefix. |
tests/test_workspace_copy.py |
| Realized workflow | dw/realize.py: realize_workflow, dw/runs.py: write_realized_workflow |
Each run directory holds workflow.json, with the run's arguments, seed and stored prompt text pinned. |
tests/test_realize.py::TestVariablesAndSeed::test_arguments_become_the_variable_defaults, tests/test_runs.py::TestRealizedWorkflow::test_the_run_directory_holds_a_realized_copy |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Run directories | dw/runs.py: open_run, workflow_identity, new_run_id |
Each execution writes into <output_dir>/<identity>/<run id>/ with a manifest.json, unless the flat layout is chosen (the test pins the flat-layout exception). |
tests/test_runs.py::TestRunDirectories::test_the_flat_layout_writes_where_it_always_did |
| A file's role (final / intermediate / input) | dw/roles.py |
Decided per step: result.role, then a final/intermediate subfolder, then data flow (a step a media-writing or unsaved consumer reads is intermediate). Engine records it at run end, with role_declared where result.role stated it; a run that predates roles recorded none, so the listing falls back on the file's own final/intermediate subfolder, then the demotion below, and a file neither places has none. At read time the gallery (GET /api/gallery, its role= filter, the metadata route) also demotes a file a later run of the same workspace read - an output: input (an unpinned latest is the newest run started before the reader) or an asset: keep_output kept from it - unless its role was declared (consumed_outputs, listed_role, once per listing; no manifest is rewritten). |
tests/test_roles.py, tests/test_roles.py::test_a_file_a_later_run_read_is_consumed, tests/test_roles.py::test_listed_role, tests/test_roles.py::test_listed_role_of_a_run_that_recorded_none, tests/test_gallery_roles.py::test_older_run_reads_its_intermediate_subfolder_as_intermediate, tests/test_gallery_roles.py::test_a_pre_roles_file_a_later_run_read_lists_as_intermediate, tests/test_gallery_roles.py::test_metadata_agrees_with_the_listing_for_an_older_run, tests/test_gallery_roles.py::test_a_file_a_later_run_read_lists_as_intermediate, tests/test_gallery_roles.py::test_a_declared_final_stays_final |
| Notes on runs, files, assets, workspaces | dw/notes.py |
A note is notes.json beside what it describes (run dir or asset dir; a file's own note, else the run's), or workspace.json at a workspace root. Only the server writes them, never the engine; at most 4000 characters, plain text. |
tests/test_notes.py |
| Run versions | dw/runs.py: open_run, run_versions, record_run_versions, copy_runs_in |
A run takes its number once, when it opens (or, copied in from another workspace, when copy_runs_in places it), so deleting a middle run leaves a gap rather than renumbering. |
tests/test_runs.py::TestRunVersions::test_a_deleted_middle_run_leaves_a_gap_rather_than_renumbering, tests/test_runs.py::TestRunVersions::test_runs_started_in_the_same_second_still_number_upward |
| Run-version surfaces | dw/runs.py: run_versions, dw/server/routes/gallery.py, dw/server/outputs.py, dw_mcp/tools_catalog.py: list_gallery, ui/src/lib/pages/GalleryPage.svelte |
Only dw/runs.py computes the number, and every other module reads it back. |
tests/test_runs.py::TestRunVersions::test_an_output_reference_can_name_a_run_by_its_version |
| What the server says about an output file | dw/server/outputs.py: output_annotations |
The gallery listing (dw/server/routes/gallery.py) and the metadata route (dw/server/routes/media.py) describe an output file's role, note, note_source, run_note, starred and version through output_annotations, and neither builds those fields on its own. |
tests/test_gallery_roles.py::test_metadata_agrees_with_the_listing_on_a_starred_noted_demoted_file |
| Result subfolders | dw/subfolders.py: subfolder_errors, dw/security.py: SUBFOLDER_PATTERN |
A step's result.subfolder is checked for shape at validation. |
tests/test_subfolders.py::TestValidationErrorsIntegration::test_validation_errors_reports_a_bad_subfolder |
| Template subfolder roles | workflows/templates/, dw/workflows/ |
Every saving step of a template is marked final or intermediate, with at least one final, and packaged builtins stay unmarked. |
tests/test_template_subfolders.py::test_every_saving_step_of_a_template_names_its_role, tests/test_template_subfolders.py::test_the_packaged_builtins_stay_unmarked |
| Frames carrying their rate | dw/media_types.py: FrameList (re-exported by dw/tasks/video_utils.py), rated_videos |
The one list of frames (or of videos) that carries .fps and .shots: a video loaded from a file (dw/argument_media.py) and a step that knows its rate but would return bare frame lists both answer it - h3_tiles' tiled refine videos, and every pipeline step whose pipeline reports its own fps (MiniMax-H3's property; dw/output_extraction.py: attach_video_frame_rate, called by pipeline.py's _run_once beside attach_audio_sample_rate, #1056) - through rated_videos, the batch and each video in it, so pair_audio and the result writer read getattr(video, "fps", None) instead of fitting and playing at the default 8 fps. A pipeline that reports no rate leaves its lists bare. No second class of that name. |
tests/test_h3_tiles.py::TestTileLoopReporting::test_the_videos_are_dws_one_frame_list, tests/test_pipeline_components.py::TestVideoFrameRate::test_a_dict_outputs_videos_carry_the_pipelines_rate, tests/test_pipeline_components.py::TestVideoFrameRate::test_a_pipeline_with_no_rate_leaves_the_lists_bare, tests/test_modular_output_properties.py::TestTheBasesOwnTakeSavesBesideIt::test_the_base_take_pairs_at_its_own_rate_with_no_padding, tests/test_video_utils.py (the FrameList cases) |
| Writing a video at its rate | dw/writers.py: encode_video, exact_frame_rate |
A fractional rate is written as the exact rational (30000/1001, 24000/1001, 60000/1001), never truncated to a whole rate, and no dw/ or dw_mcp/ module imports encode_video from diffusers.utils - every audio-carrying video write goes through dw.writers.encode_video (dw/result.py, dw/pipeline_processors/chain_spill.py - a chain's segments on disk, SegmentSpill, SegmentedFrames and SavedFrames, re-exported by chain.py). A declared fractional result.fps is therefore accepted by fps_errors. |
tests/test_fractional_fps.py::test_encode_video_with_audio_keeps_fractional_rate, tests/test_fractional_fps.py::test_no_dw_module_calls_the_diffusers_encode_video_directly, tests/test_fractional_fps.py::test_the_diffusers_hook_the_wrapper_swaps_still_exists |
| JSON records beside media | dw/media_types.py: JsonRecord, dw/result.py: Result.save_artifact |
A dict marked JsonRecord is saved whole as one .json, whatever content type the step declares, and is never split into one file per key. |
tests/test_face_track.py::test_json_record_is_saved_whole_under_a_video_content_type, tests/test_face_track.py::test_the_task_result_saves_crops_as_video_and_track_as_one_json |
| Reading a JSON record back | dw/locations.py: load_json_record |
A task reads a saved JSON record (fit_to_model's, crop_face_track's) only through load_json_record: the path is confined like any media path, then the extension must be .json, then the size is checked against MAX_JSON_SIZE, and only then is it parsed. fit._read_fit and face_track._read_track call it, never open. |
tests/test_locations.py::TestLoadJsonRecord::test_traversal_out_of_the_roots_is_refused, tests/test_locations.py::TestLoadJsonRecord::test_a_non_json_extension_is_refused, tests/test_locations.py::TestLoadJsonRecord::test_an_oversized_file_is_refused_before_it_is_read, tests/test_fit_to_model.py::TestRestore::test_fit_path_with_traversal_is_refused |
| Shots in a joined video | dw/shots.py |
Every AudioVideo constructor decides what to do with shots, and the sample spans are measured from the joined waveform. |
tests/test_shots.py::test_every_audio_video_constructor_site_is_decided, tests/test_shots.py::TestConcatVideosShots::test_an_overrun_track_is_measured_not_derived |
| Chain per-segment check and re-roll | dw/pipeline_processors/chain_check.py: check_segment, SegmentCheck, best_attempt, reroll_seed; the words measure is chain_check.check_words with the seam-head rule _seam_head, _opens_mid_line (a passed segment missing its opening words: line_opening_missing, which re-rolls via words_opening, #1308), WORDS_SIMILARITY_MIN (the ceiling for the chain check's words_similarity_min, default null = off, #1298), WORDS_FINDINGS, SEAM_TAIL_WORDS; the identity measure is chain_check.with_face_drop (face_drop against face_base, segment 1's frame 0 and its median match from frame 1 on, measured once), warn-only, behind the seam _face_scores (face_drift.face_similarities, which runs YuNet/SFace on the sampled frames only), ceiling face_drop_max default null = record only, recorded on the shot's check; a checked chain's face weights are in the plan's downloads_required (dw/plan_downloads.py: _chain_check_weights); lip sync is not a chain measure (ruling R9, bench/records/narrated.md) and lip_sync_min is an unknown key; the keys' rule is dw/pipeline_processors/chain_rules.py: chain_check_problem (torch-free, beside the look hold's chain_look_hold_problem); whether the audio is held, the floor and whether segment 1 is checked are dw/pipeline_processors/chain_config.py: chain_holds_audio, chain_audio_floor, chain_checks_segment_1, chain_checks_words, segment_line (the expected line from the chain's lines, else the <d> of the prompt) (pure: the run's ChainConfig and the cost estimate both ask them); the attempt loop is dw/pipeline_processors/chain.py: _checked_attempt, _render_attempt |
A checked segment is measured on the arithmetic analyze_seams uses (dw/tasks/assess.py: seam_measures, frame_measures) and dw/texture_bands.py, never a copy of it. Every attempt renders from the same _ChainState; nothing of an attempt (frames, audio join, shot, spill, look-hold state, segment 1's anchor or keystone) reaches the timeline until it is kept, and a re-roll runs on its own generator at reroll_seed, leaving the step's generator where attempt 0 left it. Segment 1 has no seam and is checked for the audio floor only (audio_dbfs against audio_dbfs_min, default -70; generated audio only, held audio is skipped); check_segment's hard_cut exemption is not passed by run_chain. The words measure (#1262) compares through script_check.check unchanged, on one line and no shots, and its seam-head rule takes the heard words from the Script check owners (normalize_words(numbers=False) + fold_numbers) and the head from script_check.align, never its own alignment; Whisper is transcribe_audio's cached model, levelled by script_take.level_for_transcription. The words measure runs only when words_similarity_min is set, on every segment with a line, and never when the audio is held. Absent check is off and absent retries is 0. The worst case is quoted by dw/cost_scaling.py: _worst_case_retries, N(1 + retries) renders for N segments of a chain that generates its audio, else 1 + (N - 1)(1 + retries), plus cost_scaling.words_check_term (one Whisper CPU pass per worst-case attempt. A failed words check on a segment past the first in a guide chain, one with a head trim, transcribes its untrimmed audio once more, to name line_clipped_at_head (chain.py: _segment_words; chain_check.py: _clipped_at_head, #1300); that second pass is not priced, as it runs only on a failure, on a model the first pass warmed, inside the worst case's margin); on every basis, the curated row's and, through dw/plan.py _with_worst_case, an observed or inherited history's too. |
tests/test_chain.py::TestCheckAndReroll::test_a_failed_segment_is_rerolled_with_a_new_seed_from_the_same_carry, tests/test_chain.py::TestCheckAndReroll::test_budget_exhausted_keeps_the_best_attempt_and_warns, tests/test_chain.py::TestCheckAndReroll::test_absent_check_is_todays_chain, tests/test_chain_check.py::test_the_estimate_quotes_the_worst_case_retries, tests/test_chain_check.py::test_an_inherited_estimate_quotes_the_worst_case_retries, tests/test_chain_check.py::test_the_estimate_and_the_run_agree_on_segment_1, tests/test_chain_check.py::TestAudioFloor::test_held_audio_is_never_measured, tests/test_chain.py::TestAudioFloorReroll::test_a_silent_segment_1_is_rerolled, tests/test_chain.py::TestAudioFloorReroll::test_a_silent_segment_2_only_warns_with_check_off, tests/test_chain_words.py::test_seam_head_word_of_previous_line_is_dropped, tests/test_chain_words.py::test_seam_head_number_folded_across_chunks_is_dropped, tests/test_chain_words.py::test_invented_head_word_is_kept, tests/test_chain_words.py::test_line_clipped_at_end_fails, tests/test_chain_words.py::test_line_clipped_at_head_names_a_line_opened_in_the_trimmed_head, tests/test_chain_words.py::test_no_head_finding_when_the_line_opens_after_the_trim, tests/test_chain_words.py::test_passed_segment_missing_its_opening_words_is_reported, tests/test_chain_reroll.py::test_opening_missing_rerolls_while_retries_remain, tests/test_chain_check_face_sync.py::test_face_drop_max_sets_a_failing_ceiling_without_reroll, tests/test_chain_check_face_sync.py::test_a_model_that_cannot_load_records_none, tests/test_chain_check_face_sync.py::test_face_base_excludes_frame_0, tests/test_chain_check_face_sync.py::test_faceless_segment_1_logs_once, tests/test_chain_check_face_sync.py::test_checked_chain_lists_face_weights_in_downloads, tests/test_chain_check_face_sync.py::test_lip_sync_is_not_a_chain_measure |
| Chain length and audio-source rules | dw/pipeline_processors/chain_rules.py: chain_length_problem (exactly one of segments, match_audio, duration_seconds) and match_audio_source_problem (a match_audio step names a hold_audio track or a references list); the run asks them in dw/pipeline_processors/chain_config.py: _chain_length, _plan_from_audio, _find_audio_reference; validate asks the same two in dw/step_value_checks.py: chain_length_errors, after variable substitution |
One predicate and one message per rule: the run and validate never restate either. The run alone counts the audio entries in references, on the resolved references. |
tests/test_chain.py (the match_audio and chained-segments cases), tests/test_validation.py registry order |
| Chain frame snap for a length-planned chain | dw/pipeline_processors/chain_rules.py: chain_needs_frame_snap; dw/pipeline_processors/chain_config.py: derive_frame_snaps, called from dw/workflow.py: Workflow._expand before substitution; validate's warning is dw/guides.py: frame_snap_warnings |
A duration_seconds or match_audio chain with no frame_snap takes its num_frames variable's modulus constraint as constraint:<name> before variables substitute, and the run logs it; one it cannot derive is a validate warning at chain.frame_snap, never a silent off-grid last segment. |
tests/test_chain_duration.py::TestDerivedFrameSnap::test_a_block_with_no_frame_snap_derives_it_and_says_so, tests/test_chain_duration.py::TestDerivedFrameSnap::test_validate_warns_on_a_length_planned_chain_it_cannot_snap |
| Chain look hold | dw/pipeline_processors/chain_look.py: LookHold, pickup_grade, ease_creep, ease_texture, precompensate, is_cut; built per chain in dw/pipeline_processors/chain.py |
A finished segment's frames are corrected before its carry is taken, so the next segment reads the corrected frame. A pickup the quantile gap calls a cut (is_cut) or a flat frame is never graded, and creep is only eased down, never brightened. Absent look_hold is today's chain. Numpy and torch only. |
tests/test_chain.py::TestLookHold::test_look_hold_off_is_todays_chain, tests/test_chain.py::TestLookHold::test_look_hold_pulls_a_later_segment_onto_the_carried_frame, tests/test_chain_look.py::test_pickup_grade_skips_a_cut_and_a_flat_frame |
| Chain carry pick | dw/pipeline_processors/chain_pick.py: sharpness, sharpest_index; called from dw/pipeline_processors/chain_continuity.py |
carry_pick: {"sharpest": N} takes the sharpest of a segment's last N frames (variance of a 3x3 Laplacian on luma) as the next keyframe and ends the segment there. Ties go to the latest frame, and a window of 1 is the last frame, today's chain. Numpy only. |
tests/test_chain_pick.py::test_a_blurred_last_frame_loses_to_a_sharp_one_before_it, tests/test_chain_pick.py::test_window_one_is_always_the_last_frame, tests/test_chain_pick.py::test_ties_go_to_the_latest_frame |
| Overlap windows of a long video | dw/tasks/windows.py: window_video, window_span |
Window i covers source frames [i*stride - overlap, i*stride + stride), its synthetic frames repeat the first or last frame, and its audio is cut on the source's own frames_to_samples boundaries, so adjacent windows' strides tile the track with no sample lost or repeated. |
tests/test_window_video.py::TestAudio::test_strides_tile_the_source_track_exactly, tests/test_window_video.py::TestRealPath::test_a_path_argument_keeps_its_audio, tests/test_window_video.py::TestFileMatchesInMemory::test_every_window_equals_the_in_memory_window, tests/test_read_frame_range.py |
| Keep a span of a video's frames and audio | dw/tasks/trim.py: trim_video; dw/shots.py: trimmed_shots |
Frames [start_frame, start_frame + num_frames) are kept and the audio is cut on frames_to_samples of each end at the track's own rate, so consecutive trims tile the track. A span past the clip's end is refused, never shortened. Shots are clipped to the span, sample side cleared. |
tests/test_trim_video.py::TestFramesAndAudio::test_consecutive_trims_tile_the_track, tests/test_trim_video.py::TestShots::test_shots_are_clipped_across_the_boundary |
| Repair one chain segment in place (#1169) | dw/tasks/chain_repair.py: repair_geometry (pure), chain_repair_cut, chain_repair_splice; dw/pipeline_processors/chain.py (segment_file on a kept segment's shot); workflows/templates/minimax/repair-segment.json |
The repair render holds segment k-1's tail at frame 0 and segment k+1 at its end (at num_frames - tail_frames, always 17j). The splice cuts the take itself, never the segment files: outside the repaired span the output is the take's frames, frame count and track. The segment files are found from the take's shots, not by name. Segments 2 to N: the last has the head guide only. A match_audio take's render holds the take's own track sliced under its frames, its guides picture only. The repaired seams are not hard_cut, so analyze_seams measures them. |
tests/test_chain_repair.py::TestGeometry::test_the_splice_keeps_the_take_length_and_the_grid, tests/test_chain_repair.py::TestSplice::test_the_take_outside_the_span, tests/test_chain_repair.py::TestSplice::test_a_duration_take_keeps_its_frame_count, tests/test_chain_repair.py::TestCut::test_matched_take_repair_holds_the_master_slice, tests/test_chain_repair.py::TestCut::test_last_segment_repair_has_no_tail_guide, tests/test_chain.py::TestSaveSegments::test_kept_segment_files_are_named_on_the_take_shots |
| Resume a chain from its kept segments | dw/pipeline_processors/chain_resume.py: load, check_plan, kept_count, warn_resumable; the rules are dw/pipeline_processors/chain_rules.py: chain_resume_problem, resume_reference_problem (validate asks them in dw/guides.py _chain_rule_findings, the run in ChainConfig); dw/pipeline_processors/chain.py only calls the hooks; the estimate is dw/cost_scaling.py: _chain_render_scaling |
A resume never changes the plan: every kept file must hold the frames its segment puts on the timeline and k must be under N, refused by check_plan before anything is copied. The kept files are this iteration's own siblings of the named one (own_segments, by the spill's iteration number), found by chain_repair.segment_files, copied (never moved) into the new run's spill, and the carry is read off the last of them. resume_from is an output: reference only and reaches the resume unresolved, whatever variable carries it (realize_args and the variables pass leave it; see Explicit references resolve first); a bare path is refused in the location policy's words. A chain that fails after spilling a segment reports the first file's reference as a chain_resume_hint job warning; a failed run's files resolve by name without its manifest. The estimate prices N - k renders, every rendered one checked; an unresolvable reference prices N and says so. |
tests/test_chain_resume.py::test_a_last_frame_chain_resumes_after_k_kept_segments, tests/test_chain_resume.py::test_resume_refuses_a_kept_segment_of_another_length, tests/test_chain_resume.py::test_kept_files_are_copied_not_moved, tests/test_chain_resume.py::test_a_bare_path_is_refused_as_the_location_policy_refuses_one, tests/test_chain_resume.py::test_a_fan_out_resumes_each_iteration_from_its_own_files, tests/test_plan.py::TestResumedChainEstimate::test_a_resolvable_resume_prices_the_rest |
| Rerun with variable overrides | dw/server/jobs.py: JobManager.rerun_spec merges the overrides over the stored arguments; dw/server/job_record.py: keep_drawn_seed; the route is dw/server/routes/jobs.py: rerun_job, _bound_plan_for, with dw/server/admission.py: check_bound_acknowledgement |
Admission and the bound acknowledgement check run on the MERGED arguments, never the stored ones: an undeclared override is a 400, the seed variable with new_seed is a 400, and a bound acknowledgement must name the merged plan. Without new_seed the rerun keeps the original's seed, a drawn one read from its realized workflow; when that is gone the job carries a rerun_seed_unknown warning. |
tests/test_server.py::test_rerun_overrides_merge_onto_the_stored_arguments, tests/test_server.py::test_rerun_override_of_an_undeclared_variable_is_refused, tests/test_server.py::TestBoundRerun::test_overrides_that_change_the_plan_refuse_the_original_acknowledgement, tests/test_server.py::TestBoundRerun::test_an_acknowledgement_of_the_merged_arguments_queues, tests/test_server.py::test_a_rerun_with_overrides_keeps_the_parent_s_drawn_seed, tests/test_server.py::test_a_rerun_with_overrides_says_so_when_the_drawn_seed_is_unknown |
| LUFS level match on a join | dw/tasks/joins.py: match_levels, _match_target, _lufs_or_rms, DEFAULT_MATCH_LUFS, LUFS_TRUE_PEAK_CEILING_DBTP |
match_levels: "lufs" moves each shot to the integrated-loudness target and holds a gain at the -1 dBTP true-peak ceiling, saying so (match_levels_held) rather than clipping; a shot under 0.4 s is matched by rms at its default and named (match_levels_short_shot). A sample rate is required only when some shot has a soundtrack to measure. It evens the shots; delivered loudness is normalize_audio's. |
tests/test_match_levels_lufs.py::test_a_gain_that_would_pass_the_true_peak_ceiling_is_held_and_named, tests/test_match_levels_lufs.py::test_a_shot_too_short_for_lufs_is_matched_by_rms_and_named, tests/test_match_levels_lufs.py::test_a_lufs_match_of_only_silent_entries_needs_no_sample_rate, tests/test_match_levels_lufs.py::test_concat_joins_videos_with_no_soundtrack_at_all_under_lufs |
Take attribution in check_narration |
dw/tasks/narration_check.py: align, _owners, check_narration |
Every dropped or inserted word is charged to one take: a word that replaces expected words to the take of the words it replaces; a pure insertion to the take it sits in front of (the last take after them all), except one at a take boundary whose words repeat the end or the beginning of the take before it, which is charged to that take - so a looped take, or one restarted and abandoned, fails itself, whichever copy the alignment matched, and the takes after it still align. | tests/test_narration_check.py::test_a_misheard_last_word_is_blamed_on_its_own_take, tests/test_narration_check.py::test_a_pure_insert_at_a_boundary_belongs_to_the_next_take, tests/test_narration_check.py::test_a_looped_take_shows_as_inserted_and_later_takes_still_align, tests/test_narration_check.py::test_a_repeat_aligned_after_its_first_copy_belongs_to_the_take_it_repeats, tests/test_narration_check.py::test_a_restarted_and_abandoned_take_is_charged_to_that_take |
| Narration retime and segment layout | dw/tasks/narration_retime.py: word_map, kept_span_ends, retime_narration, narration_lines |
A pause is set by the class of the punctuation it follows (comma, sentence, section - the script's when text is given, Whisper's otherwise) and a gap under min_gap_seconds is left alone; each chain boundary is a plan_segments kept-span end (segment 1 at segment_frames, then every segment_frames - guide_frames), a boundary at any word junction (one word's end to the next's start, zero-length included) stays, and one inside a word has the nearest junction moved onto it by changing the nearest eligible gap before that junction by at most boundary_slack_seconds, or is reported in mid_word_boundaries, never forced; inserted silence is the source's own room tone (find_loop_bed's best, else its quietest window, looped), never digital zero; a segment's line is the words whose midpoint falls in its kept span. |
tests/test_narration_retime.py::test_boundaries_land_in_gaps_on_the_chain_grid, tests/test_narration_retime.py::test_boundary_inside_a_long_word_is_reported_not_forced, tests/test_narration_retime.py::test_boundary_at_a_word_junction_needs_no_move, tests/test_narration_retime.py::test_boundary_inside_a_word_moves_to_the_nearest_junction, tests/test_narration_retime.py::test_comma_and_sentence_gaps_take_their_class_lengths, tests/test_narration_retime.py::test_inserted_gap_is_room_tone_not_zero, tests/test_narration_retime.py::test_lines_split_words_by_kept_span_midpoint |
metadata.dw.source on exported clips |
dw/tasks/timeline_export.py: _source_reference, _clip, summarize_bundle |
A clip whose media is a file inside a run directory of the output root carries its output: reference as metadata.dw.source (each source's own in sources mode), what chain_take and chain_output take; a file outside any run writes no source, and a bundle without one summarizes with source: null. |
tests/test_timeline_export.py::test_clips_carry_the_exported_videos_output_reference, tests/test_timeline_export.py::test_sources_mode_records_each_sources_own_reference, tests/test_timeline_export.py::test_a_video_outside_any_run_writes_no_source, tests/test_timeline_export.py::test_a_bundle_without_clip_metadata_summarizes_with_no_source |
| Join processed windows back to the source's length | dw/tasks/windows.py: join_windows; dw/tasks/video_utils.py: local_path_or_video (the source's file or loaded video, shared with h3_latent_upscale's take reader); dw/task_domains.py: window_count, window_count_problem, join_windows_errors |
Output is exactly the source's frames, each once: later windows' first overlap frames blend over the previous window's last on the open ramp dissolve_videos uses. The window count is window_count's ceil(source_frames / stride) and nothing else (it is also window_video's last index plus one). Audio is the source's own track; shots are one per window with overlap_frames on each seam, their sample side laid over that track by dw/shots.py's remeasured_shots (the same rounding as frames_to_samples, so the spans are cumulative). |
tests/test_join_windows.py::TestRoundTrip::test_unprocessed_windows_rejoin_to_the_source, tests/test_join_windows.py::TestShots::test_spans_partition_the_frames_and_tile_the_audio, tests/test_join_windows.py::TestUint8Output::test_matches_the_float32_accumulator_exactly |
A join_windows list of the wrong length, before the run |
dw/window_count_errors.py: window_count_errors, wired into validation_errors |
Owned by the task, never a template: any join_windows step whose source is probeable header-only (asset:/output: or a readable literal path), with literal num_frames/overlap and an expanded videos list, is checked against window_count_problem - the same rule the run refuses with. Anything not knowable stays silent for the run. |
tests/test_window_count_errors.py |
| Fit to a model and restore | dw/tasks/fit.py: fit_to_model, restore_to_source |
fit_to_model's record is the whole of what restore_to_source needs: the scale is inferred from the output's size and must be uniform, and restore returns the source's size times that scale and at most its frame count. | tests/test_fit_to_model.py |
| Shared image ops | dw/tasks/image_ops.py: LUMA_WEIGHTS, luma, split_alpha, join_alpha, per_frame |
grade, finish and lut take luma (float64 Rec. 709 weights, cast to the input's dtype) and alpha handling from image_ops, never from each other, so an RGBA input comes back RGBA with its alpha untouched; lut.LUMA_WEIGHTS is a re-export. An image command runs over a video or frame list only through per_frame. A media file is read by task._load_media with fetch_image(keep_alpha=True), so a file with transparency arrives RGBA; a pipeline input still flattens to RGB. |
tests/test_image_ops.py::TestLuma::test_lut_re_exports_the_one_float64_weights, tests/test_image_ops.py::TestAlpha::test_rgba_round_trip_keeps_the_alpha_bytes, tests/test_image_ops.py::TestOutputsAreByteIdentical::test_every_command_matches_the_pre_refactor_bytes, tests/test_image_ops.py::TestPerFrame::test_a_video_comes_back_as_a_video_with_the_same_frames_and_audio, tests/test_finish.py::TestRgbaFileKeepsAlpha::test_a_file_path_comes_back_rgba_with_its_alpha |
| Audio level of a deliverable | dw/audio_qc.py: warn_without_headroom, warn_if_written_above_full_scale, all_shots_silent_by_design |
A muxed video keeps the pre-encode headroom prediction and falls back to it only when the post-write probe cannot measure the file. A result a later step resets the level of (consumed_by_normalizer, dw/step_cache.py) draws neither the post-write warning nor the fallback. |
tests/test_result.py::TestNoHeadroom::test_a_clean_video_mux_drops_the_stale_prediction, tests/test_result.py::TestNoHeadroom::test_an_unprobeable_video_mux_falls_back_to_the_prediction, tests/test_result.py::TestNoHeadroom::test_a_video_a_join_level_matches_draws_no_headroom_warning, tests/test_result.py::TestTheWrittenLevel::test_a_file_that_decodes_above_full_scale_warns |
| Level-spread warning on a replaced track | dw/step_cache.py: audio_replaced_downstream, dw/tasks/joins.py: warn_on_level_spread |
A join whose track a later pair_audio takes as its video draws no level_spread warning; being that step's audio argument doesn't count. |
tests/test_step_cache.py::test_level_spread_warning_suppressed_when_audio_replaced, tests/test_step_cache.py::test_audio_replaced_downstream_false_when_it_is_the_audio_or_unread |
| Template audio-level placement | workflows/templates/minimax/music-video.json, workflows/templates/minimax/music.json |
normalize_audio (-3 dBFS) acts only on the track that goes into the final file, so the slices that condition the shots are untouched. music-video slices each shot lead_frames early and trim_video drops the run-up before the join. |
— |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Validation registry | dw/validation.py: ERROR_CHECKS, WARNING_CHECKS, run_checks |
A check is one Check in a registry, and a check that raises becomes one internal finding while the rest still run. |
tests/test_validation.py::TestExceptionPolicy::test_a_raising_check_is_one_internal_error_and_the_rest_still_run |
| Step-value checks | dw/step_value_checks.py: fps_errors, cache_errors, null_media_errors, select_errors, chain_prompts_errors, chain_length_errors |
The only caller of these checks is the error registry. | dw/validation.py: ERROR_CHECKS |
| Command result type | dw/scalar_result_validation.py: scalar_result_errors; a command declares it on register_command (returns, result_content_type) |
A command's required result content_type (application/json for a json command, result_content_type for a fixed container such as export_timeline's .otioz) is checked here from its own declaration, never by a second walk or a command-name match elsewhere. |
tests/test_content_types.py::TestExportTimelineContentType::test_a_wellformed_but_wrong_type_is_refused_naming_otioz |
| Admission | dw/server/admission.py: admit |
Validate, submit, rerun and enhance load and check a request once, and JobManager.submit does not check it again. |
tests/test_admission.py::test_a_submit_expands_once |
| Presets | dw/presets.py: apply_presets, effective_variables, preset_errors |
A presets bundle sets the variables the caller did not pass, through set_variables right before the caller's own arguments fold, so an explicit argument wins and a bundle value is validated like a caller's. The fold applies the default bundle with no usable arguments. Every reader of a written definition's variables for some arguments (constraint checks, argument warnings, host-RAM key, VRAM inheritance, observed-cost buckets, the estimate's defaults) goes through effective_variables; a reader of an expanded definition reads its folded variables. |
tests/test_presets.py::test_a_caller_value_beats_the_bundle, tests/test_presets.py::test_no_arguments_and_the_default_selector_have_one_fingerprint, tests/test_presets.py::test_a_bundle_value_breaking_a_rule_is_an_error_at_the_bundle |
| Variable constraints | dw/variable_constraints.py: apply_constraints; the grid arithmetic aligned / aligned_down |
A model's rule about a value is declared in the workflow's variable_constraints, and it is checked at validation and again before anything loads. plan_cuts (and its static check) rounds render lengths through aligned / aligned_down, never its own copy, so a planned num_frames is one the template's constraint accepts; crop_face_track pads its crops to the step's modulus/remainder grid through aligned too (padding_to_grid). |
tests/test_variable_constraints.py::TestTheGrid::test_without_snap_an_off_grid_value_is_refused_not_rounded, tests/test_variable_constraints.py::TestTheGrid::test_plan_cuts_rounds_through_the_constraint_owner, tests/test_face_track.py::TestPadding::test_padding_to_grid_rounds_through_the_constraint_owner, tests/test_variable_constraints.py::TestTheRunTimePass::test_an_illegal_value_is_refused_before_anything_loads |
| Bound acknowledgement | dw/plan.py: fingerprint, dw/server/admission.py: check_bound_acknowledgement |
A bound acknowledged_cost is refused with 409 when the plan's fingerprint or its required downloads changed. The fingerprint is keyed only by what can change the output: DOCUMENTATION_KEYS (cost, cost_drivers, vram_estimate, descriptions, catalog shape/traits), a constraint's reason and a result's role, subfolder and file name are left out, of the workflow and of each composed child; every schema key is sorted into left out or kept. A number digests by value, not spelling (12.0 as 12), and the seed stays out (#85). |
tests/test_plan.py::TestFingerprintKeys::test_the_exclusion_lists_are_pinned, tests/test_plan.py::TestTheCatalogH3Template::test_a_client_round_trip_keeps_it, tests/test_plan.py::TestFingerprintKeys::test_every_top_level_schema_key_is_sorted, tests/test_server.py::TestBoundRerun::test_a_rerun_bound_to_a_stale_plan_is_refused, tests/test_server.py::TestBoundAcknowledgement::test_a_matching_fingerprint_queues |
| Per-repo gating | dw/plan_downloads.py: _model_info, _probe_gate_blocked, assembled by dw/plan.py: build_plan |
Each gated repo, including a loras entry's repo, is probed and reported on its own. |
tests/test_plan.py::TestDownloadsRequired::test_a_gated_repo_this_token_lacks_access_to_is_blocked |
| Download sources | dw/plan_downloads.py: downloads_required, _collect_sources, _collect_component_overrides, _sub_workflow_paths (which plan.estimate walks children by too; the module never imports dw/plan.py back, and plan.py re-exports downloads_required and gate_warnings) |
downloads_required lists every model_name, loras repo (a loras entry naming a resolved weight_name is listed instead as a {repo, filename, revision} file entry built by file_entries, sized as that file), single-file URL and load_components.pretrained_model_name_or_path override (string, per-component dict, default), once each. The base repo stays listed whole: an override's replaced subfolder is not subtracted, so the list is a floor. |
tests/test_plan.py::TestDownloadsRequired::test_a_load_components_repo_override_is_listed |
| Task weights | each fetching task's *_source(arguments) (e.g. dw/tasks/remove_object.py: remover_source); dw/tasks/weights.py: argument_or_default, file_entries, package_dir; dw/tasks/registry.py: required_weights_hook, register_weights_fallback; dw/tasks/image_utils.py: processor_weights_hook; dw/security.py: validate_weight_name; dw/plan_downloads.py: _collect_task_weights, _file_info |
A model-backed task declares required_weights(arguments) on its register_command, and downloads_required lists those repos beside the pipeline model_names (a shared repo once; a reference or local-path argument none). A single-file task lists {repo, filename, revision} entries, checked per file against the cache; restore_faces also lists facexlib's weights as URL entries. A task's declaration and its loader read the same resolution of which argument wins (a single-file task's *_source(arguments), an overridable whole-repo task's argument_or_default call, a fixed repo's module constant), so they cannot differ; argument_or_default is the shared helper only where None means the default (remover_source and upscaler_source treat any falsy value as absent; spandrel_source has no default). file_entries builds a file entry (its name checked by validate_weight_name), package_dir finds facexlib's weights dir without importing it, and _file_info sizes one file. The lookup path is acyclic: dw/locations.py reads the registry's tables and the task modules import the location policy and weights.py, so the registry imports no task module (an image processor's hook reaches it through register_weights_fallback, which image_utils calls at import) and weights.py imports only dw/security.py, where validate_weight_name lives for that reason. A task module the hook lookup imports keeps transformers out of its module top (hf_pipeline is a lazy wrapper, patched by name in the tests), so a validate never pays for it. |
tests/test_task_weights.py, tests/test_task_weights.py::TestTheHookPathStaysAcyclic::test_weights_imports_only_the_security_leaf_from_dw, tests/test_task_weights.py::TestTheHookPathStaysAcyclic::test_the_registry_imports_nothing_from_dw |
| Observed cost | dw/server/observed_cost.py: declared_drivers, ObservedCosts._on_this_card, ObservedCosts.card |
observed comes only from this box's finished jobs on the named card (the one route names, for plan.estimate), else the server's own card; ObservedCosts.card owns a card's (kind, name), including the default card's name. Rows are matched by card name whatever index they ran on (a job from before jobs carried a device counts as the default card's name, so it matches any card of that model), and never writes cost. Cold and warm runs are reported separately. |
tests/test_observed_cost.py::TestColdIsNotWarm::test_the_two_are_reported_separately_each_with_its_runs, tests/test_observed_cost.py::TestByCard::test_only_rows_from_this_card_count_whichever_index_they_had, tests/test_observed_cost.py::TestByCard::test_rows_from_before_jobs_carried_a_device_do_not_count_for_another_model, tests/test_observed_cost.py::TestByCard::test_rows_from_before_jobs_carried_a_device_count_on_the_default_card |
| The estimate | dw/plan.py: estimate, _tempered; the driver scaling is dw/cost_scaling.py: scale_curated, _driver_scaling |
plan.estimate quotes the observed figure ahead of the curated one, and blends it toward the curated figure below three runs. A cost entry is measured at the workflow's variable defaults; a run moving num_inference_steps, width/height, num_frames or a driver the entry names in scales_with (per list entry, summed) scales it in proportion (scale_curated, which dw/server/deps.py reuses for an inherited figure), basis derived with scaled naming each axis, times the list-length re-price, and that scaled figure is what a thin observed one blends toward (#866). A cost entry's optional fixed_minutes (model load, warm-up, decode setup) is held out of the scaling: fixed + (minutes - fixed) x factor (fixed_minutes reads the entry _choose_entry picked, the one _price quotes from; _scaled; #1224). Observed history scaled by a driver or by a checked chain's worst-case re-rolls holds the same entry's fixed share of itself (fixed_share: fixed_minutes / minutes), not its minutes, since this box can be far faster or slower than the curated card. A preset tier with no history of its own, or too thin a bucket, is priced from (or blended toward) this box's observed default-tier figure scaled by the same rule instead, a driver the tier moves that nothing scales named in differs (#1236). |
tests/test_plan.py::TestObservedEstimate::test_history_beats_a_curated_figure, tests/test_plan.py::TestLowConfidenceObservedEstimate::test_a_single_run_blends_toward_the_curated_figure, tests/test_plan.py::TestDriverScaling::test_steps_canvas_and_frames_multiply, tests/test_plan.py::TestScalingAccuracy::test_within_thirty_percent, tests/test_plan.py::TestTiersPriceFromTheObservedDefault::test_a_thin_draft_bucket_tempers_toward_the_observed_default, tests/test_plan.py::TestDeclaredProportionalDrivers::test_a_declared_scalar_scales_in_proportion, tests/test_cost_fixed_minutes.py::test_the_fixed_share_is_not_multiplied, tests/test_cost_fixed_minutes.py::test_the_fixed_share_comes_from_the_entry_price_quoted, tests/test_chain_check.py::test_an_observed_worst_case_holds_the_fixed_share |
| Cost drivers | dw/server/observed_cost.py: declared_drivers |
A cost driver that names no declared variable is dropped. | tests/test_observed_cost.py::TestComparability::test_a_driver_naming_no_variable_is_dropped, tests/test_observed_cost.py::TestTheCatalogsDriversAreReal::test_every_declared_driver_is_a_variable_of_its_workflow |
| Raw workflow GET | dw/server/routes/library.py: get_workflow |
The raw workflow GET is served verbatim, with no observed, because the editor saves what it reads. |
— |
| VRAM projection | dw/vram_estimate.py, dw/guides.py: guide_lengths |
A template's vram_estimate is projected per step after for_each expansion, and only the largest step over the ceiling is reported. With bytes_per_guide_voxel declared, each guide the step lays in (its own non-null guides plus a guide chain's) adds its snapped frames times the canvas; an unprobeable clip is charged at num_frames, and validate, admission and run project the same number. |
tests/test_vram_estimate.py::test_only_the_largest_over_ceiling_step_is_reported_once_per_cost_entry, tests/test_vram_guides.py::test_validate_apply_and_required_agree_on_a_guided_definition |
| H3 Ref2VA VRAM numbers | workflows/templates/minimax/ |
Every Ref2VA template declares base_gb 16.0, bytes_per_voxel 28.71 and gb_per_reference 1.0. |
tests/test_h3_vram_ceiling.py::test_every_ref2va_minimax_template_declares_gb_per_reference |
| Inherited VRAM ceiling | dw/vram_inheritance.py, dw/server/deps.py: ceiling_index (the catalog parse is _catalog_listing, shared with the cost lookup below) |
A workflow with no vram_estimate is matched against the catalog by pipeline identity, and is warned, never refused. Under the ceiling it still gets a vram_config_differs warning when its offload or quantization differs from the template's, which the estimate was calibrated on (#954). |
tests/test_vram_inheritance.py::test_every_template_declaring_one_identity_declares_the_same_numbers, tests/test_vram_inheritance.py::TestConfigDiffers::test_missing_offload_under_the_ceiling_notes_it |
| Inherited cost | dw/server/deps.py: inherited_observed, dw/cost_scaling.py: chain_segment_scaling (the chain-segments ratio, shared with the catalog figure), dw/vram_inheritance.py: inherited_differences, dw/plan.py: _observed (basis inherited), dw/server/routes/jobs.py: _inherited_cost_warnings |
An inline workflow (no catalog name) is priced from the observed runs of the catalog template with the same component_type + model_name + workflow key as vram_inheritance; the template with the most cold runs wins, then name. differs lists offload, quantization, frame count, attention backend, cache, LoRA and reference count (the references list lengths, per-entry item: ones read off the for_each list; named, not scaled, #915) set differently from the template, and sets low_confidence when it lists any (#866). More pipeline steps than the template run (a for_each counted per member, against the list the bucket ran, so a fallback to the default bucket re-prices a longer list) re-price its minutes linearly, a chain's segment count other than the bucket's scales them by chain_segment_scaling (#1305), and the mean frames, inference steps, width and height per pipeline step are scaled by scale_curated itself - the one rule the catalog figure uses, so an inline copy and its template quote one request alike - with scaled naming why and low_confidence set (#851, #866). It is warned, never refused. |
tests/test_inherited_cost.py::TestPricedByPipelineIdentity::test_an_inline_copy_inherits_the_templates_runs, tests/test_inherited_cost.py::TestDifferences::test_offload_quantization_and_frames_are_named, tests/test_inherited_cost.py::TestDifferences::test_a_reference_count_other_than_the_templates_is_named, tests/test_inherited_cost.py::TestThePlanReportsIt::test_the_estimate_is_inherited_and_the_validate_answer_warns, tests/test_inherited_cost.py::TestTheTemplateAndAnInlineCopyAgree::test_the_same_minutes_within_rounding, tests/test_inherited_cost.py::TestChainSegmentsScaleTheInheritedFigure::test_the_catalog_rule_and_the_inherited_one_agree, tests/test_inherited_cost.py::TestAListLongerThanTheFallbackBucket::test_an_inline_six_shot_copy_scales_by_members |
| Projected host-RAM peak | dw/server/host_ram_need.py: memory_key, projected_peak, declared_mb, host_too_small, host_memory_block; the fit is dw/host_memory_projection.py: projected_mb; history by key is dw/server/job_history.py: peaks_for_key |
A run's memory key is the sorted pipeline identity of its pipeline steps plus every quantization_config / offload / group_offload value, with arguments substituted through replace_variables; never the workflow's name, so a renamed or inline copy shares the template's history. observed is the max peak over this machine's successful same-key runs and comes ahead of the list-run fit, unless the shape holds its pipeline across the list and the fit is the larger (the key leaves list length out, #910); with neither there is no figure. A workflow's top-level host_ram_gb (a number or variable: ref, set per tier by the presets.tier bundles of the H3 templates; declared_mb) is a floor under it, basis declared, and admission refuses a run whose declared need plus margin exceeds the host's total RAM (host_too_small, #1227). An observed or fitted figure never refuses, only the declaration does; dispatch may delay a start on it (dw/server/dispatch.py: Dispatcher._next_dispatch, #869) (Admission.host_ram_need, validate's plan.host_memory). |
tests/test_host_ram_need.py::TestMemoryKey::test_an_inline_copy_without_its_metadata_has_the_same_key, tests/test_host_ram_need.py::TestMemoryKey::test_a_different_weights_dtype_is_a_different_key, tests/test_host_ram_need.py::TestProjectedPeak::test_an_observed_peak_beats_the_fit, tests/test_host_ram_need.py::TestALongerListThanObserved::test_an_accumulating_list_takes_the_larger_fit, tests/test_host_ram_need.py::TestProjectedPeak::test_a_failed_run_is_not_observed, tests/test_host_ram_need.py::TestValidateRoute::test_an_observed_run_is_the_plans_host_memory |
| Acceleration warnings | dw/acceleration_warnings.py: acceleration_warnings, CACHE_MIN_USEFUL_STEPS, CUDA_ONLY_ATTENTION_BACKENDS (the last entry of WARNING_CHECKS) |
An acceleration setting that can't pay off (a first_block/mag cache under CACHE_MIN_USEFUL_STEPS, any taylorseer cache, compile.fullgraph with a cache, a cache on a pipeline class with no transformer or whose transformer lacks enable_cache(), a CUDA-only attention_backend on a non-CUDA device) is a warning and never a refusal, and an unresolvable value (steps, backend, device) leaves it silent. |
tests/test_validation.py::TestAccelerationWarnings::test_warnings_never_refuse, tests/test_validation.py::TestAccelerationWarnings::test_steps_that_resolve_to_nothing_are_silent, tests/test_validation.py::TestAccelerationWarnings::test_an_unknown_device_is_silent, tests/test_validation.py::TestAccelerationWarnings::test_an_unresolvable_backend_is_silent_on_mps |
| H3 sparse attention | dw/sparse_attention.py: sparse_call, install_sparse, install_sparse_for_call, apply_sparse_settings, SparseH3AttnProcessor; wired from dw/pipeline_processors/components.py: configure_components and dw/pipeline_processors/pipeline.py: apply_runtime_settings |
Whether a pipeline call runs sparse is decided once per call, all or nothing (the call's first kernel invocation included); it never changes diffusers' process-wide backend and never reaches AttentionBackendName; it installs by a processor swap plus forward hooks, never a forward wrapper; and sparse_tau is a runtime setting, not part of the weights identity (#931). |
tests/test_sparse_attention_processor.py, tests/test_sparse_attention_wiring.py, tests/test_sparse_attention_tiles.py, tests/test_sparse_attention_backends.py |
| H3 adapter partition | dw/adapter_compatibility.py; the server's index is dw/server/deps.py lora_index |
The LoRA catalog decides first, on the server only: a proven entry's workflow refuses a step in the other partition, a trial/rejected entry's only warns. A file no entry places falls to its name: an FL2VA LoRA on a reference step is refused, and one whose name says neither ref2v nor fl2v is warned. |
tests/test_h3_adapters.py::TestWhatIsRefused::test_a_keyframe_adapter_on_the_reference_path, tests/test_h3_adapters.py::TestWhatIsRefused::test_an_unrecognised_name_is_a_warning_not_a_refusal, tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_proven_entry_refuses_the_other_partition, tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_weaker_entry_only_warns, tests/test_server_loras.py::TestTheCatalogPlacesAnAdapter::test_a_trial_entry_warns_and_promoting_it_refuses |
| H3 audio hold, refine, tiled refine, guide layout, guide noise, latent anchor and regional edit | Fourteen modules, imported one way: dw/pipeline_processors/h3_rules.py holds the names and rules, torch- and diffusers-free (refine_problems, the per-argument refine rules, refine_tiles and the explicit refine_sigmas schedule among them, asked before validation and before the call; tile_plan, the tile layout on the 32 px grid, shared by validate and the run; not_h3, the step rule; snap_guide_length, guide_frame_problem, guide_chain_problems, guide_end_problem; CHAIN_CONTINUITY_MODES; the chunk constants; and the regional-edit rules mask_problems, media_problems and edits_region, the argument rules shared by validate and _with_mask; media_problems is asked by validate too, on the files that resolve then, and refuses a source_video off the 17n + 5 grid, which the pipeline would otherwise snap up over a padded source), dw/pipeline_processors/h3_mask_cells.py holds the mask-to-rows arithmetic (binarized, regenerate_cells, keep_rows, on Pillow's L conversion, torch's adaptive max pool and scipy's dilation, each imported where it runs), which h3_mask.py re-exports and h3_mask_steps.py reads from there so the blocks never import their importer, dw/pipeline_processors/h3_hold.py puts the hold and refine blocks in (insert_audio_hold, core_denoise_sequences, refines; HELD_LATENTS_OUTPUT and encoded_hold_audio, the hold block's encode handed back to it), and dw/pipeline_processors/h3_guides.py the guide layout (LAYOUT_ANCHORS, insert_guides), which imports h3_hold and never the reverse, dw/pipeline_processors/h3_anchor.py the latent statistics anchor (insert_latent_anchor, anchor_refusal; its block in h3_anchor_steps.py, imported only by anchor_block(); the arithmetic, channel_statistics and match_channel_statistics, in h3_anchor_stats.py, torch only, which h3_anchor re-exports and h3_anchor_steps reads from there, as h3_mask_cells is read, so the block never imports its importer), which imports h3_hold and h3_guides and never the reverse, and runs right after after_denoise in t2va/fl2va only (a no-op without latent_anchor, decided by the block itself; it declares idle_without = "latent_anchor" so the step-progress narrator leaves it unnamed then; chain.anchor: "segment_1" hands segments 2+ segment 1's frames in chain.py's _segment_arguments, and run_chain sets pipeline.latent_anchor_source to segment 1 for those segments only; Pipeline._with_latent_anchor emits one log line per anchored call naming that source, the quoted reference, or the step's frames; dw/guides.py's anchor_errors is the latent_anchor validation check, on h3_rules.chain_anchor_problem and the guides step rule _takes_guides_problem, asked for the anchor's name, so a step passing references (ref2va) is refused as guides is; Pipeline._with_latent_anchor refuses references at run time as _with_guides does), and dw/pipeline_processors/h3_mask.py the regional edit (insert_mask_blocks, edits_regions), which imports h3_hold and never the reverse; the blocks themselves, and the arithmetic they run, are dw/pipeline_processors/h3_hold_steps.py (encode_audio_span, refine_sigmas, and explicit_sigmas, a step's refine_sigmas list run verbatim in place of the shift grid) dw/pipeline_processors/h3_guide_steps.py (splice_guide_rows, and noised_guide_pixels, the guide_noise pixels: the layout block's _encode_guides encodes each video guide's noised pixels in place of the clean ones when guide_noise is set, on h3_rules.guide_noise_taper, one encode per guide either way (PrefixStream's paired encode under one seed is that encode, so the clean one is not run); guide_noise_errors/guide_noise_warnings in dw/guides.py validate it on h3_rules.guide_noise_problem and _takes_guides_problem, and Pipeline._with_guide_noise drops a 0 and refuses at run time) and dw/pipeline_processors/h3_mask_steps.py (the source-latents and mask-blend blocks), which import diffusers when they load, so the three owners import them only inside blocks(), guide_blocks() and h3_mask.blocks() (#790), and h3_guide_steps imports h3_hold_steps and never the reverse (_with_guides in dw/pipeline_processors/pipeline_h3.py - the H3Arguments mixin that gives Pipeline its per-call H3 argument wrappers _with_held_audio, _with_guides, _with_guide_noise, _with_latent_anchor, _with_mask and _check_refine, still named Pipeline._with_* - adds its blocks for a guides call, and _with_mask checks a regional edit's arguments and media and builds the regenerate mask for a source_video call), dw/guides.py: the guides validation check and the chain continuity: "guide" check, guide_chain_errors, which asks the guides check's own step rule (_takes_guides_problem, built on h3_rules.not_h3 and REFERENCE_WORKFLOWS) and reads the mode names from h3_rules.CHAIN_CONTINUITY_MODES, the names chain_continuity.py's CONTINUITY_MODES is zipped against (the three continuity classes, split out of chain.py as the chain's settings and segment plan are into chain_config.py: ChainConfig, plan_segments, chain_segment_count) (both registered in dw/validation.py; a previous_result: guide is checked at run time), dw/pipeline_processors/h3_tiles.py: the tiled refine (the tile loop, the latent-space feather blend, the per-tile crop of one whole-canvas noise draw, the decode-free graph variant, and the one spatially tiled decode), which imports h3_rules, h3_hold and media_types (for FrameList; and events, progress) and is imported by pipeline.py and, lazily, components.py (which attaches the variant at load, behind h3_hold.core_denoise_sequences, and calls insert_mask_blocks at load), never by h3_rules or h3_hold, dw/hold_audio.py: hold_audio_errors (a video is held by its soundtrack), refine_strength_errors, mask_errors (Check regional_edit, which probes the source and mask through ValidationContext.probe the way dw/guides.py probes guide videos) |
The one place dw modifies a diffusers modular pipeline's block graph: three blocks go in, before set_timesteps, denoise and after_denoise, in every H3 core-denoise sequence (the whole graph and the workflow=-pruned flat graph), and four more for a regional edit and the audio pass: the source block before set_timesteps, the blend block inside the denoise loop after update, the audio-pass block (DwH3AudioPassStep) before denoise and the release block (DwH3ReleaseVideoStep) before after_denoise. All are no-ops without their arguments (hold_audio / refine_strength, source_video, dw_held_video). The tiled refine adds a second graph built at load beside them (h3_tiles.decode_free_blocks: the decode and text-encoder blocks dropped, every kept block the graph's own instance, so hold and refine stay in) and swaps it into pipeline._blocks only for the duration of the one text-encode call and the tile calls (swapped_blocks, restored on failure). Every tile runs the hold block, but the track is encoded once per refine: the first tile is asked for the block's encode (HELD_LATENTS_OUTPUT) and every later tile's hold_audio carries it (encoded_hold_audio, an EncodedAudioReference), which the block writes in place of running the audio VAE. A hold_audio path is opened only after locations.validate_media_path against the step's workflow directory. dw/introspection.py builds the same graph without weights, hold included, so get_pipeline_signature lists a modular pipeline's block inputs. The copies of diffusers behaviour are pinned to the installed diffusers: shifted_sigma_grid (the spacing refine_sigmas rests on) to stock set_timesteps, GUIDE_FRAMES_PER_CHUNK/GUIDE_LATENTS_PER_CHUNK to the H3 video VAE's clip_length and latents per clip, and every name in LAYOUT_ANCHORS (_fill_audio_positions among them) to before_denoise. |
tests/test_h3_hold_audio.py::TestAnchors::test_inserted_next_to_the_anchors_in_every_shape, tests/test_h3_hold_audio.py::TestHoldAudioLocation::test_a_path_outside_the_roots_is_refused_at_the_call, tests/test_h3_hold_audio.py::TestRefusals::test_a_non_h3_step_is_refused, tests/test_h3_hold_audio.py::TestSignature::test_the_h3_signature_shows_hold_audio, tests/test_h3_refine.py::TestAnchors::test_inserted_before_denoise_in_every_shape, tests/test_h3_tiles.py::TestDecodeFree::test_variant_has_no_decode_and_shares_blocks, tests/test_h3_tiles.py::TestTileLoop::test_blocks_are_restored_when_a_tile_fails, tests/test_h3_tiles.py::TestTileLoop::test_the_track_is_encoded_by_one_tile_and_handed_to_the_rest, tests/test_h3_hold_audio.py::TestHoldBlock::test_a_carried_encode_is_written_without_encoding_again, tests/test_h3_tiles.py, tests/test_h3_rules.py (tile_plan and the refine_tiles rules), tests/test_h3_mask.py, tests/test_h3_refine.py::TestSigmaGridDrift::test_the_full_grid_is_stock_set_timesteps, tests/test_h3_refine.py::TestSigmaListOneOwner::test_the_runtime_message_is_the_owners, tests/test_h3_refine.py::TestSigmaListSchedule::test_the_block_runs_the_list_unshifted, tests/test_h3_guides.py::TestChunkDrift::test_frames_per_chunk_is_the_vae_clip_length, tests/test_h3_guides.py::TestChunkDrift::test_latents_per_chunk_is_the_vae_tokens_per_clip, tests/test_h3_guides.py::TestAnchors::test_installed_diffusers_has_every_anchor, tests/test_guide_chain_validation.py::TestRefused::test_the_step_rule_is_the_guides_rule, tests/test_guide_chain_validation.py::TestRefused::test_every_run_mode_is_known_to_validate, tests/test_h3_rules.py::test_h3_rules_imports_neither_torch_nor_diffusers, tests/test_h3_rules.py::test_the_block_owners_import_no_diffusers, tests/test_h3_anchor.py::TestInsert::test_follows_after_denoise, tests/test_h3_anchor.py::TestStep::test_without_an_anchor_the_latents_are_untouched, tests/test_h3_anchor.py::TestWithLatentAnchor::test_no_anchor_logs_nothing, tests/test_h3_anchor.py::TestWithLatentAnchor::test_a_chain_segment_logs_its_label_and_segment_1, tests/test_modular_progress.py::test_the_anchor_step_is_idle_without_latent_anchor, tests/test_h3_anchor.py::TestAnchorErrors::test_an_unknown_anchor_names_the_values, tests/test_h3_guide_noise.py |
| H3 audio pass (#1192) | dw/pipeline_processors/h3_rules.py: is_audio_pass (a source_video with no mask and no hold_audio), the audio-pass rules inside mask_problems (_audio_pass_problems, _listen_scale_problems, LISTEN_SCALES) and listen_canvas; dw/pipeline_processors/h3_mask.py: insert_mask_blocks puts the pair in beside the source block, passes_audio, listen_frames (the area resample); dw/pipeline_processors/h3_mask_steps.py: DwH3AudioPassStep (before denoise), DwH3ReleaseVideoStep (before after_denoise), on h3_hold_steps.row_timestep_plan; dw/pipeline_processors/pipeline_h3.py: with_audio_pass; dw/output_extraction.py: held_videos |
No video row is ever stepped: the pass block widens num_condition_video_rows over every video row after set_timesteps and pins them at t = 1.0 in the row plan, the source block's clean encode held at every step; the audio keeps the stock audio schedule from full noise. No refine argument runs with it. The result plays the full-size source frames (HELD_VIDEO_OUTPUT), never the decode, the way dw_held_audio stands in for a held track. listen_scale moves the encode canvas only, keeping the source's aspect (listen_canvas snaps the shorter side and derives the longer). The result keeps the source's own frame rate (HELD_VIDEO_RATE_OUTPUT, read by h3_guides.source_frame_rate), never relabelled to 24: a source at another rate is conditioned on a copy retimed to 24 fps over its own duration (h3_rules.retimed_indices, nearest frame, the count snapped up onto the 17n + 5 grid), and the generated track is trimmed to the source's duration (output_extraction.held_duration_audio); a 24 fps source is conditioned on as it is. A source with no rate and no soundtrack to time it is refused. |
tests/test_h3_audio_pass.py::test_the_output_frames_are_the_sources_own, tests/test_h3_audio_pass.py::test_listen_scale_half_runs_the_kept_rows_at_half_canvas, tests/test_h3_audio_pass.py::test_a_mask_or_hold_audio_keeps_todays_meaning, tests/test_h3_audio_pass.py::TestAudioPassBlock::test_the_loop_steps_audio_and_never_a_video_row, tests/test_h3_audio_pass.py::TestSourceFrameRate::test_a_30_fps_source_plays_at_30_under_audio_of_its_own_length, tests/test_h3_audio_pass.py::TestSourceFrameRate::test_a_24_fps_source_is_unchanged |
| IC-LoRA reference scale | workflows/templates/ltx2/ |
The five conditioning templates run at reference_downscale_factor: 1, and the two upscalers at 2. |
tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_reference_is_encoded_at_the_output_resolution, tests/test_ltx2_ic_loras.py::TestTheGenerativeUpscaleMatchesTheSameCard::test_it_loads_the_upscaler_at_factor_two |
| IC-LoRA numbers | workflows/templates/ltx2/ |
Every number in the IC-LoRA templates comes from the vendor's model card. | tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_strength_is_the_cards_default, tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_defaults_are_the_trained_bucket |
| IC-LoRA prompt genre | prompts/ltx2/ |
A stored prompt tagged ic-lora is checked against its trained caption form, not against the T2V paragraph rule. |
tests/test_ltx_prompt_library.py::test_an_ic_lora_prompt_is_in_its_trained_form |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Step cache | dw/step_cache.py, dw/workflow_run.py: cache_lookup |
A seeded rerun with unchanged inputs is served from the cache, marked reused: true, and writes nothing; POST /api/memory/clear (dw/server/routes/system.py: clear_memory) drops it. A step's key is only what can change its output: workflow metadata (cost_drivers, vram_estimate, descriptions) never reaches it and a result's role is dropped (NON_OUTPUT_RESULT_KEYS); attention backend, cache config, dtype and LoRA stay, and so do subfolder and file_base_name, because a hit republishes the cached paths. |
tests/test_workflow_step_cache.py::TestOutputKeys::test_the_exclusion_list_is_pinned, tests/test_workflow_step_cache.py::TestOutputKeys::test_metadata_alone_is_a_hit, tests/test_workflow_step_cache.py::test_cache_hit_marks_its_manifest_entry_and_event_reused, tests/test_workflow_step_cache.py::test_second_run_with_unchanged_step_reuses_cached_result |
| No seed, no cache | dw/workflow_run.py: prepare_run |
A workflow that sets no seed skips the cache. |
tests/test_workflow_step_cache.py::TestCacheHits::test_an_unseeded_workflow_has_no_hits |
| A different result | dw/server/routes/jobs.py, dw_mcp/tools_jobs.py: rerun_job |
rerun with new_seed draws a fresh seed into the workflow's seed variable. |
tests/test_rerun_new_seed.py::test_a_rerun_with_a_new_seed_draws_one_into_that_variable |
| Worker protocol | dw/worker_protocol.py: parse_reply |
Every command and reply is a frozen dataclass that travels as a wire dict. A run event's keys sit beside the wire's own type, so an event's type key (cache_stats' cache type) crosses as event_type and comes back as type. |
tests/test_worker_messages.py::test_from_wire_inverts_to_wire, tests/test_cache_stats.py::test_the_event_crosses_the_worker_wire_with_its_cache_type, tests/test_worker_messages.py::test_an_unknown_reply_type_is_kept_whole_rather_than_raised |
| Persistent worker | dw/worker.py, dw/worker_manager.py, dw/serve.py |
Jobs run in spawned workers (one per card) that keep models loaded, so a change to engine code needs a server restart. | — |
| Worker PyTorch settings | dw/__init__.py: configure_torch, called by dw/worker.py; the defaults are dw/settings.py: Settings |
Every process that runs a pipeline applies the settings file's enable_tf32, cudnn_benchmark and cudnn_deterministic itself: a worker when it starts and again at each job start, since the parent's startup runs no pipeline. The defaults (TF32 and benchmark off, deterministic on) give the same output for the same seed (#1238). |
tests/test_torch_settings.py::test_worker_initialisation_applies_the_torch_settings, tests/test_torch_settings.py::test_defaults_make_output_reproducible |
| Worker card pinning | dw/devices.py: worker_environment, pinned_environment, resolve_serve_devices, check_device_present, label_of |
A worker named with a CUDA index is spawned with CUDA_VISIBLE_DEVICES set to that card and DW_DEVICE=cuda (a bare cuda pins nothing), so index-0 reads inside it measure its own card; --devices/devices takes one entry per worker (a card named twice, or a bare cuda in a list of several, is refused), a card the box lacks is refused at startup naming the cards present (in a list of several it is dropped with a warning and the server starts on the rest; only a list with no card present refuses), and a job stores the card it ran on as device_ordinal and device_card (#693), job and history alike: rerun affinity reads the ordinal, observed cost buckets by the card, and the device label "cuda:N <card name>" is only label_of joining the two - nothing splits a label back apart. |
tests/test_worker_manager.py::TestWorkerPinning::test_a_cuda_index_pins_the_worker_through_the_environment, tests/test_worker_manager.py::TestWorkerPinning::test_a_bare_cuda_sets_no_pin, tests/test_serve_main.py::TestConfigureDevices::test_a_refusal_exits_with_code_2, tests/test_serve_main.py::TestResolveServeDevices::test_a_card_the_machine_lacks_is_refused_naming_the_cards_present, tests/test_serve_main.py::TestResolveServeDevices::test_two_entries_are_both_kept, tests/test_serve_main.py::TestResolveServeDevices::test_a_card_gone_from_a_list_is_dropped_and_reported, tests/test_serve_main.py::TestConfigureDevices::test_a_pulled_card_in_the_setting_starts_on_the_rest_and_warns, tests/test_serve_main.py::TestResolveServeDevices::test_a_card_named_twice_is_refused |
| Failed-run reporting | dw/worker.py, dw/worker_protocol.py: Failed, Cancelled |
A failed or cancelled run's reply still carries the manifest of the steps that ran. | tests/test_worker_execute.py::test_failure_carries_the_manifest_of_the_steps_that_ran, tests/test_worker_execute.py::test_cancellation_carries_the_manifest_too |
| Failure path redaction | dw/path_redaction.py: redact_paths, called by dw/worker.py |
A failed run's message and traceback name a file under the job's asset roots or output directory by its asset:/output: reference, never its absolute server path. |
tests/test_worker_execute.py::test_a_failure_names_an_asset_by_reference_not_by_server_path, tests/test_path_redaction.py |
| Run-time warnings | dw/events.py: emit_warning |
A warning found at run time is emitted as an event, so it reaches the caller and not just the log. | tests/test_concat_videos.py::TestWarningsReachTheCaller::test_the_level_spread_warning_is_emitted_as_an_event |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Opening media | dw/media.py, dw/media_readers.py (the frame readers, re-exported by dw/media.py, which imports from it and never the reverse) |
These are the only modules that call av.open. They import no torch and no step types. |
tests/test_media_layering.py::test_av_open_appears_only_in_media, tests/test_media_layering.py::test_media_imports_no_step_types (no torch: —) |
| Frame moments and crops | dw/media_frames.py: moment_index, parse_crop |
One at grammar ("frame:N", or seconds at the clip's fps; negative counts back from the end for get_output_frames and is refused by compare_outputs) and one [x, y, width, height] crop parse serve get_output_frames and compare_outputs. Only the bounds policy is the caller's: get_output_frames refuses a moment past the end and clamps a crop (_moment_to_index, resolve_crop_box); compare_outputs blanks a past-end cell and refuses a crop that doesn't fit an output. |
tests/test_media_compare.py::test_resolve_moments_refuses_what_get_output_frames_refuses, tests/test_media_compare.py::test_crop_box_refuses_a_malformed_box_as_resolve_crop_box_does |
| Output comparison | dw/media_compare.py (texture bands from dw/texture_bands.py): compare, psnr, ssim, mean_abs_diff, check_request, crop_box |
Metrics always compare the same pixels: one shared crop is the measured region, and per-output crops are display only, leaving metrics on the full frame. An output whose size differs from the baseline's, or a moment past its end, gets null metrics with a reason; nothing is resized to force a number. SSIM is on Rec. 601 luma (SSIM's convention, not image_ops.luma). It runs in the server process and decodes through dw/media.py and open_bounded. |
tests/test_compare_route.py::test_a_shared_crop_is_the_region_measured, tests/test_compare_route.py::test_per_output_crops_are_display_only, tests/test_compare_route.py::test_a_size_mismatch_gets_no_numbers_and_says_why, tests/test_compare_route.py::test_a_moment_past_the_shorter_clip_is_a_blank_cell |
| Signal processing | dw/dsp.py |
It measures and transforms waveforms, decides nothing, and imports nothing from dw. |
tests/test_media_layering.py::test_dsp_imports_nothing_from_dw |
| Assessment probes | dw/tasks/assess.py, dw/assessment_rules.py |
Exactly three registered commands are assessment probes. Each measures a finished file, and every rule names a real probe field. | tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_assessment_is_exactly_the_three_probes, tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_every_rule_names_a_registered_json_command |
| Script check | dw/tasks/script_check.py: check; dw/tasks/script_findings.py: the rule constants and the finding builders check calls (imported one way, re-exported by script_check); dw/tasks/script_take.py: guard_words, word_level, shot_spans; dw/script_lines.py: parse_lines, parse_shots, shot_names_error; number-word folding is dw/script_lines.py: fold_numbers, normalize_words(numbers=), folded across chunks by dw/tasks/script_take.py: _folded_heard; a prompt's dialogue line is dw/script_lines.py: dialogue_text |
check_script aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with assessment_rules.finding() and reads the guard's floor and window (DEAD_AIR_FLOOR_DBFS, DEAD_AIR_WINDOW) rather than copying them; the loudest-window measure is dsp.loudest_window_dbfs and the looped-bed self-correlation dsp.repeat_correlation (script_findings.is_looped_bed only decides on it) (script_take.word_level wraps it; the guard, the transcription levelling, the floor-gated audible_runs and the shot placement (with clamp_to_shot, a line's timing kept inside its shot) are dw/tasks/script_take.py, which script_check reads one way - nothing there reads the rule tables), which transcribe_audio's suspect: "silent" mark reads with the same floor and window; the repetition-loop detector (script_lines.looped_texts, REPEAT_RUN_MIN, REPEAT_MAX_PERIOD) lives in dw/script_lines.py and backs both check_script's repetition discard and transcribe_audio's suspect: "repetition", and script_lines.suspect_runs/SUSPECT_LEFT_OUT are what plan_cuts groups and leaves out suspect chunks by - so neither task imports the other. Number words are folded to digits in the one normaliser both sides of every comparison use (normalize_words, fold_numbers, #1297), so 'Article One' and 'Article 1' are the same words for check_script and the chain's check_words alike; a caller with one heard word per chunk normalises each with numbers=False and folds the joined list (guard_words through _folded_heard), never folding per chunk. The line a dialogue segment speaks is dialogue_text's <d> bodies, not strip_markup, which keeps the prose. It is not an assessment probe and adds nothing to RULES. Its literal lines is refused at validation through task_problems (script_lines_errors), which calls the same parse_lines the task runs, and a literal shots through the same parse_shots and shot_names_error; the parsers live in dw/script_lines.py, not the task, so that task_problems reaches them without importing script_check (the import_cycles ratchet in scripts/arch_metrics.py). A line's shot may be the short @name of the one recorded shot that carries it (script_lines.matching_shot_names, with_full_shot_names, shared by the task and the literal-shots check); which member key a recorded name carries comes from references.member_keys, beside MEMBER_SEPARATOR and for_each.member_name, never a re-derived pattern: a key ends at the first character outside references.VARIABLE_NAME_CHARACTERS, the set security.validate_variable_name (which checks a for_each entry's name) builds its pattern from. With audio an output:/asset:/path whose run manifest records shots, dw/shot_span_preflight.py: script_shot_name_errors (the script_shot_names error check) runs shot_names_error at validation, reading the map through runs.shots_beside. Its shot map comes from assess.resolve_shots, the one the probes use, never a copy; it places each shot on the soundtrack with assess.sample_span, decides repeated shot names with shots.duplicate_shot_names, and asks locations.is_http_url whether the take is a URL. |
tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag, tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found, tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference, tests/test_script_check.py::TestShotValidation::test_a_short_member_name_matches_the_one_shot_carrying_it, tests/test_script_check.py::TestShotValidation::test_a_valid_variable_name_comes_back_whole_as_a_member_key, tests/test_shot_span_preflight.py::TestScriptShotNames::test_a_wrong_shot_name_is_an_error_at_validate_time, tests/test_script_lines_words.py::test_number_words_equal_digits, tests/test_script_lines_words.py::test_guard_words_folds_number_words_across_chunks, tests/test_script_lines_words.py::test_dialogue_text_joins_bodies_in_order |
| Audio sync against a source slice | dw/tasks/measure_sync.py: measure_sync; dw/dsp.py: envelope_lag |
measure_sync cross-correlates the onset envelopes of a shot's audio and its source slice and answers the lag and a 0-1 confidence. A positive lag means the audio is LATE (its events fall after the reference's), negative early; lag_seconds is None for a silent or flat track. It builds its findings (sync_unmeasurable, sync_low_confidence, audio_out_of_sync) with assessment_rules.finding() over module-local rule dicts and adds nothing to RULES; it reads the silence floor from beats.SILENT_DBFS. envelope_lag imports nothing from dw. |
tests/test_measure_sync.py::TestLag::test_late_audio_is_a_positive_lag, tests/test_measure_sync.py::TestLag::test_early_audio_is_a_negative_lag, tests/test_measure_sync.py::TestUnmeasurable::test_a_silent_reference_has_no_lag |
| Lip sync of a picture to its audio | dw/tasks/lip_sync.py: lip_sync_score, measure_lip_sync, the YuNet seam _faces and _mouth_box |
The mouth box is cut from YuNet's two mouth-corner landmarks (the corners' span, run down by their distance), and YuNet is loaded only through face_track.face_detector_model. The mouth signal (mean absolute grey change in the box from the frame before) is Pearson-correlated with the audio's per-frame RMS envelope at whole-frame lags: correlation is the best r within SYNC_WINDOW_FRAMES (3) of lag 0, lag_seconds/best_correlation the best lag within max_lag_seconds (default 1, the task refuses above MAX_LAG_SECONDS_CAP 2) and its r; the search never reaches past len(frames) - MIN_PAIRS. A positive lag means the audio is LATE, as measure_sync's is. A take-level indicator only, never a chain measure: the calibration found no per-segment threshold (ruling R9, bench/records/narrated.md). lip_sync_score takes frames and a waveform in memory, so a calibration script calls it without a file; it builds no findings and sets no threshold. |
tests/test_lip_sync.py::test_synced_mouth_scores_high_at_zero_lag, tests/test_lip_sync.py::test_shifted_audio_scores_at_its_shift, tests/test_lip_sync.py::test_no_face_returns_none, tests/test_lip_sync.py::test_the_mouth_box_comes_from_the_mouth_corners, tests/test_lip_sync.py::test_max_lag_over_the_cap_is_refused, tests/test_lip_sync.py::test_reach_is_capped_by_the_frames |
| Sung phrases from voiced energy | dw/tasks/phrases.py: analyze_phrases, voiced_power, voiced_runs; dw/task_problems.py: phrases_problems, phrases_errors |
analyze_phrases times a song's sung phrases with no model: a stereo file's centre-weighted band power (max(0, P_mid - P_side), 250-3500 Hz, 20 ms frames) or, for mono, centre: false or identical channels, the downmix's band power (method band). The voiced floor is voice_attribution.voiced_floor_dbfs, the rule attribute_voices uses, unless floor_db is given. It answers a dict - {phrases, duration_seconds, method, warnings} - so a step saves one JSON artifact. Its ranges are the registration's domains; phrases_problems (a real boolean centre) runs at validate and at run time. |
tests/test_analyze_phrases.py::TestCentre::test_centred_bursts_under_hard_panned_noise_are_found, tests/test_analyze_phrases.py::TestCentre::test_bursts_panned_hard_to_one_side_are_not_centre, tests/test_analyze_phrases.py::TestRefusals::test_validate_refuses_a_missing_audio_and_a_negative_gap |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| Path validators | dw/security.py: validate_path, validate_workflow_path, validate_output_path |
Filesystem access goes through a path validator that is given a base directory. | tests/test_security.py::test_path_validation, .github/codeql/dw-security/DwPathInjection.ql |
| URL and argument validators | dw/security.py: validate_url, sanitize_command_args |
A URL must be http or https, and a subprocess argument must not carry a shell metacharacter. | tests/test_security.py::TestValidateUrl::test_dangerous_schemes_are_rejected, tests/test_security.py::TestSanitizeCommandArgs::test_every_shell_metacharacter_is_rejected |
| Locations from a workflow | dw/locations.py |
A local path in a workflow's media arguments (named like their media, or declared by the task's @register_command(media_arguments=...), read through TASK_MEDIA_ARGUMENTS) is confined to a known root, a location written as a URL is one security.validate_url accepts, and a URL must not name a host inside the deployment. A host that does not resolve is refused. The requests themselves are dw/outbound.py's. |
tests/test_locations.py::TestMediaPathContainment::test_absolute_path_outside_every_root_is_refused, tests/test_locations.py::TestMediaPathContainment::test_a_url_that_is_not_http_is_refused, tests/test_locations.py::TestMediaPathContainment::test_an_upper_case_https_url_is_not_refused_as_another_scheme, tests/test_security_ssrf.py::TestInternalAddressSpellings::test_a_name_that_resolves_inside, tests/test_locations.py::TestTaskMediaArguments::test_an_unreadable_source_is_an_error_at_its_path |
| Outbound requests | dw/outbound.py: safe_get, safe_post |
Every outbound request is validated per hop against dw/locations.py's policy, resolved once and dialed at the address the policy checked, capped and timed. No other module under dw/ makes an HTTP request or builds a session or adapter. |
tests/test_outbound.py::TestSafeRequest::test_one_lookup_per_hop_and_the_dial_is_the_checked_address |
| Trust gate | dw/trust.py: require_trusted_dotted_name |
In an untrusted workflow (the default), a dotted type outside the allowlist is a validation error before anything is imported. | tests/test_security_trust_gate.py::TestValidationRefusesBeforeImport::test_a_dotted_type_is_a_validation_error |
| Remote generate code | dw/trust.py: require_trusted_generate_arguments; dw/tasks/text_generation.py: _validated_generate_kwargs, _validated_chat_template_kwargs |
Untrusted, generate_kwargs may not set custom_generate or trust_remote_code (transformers would download and run a repo's custom_generate/generate.py); both it and chat_template_kwargs hold only scalars (or, for generate_kwargs, lists of them) and none of the pipeline call's own keys. All checked before the model loads. |
tests/test_security_trust_gate.py::TestRemoteGenerateCodeUntrusted::test_refused_before_the_model_loads, tests/test_text_generation.py::TestGenerateKwargsAreFiltered::test_a_structural_key_is_refused_before_any_load, tests/test_text_generation.py::TestGenerateKwargsAreFiltered::test_a_nested_value_is_refused |
| Hub-file system prompts | dw/tasks/text_generation.py: resolve_system_prompt |
A system_prompt.from_hub file is pinned to a full commit, asked of the Hub anonymously before every use (failing closed on any error, offline or under HF_HUB_OFFLINE), downloaded with token=False, and refused over 1 MiB or when the Hub reports no size. |
tests/test_text_generation.py::TestSystemPromptFromHub::test_a_file_the_hub_will_not_serve_publicly_is_refused, tests/test_text_generation.py::TestSystemPromptFromHub::test_the_download_never_carries_the_server_token, tests/test_text_generation.py::TestSystemPromptFromHub::test_a_file_of_unknown_size_is_refused_before_downloading |
| CodeQL path-injection model | .github/codeql/dw-security/DwPathSanitizers.qll, .github/workflows/codeql.yml |
The local pack models the dw/security.py validators (and dw/locations.py's validate_media_path) as sanitizers, so a validator that moves is re-modelled in the same commit. |
.github/codeql/dw-security/DwPathInjection.ql |
| Archives never follow links | dw/server/outputs.py: zip_download |
A gallery listing drops a symlink that leaves its root, and an archive skips one. | tests/test_security_symlinks.py::TestOutputs::test_the_archive_route_does_not_follow_the_link, tests/test_security_symlinks.py::TestOutputs::test_the_gallery_listing_does_not_enumerate_the_link |
| Engine/server import direction | dw/workspace.py: EXPORTS_SUBDIR |
No engine module imports dw.server except the entry point dw/serve.py, and a constant both sides need lives engine-side. |
docs/stabilization/baseline.json: import_cycles ratchet (indirect: it fails only on an import that closes a cycle) |
| Module and function size | scripts/arch_metrics.py: SIZE_WARNING, SIZE_CEILING |
A module stays at or under 1,100 lines, and it warns above 1,000. A function stays at or under 150 lines. | docs/stabilization/baseline.json: modules_over_size_ceiling, functions_over_150_lines |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| App factory | dw/server/app.py: create_app |
create_app builds the JobManager and app.state, installs the middleware and registers the routes, and all state lives in the JobManager. |
— |
| Routers | dw/server/routes/*.py, dw/server/routes/__init__.py: ROUTERS, include_routers, include_file_routes |
Routers register in ROUTERS order, because a greedy {name:path} route must come after its specific siblings. The files router comes later, after /mcp. |
tests/test_server_downloads.py::test_download_workflow_sets_content_disposition_attachment |
| Job queue | dw/server/jobs.py: JobManager; dw/server/job_results.py: consume_results |
A dispatcher thread hands queued jobs, in queue order, to the pool's workers (Worker pool dispatch); each worker runs one job at a time, and consume_results folds that worker's replies into the job, naming files relative to the job's own output_dir. A job carries its own output_dir, asset_dir and workflow_dir, so it stays in its workspace. |
tests/test_server_workspaces.py::TestRunning::test_rerun_stays_in_the_workspace_it_ran_in (FIFO: —) |
| Worker pool dispatch | dw/server/dispatch.py: Dispatcher._next_dispatch, Dispatcher._host_ram_wait (mixed into JobManager); dw/server/pool.py: WorkerSlot, check_fits, largest_ceiling_gb, host_ram_wait; dw/vram_estimate.py: required_vram_gb; dw/server/admission.py: pool_capacity |
--devices runs one worker per card; a queued job goes to a free card that fits what it needs (none starts while a cache clear holds dispatch, dw disk caches), a job that fits no free card keeps its place while a smaller one behind it takes the card, a job bigger than every card is refused at submit, a job that fits a free card is also held while its projected host-RAM peak exceeds MemAvailable less what running jobs have yet to add plus the idle worker's own RSS (another dw.serve's running jobs on the host count against MemAvailable too, through per-server claim files under a flock (dw/host_memory.py: HostClaims, #1227), a job alone is never held, no figure or no RAM reading means no gate, a smaller job behind a held one starts, and the dispatcher re-checks every 10 s; #869), and cancel or a crash touches only the worker running that job. On a backend no cost entry describes, a declared vram_estimate is held at admission to the pool's largest card, so a job the first card cannot hold is not refused when a larger card can hold it. |
tests/test_worker_pool.py::TestConcurrency::test_two_workers_run_two_jobs_at_once, tests/test_worker_pool.py::TestBackfill::test_a_job_too_big_for_the_free_card_waits_while_a_smaller_one_takes_it, tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_too_big_for_every_card_is_refused_naming_the_largest, tests/test_worker_pool.py::TestCancel::test_cancel_reaches_the_worker_running_the_job, tests/test_worker_pool.py::TestCrashIsolation::test_a_crashed_worker_fails_only_its_own_job, tests/test_worker_pool.py::TestOomRanking::test_each_dispatch_scores_its_worker_highest, tests/test_worker_pool.py::TestFitAtSubmit::test_a_job_only_the_larger_card_holds_is_admitted, tests/test_worker_pool.py::TestHostRamGate::test_a_job_the_host_cannot_hold_beside_the_running_one_is_held, tests/test_worker_pool.py::TestHostRamGate::test_a_job_alone_is_never_held, tests/test_worker_pool.py::TestHostRamGate::test_a_smaller_job_starts_past_a_held_one |
| Card affinity | dw/server/pool.py: choose_slot; dw/server/jobs.py: JobManager.route; dw/worker_protocol.py: workflow_identity |
Among the free cards a job fits it goes to the rerun's original card, else the card whose worker last ran its workflow identity (workflow_identity, the one rule the worker caches by too), else one with no worker running, else the first in --devices order; a cache probe and plan.estimate (priced_for) ask about the card route names, and memory is read and cleared per card. |
tests/test_worker_pool.py::TestIdentityAffinity::test_a_second_run_lands_on_the_card_that_ran_the_workflow_first, tests/test_worker_pool.py::TestRerunAffinity::test_a_rerun_lands_on_the_original_card_when_it_is_free, tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to, tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle |
| Per-card memory and cache | dw/server/worker_memory.py: memory_status, clear_memory, probe_cache |
Each card keeps its own last memory reading; an idle card's worker is asked live, a busy one answers with the run's last report and says why it is not live, and a clear leaves a card running a job alone. Every request takes the card's lock with a bounded wait, and a card's lock is never taken while the JobManager's lock is held. | tests/test_worker_pool.py::TestMemoryPerCard::test_a_named_card_running_a_job_is_refused_though_the_other_is_idle, tests/test_worker_pool.py::TestProbeRouting::test_the_probe_goes_to_the_worker_the_job_would_be_routed_to, tests/test_server.py::test_memory_says_why_a_reading_is_not_the_worker_s |
| dw disk caches | dw/disk_caches.py; dw/server/jobs.py: JobManager.hold_dispatch; dw/server/routes/system.py: clear_cache, delete_cache_entry |
Members inductor, triton, quantized. Clear only a root carrying the .dw-cache marker, never while a job is running or queued (quantized also not during a model download); an entry delete takes only a 64-hex id, checked before any path is built; clear renames into .clearing-* then deletes. The busy check and the dispatch hold are one step in JobManager.hold_dispatch, kept for the whole clear or entry delete: a job submitted meanwhile is accepted but stays queued until it ends. |
tests/test_disk_caches.py, tests/test_disk_caches.py::test_job_submitted_during_a_clear_waits_for_it |
| Job-to-run link | dw/server/job_record.py, dw/server/jobs.py: JobManager.realized |
A job records run_id, run_dir and run_version, and JobManager.realized reads that run's workflow.json, confined to the output root. |
tests/test_server_jobs.py::test_realized_reads_the_file_the_run_wrote, tests/test_server_jobs.py::test_realized_refuses_a_run_dir_that_escapes_the_output_root |
| Job history | dw/server/job_history.py: JobHistory, _ADDED_COLUMNS, close_instance, close_dead_servers, hold_server_lock |
Jobs persist in jobs.sqlite, in WAL mode, so the Jobs view survives a restart. A database from an older release gains each column it lacks from the _ADDED_COLUMNS table when it is opened. Every admitted job has a row from submit on, stamped with its server's instance id; each server holds servers/<instance>.lock beside the database for its life. A startup closes the queued and running rows of every server whose lock it can take as cancelled, error starting server restart: (a running one names its run directory), and leaves a live sibling's rows alone; readers serve closed rows only (#1220). |
tests/test_server.py::test_job_history_survives_restart_and_reruns, tests/test_restart_tombstones.py, tests/test_job_history_wal.py::test_the_jobs_database_uses_wal_mode, tests/test_server_jobs.py::test_a_v0_13_0_database_gains_the_columns_added_since |
| Job fields, live and persisted | dw/server/job_record.py: Job.summary, Job.detail; dw/server/job_history.py: _RECORDED, _ADDED_COLUMNS, recent_summaries, _to_detail |
A job field is declared once per side: on Job (and its summary()/detail()), and in history as an _ADDED_COLUMNS entry plus one (column, getter) in _RECORDED. History readers take columns by name, never by position, and a historical job serves the same summary and detail keys a live one does, apart from the live-only poll state and the history-only markers the parity test declares. |
tests/test_job_history_parity.py::test_history_serves_the_keys_a_live_job_does |
| HTTP security | dw/server/http_security.py: install_middleware, query_token_ok |
Every API route needs the token, and only a route marked query_token_ok takes it as ?token=. |
tests/test_security_auth.py::TestTheTokenGate::test_every_api_spelling_needs_the_token, tests/test_security_auth.py::TestTheTokenGate::test_the_query_token_is_refused_where_it_is_not_allowed |
| Prompt enhancement | dw/server/enhancers.py: PRESETS, _T2I_SYSTEM_PROMPT |
PRESETS is the Enhance panel's menu (label, curated LLMs, preselect hints); a preset's family-specific knowledge, its spec, lives in a builtin under dw/workflows/, never in Python. t2i's generic, family-agnostic system prompt stays in Python. A third preset moves the menu into builtin metadata (#696). |
tests/test_prompt_library.py::test_every_enhancer_preset_targets_a_known_family, tests/test_server.py::TestEnhance::test_presets_are_listed, tests/test_server.py::TestEnhance::test_enhance_workflows_are_schema_valid, tests/test_server.py::TestEnhance::test_an_unknown_preset_is_a_client_error |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
dw_mcp stays torch-free |
dw_mcp/ |
dw_mcp reaches dw.serve over HTTP and imports no dw module, because dw/__init__.py pulls in torch. |
tests/test_mcp_server.py::TestStartupWeight::test_the_server_starts_without_importing_the_engine |
| Tool surface | dw_mcp/server.py, dw_mcp/tools_*.py |
Only these modules import the MCP SDK, and each tool body is a one-line call into a handler. | tests/test_mcp_server.py::test_the_wiring_table_covers_every_registered_tool, tests/test_mcp_server.py::test_the_stated_tool_count_is_the_registered_one |
| Surface text budget | the tool docstrings in dw_mcp/tools_*.py, the instructions in dw_mcp/server.py |
The instructions and each tool description stay at or under 2,048 characters, and the whole surface stays within its token budget. | tests/test_mcp_server.py: SURFACE_BUDGET, CLIENT_TEXT_LIMIT; tests/test_mcp_server.py::test_no_text_the_agent_reads_is_cut_off_by_the_client, tests/test_mcp_server.py::test_the_tool_surface_fits_the_budget |
| Guide pointers in resident text | the tool docstrings in dw_mcp/tools_*.py, the instructions in dw_mcp/server.py, each plugin SKILL.md and its references/ |
Every get_guide(...) / guide-section pointer opens a real section through dw/server/guides.py's get_guide, and the tools that point at a guide keep their pointer when text is trimmed to fit the budget. |
tests/test_guide_pointers.py (TOOLS_THAT_POINT) |
| Spending needs consent | each handler behind a tool that takes acknowledged_cost (dw_mcp/diagnose.py, models.py, prompts.py, workspaces.py) |
Every tool that takes acknowledged_cost refuses until it is set; delete_workspace's refusal is the server's 409. |
tests/test_mcp_twins.py::test_every_tool_that_takes_acknowledged_cost_refuses_without_it |
| Copies of engine rules | the constants and word lists in dw_mcp/ and dw/run.py |
dw_mcp cannot import its owners, so each copy equals its owner; a copy that needs no twin is deleted, not pinned. |
tests/test_mcp_twins.py |
| The CLI is a client of the server | dw/run.py |
dw.run queues on dw.serve through dw_mcp.client, so dw imports dw_mcp and never the reverse. |
tests/test_mcp_server.py::TestStartupWeight::test_the_server_starts_without_importing_the_engine, tests/test_mcp_twins.py |
| Inline images | dw/server/inline_media.py |
Fitting an image to a longest side and halving it under a base64 budget happens on the server, for /image and /frames alike; dw_mcp forwards the size and budget and imports no Pillow. |
tests/test_server_gallery_image.py, tests/test_mcp_server.py::TestStartupWeight::test_the_server_starts_without_importing_the_engine |
| Workflow patch | dw/library.py: merge_patch; dw/server/routes/library.py: patch_workflow |
A merge patch is applied on the server under the lock PUT takes; no client reads, merges and writes back. |
tests/test_server_library_path.py::TestPatchWorkflow::test_patch_merges_onto_the_stored_definition |
| Deleting a job's run | dw/server/jobs.py: JobManager.run_location; dw/server/outputs.py: delete_run_directory |
The server resolves a job's run directory against the root the job ran in, and refuses a job still queued or running. | tests/test_server_jobs.py::TestDeleteAJobsRun::test_a_running_job_is_409 |
| Level findings | dw/server/assess.py: level_findings |
Gallery metadata reports level problems from dw/audio_qc.py's thresholds; no client restates a threshold. The silent-by-design exemption to the near-silent rule is dw/audio_qc.py: all_shots_silent_by_design, used by this row and by the write-time warning. |
tests/test_server_assess.py::TestMetadataLevelFindings::test_level_findings_read_audio_qc_thresholds |
| API errors | dw_mcp/client.py |
An API failure becomes a message a person can act on, here and nowhere else. | tests/test_mcp_client.py::test_a_400_surfaces_the_servers_detail_verbatim |
| Concept | Owner | Rule | Enforced by |
|---|---|---|---|
| The UI reads engine fields | ui/src/lib/plan.ts: describePlan, ui/src/lib/results.ts: sectionBySubfolder |
The UI reads plan, version and subfolder as fields the server sends and derives nothing of its own, and it never sends acknowledged_cost. |
ui/src/lib/plan.test.ts, ui/src/lib/results.test.ts |
| Reference prefixes in the UI | ui/src/lib/references.ts |
The only module in ui/src that spells a reference prefix; each equals dw/references.py's. |
tests/test_ui_twins.py::test_the_ui_spells_every_reference_prefix_the_engine_does; prefix_literals in ui/scripts/arch-metrics.mjs |
| Output kinds on the job page | dw/server/outputs.py: MEDIA_KINDS, output_kinds |
The page renders an output by the output_kinds the job detail carries, never by its extension. |
tests/test_server.py::test_a_job_reports_each_output_files_media_kind, ui/src/lib/pages/JobPage.test.ts |
| The editor's live reference check | ui/src/lib/flow.ts: danglingReferenceDetails, owned by dw/previous_results.py and dw/for_each.py |
The editor warns as the author types, so it keeps a copy of the reference rules; one case file is run through both, the engine deciding each case. | tests/test_ui_twins.py::test_the_engine_decides_each_shared_reference_case, ui/src/lib/flow.test.ts |
| Name segments in the UI | ui/src/lib/names.ts: isNameSegment |
Workspaces, folders and stored names follow dw/security.py's WORKSPACE_NAME_PATTERN and length cap, counted in code points. |
tests/test_ui_twins.py::test_the_engine_decides_each_shared_workspace_name_case, ui/src/lib/names.test.ts |
| UI copies of engine rules | the constants tests/test_ui_twins.py reads |
A list the UI must hold equals its owner, read from the TS source; a rule tested on both sides reads one case file in tests/fixtures/. |
tests/test_ui_twins.py |
| UI overlays and suggestions | ui/src/lib/ui/: ConfirmDialog, Modal, Popover, Suggest |
Bits UI is imported only here; pages and components use the wrappers, which own focus, Escape, outside presses and scroll lock. Overlay styles and the stacking scale (--layer-*) live in ui/src/app.css. |
ui/eslint.config.js (no-restricted-imports), ui/src/lib/ui/*.test.ts, ui/e2e/chrome.spec.ts |
| Escape on a page under an overlay | ui/src/lib/ui/layers.svelte.ts: holdLayer, overlayOpen |
Every open overlay holds a layer; a page's own Escape handling acts only when none is open. | ui/src/lib/pages/GalleryPage.test.ts (an Escape that closes a confirm leaves the selection) |
| The editors' document | ui/src/lib/editorShell.svelte.ts: DocumentEditor |
The workflow and prompt editors keep one rule each for the view (remembered per editor), the JSON draft (a failed parse pins it), dirty against the last load or save, the save path, and the tab-close guard. | ui/src/lib/editorShell.test.ts, ui/src/lib/pages/PromptEditorPage.test.ts, ui/src/lib/pages/EditorPage.test.ts |
| UI polling | ui/src/lib/poll.ts: poll, sleep |
A page that refreshes on a timer starts it through poll and stops it with the function poll returns. |
ui/src/lib/poll.test.ts |
| Workspace state in the UI | ui/src/lib/workspaceState.svelte.ts: workspace |
The current workspace is one $state; api.ts reads it from here, so the request layer imports no page state. |
import_cycles in ui/scripts/arch-metrics.mjs |
| The UI's response contract | dw/server/api_models.py (the models; it re-exports dw/server/api_models_base.py: ApiModel, send_rejected_responses, and the library routes' models in dw/server/api_models_library.py); scripts/dump_openapi.py |
Every JSON route api.ts calls declares a response model, and types.ts re-exports the types generated from it (ui/src/lib/generated/). The payload does not change: an absent key stays absent (response_model_exclude_unset), a sometimes-sent key is sometimes(). Runtime is lenient - an undeclared key passes and a rejected response is logged and sent as built; tests, the e2e server and the dump run strict (DW_STRICT_RESPONSES=1). The document is generated under the FastAPI and Pydantic constraints-openapi.txt pins: CI installs them, and the dump refuses to write under others. |
tests/test_api_contract.py, tests/test_ui_twins.py::test_every_json_route_the_ui_calls_declares_its_response, CI's "Response contract" step |
Where --live appears |
ui/src/app.css (its header) |
--live marks machine state only and --select the user's place only (focus ring, sidebar), each in the files ui/scripts/design-rules.test.ts lists. |
ui/scripts/design-rules.test.ts |
| UI architecture ratchet | ui/scripts/arch-metrics.mjs |
No metric rises above docs/stabilization/ui/baseline.json. |
the ui job in .github/workflows/ci.yml; npm run preflight |