Skip to content

Generate readable subtitles from word-level timings - #42

Open
davidvanleeuwen wants to merge 1 commit into
mainfrom
fix/readable-generated-subtitles
Open

davidvanleeuwen wants to merge 1 commit into
mainfrom
fix/readable-generated-subtitles

Conversation

@davidvanleeuwen

Copy link
Copy Markdown
Member

✨ What changes

Generated subtitles currently reuse whole transcription segments, producing long blocks and estimated word highlighting. This change builds shorter cues from actual word timings, while preserving every word in both VTT and transcript JSON.

  • Request word timestamps from the pinned Voxtral V2 model and normalize its word-level segments response. Other model overrides retain segment requests.
  • Share two-line, 42-character layout across transcription and translation, preferring punctuation and pauses.
  • Let brief cues remain visible during available silence, targeting 17 characters/second without moving word timestamps, overlapping the next cue, or passing the known media end.
  • Keep supplied VTT unchanged; use estimated timing when complete, ordered word timings are unavailable. Translated words remain estimates.

👀 Example

A single long paragraph becomes several shorter cues. A word spoken from 1.00–1.10 can remain displayed until 1.83 when there is silence, while its JSON word timing stays 1.00–1.10.

🔎 Additional checks

A 6.6-second synthetic speech recording was sent to the real Mistral API: word mode returned 16 word entries, which are retained exactly in the JSON and grouped into two display cues. The recorded, public-safe response is included as a regression fixture, including a zero-duration word from provider rounding.

The subtitle changes were tested independently of running development databases. Dense speech can still exceed the target reading speed; the builder preserves text rather than dropping words or delaying later speech. Browser fitting is handled separately in Components.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant