Skip to content

fix: group transcript captions into readable sentences - #2259

Open
pavzagor wants to merge 4 commits into
CapSoftware:mainfrom
pavzagor:codex/fix-shitty-transcription
Open

fix: group transcript captions into readable sentences#2259
pavzagor wants to merge 4 commits into
CapSoftware:mainfrom
pavzagor:codex/fix-shitty-transcription

Conversation

@pavzagor

@pavzagor pavzagor commented Sep 9, 2026

Copy link
Copy Markdown

The Transcript tab currently renders every subtitle cue as a separate row, splitting speech at commas, short pauses, and every eight words. This groups adjacent cues into sentence rows while preserving speaker boundaries and exact outer timestamps. Existing transcripts, translated captions, and provisional live transcripts benefit without retranscription.

Subtitle files retain their original timing. Owners can select Edit transcript to expose the original cues for precise corrections; edits never save a merged sentence under a single cue ID. Timestamped copying uses the readable sentence rows. Grouping also stops at long silence, overlaps, and bounded length/duration when punctuation is missing.

Depends on #2243 (AssemblyAI diarization). This branch is based on that PR's head, 2766dc0; merge that PR first. Until then, GitHub's cumulative diff includes the prerequisite. Review only this follow-up's four files.

English before/after proof

A fresh AssemblyAI transcription of NASA's public JFK archival clip produces 12 caption fragments before → 3 sentence rows after, preserving all 72 words. Selecting the matching row on either side seeks the embedded source video to 12.850 seconds.

Watch/download the before/after MP4 · GitHub video page · Source clip · All proof files and methodology

English transcript before and after

The proof uses the actual old/new React components and real ASR output, with storage/auth hooks mocked. It is not production footage or proof of a persisted backend edit. Full local share-page verification was blocked by unavailable MySQL at 127.0.0.1:3306. NASA's clip splices two speeches; A/B are the unmodified ASR labels.

Validation

  • 91 focused tests passed: sentence grouping/UI, diarization, VTT, text formatting, translation, caption generation, and transcript editing.
  • Scoped Biome check and git diff --check passed.
  • Web tsc --noEmit passed in the existing development checkout; the isolated checkout reused dependencies and was unsuitable for the workspace reference type check.
  • Browser checked sentence seeking, fragment editing controls, and 360px/393px layouts without horizontal overflow. Timestamped copying is covered by the UI tests.
  • No migration, new environment variable, or paid reprocessing is required by this change.

RetriggerView in GreptileConfidence Score: 4/5

The PR should not merge until sentence-boundary detection handles punctuation used by every supported transcript language, particularly Arabic questions.

Findings

  1. P1 Arabic questions merge together
Prompt To Fix All With AI
### Issue 1
apps/web/lib/transcript-sentences.ts:10
Arabic is a supported transcript language, but this sentence check does not recognize the Arabic question mark (`؟`). Adjacent Arabic questions are therefore merged until another recognized terminator or safety limit appears, producing incorrect sentence rows in original and translated Arabic transcripts. Add the missing terminator and a focused test.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

  • Adds bounded grouping based on punctuation, speakers, timing gaps, overlaps, duration, and text length.
  • Applies grouping to canonical, translated, and provisional live transcript views.
  • Adds an explicit edit mode that continues to save against original cue IDs.
  • Adds focused grouping and component-level interaction tests.

Review fix: multilingual sentence endings

Addressed the Arabic question-mark finding in decfd40 with Unicode Sentence_Terminal. Regression tests cover Arabic, quoted Arabic and Hindi, plus original/translated Arabic rendering and exact seeking. All 91 focused tests, TypeScript, scoped Biome and whitespace checks pass. The approved English output remains byte-for-byte equivalent as parsed sentence entries (12 fragments → 3 sentences).

Review-fix walkthrough · Validation log · English parity check

Arabic and Hindi sentence boundaries before and after

This additional proof uses the actual component with a synthetic multilingual fixture and mocked storage/auth.

Comment thread apps/web/lib/transcript-sentences.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant