Skip to content

remarks: review anything by talking over a screen recording

The problem

Writing notes while reviewing a site breaks the flow you are judging, and the notes come out vague: which button, which page, slow how? A screen recording keeps everything, but nobody rewatches twenty minutes to find the eight things that were said.

The method

The subtitle file of a narrated recording is already an index: every line of speech has a start and end time, and while you say "this heading is far too small" the heading is on screen. Transcribe the recording, then take one still from the video during each remark. The output is a plain review.md: number, time, the words, the picture.

  • Cut sentences rejoined: cues are joined until a sentence ends or the speaker pauses for 1.5 s, because Whisper splits sentences.
  • Noise dropped: [BLANK_AUDIO], (music) and the like are not remarks.
  • Still while speaking: halfway by default, --at 0 to --at 1; seeks past the end are pulled back to a real frame.
  • Words stay words: markdown characters in speech are escaped.
  • Never half a review: built in a temp folder, renamed into place, an existing review never overwritten.

Sample

for start, end, words in join(cues(srt_text)):          # rejoined remarks
    t = start + (end - start) * 0.5                      # while it was said
    subprocess.run(["ffmpeg", "-ss", f"{t:.3f}", "-i", video,
                    "-frames:v", "1", f"frames/{n:03d}.jpg"])

Use it:

remarks REC.mov                              # REC.srt or REC.vtt beside it
remarks REC.mov --transcribe ./whisper-srt   # offline whisper.cpp on the way

--transcribe CMD runs CMD VIDEO and reads SRT from its output, so any speech-to-text tool fits.

Limits

  • One still per remark; something that changes while you speak gets one moment.
  • Speech-to-text mishears names; the transcript is quoted as given.
  • A speaker who never pauses, with a transcriber that never punctuates, gives one long remark.

Code

methods/remarks/ in the methods repository: the script, whisper-srt, a README and a 25-check test that proves each still shows what was on screen. Tried with real speech through real whisper.cpp.