# PHYS1002 Lecture Video Workflow

This runbook turns a downloaded course recording into ordered, compact lecture
clips with transcripts, polished English descriptions, a paste-ready Canvas
page, and optional SharePoint/Canvas publication. It is based on the proven
MSDM5051 workflow but keeps the scientific and editorial decisions specific to
PHYS1002.

For first-time setup on another computer or resumption of a transferred run,
read [`lecture-video-portability.md`](lecture-video-portability.md) completely.
It supplies dependency setup, portable path variables, a read-only preflight,
artifact schemas, source-to-clip timing, template adaptation, and recovery.
The current computer's absolute paths below are historical locations; use the
resolved local paths from that guide.

## One-command interface

Use one of these commands in Codex:

```text
/lecture-video latest
/lecture-video latest --publish
/lecture-video /absolute/path/to/recording.ts MMDD
```

The command is an agent workflow trigger, not a literal shell program. The
agent performs discovery, transcription, semantic analysis, clipping,
encoding, documentation, QA, cleanup, and the scoped Git commit without asking
the user to restate this runbook. `--publish` also requests the external upload
and Canvas steps.

Every invocation must also read and execute
[`lecture-video-bilingual-subtitle-workflow.md`](lecture-video-bilingual-subtitle-workflow.md)
completely unless the user explicitly requests videos without burned
subtitles.

When the user requests English slide replacement or a PDF inset, also read
[`lecture-video-pdf-overlay-workflow.md`](lecture-video-pdf-overlay-workflow.md)
completely. This optional extension includes transcript-first page matching;
do not enable it merely because an exported PDF exists.

## Stable defaults

- Source: newest `~/Downloads/*.ts`, unless the command gives a path.
- BiliNote checkout: resolve `BILINOTE_ROOT` on the current computer. The
  historical location is `BiliNote`; BiliNote
  and its local transcription helper are separate from this repository.
- Output directory: repository-root `videos/`.
- Date prefix: lecture date in `MMDD` form, confirmed from the recording, course
  schedule, or explicit user input rather than assumed from download time.
- Scope: the instructor's substantive lecture, including useful questions,
  demonstrations, worked reasoning, and the closing recap.
- Exclude: waiting, attendance, class breaks, equipment or cursor setup,
  repeated low-information transitions, and any separate tutorial or problem
  session.
- Chapter size: one coherent idea, comparison, example, or worked argument,
  normally about 3–15 minutes. Do not cut on a fixed-duration grid.
- Filename: `MMDD_PP_Polished_English_Title.mp4`, with consecutive two-digit
  part numbers.
- Video: H.264 High Profile at the source frame rate, using `libx264`, slow
  preset, CRF 30, and `yuv420p`.
- Crop: inspect every new recording rather than inheriting the previous crop.
  When a crop improves focus, use even-pixel coordinates and dimensions, apply
  it directly to the original source, and encode at the resulting native crop
  resolution. Do not scale the crop back to 1080p or another standard size.
- Audio: AAC mono at 56 kbps.
- Web playback: MP4 with `+faststart`.
- Student-facing language: polished English unless requested otherwise.
- Webpage title: propose one polished English title of no more than 10 words
  for every lecture. Print it in the chat handoff for the user to paste into
  the Canvas title input. Do not insert it into the page HTML or store it as a
  heading in `MMDD_canvas_page.txt`. Treat it as a catalog label rather than a
  promotional headline: use concrete content keywords and represent the
  lecture's scope, especially its opening and closing substantive topics. A
  `From ... to ...` construction is useful when the lecture spans a range.
- Burned subtitles: English and Traditional Chinese, following the required
  separate bilingual-subtitle workflow.
- Student page: simple paste-ready HTML stored in a `.txt` file.

These defaults can be adjusted when the recording or upload destination
requires it. Record every deliberate deviation in the manifest.

## Expected output package

For lecture date `MMDD`, create:

```text
videos/
  MMDD_01_First_Title.mp4
  MMDD_02_Second_Title.mp4
  ...
  MMDD.transcript.raw.json
  MMDD_environment.json
  MMDD_edit_plan.json
  MMDD_transcript.md
  MMDD_bilingual_subtitles.json
  subtitles/MMDD_01_bilingual.ass
  subtitles/MMDD_02_bilingual.ass
  ...
  MMDD_segments.json
  MMDD_scientific_review.md
  MMDD_verification.json
  MMDD_summaries.md
  MMDD_canvas_page.txt
scripts/
  prepare_MMDD_lecture.py
```

`MMDD_segments.json` is the production log. Include the source path, recording
duration, processed cutoff, excluded ranges and reasons, selected source
ranges, encoding settings, final filenames, verified durations, sizes,
bitrates, and stream properties. Keep source/PDF content hashes, environment
versions, specific instructor approvals and the executable edit plan with the
package, following the portability guide's schemas and handoff inventory.

`MMDD_summaries.md` is student-facing. It contains only the lecture heading,
ordered titles, and one polished paragraph per part. Do not include filenames,
source ranges, break notes, edit decisions, or QA details.

`MMDD_canvas_page.txt` is paste-ready HTML with this structure:

```html
<p>One polished paragraph summarizing the whole lecture.</p>

<hr>

<h2>Part 01: First Title</h2>

<p>One polished paragraph describing this clip.</p>

<p><video id="lecture-video-MMDD-01" style="width: 100%; height: auto; display: block;" controls="controls" playsinline="playsinline" aria-label="Part 01: First Title" src="CANVAS_VIDEO_URL_01">Your browser does not support HTML5 video.</video></p>

<details style="margin: 1rem 0 1.5rem; border: 1px solid #c7cdd1; border-radius: 6px; padding: 0.65rem 0.85rem; font-size: 0.85rem;">
<summary style="cursor: pointer;"><strong>English transcript</strong></summary>
...
</details>

<details style="margin: 1rem 0 1.5rem; border: 1px solid #c7cdd1; border-radius: 6px; padding: 0.65rem 0.85rem; font-size: 0.85rem;">
<summary style="cursor: pointer;"><strong>中文字幕</strong></summary>
...
</details>
```

Generate this structure with `scripts/build_canvas_lecture_page.py`; do not ask
an LLM to concatenate hundreds of subtitle cues. Repeat the heading, summary,
native-player placeholder, English transcript disclosure, and Traditional
Chinese transcript disclosure for every part. The English box comes first.
Both `<details>` elements omit the `open` attribute and are therefore collapsed
by default. Use plain clip-local timestamps; do not make them links, because a
standard media-fragment link opens another media resource rather than seeking
the player already on the Canvas page.

This file is the canonical page source for the later Canvas HTML round trip. It
supplies the global summary, headings, descriptions, part order, player
structure, and proofread transcript text. Before Canvas assigns the media
attachment IDs, the only unknowns are the `CANVAS_VIDEO_URL_PP` placeholders.
The proposed webpage title is delivered separately in chat for the Canvas title
input, so never add it as an `<h1>` or otherwise duplicate it in the page body.
The first heading in the HTML must be `<h2>`.

## Phase 1: discover and inspect

1. Resolve the source to an absolute path. For `latest`, sort `.ts` files in
   `~/Downloads` by modification time and use the newest.
2. Inspect the source with `ffprobe`: duration, dimensions, frame rate, codecs,
   audio channels, and bitrate.
3. Sample representative frames across the full recording, including the
   beginning, middle, end, and different instructor positions. Decide whether
   distracting audience, ceiling, wall, or unused-room areas justify a crop.
   Do not assume the previous lecture's camera geometry still applies.
   At this inspection stage, also check the language of the **main slide body**
   across representative topics and teaching-panel views. English is the
   intended slide language; occasional Chinese words, quotations, names or
   bilingual subtitles do not count as a wrong-screen recording. If the main
   slide body is Chinese for a sustained portion, alert the instructor and ask
   whether to replace that panel with the English PDF **before transcription,
   chapter production or encoding begins**. Pause production for that choice;
   do not assume a clear picture means the correct screen was recorded, and do
   not automatically enable PDF replacement. An explicit request to replace an
   identified recording already supplies the required authorization. Record
   the inspection, affected portions and instructor's decision in the manifest.
4. When cropping, define a single stable rectangle for the recording. Keep the
   teaching displays, instructor, and ordinary gestures; exclude seated
   audience heads in normal frames when practical. An occasional person who
   stands up does not require a tighter crop. Use even `x`, `y`, width, and
   height values for `yuv420p`, and record the geometry and rationale in the
   manifest.
5. Confirm the lecture date from a visible slide, spoken course context, the
   course schedule, or explicit user input. A generated filename is supporting
   evidence, not sufficient evidence by itself.
6. Read BiliNote's current transcription skill and implementation, resolved
   under the current `BILINOTE_ROOT`:

   ```text
   BILINOTE_ROOT/.agents/skills/video-semantic-clips/SKILL.md
   ```

7. Reuse the configured BiliNote transcriber. Do not silently substitute
   another speech-to-text system or alter its model configuration.
   Use that skill's transcriber implementation; PHYS1002's full-workflow
   instruction selects all meaningful main-lecture chapters without the
   BiliNote skill's chapter-selection pause. PHYS1002 also supplies the English
   titles, output directory, subtitle rules, and approved scientific-edit policy.

## Phase 2: transcribe

Write the result into this repository so the original recording remains
untouched. Define these variables using the portability guide first:

```sh
cd "$BILINOTE_ROOT"
"$LECTURE_PYTHON" \
  "$BILINOTE_ROOT/.agents/skills/video-semantic-clips/scripts/transcribe_video.py" \
  "$LECTURE_SOURCE" \
  --output "$PHYS1002_ROOT/videos/$LECTURE_DATE.transcript.raw.json" \
  --project-root "$BILINOTE_ROOT" --device auto
```

The expected configuration is faster-whisper `large-v3`. Prefer CUDA when
available. If the sandbox hides the GPU or prevents the BiliNote logger from
writing, request the narrow permission needed for this command rather than
changing the transcriber. Never modify the source `.ts` file.

The full recording may be transcribed internally to identify breaks and
speaker transitions. The readable transcript should contain only selected
lecture ranges.

After the chapter ranges are final, run the required bilingual-subtitle
workflow: proofread the ASR in clip-scoped batches, review the Traditional
Chinese translation, and generate the timed subtitle sources before encoding.

After the proofread bilingual JSON and polished summaries exist, assemble the
placeholder Canvas page mechanically:

```sh
python -B scripts/build_canvas_lecture_page.py MMDD
```

The script reads `MMDD_summaries.md`, `MMDD_bilingual_subtitles.json`, and the
existing overview paragraph in `MMDD_canvas_page.txt`, then rewrites the Canvas
page with one native-player placeholder and two default-collapsed transcript
boxes per part. Transcript assembly is not an LLM task.

## Phase 3: identify meaningful chapters

Analyze timestamped ASR in consecutive windows and reconcile the full lecture
globally. Use all available evidence:

- topic changes and explicit verbal transitions;
- visible section titles, slide changes, demonstrations, and equations;
- questions that start or conclude a coherent line of reasoning;
- instructor recaps and statements that the lecture or a section has ended;
- speaker changes, silence detection, and room activity around breaks;
- short frame samples immediately before and after uncertain boundaries;
- relevant current PHYS1002 slides when they clarify a heading or term.

Choose boundaries at complete sentences or nearby natural pauses. Preserve
short transitions when they introduce the next idea. If a single topic is
split by a break, encode its separated source ranges as one concatenated
student-facing clip when that gives a more coherent result.

For Part 01, inspect the original audio with `silencedetect` and begin at the
first real instructor speech with only a small natural lead-in. Whisper can
absorb initial silence into its first timestamp, so do not trust that timestamp
alone.

Check scientific names and claims while writing titles and summaries. Fix
obvious ASR errors such as planet names, technical terms, and historical names;
do not repeat a mistaken transcript wording in student-facing copy. Summaries
should accurately describe what was taught without silently adding material
that is absent from the recording.

## Phase 4: create the preparation script

Use the newest verified `scripts/prepare_MMDD_lecture.py` as the working
reference. These are date-specific implementations, not generic launchers.
Follow the portability guide's template-adaptation checklist, then update:

- source path, lecture date, duration, and cutoff;
- ordered clip IDs and polished English titles;
- one or more source ranges per clip;
- the inspected crop decision and, when used, its source-relative `x`, `y`,
  width, and height;
- one polished English webpage title of no more than 10 words, for the final
  chat handoff only;
- one polished description per clip and one global lecture summary;
- excluded ranges and explicit production reasons;
- expected encoding and verification properties.

Titles and descriptions are editorial summaries, not lightly cleaned ASR.
Remove filler, false starts, classroom logistics, and recognition errors.

The script generates the readable transcript, manifest, summaries, and Canvas
HTML source, then encodes and verifies every MP4. It must reuse already verified
outputs and allow selected failed clips to be forced:

```sh
python -u scripts/prepare_MMDD_lecture.py --encode
python -u scripts/prepare_MMDD_lecture.py --encode --select 3,7 --force
```

## Phase 5: encode

For each continuous range, decode from the original recording with a pre-roll
(normally 10 seconds before the intended cut, or source start). TS seeking can
lose the first decodable pictures while preserving audio. Trim video and audio
to the intended cut **on the same source-relative timeline**, then subtract the
same cut timestamp from both streams. Do not independently reset their first
decoded frames with `PTS-STARTPTS`: this can hide missing video and introduce
an A/V offset even when output duration and audio checks pass. For example,
after seeking to `cut - 10`, use `trim=start=10:end=...`,
`setpts=PTS-10/TB`, `atrim=start=10:end=...`, and `asetpts=PTS-10/TB`.
Verify that the first retained frame is within one source frame of the cut;
increase pre-roll or decode from source start if necessary. Apply the inspected
crop when required, and encode in one generation with:

```text
-c:v libx264 -preset slow -crf 30
-profile:v high -level 4.1 -pix_fmt yuv420p
-c:a aac -ac 1 -b:a 56k
-movflags +faststart
```

For a topic with multiple ranges, open each range as a separate FFmpeg input,
apply the same crop to every video input, preserve the common audio/video time
origin for each range, and join them with the `concat` filter. Inspect both
sides of every join.

Lecture video is mostly static, so CRF encoding gives good compression without
a high target bitrate. Preserve the crop's native pixels and the source frame
rate so projected labels, motion, and handwriting remain natural. Never crop a
previously encoded MP4: crop and encode directly from the original TS or other
camera source to avoid an unnecessary lossy generation. Do not resize after
cropping. Use two-pass encoding only when an upload destination imposes a hard
size budget, and document that exception.

Burn the required bilingual subtitles during this same direct-from-source
encoding generation. Do not add subtitles by re-encoding a generated MP4.

Transport streams can emit missing-reference-picture warnings during seeking.
They are acceptable only when the finished MP4 passes duration, stream, audio,
and visual checks.

## Phase 6: verify

Verification is mandatory for every output:

1. `ffprobe` confirms MP4, source-rate H.264 High Profile video, AAC mono audio,
   the exact native crop dimensions (or original dimensions when uncropped),
   `yuv420p`, and duration within 0.75 seconds of the selected source ranges.
2. Record duration, size, bitrate, codecs, dimensions, frame rate, pixel format,
   and audio channels in `MMDD_segments.json`.
3. Visually inspect the first and last frame of every clip, sampling the actual
   video-stream duration rather than a potentially longer audio/container tail.
   Check video-to-source alignment as well as audio alignment near the start
   and end. Matching audio alone cannot rule out a shifted video track.
4. Inspect frames immediately before and after every concatenated join.
5. Confirm Part 01 begins at speech rather than a long silent lead-in.
6. Confirm numbering is consecutive and identical in filenames, the manifest,
   summaries, transcript, and Canvas source.
7. Confirm the student-facing summaries and Canvas source contain no
   source-recording edit timecodes, exclusions, edit notes, join notes, or QA
   details. Clip-local timestamps inside the student transcript boxes are
   expected.
8. Check that titles and summaries use correct astronomical terminology and do
   not preserve obvious ASR hallucinations.
9. For cropped output, sample frames across every clip and confirm the crop
   remains stable, retains both teaching displays and normal instructor
   gestures, and generally excludes seated audience heads. Confirm there is no
   scaling by matching the output dimensions exactly to the crop width and
   height.
10. Complete the subtitle validation, visual inspection, audio spot-check, and
    full decode checks required by the bilingual-subtitle workflow.

## Phase 7: optional SharePoint and Canvas publication

Run this phase only when the trigger contains `--publish` or the user otherwise
explicitly requests publication.

1. Prefer a connected, user-authorized browser surface with the user's existing
   signed-in session. Do not inspect credential stores, cookies, or profiles.
2. Upload the ordered MP4 files to the user-specified SharePoint folder.
3. Set view-only sharing as requested and as permitted by the institution.
4. Begin with the generated `CANVAS_VIDEO_URL_PP` placeholders. Do not replace
   them with guessed links before Canvas has assigned the attachment IDs.
5. Use a video-first HTML round trip so Canvas assigns and preserves the media
   attachment IDs:
   - Create a blank Canvas page and insert the uploaded videos with the Rich
     Content Editor. Canvas insertion order is not an identity check.
   - Switch to the HTML editor and copy all Canvas-generated HTML. The pasted
     HTML may contain only media elements or may already contain some or all of
     the page text; both forms are valid media-input files.
   - Parse the pasted HTML as a document fragment. Accept Canvas attachment
     iframes, existing `<video>` elements, and
     `<video><source></video>` markup. Obtain the numeric attachment ID from a
     `media_attachments_iframe/ATTACHMENT_ID` URL, a direct
     `/courses/COURSE_ID/files/ATTACHMENT_ID/download` URL, or the corresponding
     Canvas media metadata. Never invent an ID.
   - Whether the media-input file contains only videos or also includes page
     text, use it only to recover the media identities and URLs. Keep
     the local `videos/MMDD_canvas_page.txt` as the canonical summaries and
     transcript source; never duplicate pasted text around it.
   - Match each Canvas media title's exact MP4 filename to the lecture manifest,
     then reorder by manifest part number. Do not infer identity from pasted
     iframe order: uploads can appear out of order. Reject missing, duplicate,
     unknown, or ambiguous filenames and attachment IDs. For pasted Canvas
     iframes, run `scripts/sort_canvas_media_by_filename.py MMDD RAW_HTML`
     first, then pass its `videos/MMDD_canvas_media.txt` output to the page
     assembler. Stop rather than assigning an unidentifiable video by position.
     For enhanced variants with a separate production manifest, pass
     `--manifest PATH_TO_VARIANT_MANIFEST --output PATH_TO_VARIANT_MEDIA` to
     the sorter. Pass the variant's `--base-page` and `--output` to the page
     assembler as well. Keep original-version media inputs and page source
     intact; new uploads receive their own attachment IDs even when the lecture
     date and part numbers are unchanged.
   - Fill the placeholders by running the deterministic assembler again:

     ```sh
     python -B scripts/build_canvas_lecture_page.py MMDD --media-html videos/MMDD_canvas_media.txt
     ```

     The script replaces each Canvas iframe, or normalizes each existing video
     URL, to the browser's native HTML5 player. It makes the player exactly as
     wide as the page and omits a fixed height and aspect ratio so the intrinsic
     video dimensions are retained:

     ```html
     <p><video style="width: 100%; height: auto; display: block;" controls="controls" playsinline="playsinline" src="https://canvas.ust.hk/files/ATTACHMENT_ID/download?download_frd=1">Your browser does not support HTML5 video.</video></p>
     ```

     Prefer native HTML5 only when its **decoded resolution** has been verified
     against the uploaded file. A functioning `/files/ID/download` URL is not
     evidence of original quality: it may serve a 360p derivative while the
     Canvas player has 1080p adaptive streams. CSS width changes display size,
     not decoded resolution. Inspect `videoWidth`/`videoHeight` after metadata
     loads, and compare with the original Canvas player's Quality menu.

     Use the Canvas host and attachment ID from the original media markup. A
     verified stable HD MP4 URL can retain the native player; do not embed
     temporary signed CDN URLs, `blob:` URLs, or a DASH manifest as if it were
     an ordinary MP4. Never guess a bitrate or resolution query parameter.
     If HD is available only through Canvas's adaptive player, or the native
     player fails sanitization, retain the original iframe using:

     ```sh
     python -B scripts/build_canvas_lecture_page.py MMDD --media-html videos/MMDD_canvas_media.txt --player canvas
     ```

     The Canvas fallback uses a page-width responsive 16:9 wrapper, appropriate
     for the current 1920×1080 package. For a differently shaped future video,
     adjust the wrapper to its verified native aspect ratio. Verify the wrapper
     survives Canvas sanitization and retains controls at narrow widths.
     Canvas can initially expose only 360p while HD processing is incomplete;
     recheck available qualities before concluding the upload lost resolution.
     Students can select Settings → Quality → 1080p when available. Do not
     claim an instructor-wide forced-HD default unless actually verified.
6. Report the path to the completed `videos/MMDD_canvas_page.txt`. Do not paste
   the large HTML document into the chat unless the user explicitly asks; the
   user will open the local `.txt` file and copy it into Canvas.
7. Print the proposed English webpage title in chat, separately from the HTML,
   so the user can paste it into the Canvas title input. Confirm it contains no
   more than 10 words and helps students locate the page by clearly reflecting
   the lecture's beginning, ending, and principal content range.
8. Save once, reopen the page, and copy the saved HTML back out. Confirm Canvas
   retained the chosen player, its verified source URL, page-width style, and
   two independently collapsible transcript boxes. Confirm both boxes are
   closed by default, then visually verify one desktop width and one
   narrow/mobile width. For native players, check decoded dimensions for every
   clip. For Canvas players, check HD availability for every clip and verify
   actual HD decoding on representative clips, including after seeking.

If a signed-in browser or required connector is unavailable, leave the verified
local package and paste-ready HTML intact and report the exact missing
capability. Never recover credentials from local browser data.

## Phase 8: cleanup and Git

- Track and stop only transcription, FFmpeg, preview, browser, or helper
  processes started by this task.
- Remove task-specific temporary frames, contact sheets, comparison encodes,
  browser profiles, and Python bytecode caches.
- Never delete, overwrite, or modify the original `.ts` recording.
- Never stage or commit generated MP4 files.
- Stage and commit only the preparation script, workflow documentation, raw
  and readable transcripts, bilingual subtitle sources, manifest, summaries,
  and Canvas page source that belong to the completed run.
- Preserve unrelated user changes and untracked files.

## 0902 reference result

The first PHYS1002 run used the 2 September 2026 opening lecture as its
reference. It divided the substantive material into nine semantic chapters,
removed the class break and break-time Canvas logistics, preserved the opening
speech and closing recap, and used the inspected native crop
`1668:840:66:36` from the 1920×1080 original. The crop follows the outer edges
and highest top edge of the two projection screens and normally excludes
student heads without scaling. The nine verified clips total approximately
277 MiB. The reference implementation and artifacts are:

```text
scripts/prepare_0902_lecture.py
videos/0902_segments.json
videos/0902_summaries.md
videos/0902_canvas_page.txt
videos/0902_transcript.md
```