From 2ea3fdee3008b6799db90da6e6097e7e06211db6 Mon Sep 17 00:00:00 2001
From: Ben <ben@ben.us.com>
Date: Thu, 14 May 2026 00:02:25 -0700
Subject: [PATCH] chore: add Copilot instructions (project guidance)

---
 .github/copilot-instructions.md | 67 +++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100644 .github/copilot-instructions.md

diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
new file mode 100644
index 0000000..41167aa
--- /dev/null
+++ b/.github/copilot-instructions.md
@@ -0,0 +1,67 @@
+The repository is a browser-only local video key-frame extractor (suggested Vite + React + TypeScript).
+
+Be concise and only change files relevant to this feature-area unless asked to expand the scope.
+
+Quick context (what this project is and why)
+- Goal: let a user pick a local video file, scan for visually significant frames, and copy/download a small set (4–10) of full-resolution stills. No backend; everything runs in the browser.
+- Primary design choices: use HTMLVideoElement + Canvas API for V1 sampling; avoid heavy WASM tools (ffmpeg.wasm, OpenCV.js) for initial implementation. Move analysis to a Worker/OffscreenCanvas in V2.
+
+Where to look (key files/dirs)
+- `design-document.md` — authoritative implementation notes and algorithms (frame difference, histogram, phash), default settings, and suggested file layout.
+- Suggested source layout (not yet implemented): `src/app`, `src/components`, `src/media`, `src/analysis`, `src/workers`, `src/types` — use these paths if adding code.
+
+High-level architecture
+- File picker → createObjectURL → hidden `<video>` element → sampling loop (seek → draw to small analysis canvas) → compare to last accepted key-frame → collect candidate timestamps → rank/time-bucket → render full-res frames only for chosen candidates → Copy/Download.
+
+What a PR should do (examples of correct changes)
+- Add `src/media/seekVideo.ts` which implements the seek+promise pattern (listen for `seeked` and `error`). See `design-document.md` for example.
+- Add `src/analysis/frameDifference.ts` that implements luminance-based diff with the pixelDeltaThreshold and changedPixelRatioThreshold defaults from `design-document.md`.
+- UI work should use React + TypeScript; keep state minimal (video meta, settings, candidates, progress). Prefer small focused components listed in the suggested file structure.
+
+Project-specific patterns and conventions
+- Use a small analysis canvas for pixel comparisons (defaults: 160×90). Do not store or compare full-resolution frames during scanning.
+- Compare each sampled frame against the last accepted key-frame, not just the immediately previous sample. Enforce a minimum time gap (default 2s) between accepted frames.
+- Keep candidate blobs count small (4–10 final images). Avoid persisting many full-res canvases in memory.
+- Clipboard writes must be triggered by user gestures and require secure contexts (HTTPS / localhost). Provide a download fallback on clipboard failure.
+
+Important defaults and constants (from design-document.md)
+- ANALYSIS_W = 160, ANALYSIS_H = 90
+- SAMPLE_INTERVAL = 0.5 (seconds)
+- PIXEL_DELTA_THRESHOLD = 32
+- CHANGED_PIXEL_RATIO_THRESHOLD = 0.18
+- MIN_SECONDS_BETWEEN_ACCEPTED = 2.0
+
+Build / dev / test workflows
+- Recommended dev setup: Vite + React + TypeScript. Example commands to put into README when scaffolding (these are the expected scripts if project is bootstrapped):
+  - `npm install`
+  - `npm run dev` (Vite dev server)
+  - `npm run build`
+  - `npm run preview`
+
+Debugging tips specific to this repo
+- The video decoding/seek behavior is the primary cause of flakiness: verify `video.duration` and `video.currentTime` after `seeked` events. Use conservative bounds (Math.min(time, duration)).
+- If no candidates are found, test with a lower threshold or ensure the sampling interval covers the video's content.
+- For clipboard issues, test locally over `localhost` or via an HTTPS preview server (Vite's dev server is fine for local testing).
+
+Integration points and possible pitfalls
+- Browser codec limitations: prefer MP4/H.264 for broad compatibility. WebCodecs can be added later but is optional.
+- Worker migration: initial implementation may keep `seek` and `drawImage` on main thread; move only pixel analysis to the worker (send ImageData buffers). See the Worker message shapes in `design-document.md`.
+
+Small concrete examples to copy into PRs
+- Seek helper (expected file: `src/media/seekVideo.ts`): the repo expects a promise-based `seekVideo(video, time)` which resolves on `seeked` or rejects on `error`.
+- Analysis function (expected file: `src/analysis/frameDifference.ts`): implement `frameDifferenceRatio(a: ImageData, b: ImageData, pixelDeltaThreshold)` returning the fraction of pixels changed.
+
+What to avoid
+- Do not read the entire File into memory with FileReader.readAsDataURL. Use `URL.createObjectURL(file)` and `URL.revokeObjectURL()`.
+- Do not generate or buffer many full-resolution canvases during scanning.
+- Avoid adding ffmpeg.wasm / OpenCV.js as a V1 dependency — explain the tradeoffs in the PR description if proposing them.
+
+Testing guidance
+- Unit-test pure analysis code (frameDifference, histogramDiff, ranking). These are deterministic and fast.
+- For integration/e2e, run the app in Vite dev and manually test with representative videos (MP4/H.264). Document repro steps for flakiness (seek failures, large files).
+
+When in doubt
+- Prefer minimal, well-scoped changes that mirror the design-doc algorithms. Reference specific sections of `design-document.md` in PR descriptions.
+
+Ask for feedback
+- If any section of this file is unclear or you want more examples (component props, state types, worker message shapes), tell me which area and I will expand with code snippets or tests.