chore: add Copilot instructions (project guidance)

2026-05-14 00:02:25 -07:00
commit 2ea3fdee30
1 changed files with 67 additions and 0 deletions
@@ -0,0 +1,67 @@
 The repository is a browser-only local video key-frame extractor (suggested Vite + React + TypeScript).
 Be concise and only change files relevant to this feature-area unless asked to expand the scope.
 Quick context (what this project is and why)
 - Goal: let a user pick a local video file, scan for visually significant frames, and copy/download a small set (4–10) of full-resolution stills. No backend; everything runs in the browser.
 - Primary design choices: use HTMLVideoElement + Canvas API for V1 sampling; avoid heavy WASM tools (ffmpeg.wasm, OpenCV.js) for initial implementation. Move analysis to a Worker/OffscreenCanvas in V2.
 Where to look (key files/dirs)
 - `design-document.md` — authoritative implementation notes and algorithms (frame difference, histogram, phash), default settings, and suggested file layout.
 - Suggested source layout (not yet implemented): `src/app`, `src/components`, `src/media`, `src/analysis`, `src/workers`, `src/types` — use these paths if adding code.
 High-level architecture
 - File picker → createObjectURL → hidden `<video>` element → sampling loop (seek → draw to small analysis canvas) → compare to last accepted key-frame → collect candidate timestamps → rank/time-bucket → render full-res frames only for chosen candidates → Copy/Download.
 What a PR should do (examples of correct changes)
 - Add `src/media/seekVideo.ts` which implements the seek+promise pattern (listen for `seeked` and `error`). See `design-document.md` for example.
 - Add `src/analysis/frameDifference.ts` that implements luminance-based diff with the pixelDeltaThreshold and changedPixelRatioThreshold defaults from `design-document.md`.
 - UI work should use React + TypeScript; keep state minimal (video meta, settings, candidates, progress). Prefer small focused components listed in the suggested file structure.
 Project-specific patterns and conventions
 - Use a small analysis canvas for pixel comparisons (defaults: 160×90). Do not store or compare full-resolution frames during scanning.
 - Compare each sampled frame against the last accepted key-frame, not just the immediately previous sample. Enforce a minimum time gap (default 2s) between accepted frames.
 - Keep candidate blobs count small (4–10 final images). Avoid persisting many full-res canvases in memory.
 - Clipboard writes must be triggered by user gestures and require secure contexts (HTTPS / localhost). Provide a download fallback on clipboard failure.
 Important defaults and constants (from design-document.md)
 - ANALYSIS_W = 160, ANALYSIS_H = 90
 - SAMPLE_INTERVAL = 0.5 (seconds)
 - PIXEL_DELTA_THRESHOLD = 32
 - CHANGED_PIXEL_RATIO_THRESHOLD = 0.18
 - MIN_SECONDS_BETWEEN_ACCEPTED = 2.0
 Build / dev / test workflows
 - Recommended dev setup: Vite + React + TypeScript. Example commands to put into README when scaffolding (these are the expected scripts if project is bootstrapped):
  - `npm install`
  - `npm run dev` (Vite dev server)
  - `npm run build`
  - `npm run preview`
 Debugging tips specific to this repo
 - The video decoding/seek behavior is the primary cause of flakiness: verify `video.duration` and `video.currentTime` after `seeked` events. Use conservative bounds (Math.min(time, duration)).
 - If no candidates are found, test with a lower threshold or ensure the sampling interval covers the video's content.
 - For clipboard issues, test locally over `localhost` or via an HTTPS preview server (Vite's dev server is fine for local testing).
 Integration points and possible pitfalls
 - Browser codec limitations: prefer MP4/H.264 for broad compatibility. WebCodecs can be added later but is optional.
 - Worker migration: initial implementation may keep `seek` and `drawImage` on main thread; move only pixel analysis to the worker (send ImageData buffers). See the Worker message shapes in `design-document.md`.
 Small concrete examples to copy into PRs
 - Seek helper (expected file: `src/media/seekVideo.ts`): the repo expects a promise-based `seekVideo(video, time)` which resolves on `seeked` or rejects on `error`.
 - Analysis function (expected file: `src/analysis/frameDifference.ts`): implement `frameDifferenceRatio(a: ImageData, b: ImageData, pixelDeltaThreshold)` returning the fraction of pixels changed.
 What to avoid
 - Do not read the entire File into memory with FileReader.readAsDataURL. Use `URL.createObjectURL(file)` and `URL.revokeObjectURL()`.
 - Do not generate or buffer many full-resolution canvases during scanning.
 - Avoid adding ffmpeg.wasm / OpenCV.js as a V1 dependency — explain the tradeoffs in the PR description if proposing them.
 Testing guidance
 - Unit-test pure analysis code (frameDifference, histogramDiff, ranking). These are deterministic and fast.
 - For integration/e2e, run the app in Vite dev and manually test with representative videos (MP4/H.264). Document repro steps for flakiness (seek failures, large files).
 When in doubt
 - Prefer minimal, well-scoped changes that mirror the design-doc algorithms. Reference specific sections of `design-document.md` in PR descriptions.
 Ask for feedback
 - If any section of this file is unclear or you want more examples (component props, state types, worker message shapes), tell me which area and I will expand with code snippets or tests.