68 lines
5.2 KiB
Markdown
68 lines
5.2 KiB
Markdown
The repository is a browser-only local video key-frame extractor (suggested Vite + React + TypeScript).
|
||
|
||
Be concise and only change files relevant to this feature-area unless asked to expand the scope.
|
||
|
||
Quick context (what this project is and why)
|
||
- Goal: let a user pick a local video file, scan for visually significant frames, and copy/download a small set (4–10) of full-resolution stills. No backend; everything runs in the browser.
|
||
- Primary design choices: use HTMLVideoElement + Canvas API for V1 sampling; avoid heavy WASM tools (ffmpeg.wasm, OpenCV.js) for initial implementation. Move analysis to a Worker/OffscreenCanvas in V2.
|
||
|
||
Where to look (key files/dirs)
|
||
- `design-document.md` — authoritative implementation notes and algorithms (frame difference, histogram, phash), default settings, and suggested file layout.
|
||
- Suggested source layout (not yet implemented): `src/app`, `src/components`, `src/media`, `src/analysis`, `src/workers`, `src/types` — use these paths if adding code.
|
||
|
||
High-level architecture
|
||
- File picker → createObjectURL → hidden `<video>` element → sampling loop (seek → draw to small analysis canvas) → compare to last accepted key-frame → collect candidate timestamps → rank/time-bucket → render full-res frames only for chosen candidates → Copy/Download.
|
||
|
||
What a PR should do (examples of correct changes)
|
||
- Add `src/media/seekVideo.ts` which implements the seek+promise pattern (listen for `seeked` and `error`). See `design-document.md` for example.
|
||
- Add `src/analysis/frameDifference.ts` that implements luminance-based diff with the pixelDeltaThreshold and changedPixelRatioThreshold defaults from `design-document.md`.
|
||
- UI work should use React + TypeScript; keep state minimal (video meta, settings, candidates, progress). Prefer small focused components listed in the suggested file structure.
|
||
|
||
Project-specific patterns and conventions
|
||
- Use a small analysis canvas for pixel comparisons (defaults: 160×90). Do not store or compare full-resolution frames during scanning.
|
||
- Compare each sampled frame against the last accepted key-frame, not just the immediately previous sample. Enforce a minimum time gap (default 2s) between accepted frames.
|
||
- Keep candidate blobs count small (4–10 final images). Avoid persisting many full-res canvases in memory.
|
||
- Clipboard writes must be triggered by user gestures and require secure contexts (HTTPS / localhost). Provide a download fallback on clipboard failure.
|
||
|
||
Important defaults and constants (from design-document.md)
|
||
- ANALYSIS_W = 160, ANALYSIS_H = 90
|
||
- SAMPLE_INTERVAL = 0.5 (seconds)
|
||
- PIXEL_DELTA_THRESHOLD = 32
|
||
- CHANGED_PIXEL_RATIO_THRESHOLD = 0.18
|
||
- MIN_SECONDS_BETWEEN_ACCEPTED = 2.0
|
||
|
||
Build / dev / test workflows
|
||
- Recommended dev setup: Vite + React + TypeScript. Example commands to put into README when scaffolding (these are the expected scripts if project is bootstrapped):
|
||
- `npm install`
|
||
- `npm run dev` (Vite dev server)
|
||
- `npm run build`
|
||
- `npm run preview`
|
||
|
||
Debugging tips specific to this repo
|
||
- The video decoding/seek behavior is the primary cause of flakiness: verify `video.duration` and `video.currentTime` after `seeked` events. Use conservative bounds (Math.min(time, duration)).
|
||
- If no candidates are found, test with a lower threshold or ensure the sampling interval covers the video's content.
|
||
- For clipboard issues, test locally over `localhost` or via an HTTPS preview server (Vite's dev server is fine for local testing).
|
||
|
||
Integration points and possible pitfalls
|
||
- Browser codec limitations: prefer MP4/H.264 for broad compatibility. WebCodecs can be added later but is optional.
|
||
- Worker migration: initial implementation may keep `seek` and `drawImage` on main thread; move only pixel analysis to the worker (send ImageData buffers). See the Worker message shapes in `design-document.md`.
|
||
|
||
Small concrete examples to copy into PRs
|
||
- Seek helper (expected file: `src/media/seekVideo.ts`): the repo expects a promise-based `seekVideo(video, time)` which resolves on `seeked` or rejects on `error`.
|
||
- Analysis function (expected file: `src/analysis/frameDifference.ts`): implement `frameDifferenceRatio(a: ImageData, b: ImageData, pixelDeltaThreshold)` returning the fraction of pixels changed.
|
||
|
||
What to avoid
|
||
- Do not read the entire File into memory with FileReader.readAsDataURL. Use `URL.createObjectURL(file)` and `URL.revokeObjectURL()`.
|
||
- Do not generate or buffer many full-resolution canvases during scanning.
|
||
- Avoid adding ffmpeg.wasm / OpenCV.js as a V1 dependency — explain the tradeoffs in the PR description if proposing them.
|
||
|
||
Testing guidance
|
||
- Unit-test pure analysis code (frameDifference, histogramDiff, ranking). These are deterministic and fast.
|
||
- For integration/e2e, run the app in Vite dev and manually test with representative videos (MP4/H.264). Document repro steps for flakiness (seek failures, large files).
|
||
|
||
When in doubt
|
||
- Prefer minimal, well-scoped changes that mirror the design-doc algorithms. Reference specific sections of `design-document.md` in PR descriptions.
|
||
|
||
Ask for feedback
|
||
- If any section of this file is unclear or you want more examples (component props, state types, worker message shapes), tell me which area and I will expand with code snippets or tests.
|