Frame selection is the whole game: notes on making LLMs watch video
An LLM can afford about 150 images per video, so which frames you keep decides everything. Engineering notes from claude-real-video: adaptive scene detection, three-channel deduplication, and fusing frames with the transcript on one clock.
Read full article →