Overnight video-to-RAG pipeline for 2,000+ saved clips

What they built

u/Jeidoz uses a local Qwen3.6 35B (Q4, 112k context) for small, specific agentic tasks. They built scripts so the model can understand videos longer than two minutes.

The plan is to let it run for a few nights through 2,000+ saved TikTok and YouTube videos, turning each into RAG-ready metadata.

Setup

• Model: Qwen3.6 35B Q4 with a 112k context window (vision + tool calling built in)
• ffmpeg, scenedetect, Whisper

How it works

1. Split videos into scenes (scenedetect, ffmpeg)
2. Transcribe audio with Whisper
3. The vision-capable model writes searchable metadata for the RAG system

Do it yourself

1. Use scenedetect to cut each video into scenes and grab keyframes
2. Transcribe audio with Whisper
3. Send keyframes and transcript to a local vision model and ask for title, topics and summary as JSON
4. Embed the output into a local vector store
5. Run it in batches overnight

Describe this video from the keyframes and transcript. Return JSON: {"title":..., "topics":[...], "summary":..., "useful_for":...}