Local AI use cases with Qwen
Qwen is Alibaba's family of open-weight models. It ranges from tiny to very large and includes coder and vision variants that are widely used for local tool calling and coding.
Rebel BitesHome RAG assistant for embedded-systems docs on Proxmox
Ollama, llama.cpp, Open WebUI, BGE-M3 and SearXNG in a GPU-passthrough VM. Moving from 6 GB to 16 GB of VRAM changed which models could run.
Offline meeting minutes with whisper.cpp + Ollama
Turns a 1-hour meeting into structured minutes in under 4 minutes on an RTX 3060, with zero network requests. Bash and Python pipeline included.
Overnight video-to-RAG pipeline for 2,000+ saved clips
A local Qwen3.6 35B with ffmpeg, scenedetect and Whisper indexes saved TikToks and YouTube videos, so you can later find 'that one plumbing tutorial'.
A private ChatGPT for a family of four on a Mac mini
One always-on M4 Mac mini serves Open WebUI to every phone and laptop in the house. Kids get a tutor preset, and three AI subscriptions are gone.
Hybrid workflow: local for private and repetitive, cloud for hard
A Linux sysadmin stopped trying to replace cloud LLMs. Ollama now handles private docs, overnight article summaries and VS Code autocomplete.
Fully local Home Assistant voice assistant on a GPU
Wake word to spoken answer in 2-3 seconds, with zero audio leaving the house. Built from faster-whisper, Piper and Ollama, with a cloud fallback.
Replacing Google Nest Minis with local Assist on llama.cpp
After a year of Nest Minis getting dumber, a household moved to fully local Assist. Higher-quant GGUF models from Hugging Face are what finally fixed tool calling.
Mac Studio as an always-on, LLM-driven smart-home brain
Phrases like 'it's stuffy in here' become real device calls, run by Ollama with qwen3:14b on a Mac Studio. €15 ESP32 satellites keep all audio on the LAN.
Security camera descriptions and search with Frigate GenAI
Frigate sends camera frames to a local vision model (qwen3-vl via Ollama or llama.cpp) for searchable event descriptions, review summaries and chat.
14 months local-first: what still breaks
An honest report from a three-machine home lab. Local handles about 85% of tasks, but four kinds of work still fail, and that cut cloud spend from $187 to about $25 a month.
Aider + qwen2.5-coder:14b as a local Claude Code alternative
A tic-tac-toe test on an RTX 5060 Ti 16 GB. The game worked on the first pass, but the tests and input validation needed several rounds of prompting.
Copilot-style coding assistant from a headless RTX 3090 box
qwen2.5-coder:7b on a home server, reached from a laptop over an SSH tunnel and wired into VS Code through Continue for chat, edits and autocomplete.