Local & private AI for homelab
Home servers, NAS boxes and hypervisors running local AI alongside other self-hosted services.
Rebel BitesHome RAG assistant for embedded-systems docs on Proxmox
Ollama, llama.cpp, Open WebUI, BGE-M3 and SearXNG in a GPU-passthrough VM. Moving from 6 GB to 16 GB of VRAM changed which models could run.
Self-hosted bookmarks that tag themselves with local AI (Karakeep)
Karakeep (formerly Hoarder) runs on a Debian LXC, and gemma3:4b in Ollama tags each bookmark. Getting reliable tags from a small model takes one key setting.
Paperless AI on an 8 GB RTX 3060 Ti shared with Frigate
Gemma 3 4B with embeddinggemma answers over 1,318 documents in about 5 seconds, on a GPU that's also running Frigate and Plex.
Self-hosted AI email assistant on Linux (n8n + Ollama)
Classifies IMAP mail with llama3.1:8b, routes it, saves reply drafts and extracts action items as JSON. Reports 85-95% classification accuracy.
Leaving Google Photos for Immich with local AI search
CLIP smart search and face recognition run on the author's own NAS. A GTX 1070 turned days of indexing into one night.
Copilot-style coding assistant from a headless RTX 3090 box
qwen2.5-coder:7b on a home server, reached from a laptop over an SSH tunnel and wired into VS Code through Continue for chat, edits and autocomplete.