Obsidian vault Q&A with Mini-RAG and llama.cpp

What they built

Mini-RAG lets you open a chat about a note or folder from the right-click menu or command palette, with answers retrieved from your own notes by a local model.

Setup

• Provider: llama.cpp (llama-server) or Ollama
• With llama.cpp: one server for chat, a second with --embeddings for a dedicated embedding model
• Settings: temperature, top-K retrieved chunks (1-10), optional on-disk embedding cache (off by default)

How it works

1. llama-server serves one model per process, so the chat model follows the chat server
2. Note chunks are embedded and the top-K excerpts go along with each question
3. If no embedding model is set, it stuffs as much of the note as fits the context

Do it yourself

1. Start a llama.cpp chat server on port 8080
2. Start an embedding server on port 8081
3. In Mini-RAG options pick llama.cpp, set both URLs and enable the dedicated embedding model
4. Right-click a note or folder → open Mini-RAG chat

llama-server -hf bartowski/Qwen2.5-3B-Instruct-GGUF:Q4_K_M -ngl 99 -c 8192 --port 8080
llama-server -hf nomic-ai/nomic-embed-text-v1.5-GGUF:Q8_0 --embeddings --port 8081