14 months local-first: what still breaks

What they built

Sam Hartley has run a local-first AI setup for 14 months across a Mac, a Windows PC and an Ubuntu box. He writes about the tasks where local still loses, because 'nobody's building the my local LLM produced garbage blog post'.

Setup

• Mac, 16 GB RAM: Qwen 3.5 9B, Granite 3.2 8B (orchestration, writing)
• WinPC with RTX 3060 12 GB: Qwen 3 Coder 30B, DeepSeek R1 8B (code, reasoning)
• Ubuntu PC, 128 GB RAM: larger DeepSeek R1, MinCPM-V (heavy tasks, vision)

Results & numbers

• Local handles ~85% of tasks; electricity ~$12/month
• Cloud spend fell from $187 to ~$25/month; over $2,000 saved in a year
• A distributed-systems bug: 40 minutes wasted locally versus 30 seconds in a cloud model

Gotchas

• Complex multi-step reasoning: local models pattern-match instead of tracing logic
• No current information without a RAG/search pipeline
• Documents over ~3 pages lose critical details
• Small local vision models fail on diagrams and UI mockups
• Long-form text drifts after ~1,500 words

“free output that's wrong isn't free — it costs your time to fix it.”— Sam Hartley

Do it yourself

1. Route quick questions and summaries to a ~9B local model
2. Use a ~30B coder for code under ~200 lines
3. Use local for data extraction ('surprisingly good, even at 7B')
4. Send multi-file debugging, current-info questions and complex vision to the cloud