Copilot-style coding assistant from a headless RTX 3090 box

What they built

Privacy concerns kept the author away from Copilot even after seeing its productivity boost. Instead they run the model on a headless home desktop and use it from their laptops through VS Code's Continue plugin.

Setup

• Headless Ubuntu desktop with NVIDIA RTX 3090
• Ollama with qwen2.5-coder:7b (7.6B, Q4_K_M, 4.7 GB)
• VS Code + Continue on the laptop, through an SSH port forward

Results & numbers

• No sluggishness reported, and satisfied with the quality of assistance

Gotchas

• qwen2.5-coder:32b (Q4_K_M, ~20 GB) fit the GPU but was too big as a coding agent; Continue sometimes froze

Do it yourself

1. Install Ollama on the GPU machine and set OLLAMA_HOST=0.0.0.0:11434 via systemctl edit ollama.service
2. ollama run qwen2.5-coder:7b
3. From the laptop: ssh -L 11434:localhost:11434 user@server
4. Add the config below to ~/.continue/config.yaml

name: Local Config
version: 1.0.0
schema: v1
models:
  - name: Qwen
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles: [chat, edit, apply, autocomplete]