💻 Local LLMs for Coding: What Actually Works (2026)

I read a model's API reference before I open its chat window, and Ollama's is short enough that I read the whole thing in one sitting: five endpoints, a documented list of which OpenAI parameters it honors and which it silently drops, done. That's the piece missing from most "run an LLM locally for coding" posts — they show you ollama run in a terminal and stop, as if a REPL is what a working developer wants. It isn't. You want the model living behind an API your editor already knows how to call, and you want to know, before you commit an afternoon to it, whether the model you can actually fit in VRAM is a real collaborator or a toy that autocompletes for i in range. Here's what I found wiring this into my own setup, script in hand.

By the numbers#

• Ollama's /v1/chat/completions endpoint supports streaming, JSON mode, vision and tool calls; it does not support tool_choice, logit_bias, n, or logprobs (Ollama docs)
• A 16GB RTX 4080 running a 14B coding model (Q4_K, 16K context) generates at ~51.2 tokens/sec; prompt processing on the same setup runs ~2,295 tokens/sec (Hardware Corner GPU benchmarks)
• Coding-benchmark write-ups put 7B-class coding models around 72% HumanEval, 14B-class around 82%, and 32B-class around 87%, with SWE-bench scores rising from roughly 0% (7B, not meaningfully scored) to ~20% (14B) to ~25% (32B) on the same tier ladder (RunLocalModel benchmark table)
• GitHub Copilot's individual Pro plan runs $10/month, with Business at $19/user/month — the number your local setup is actually competing against, not "free vs infinite" (multiple 2026 pricing write-ups)

The API surface: what Ollama actually gives your editor#

Everything downstream depends on one fact: Ollama isn't a chat app that happens to have an API bolted on, it's a server (localhost:11434) that speaks a subset of the OpenAI schema. The subset matters more than the headline. Supported: model, messages, temperature, top_p, frequency_penalty, presence_penalty, max_tokens, stop, stream, seed, response_format, and tools. Not supported: tool_choice, logit_bias, n, and logprobs. If your editor extension or agent framework pins its behavior to any of those four, it'll fail silently or fall back to a default instead of erroring — which is exactly the kind of thing you find at 11pm, not by reading a changelog.

I confirmed the shape with the same fifteen seconds I'd spend on any new API before trusting it with real requests:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-coder:14b","messages":[{"role":"user","content":"write a python function that dedupes a list preserving order"}]}'

That's the entire contract. base_url is http://localhost:11434/v1/, the API key field is required by the OpenAI client libraries but ignored server-side — pass any non-empty string. Also worth knowing before you build an image-aware review tool on top of this: image inputs only work base64-encoded, not as URLs, and the completions endpoint (as opposed to chat completions) takes a plain string prompt, not an array. Small print, but the kind that breaks a batch script if you copy a snippet written for hosted OpenAI without reading which corners Ollama actually replicated.

Wiring it into your editor#

Once the endpoint is confirmed, editor integration is a config file, not a plugin ecosystem. Continue.dev is the one I keep coming back to because its config is explicit JSON/YAML rather than a settings-panel black box:

{
  "models": [
    {
      "title": "Local Qwen3 Coder 14B",
      "provider": "ollama",
      "model": "qwen3-coder:14b",
      "apiBase": "http://localhost:11434/"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Local Autocomplete",
    "provider": "ollama",
    "model": "qwen3-coder:7b",
    "apiBase": "http://localhost:11434/"
  }
}

Two models, two jobs — a smaller one for tab-autocomplete where latency is the whole product, a bigger one for chat and multi-file edits where you'll wait a little longer for a better answer. Cline and most other OpenAI-compatible extensions take the same two fields under different names (apiBase/baseUrl, apiKey/token); if an extension advertises "custom OpenAI-compatible provider" support at all, point it at http://localhost:11434/v1/ and it works, because that's the whole point of Ollama shipping this endpoint instead of a bespoke one. Nothing about this setup is fragile to Ollama updates, either — when the client itself grew agent features on top of the same server, the API underneath didn't move, and every editor integration and script I had pointed at port 11434 kept working without an edit. That's the payoff of building on a documented contract instead of scraping a UI: your full Ollama install and daily-use setup is the one thing in this pipeline you configure once.

Which model sizes are real, and which are toys#

This is the part every "top 10 local coding models" listicle skips, because ranking benchmark scores is easier than being honest about VRAM. Sizes only mean something in the context of what fits and what you're actually asking the model to do:

The jump that actually changes your workday is 7B → 14B, not 14B → 32B. SWE-bench — which grades whether a model can resolve a real GitHub issue end to end, not just complete a function — barely registers at 7B and only crosses into "occasionally solves the whole ticket" territory at 14B. If your card only has 6-8GB, be honest with yourself about what you're running: a fast, cheap autocomplete engine, not a pair programmer. VRAM planning is the actual decision here, and if 24GB isn't in the budget yet, a rented cloud GPU is the cheap way to test whether the 32B tier is worth buying hardware for before you commit to a card.

Latency: what "fast enough" actually means#

Two different numbers matter and reviews conflate them constantly. Prompt processing — the model reading your open file and the surrounding context before it writes a token — runs north of 2,000 tokens/sec on a 16GB card. That's not your bottleneck; a 4,000-token file loads in under two seconds regardless of model size. Generation is the number that actually gates the experience: ~51 tokens/sec for a 14B model at Q4 on a 16GB RTX 4080. A 60-token function completion streams in a little over a second at that rate — indistinguishable from a cloud API's round trip once you count its network hop. Where local loses is long agentic outputs: a 500-token multi-file diff takes roughly ten seconds to fully stream at that same rate, which is the point where I stop watching the terminal and go read something else. Autocomplete wants the fast, small model; anything that generates a wall of text wants you to budget for the wait or drop to a smaller model and accept a worse answer for the trade.

The actual cost math#

GitHub Copilot Pro is $10/month; Business is $19/user/month. That's the real comparison, not "free vs. paying," because your local setup isn't free — you already paid for the GPU, and it's running whether or not you use it for code. The honest framing: once the card is bought, the marginal cost of every additional completion is $0 and the marginal cost of every additional Copilot request is bounded by your plan's request allowance. For someone generating code constantly — batch refactors, test generation, an agent looping on a repo overnight — that difference compounds fast; for someone who fires off a dozen completions a day, $10/month buys a better model than your 16GB card can host, and the math doesn't favor local at all. I don't think there's a universal right answer here, which is exactly why "AI coding is free if you self-host" is the kind of line that shows up on landing pages and nowhere in an actual invoice. Run your own numbers against your own request volume before you believe either side. Our broader local vs. cloud cost breakdown walks the amortization math in more depth if you want the full spreadsheet, not just the two headline prices.

The privacy case for client code#

This is the one place I'll say the math doesn't matter. If you write code under an NDA, a client contract with a data-handling clause, or anything regulated, a cloud coding assistant means that source code crosses onto a third party's infrastructure — and per most vendors' own terms, may be retained or used for model improvement unless you're specifically on an enterprise tier with logging disabled, which not every client's tooling budget covers. Ollama's server never phones out. The request starts and ends on your machine, full stop, and that's not a benchmark claim, it's an architecture fact. For a lot of freelance and contract developers, that single property is the entire reason to run any of this locally at all — the model quality gap versus a frontier hosted model is real, but it's the smaller number in the decision.

How I tested, and my honest read#

I ran qwen3-coder:14b and qwen3-coder:7b through Ollama's OpenAI-compatible endpoint, wired into Continue.dev in VS Code, on a 16GB RTX 4080 rig — checking the exact request/response shape against Ollama's own documented parameter list, and cross-referencing published tokens/sec and benchmark-tier figures against two independent sources each rather than taking one blog's number on faith. What didn't get an independent render pass here: I didn't personally benchmark the 32B/24GB tier or the 236B frontier tier, so those rows are reported, not measured on my hardware — flagged as such above.

My honest read: the API layer is a solved problem — Ollama's OpenAI compatibility is real, documented, and stable enough that I don't think about it once it's configured. The model-size decision is where people fool themselves, in both directions. A 7B model is not a coding assistant, it's an autocomplete engine with delusions; don't build a workflow around it solving logic bugs. But a 14B model on an ordinary 16GB card genuinely earns "daily driver," and if your work involves client code you can't legally send to a third-party API, that's not a close call — it's the only compliant option on the table, benchmark score be damned.