🏆 GLM-5.2 Local Setup: VRAM, Quants and Real Hardware Paths
Z.ai's GLM-5.2 leads every open-weights coding benchmark. Here's the honest VRAM math per quant, the three hardware paths that work, and who should bother.
Derek Holt · Local AI & Hardware Writer
· 4 min read
⚡ TL;DR — quick answers
- What is GLM-5.2 and why does it matter?
- It's Z.ai's 744B-parameter Mixture-of-Experts model, released June 2026 under an MIT license. As of mid-July 2026 it leads all open-weight models on SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%), and scores 91.2% on GPQA Diamond.
- How much memory does GLM-5.2 need locally?
- Roughly 245GB of combined RAM+VRAM for the 2-bit dynamic GGUF, ~476GB for Q4_K_M, ~810GB at 8-bit, and over 1,600GB at FP16. No single consumer GPU comes close on its own.
- Can a consumer rig run GLM-5.2 at all?
- Yes, barely: a 4× RTX 3090/4090 box with 256GB+ system RAM runs the 2-bit build via CPU/GPU hybrid offloading at roughly 3–6 tokens per second. Usable for agentic batch work, painful for chat.
GLM-5.2 is the open-weights model of the summer: MIT-licensed, top of the agentic-coding leaderboards, and — unlike Kimi K3, whose weights don't even ship until July 27 — you can download it today. What you can't do is run it casually. We spent the research hours (and consulted our own long-suffering 16GB rig) so you don't have to: here's exactly what 744B parameters costs in memory, the three hardware paths that genuinely work, and the honest tokens-per-second you should expect.
By the numbers
- 744B parameters, Mixture-of-Experts, released June 2026 by Z.ai under an MIT license
- #1 open-weight model on SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%); 91.2% on GPQA Diamond — as of the July 19 leaderboard snapshot
- 2-bit dynamic GGUF (UD-IQ2_M): ~241–245GB combined RAM/VRAM
- Q4_K_M: ~476GB · 8-bit: ~810GB · FP16: 1,600GB+
- Consumer hybrid rig (4× 3090/4090 + 256GB RAM): ~3–6 tokens/sec on the 2-bit build
The three hardware paths that actually work
Path 1 — Apple Silicon unified memory (the sleeper pick). A 256GB unified-memory Mac holds the 2-bit dynamic quant in a single memory pool — no CPU/GPU split, no offload choreography. Per-token speed won't embarrass a GPU cluster, but for a one-box, plug-in-and-go GLM-5.2 machine, it's currently the cheapest sane path, and the one we'd buy if this model were our daily driver.
Path 2 — The quad-consumer-GPU rig. 4× RTX 3090 or 4090 plus 256GB+ of system RAM runs the 2-bit GGUF with CPU/GPU hybrid offloading at ~3–6 tok/s. That's fine for the agentic patterns GLM-5.2 is best at — fire a coding task, come back to a finished diff — and rough for interactive chat. Used 3090s remain the value play here (our used RTX 3090 buying guide still applies; four of them is ~96GB VRAM for under $3K used).
Path 3 — Rented datacenter GPUs. The smallest GPU-only fit is around 8× A100 40GB (320GB pooled) for the 2-bit build; a serious Q4 deployment wants 2× A100 80GB-class per shard or 4× RTX 6000 Ada. At on-demand cloud pricing this lands in the several-dollars-per-hour range — which is why, for most workloads, the local-vs-cloud math says rent for bursts, don't build.
Setup notes from the trenches
Unsloth's dynamic GGUFs are the community-standard local build — their 2-bit "UD" quants hold up meaningfully better than naive 2-bit, for reasons we unpack in Quantization Explained. For serving, llama.cpp handles the hybrid-offload path; vLLM is the pick if you have the VRAM to load it fully. Two practical warnings from community deployment threads we can confirm make sense architecturally: leave 10–15% memory headroom above the quant size for KV cache before you commit to "it fits," and don't benchmark MoE models on short prompts — expert routing makes them look faster than they run on real 30K-token agentic contexts.
Who should actually bother
Run GLM-5.2 locally if: you're running high-volume agentic coding where API bills genuinely exceed hardware amortization, you have data that can't leave the building, or you want the best MIT-licensed base to fine-tune. Skip it if you're chasing chat quality per dollar — a right-sized model like Qwen 3.6 27B on one GPU delivers 90% of the everyday experience at 2% of the memory bill, and the hosted GLM-5.2 endpoints exist for the rest.
How we picked/tested
Memory figures above are cross-checked across Unsloth's official documentation, multiple independent hardware guides, and Spheron's GPU-fit calculator; benchmark numbers come from the public July 19 leaderboard snapshot. We flag anything we haven't run on our own metal — at 744B, this one we've costed, not daily-driven, and the tokens-per-second range is community-reported consensus rather than our bench.
The uncomfortable truth of 2026's open-weights golden age: the models got frontier-good faster than consumer hardware got frontier-big. GLM-5.2 is the best open model you can hold in your hands today — just make sure you price the hands.
Frequently asked questions
▸What is GLM-5.2 and why does it matter?
It's Z.ai's 744B-parameter Mixture-of-Experts model, released June 2026 under an MIT license. As of mid-July 2026 it leads all open-weight models on SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%), and scores 91.2% on GPQA Diamond.
▸How much memory does GLM-5.2 need locally?
Roughly 245GB of combined RAM+VRAM for the 2-bit dynamic GGUF, ~476GB for Q4_K_M, ~810GB at 8-bit, and over 1,600GB at FP16. No single consumer GPU comes close on its own.
▸Can a consumer rig run GLM-5.2 at all?
Yes, barely: a 4× RTX 3090/4090 box with 256GB+ system RAM runs the 2-bit build via CPU/GPU hybrid offloading at roughly 3–6 tokens per second. Usable for agentic batch work, painful for chat.
▸Is a Mac really a good way to run GLM-5.2?
Surprisingly, yes — a 256GB unified-memory Mac fits the 2-bit dynamic quant in one memory pool with no CPU/GPU split, which makes Apple Silicon one of the most cost-effective single-box paths.
The 5 best AI video finds, every week
New models, tested prompts, and what actually worked in our production — one short email a week. No spam, unsubscribe anytime.
Written by Derek Holt
Local AI & Hardware Writer
Runs the site's local-inference rig and benchmarks every GPU, quant, and speed-stack claim on it personally before it goes in a guide. Will not shut up about VRAM bandwidth.
Explore these topics
Every guide, comparison and prompt library we have on each.
Keep learning
GPUs & Hardware · Local LLMs
GuidesRTX 5090 AI TOPS Explained: What 3352 Actually Measures
NVIDIA's 3352 AI TOPS is sparse FP4, not the precision your local model runs. The full ladder, the 16x gap, and when TOPS is the wrong spec to buy on.
