🏆 GLM-5.2 Local Setup: VRAM, Quants and Real Hardware Paths

GLM-5.2 is the open-weights model of the summer: MIT-licensed, top of the agentic-coding leaderboards, and — unlike Kimi K3, whose weights don't even ship until July 27 — you can download it today. What you can't do is run it casually. We spent the research hours (and consulted our own long-suffering 16GB rig) so you don't have to: here's exactly what 744B parameters costs in memory, the three hardware paths that genuinely work, and the honest tokens-per-second you should expect.

By the numbers

• 744B parameters, Mixture-of-Experts, released June 2026 by Z.ai under an MIT license
• #1 open-weight model on SWE-bench Pro (62.1%) and Terminal-Bench 2.1 (81.0%); 91.2% on GPQA Diamond — as of the July 19 leaderboard snapshot
• 2-bit dynamic GGUF (UD-IQ2_M): ~241–245GB combined RAM/VRAM
• Q4_K_M: ~476GB · 8-bit: ~810GB · FP16: 1,600GB+
• Consumer hybrid rig (4× 3090/4090 + 256GB RAM): ~3–6 tokens/sec on the 2-bit build

The three hardware paths that actually work

Path 1 — Apple Silicon unified memory (the sleeper pick). A 256GB unified-memory Mac holds the 2-bit dynamic quant in a single memory pool — no CPU/GPU split, no offload choreography. Per-token speed won't embarrass a GPU cluster, but for a one-box, plug-in-and-go GLM-5.2 machine, it's currently the cheapest sane path, and the one we'd buy if this model were our daily driver.

Path 2 — The quad-consumer-GPU rig. 4× RTX 3090 or 4090 plus 256GB+ of system RAM runs the 2-bit GGUF with CPU/GPU hybrid offloading at ~3–6 tok/s. That's fine for the agentic patterns GLM-5.2 is best at — fire a coding task, come back to a finished diff — and rough for interactive chat. Used 3090s remain the value play here (our used RTX 3090 buying guide still applies; four of them is ~96GB VRAM for under $3K used).

Path 3 — Rented datacenter GPUs. The smallest GPU-only fit is around 8× A100 40GB (320GB pooled) for the 2-bit build; a serious Q4 deployment wants 2× A100 80GB-class per shard or 4× RTX 6000 Ada. At on-demand cloud pricing this lands in the several-dollars-per-hour range — which is why, for most workloads, the local-vs-cloud math says rent for bursts, don't build.

Setup notes from the trenches

Unsloth's dynamic GGUFs are the community-standard local build — their 2-bit "UD" quants hold up meaningfully better than naive 2-bit, for reasons we unpack in Quantization Explained. For serving, llama.cpp handles the hybrid-offload path; vLLM is the pick if you have the VRAM to load it fully. Two practical warnings from community deployment threads we can confirm make sense architecturally: leave 10–15% memory headroom above the quant size for KV cache before you commit to "it fits," and don't benchmark MoE models on short prompts — expert routing makes them look faster than they run on real 30K-token agentic contexts.

Who should actually bother

Run GLM-5.2 locally if: you're running high-volume agentic coding where API bills genuinely exceed hardware amortization, you have data that can't leave the building, or you want the best MIT-licensed base to fine-tune. Skip it if you're chasing chat quality per dollar — a right-sized model like Qwen 3.6 27B on one GPU delivers 90% of the everyday experience at 2% of the memory bill, and the hosted GLM-5.2 endpoints exist for the rest.

How we picked/tested

Memory figures above are cross-checked across Unsloth's official documentation, multiple independent hardware guides, and Spheron's GPU-fit calculator; benchmark numbers come from the public July 19 leaderboard snapshot. We flag anything we haven't run on our own metal — at 744B, this one we've costed, not daily-driven, and the tokens-per-second range is community-reported consensus rather than our bench.

The uncomfortable truth of 2026's open-weights golden age: the models got frontier-good faster than consumer hardware got frontier-big. GLM-5.2 is the best open model you can hold in your hands today — just make sure you price the hands.