_____ _                         _                 _         _
  / ____| |                       | |               | |       | |
 | |    | |__  _ __ _   _ ___  ___| | __ _ _ __ ___ | |__   __| | __ _
 | |    |  _ \|  __| | | / __|/ _ \ |/ _` |  _   _ \|  _ \ / _  |/ _` |
 | |____| | | | |  | |_| \__ \ (_) | | (_| | | | | || |_) | (_| | (_| |
  \_____|_| |_|_|   \__, |___/\___/|_|\__,_|_| |_| |_|___/ \__,_|\__,_|
                      __/ |
                     |___/       W R I T E U P S

<<< Back to Index

Unified Memory Hardware for Local LLM Inference — Buyer's Guide (2026)

_Last reviewed: 2026-03-02_

Why Unified/Shared Memory?

Traditional discrete GPUs (like NVIDIA RTX 4090) have dedicated VRAM — fast but limited (24GB). When your model doesn't fit, you're stuck with slow CPU offloading or expensive multi-GPU setups without true memory pooling (no NVLink on consumer cards).

Unified memory architectures — where CPU and GPU share the same RAM pool — change the equation. You trade some raw compute speed for massive usable memory, letting you run much larger models on a single machine.

Key Visualizations

Hardware Options Compared

The Contenders

For Comparison: Discrete GPU Setups

Model Requirements vs. Hardware Capacity

State of the Art Models (2026) — Memory Requirements

Flagship Models — The Biggest Open Models You Can Run Locally (2026)

These are the frontier-class open-weight models. Running them locally is the whole reason unified memory matters.

What Can You Actually Run on Each Machine?

Key insight: A 128GB Strix Halo machine can run every model up to ~120GB at Q2. That includes Llama 4 Scout and Qwen3 235B — models that would require $50,000+ in discrete GPUs at full precision. The M3 Ultra at 512GB is the only consumer-ish machine that can run GLM-5 and DeepSeek R1 without a server rack.

MoE Offloading — A Game Changer for Unified Memory

Mixture-of-Experts models are uniquely suited to unified memory because only a fraction of experts are active per token. With MoE-aware inference (e.g., llama.cpp's --override-kv or Unsloth's dynamic quantization), you can:

• Keep active experts on GPU/fast memory
• Offload inactive experts to system RAM with minimal speed penalty
• Run 671B+ parameter models on hardware that can't hold the full weights in fast memory

This is why Qwen3 235B runs at 6 tok/s on a single RTX 3060 + 128GB RAM, and DeepSeek R1 can run on 20GB RAM (slowly). Unified memory machines make this offloading nearly free since CPU and GPU share the same RAM.

See our [Mixture of Experts Explainer](mixture-of-experts-explainer.html) for a deeper dive into how this works.

Performance Characteristics

Inference Speed (Approximate tok/s for 70B-class Q4 models)

The Key Tradeoff

Discrete GPUs have vastly higher memory bandwidth (1,000+ GB/s per card) and compute throughput. They're faster per-token when the model fits in VRAM.

Unified memory machines have lower bandwidth (256–819 GB/s) but much larger capacity. They win when the model *doesn't fit* in discrete VRAM, avoiding the devastating performance cliff of CPU offloading.

Cost Efficiency Analysis

Price per GB of Usable LLM Memory

Recommendations

Best Value: Framework Desktop or Bosgame M5 (Strix Halo, 128GB)

• $1,700–$2,000 for 128GB unified memory
• Runs 70B models comfortably, Llama 4 Scout easily
• Best $/GB ratio of any option
• Linux-friendly (Framework especially)
• Limitation: lower memory bandwidth (~256 GB/s) means slower tok/s than Apple Silicon

Best Performance: Mac Studio M3 Ultra (192GB+)

• $5,600+ for 192GB, up to 512GB
• 819 GB/s bandwidth = fastest unified memory inference
• Runs models nothing else can touch (235B+ MoE at 512GB)
• Limitation: macOS ecosystem, expensive upgrades, proprietary

Best Portability: Lenovo Yoga Pro 7a or ASUS ProArt PX13

• $2,500–$4,050 for a 128GB laptop
• Same Strix Halo silicon as desktops
• Run 70B models on a plane
• Limitation: thermal throttling under sustained load, soldered RAM

Keep Your RTX 4090 If...

• You primarily run 32B and smaller models (fastest tok/s)
• You value raw speed over model size
• You're willing to wait for RTX 5090 48GB rumors or price drops

Avoid Multi-GPU Consumer Builds

• No NVLink on RTX 40/50 series = no true memory pooling
• $15,000–$25,000 for 96GB split across 4 cards
• A $2,000 Framework Desktop gives you 128GB unified for 1/10th the price

Bottom Line

The unified memory revolution means you no longer need a server rack to run frontier-class open models locally. A $2,000 Strix Halo mini PC can run models that would require $20,000+ in discrete GPU hardware. The tradeoff is speed — but for most personal/small-team inference, the speed difference is acceptable.

If buying today for LLM inference: Strix Halo 128GB (Framework or Bosgame) is the highest-value purchase. If budget allows and you want maximum headroom, Mac Studio M3 Ultra with 192GB+ is the performance king.

Sources

• Framework Desktop specs/pricing: framework.com, tomshardware.com, servethehome.com
• Bosgame M5 pricing: notebookcheck.net, videocardz.com
• Apple Mac Studio specs/pricing: apple.com, pcmag.com, macrumors.com
• AMD Strix Halo benchmarks: amd.com, techpowerup.com, llm-tracker.info
• Model requirements: huggingface.co, apxml.com, ai.google.dev, gradientflow.com
• Multi-GPU builds: a16z.com, introl.com, cloudninjas.com

<<< Back to Index