Running Stable Diffusion Locally: How Much GPU Memory You Actually Need

For local image generation, 8 GB of VRAM handles Stable Diffusion 1.5 and SDXL, 16 GB covers SD 3.5 Medium and FLUX.2 [klein] 4B, and the heavyweights (SD 3.5 Large and FLUX.2 [klein] 9B at full precision) call for 18 to 29 GB. This guide pulls the official numbers into one place, explains where the memory goes, and maps each workload to a graphics card.

The numbers at a glance

The figures below come from model cards and vendor announcements. Treat them as guidance for a single image at the model's standard size, not a hard floor for every setup. Add-ons and higher resolutions push usage up.

Why two numbers for FLUX.2 [klein]? The Hugging Face model card describes loading the full pipeline, text encoder included. The ComfyUI team measured inside its own app, which shuffles components in and out of memory as needed. Both are accurate under their own conditions. When you are planning a purchase, the higher figure is the safer one.

Licensing differs too, which matters if you plan to sell your output or build a product. SD 1.5 uses CreativeML OpenRAIL-M. SD 3.5 is under the Stability AI Community License, which allows commercial use for organisations and individuals with under $1M in annual revenue. FLUX.2 [klein] 4B is Apache 2.0, while the 9B model is non-commercial only.

Where the memory goes

A common mistake is looking at the size of the model file and assuming that is the whole requirement. In practice, the entire pipeline plus working data has to fit.

Pipeline components

A text encoder turns your prompt into numbers, the diffusion model builds the image step by step in a compressed latent space, and a VAE decodes that into pixels. Older models use small encoders. Newer ones rely on large language models: SD 3.5 runs two CLIP encoders plus T5-XXL, and FLUX.2 [klein] 9B uses an 8 billion parameter Qwen3 model. That is why Stability AI's 9.9 GB figure for SD 3.5 Medium does not describe the full pipeline.

Resolution and batch size

Bigger images mean more intermediate data at every step. The weights stay the same size, so memory does not scale perfectly with pixel count, yet jumping from 1024 × 1024 to 2048 × 2048 can hit the ceiling on a card that handled the base size easily. Batch size works the same way: generating ten images in one pass needs far more memory than ten runs of one image each.

Weight format

The same parameter count can take very different amounts of space. FP16 and BF16 store each value in 16 bits, FP8 in 8 bits, and 4-bit formats go further. NVIDIA and Stability AI report that converting SD 3.5 Large to FP8 with TensorRT cuts memory use by 40%, from over 18 GB down to 11 GB, and runs 2.3 times faster than BF16 in PyTorch. The catch: NVIDIA lists FP8 support for GeForce RTX 40 Series and newer, and RTX 50 adds FP4 on top. Compressed weights can cost some fine detail, so check faces, small text and textures on your own prompts.

Extras

ControlNet (pose, edges, depth), IP-Adapter (image reference), refiners and upscalers each load their own models. There is no fixed overhead: it depends on the module and whether your app keeps everything resident or loads it in turns. Using a ready-made LoRA is light, but training your own is a separate job with higher requirements, so check the documentation of the trainer you plan to use.

Check your own GPU in five minutes

Before upgrading anything, measure your real usage.

1. Find your VRAM. In Windows, open Task Manager, go to Performance and select GPU. "Dedicated GPU memory" is your VRAM. On NVIDIA cards, running nvidia-smi in a terminal shows the same.
2. Run one image at batch size 1 at the model's native resolution. Ignore the first run, since it includes loading weights from disk.
3. Watch the peak on the second run. If dedicated memory hits the ceiling while "Shared GPU memory" climbs, the app is spilling into system RAM. You still get an image, just much more slowly.
4. Change one setting at a time: resolution, batch, weight format, extra modules. That shows exactly what eats your headroom.

ComfyUI is the easiest place to start. Its developers list support for NVIDIA, AMD (RDNA 2 through RDNA 4 via ROCm), Intel Arc and Apple Silicon, and say smart VRAM and RAM management lets even the largest open models run on as little as 4 GB of VRAM and 8 GB of RAM, though not quickly. Supported image models include SD 1.5, SDXL, SD 3.5, FLUX.1 and FLUX.2.

Matching a graphics card to the workload

VRAM matters more here than in gaming. A shortfall in a game costs frame rate; in image generation it either slows you down dramatically through offloading or stops a model from loading at all. Memory sizes below are from NVIDIA's official spec comparison.

Just exploring? 8 to 12 GB is enough to learn the ropes with SD 1.5 and SDXL.

Working with 2025 and 2026 models regularly? Aim for 16 GB. Black Forest Labs says FLUX.2 [klein] 4B is accessible on RTX 3090 or RTX 4070 class cards and above.

Running the largest open models uncompressed? Look at 24 to 32 GB. The FLUX.2 [klein] 9B card names RTX 4090 and above.

Previous generation cards still have a role. The RTX 4090 and RTX 3090 both carry 24 GB of GDDR6X, which keeps them useful for heavy models if you already own one. Keep in mind that the RTX 3090 predates the FP8 support NVIDIA lists from RTX 40 onward, so the FP8 TensorRT build of SD 3.5 Large will not bring the same memory savings there.

If you need more than 32 GB for training or several services running side by side, professional GPUs are the next step. For background on how GPUs and CPUs accelerate AI workloads, see our guide to AI acceleration in PC hardware.

The rest of the system

The GPU does most of the work, but it does not work alone. When VRAM runs short, apps move components into system RAM, so you want enough RAM to hold whatever does not fit on the card. Models weigh tens of gigabytes, so a fast NVMe SSD shortens initial loading and switching between them. The CPU plays a minor role once generation runs on the GPU.

If you are specifying a PC specifically for local image generation, the HYPERPC configurator lets you choose the GPU memory, system RAM and storage your models need.

Bottom line

• 8 GB: SD 1.5 and SDXL, a solid starting point.
• 12 GB: SDXL with add-ons, SD 3.5 Large in FP8 on RTX 40 or newer.
• 16 GB: the practical baseline for current models, including SD 3.5 Medium and FLUX.2 [klein] 4B.
• 24 to 32 GB: SD 3.5 Large uncompressed and FLUX.2 [klein] 9B.

Measure your own workflow before you buy. Resolution, batch size, weight format and add-ons affect real usage as much as the model name does.