Skip to content
Convertto

VRAM Calculator — Will This Model Fit On My GPU?

Work out exactly how much video memory a local model needs, including the KV cache that scales with context length.

A local model needs memory for three things: the weights (parameters × bits per weight ÷ 8), the KV cache (2 × layers × KV heads × head dimension × context length × concurrent sequences × bytes per element), and about a gigabyte of runtime overhead. This calculator computes all three for twelve real architectures at eight quantisation levels, and shows how the requirement grows with context length — the part every VRAM table omits.

Runs in your browser
Rule of thumb
Parameters in billions × 0.6 ≈ GB at Q4_K_M, before the KV cache
The trap
The KV cache grows linearly with context and can exceed the weights past 32k tokens
Best trade
A larger model at Q4 usually beats a smaller model at Q8 for the same memory
Privacy
Runs entirely in your browser — nothing is uploaded
Cost
Free, unlimited, no sign-up

Frequently asked questions

Why does my model fit at 4k context but run out of memory at 32k?

The KV cache. It stores a key and value vector for every token in the context, in every layer, and it scales linearly — eight times the context is eight times the cache. On a 70B model at 32k that can be tens of gigabytes on top of the weights.

Is a bigger model at Q4 better than a smaller one at Q8?

Almost always, down to about Q4. A 70B at Q4_K_M consistently outperforms a 34B at Q8 in the same memory. Below Q3 the relationship breaks down — the degradation becomes severe enough that the smaller, better-quantised model wins.

What is grouped-query attention and why does it matter here?

It lets several attention heads share one set of key and value projections, so an 8-KV-head model caches a quarter of what a 32-KV-head model of the same size would. It is the single biggest factor in whether long context is affordable, which is why an older 7B model can need more cache than a newer 8B one.

How much extra memory does fine-tuning need?

Full fine-tuning needs roughly four times the weights on top: gradients at the weight precision plus two fp32 optimiser moments per parameter. LoRA avoids nearly all of it by training a small adapter and freezing the base weights — which is why almost nobody full-fine-tunes on consumer hardware.

How to use the llm vram calculator

  1. 1Choose the model.
  2. 2Choose the quantisation.
  3. 3Set the context length.
  4. 4Choose the kv cache precision.
  5. 5Set the concurrent sequences.
  6. 6Turn "Fine-tuning rather than inference" on or off as needed.
  7. 7Set the parameters (billions).
  8. 8Set the layers.
  9. 9Set the hidden size.
  10. 10Set the attention heads.
  11. 11Set the key/value heads.
  12. 12The result appears immediately — copy or download it.

Sources & specifications

Embed this tool

Put the working llm vram calculator on your own site. It runs in your visitors' browsers exactly as it does here — free, no account, nothing uploaded.

Share this tool

Related tools

GGUF Quantisation Size CalculatorWork out the download size of a quantised model at every GGUF level, and what quality each one costs you.LLM Model ComparisonEvery current model side by side: price per million tokens, context window, max output, modalities and cache rates.What Is My Screen Resolution?Check your screen resolution, browser viewport, device pixel ratio, colour depth and GPU.AI API Cost CalculatorWork out what a prompt costs per call, per day and per month, and compare the same workload across every major model.AI Writing Pattern CheckerMeasure sentence-length variance, vocabulary richness, marker phrases and typographic tells, with per-sentence scoring.Chat Export FormatterDrop in an exported conversations file and get clean Markdown, HTML or plain text — one document per conversation.

Last updated

More ai & llm tools