What is llama.cpp?

llama.cpp is an open-source C/C++ inference engine for running large language models locally on CPUs and consumer GPUs, using the GGUF format.

Primary sources

Specifications cited by the tools below whose own titles define llama.cpp.

2 tools that work with llama.cpp

Files never leave your browser.

AI & LLM Tools2

GGUF Quantisation Size CalculatorWork out the download size of a quantised model at every GGUF level, and what quality each one costs you.LLM VRAM CalculatorWork out exactly how much video memory a local model needs, including the KV cache that scales with context length.

Related terms

Terms that appear alongside llama.cpp on the same tools.

← All glossary terms Browse every tool →