llama.cpp
VerifiedOpen sourceRun LLMs locally with pure C/C++
Best for github · local-llm · open-source
4.9
Toolora score — editorial, not public reviewsToolora score97/100
Strengths
- CPU and GPU backends
- GGUF model format
Skip if
you want hosted support, not a repo to run
llama.cpp enables efficient local inference of Llama and compatible models on consumer hardware with minimal dependencies.
Key features
- CPU and GPU backends
- GGUF model format
- Quantisation support
- Server and CLI modes
Why it is on Toolora
We list llama.cpp for people who need github · local-llm · open-source. Skip it if you want hosted support, not a repo to run. It is the start-here pick when you need to chat or code with a model that never leaves this computer.
Closest alternatives on Toolora: Ollama, LM Studio, Jan.
The pick when you need to…
Try instead
CompareOpen-source option with 165k GitHub stars.
Also built for local llm.
Also built for local llm.
Featured in guides
llama.cpp
Free / open source