llama.cpp
Open sourceRun LLMs locally with pure C/C++
Best for github · local-llm · open-source
Our own editorial score, not public reviews.
Strengths
- CPU and GPU backends
- GGUF model format
Skip if
you want hosted support, not a repo to run
llama.cpp enables efficient local inference of Llama and compatible models on consumer hardware with minimal dependencies.
Key features
- CPU and GPU backends
- GGUF model format
- Quantisation support
- Server and CLI modes
Why it is on Toolora
We list llama.cpp for people who need github · local-llm · open-source. Skip it if you want hosted support, not a repo to run. It is the start-here pick when you need to chat or code with a model that never leaves this computer.
Closest alternatives on Toolora: Ollama, Open WebUI, LM Studio.
The pick when you need to…
Try instead
All llama.cpp alternativesOpen source, with 165k GitHub stars.
Open WebUI
★ 124kOur pickSelf-hosted ChatGPT-style UI for local models
AI ToolsOpen source, with 124k GitHub stars.
Also covers local llm and inference.
Featured in guides
llama.cpp
Free / open source