TToolora

Side by side

Compare tools

3 tools selected

← Keep browsing
verified

vLLM

High-throughput LLM serving engine

4.8
Open source
Get it here↗

Free / open source

verified

llama.cpp

Run LLMs locally with pure C/C++

4.9
Open source
Get it here↗

Free / open source

featured

Ollama

Run open models locally with one command

4.8
Free
Get it here↗

Free

SpecvLLMllama.cppOllama
CategoryAI ToolsAI ToolsAI Tools
PricingFree / open sourceFree / open sourceFree
Rating4.8 (3,200)4.9 (5,400)4.8 (7,300)
Community94/10097/10096/100
Tierverifiedverifiedfeatured
Best forgithub, inference, serving, productiongithub, local-llm, open-source, inferencelocal-llm, cli, developer, github
Top features
  • ·PagedAttention memory management
  • ·OpenAI-compatible server
  • ·Tensor and pipeline parallelism
  • ·CPU and GPU backends
  • ·GGUF model format
  • ·Quantisation support
  • ·One-line model pulls
  • ·Local REST API
  • ·macOS, Windows, Linux