TToolora

Side by side

Compare tools

2 tools selected

← Keep browsing

llama.cpp

Run LLMs locally with pure C/C++

Toolora score97Open source
Get it here↗

Free / open source

vLLM

High-throughput LLM serving engine

Toolora score94Open source
Get it here↗

Free / open source

Specllama.cppvLLM
CategoryAI ToolsAI Tools
PricingFree / open sourceFree / open source
Toolora score97/10094/100
Best forgithub, local-llm, open-source, inferencegithub, inference, serving, production
Top features
  • ·CPU and GPU backends
  • ·GGUF model format
  • ·Quantisation support
  • ·PagedAttention memory management
  • ·OpenAI-compatible server
  • ·Tensor and pipeline parallelism