llama.cpp
LLM inference in plain C/C++ — run capable models on everyday hardware.
What it does
Why it matters
When capable open models appeared, the question was who could actually run them. llama.cpp's answer — a dependency-light C++ codebase with aggressive quantization — put LLM inference on hardware people already owned. It bootstrapped the entire local-AI ecosystem: Ollama, LM Studio, and countless tools sit on its foundations.
Engineering notes
- Quantization as a first-class design. K-quants trade precision for memory with unusual care; the format work (GGUF) standardized what everyone else builds on.
- Portable by construction. CPU-first with optional GPU backends (CUDA, Metal, Vulkan) — the same binary adapts to the hardware it finds.
- The ggml tensor library. A minimal, dependency-free tensor core underneath — the project builds its own foundations rather than inheriting framework bloat.
Limitations
- It's infrastructure, not an application — expect to build your own serving, batching, and eval layers on top.
- The pace of model-architecture support means bleeding-edge models can lag by days or weeks.
Related projects
Claude Code
Anthropic's agentic coding tool that works in your terminal.
Claude Code brought the agent loop to the command line: it reads your codebase, plans changes, runs commands, and iterates — with you directing, not typing. The reference implementation of terminal-native AI engineering.
v0
Generate UI with prompts, from the makers of Next.js.
Vercel's v0 turns text descriptions into working React components with shadcn/ui and Tailwind. The fastest path from 'I need a landing page' to deployable UI — and a glimpse of generative UI as a primitive.
Canonical URL: https://notvibescoded.com/projects/llama-cpp