Depth maps power robotics, AR placement, and scene understanding. Most reference implementations still drag a Python runtime and a multi-gigabyte PyTorch install to production.
depth-anything.cpp is LocalAI maintainer Emanuele Ciocca's C++17 port of ByteDance Depth Anything 3. One GGUF file. No Python at inference. And on CPU it is faster than PyTorch on the same model.
What you get from Depth Anything 3
Depth Anything 3 estimates metric depth in meters from a single RGB photo, plus:
| Output | Use case |
|---|---|
| Per-pixel depth map | Obstacle detection, scene layout |
| Confidence + sky mask | Filter unreliable regions |
| Camera intrinsics / extrinsics | AR alignment, photogrammetry glue |
| Point cloud / GLB export | 3D viewers, simulation |
The C++ port targets dependency-free deployment: ship a binary and a quantized GGUF, not a conda environment.
Benchmark numbers that matter on CPU
LocalAI published apples-to-apples numbers on a Ryzen 9 9950X3D (16 threads, 504x336 input):
| Engine | Quant | Model size | Load (ms) | Infer (ms) | Peak RAM (MB) | vs PyTorch |
|---|---|---|---|---|---|---|
| PyTorch | f32 | 516 MB | 749 | 416.9 | 1328 | 1.00x |
| C++/ggml | f32 | 393 MB | 112 | 346.4 | 614 | 1.20x |
| C++/ggml | q8_0 | 142 MB | 40 | 319.4 | 363 | 1.31x |
| C++/ggml | q4_k | 99 MB | 25 | 395.2 | 320 | 1.05x |
The q4_k build is a 99 MB file with near-lossless depth quality (correlation 1.0 vs reference in parity tests). That is the profile you want on a factory NUC or a VPS without a GPU.
Ciocca's engineering write-up is honest: the big GEMMs are close to a wash because everyone calls similar BLAS kernels. The wins come from host-side work Python never optimized (cached positional embeddings, faster load path) and from not booting an interpreter to run inference.
On GPU, the story is flatter: with ggml CUDA and flash attention, depth-anything.cpp roughly ties tuned PyTorch and mainly wins cold start.
LocalAI v4.5 integration
LocalAI v4.5.0 wires depth-anything.cpp as a native Go gRPC backend via purego (no Python at inference).
Three access patterns:
| Interface | Returns |
|---|---|
POST /v1/depth | Full depth surface (map, stats, camera matrices) |
Depth gRPC RPC | Same typed payload for service meshes |
GenerateImage | Min-max normalized grayscale depth PNG |
Eight GGUF variants live at mudler/depth-anything.cpp-gguf (base/small/large/giant plus mono-large, multiple quants). Builds cover cpu, CUDA 12/13, Intel SYCL, Vulkan, and L4T arm64.
The release also adds Depth Anything V2 gallery models and a CED audio event detector (527 AudioSet classes). Depth and sound tagging in one LocalAI install is a useful combo for edge agents that need both scene geometry and anomaly audio.
When I would deploy this
Good fits:
- Warehouse or retail robots that need metric depth on CPU-only edge boxes
- AR try-on pipelines that already run LocalAI for LLM/VLM tasks
- Privacy-sensitive sites that cannot ship frames to a cloud depth API
- CI parity checks against PyTorch reference outputs (bit-exact correlation)
Weak fits:
- Training or fine-tuning depth models (still a Python workflow)
- Sub-millisecond depth on every frame of a 4K 60fps stream without GPU help
- Teams with zero appetite for operating native binaries (use a hosted API instead)
Build and embed options
The upstream repo supports CLI (da3-cli), static/shared libraries, and optional CUDA/Metal/Vulkan backends. Key CMake flags include DA_GGML_LLAMAFILE for tinyBLAS CPU kernels and DA_BUILD_TESTS for the 37-test parity suite.
If you are already on the LocalAI gallery pattern, this is another GGUF + engine drop-in, same mental model as whisper.cpp and llama.cpp ports.
Depth on CPU without Python is the kind of boring infrastructure win that unlocks real edge products. If you are wiring perception into voice agents, robotics, or on-prem automation, book a free call and we can map where metric depth fits your stack.