What if you could see a robot's LiDAR through your own eyes and tap the floor to send it somewhere? That is not a concept video. It is an MIT-licensed repo you can clone on a MacBook.
spectacles-dimensional-os connects Snap Spectacles (2024 dev kit) to a Unitree Go2 through Dimensional OS (DimOS), an open-source robot stack. Developer Johannes Tscharn built it to control and visualize quadrupeds in real space, with optional voice-agent mode layered on top.
I do not own a Go2 or Spectacles dev kit. I still care about this project because it is a clean reference for spatial agent UX: registration, streaming perception, goal submission, and emergency stop in one loop.
System overview in one paragraph
DimOS owns the robot: connection, odometry, navigation, LiDAR, motion. A DimOS module ARBridge (dimos-ar) exposes a JSON WebSocket on port 8787. The Spectacles Lens is a client. It connects, completes registration to solve the shared world ↔ odom transform, then drives and visualizes the robot in AR.
After registration, Manual Mode anchors a spatial controller in front of the robot. Tilt and rotate your hand (or pinch-drag a navigation marker) to aim a direction arrow. The Lens streams world-frame nav_goal messages. DimOS plans a path. The robot walks. Live pose updates keep the AR marker aligned with the physical machine. Optional LiDAR layers show obstacles and height bands.
Two control modes worth stealing
| Mode | Input | Best for |
|---|---|---|
| Manual spatial targeting | Hand tilt, pinch-drag marker | Precise nav goals, demos, safety-critical steering |
| Voice agent | Spoken commands to an LLM-based agent | High-level tasks ("go to the kitchen") when you trust the planner |
Manual mode is the product lesson. Most teleop UIs are gamepad-think on a flat screen. This one meets the robot where it occupies space. You see planned paths as overlays. You emergency-stop with a gesture instead of hunting for a laptop key.
Agent mode is the research flex. It is also the failure mode if you skip verification. The repo's split lets you demo spatial UI without betting safety on an LLM on day one.
Setup reality check
Supported OS for development: macOS (primary), Linux (generally supported, less tested), Windows not supported.
Installation runs through scripts/setup.sh, which pulls DimOS dependencies and the WebSocket bridge. You need:
- Snap Spectacles dev kit with Lens runtime
- Unitree Go2 (Pro/Air tested) or replay mode without hardware
- A machine that can run DimOS modules locally
G1 humanoid support is listed but explicitly not fully tested. The maintainer is looking for collaborators, which is honest and rare in robotics READMEs.
Why this matters outside robotics Twitter
If you build voice agents or ops automation, the same primitives show up:
- Shared coordinate frames (world ↔ odom ↔ device) are integration work, not ML
- Streaming perception (LiDAR point clouds) needs throttling and graceful degradation in AR
- Human-in-the-loop goals beat fully autonomous loops for early deployments
- Replay mode is how you test UX without burning hardware hours
AlphaSignal paired this story with phone-sized 27B models and Canadian research funding. The pattern is frontier capability moving closer to where humans already are (pocket, lab, glasses), not locked in a datacenter dashboard.
Limits I would not hand-wave
- Dev-kit Spectacles are not consumer scale. This is a builder project.
- LLM voice mode needs the same guardrails as any embodied agent: geofencing, estop, rate limits
- LiDAR visualization in AR is bandwidth and comfort sensitive (latency, flicker, field of view)
- Unitree plus Snap is a hardware stack most agencies will not ship next quarter
Treat it as a reference architecture for spatial agents, not a turnkey product.
If you are prototyping embodied agents
Clone the repo, run replay mode, and study the WebSocket schema before you write your own AR client. The valuable artifact is not the demo video. It is the separation between DimOS (robot truth) and Lens (human intent).
If you are designing voice or vision agents for physical ops and want help scoping safety layers and integration work, book a free call. The model is the easy part. Registration and estop are not.