Vggt Mps

vggt-mps

Port of Facebook Research's [VGGT](https://github.com/facebookresearch/vggt) (Visual Geometry Grounded Transformer) to Apple Silicon via PyTorch's MPS backend. Takes single or multi-view images and produces depth maps, camera poses, and 3D point clouds.

Version	2.0.0
Python	3.10+
Platform	macOS 13+ on Apple Silicon (M1/M2/M3)
Model	facebook/VGGT-1B (1B params, ~5 GB on disk)
License	MIT
PyPI	Not yet published

What it produces

Given N input images, VGGT predicts:

Output	Description
Depth maps	Per-pixel depth estimation
Camera poses	6-DOF camera parameters for each view
3D point clouds	Dense reconstruction (exportable as PLY, OBJ, GLB)
Confidence maps	Per-pixel reliability scores

Architecture

The upstream VGGT model is a 1B-parameter transformer trained on multi-view geometry tasks. This repo wraps it with:

• MPS device detection and dtype handling (float32 for Metal compatibility)
• A sparse attention module (vggt_sparse_attention.py) that patches the model at runtime for O(n) memory scaling instead of O(n^2)
• A unified CLI (vggt command with subcommands)
• A Gradio web interface
• An MCP server for Claude Desktop integration

vggt-mps/
  src/
    vggt_core.py                # Core VGGT processing
    vggt_sparse_attention.py    # Runtime sparse attention patch
    config.py                   # Centralized configuration
    visualization.py            # 3D visualization
    commands/                   # CLI subcommands (demo, reconstruct, test, benchmark, web)
    utils/                      # Model loader, image utils, export
  tests/                        # MPS, sparse attention, integration tests
  repo/vggt/                    # Vendored upstream VGGT source

Sparse attention

The sparse attention module replaces standard O(n^2) cross-view attention with a covisibility-masked variant. No retraining required -- it patches the loaded