SfM Backends

Structure-from-Motion (SfM) is the first stage of the pipeline. It takes your images and estimates where each camera was positioned in 3D space. These camera poses are then used to train the Gaussian Splat model. Two SfM backends run today:

Backend comparison

Backend	Speed	Quality	Best for
`colmap_sift_glomap`	Fast	Excellent	Every upload, ARKit captures included (default)
`prebuilt_colmap`	Instant	Yours	A COLMAP model you already built

prebuilt_colmap is selected automatically when your zip contains a sparse/ model — it is not something you request via sfm.backend. The pipeline also registers none_arkit, a pose passthrough, but no API input gives it the model it copies, so selecting it fails the scene (refunded; NoneArkitBackend.run in sfm_backends/none_arkit.py).

The API also accepts legacy short-names for backward compatibility: glomap and colmap both resolve to colmap_sift_glomap, and none resolves to none_arkit. Prefer the canonical names in new code; the colmap alias emits a deprecation warning and may be removed.

`colmap_sift_glomap` (default)

GPU SIFT feature extraction (via COLMAP) combined with GLOMAP’s modern global mapper, which solves all camera poses simultaneously rather than incrementally.

{
  "sfm": {
    "backend": "colmap_sift_glomap"
  }
}

Advantages:

Significantly faster than incremental COLMAP
GPU SIFT for feature extraction
Global optimization is more robust for video walkthroughs
Default for all presets

When to use: Almost always. This is the recommended backend for video input and most photo sets. If you don’t override sfm.backend, this is what runs.

Options

Set these in sfm.options. SfM reads only this object: the top-level sfm.matcher is ignored, and sfm.max_image_size sets only the training resolution (SfmRequest.from_api_payload in sfm_backends/configs.py).

Key	Type	Default	Description
`matcher`	`string`	By input type	`exhaustive` or `sequential`. Unset, photos get `exhaustive` and video `sequential`. `exhaustive` is for `standard` and above, as with `sfm.matcher`
`max_image_size`	`integer`	`1600`	Longest edge, in pixels, for feature extraction: `400`-`4096`

The backend also accepts tuning keys for COLMAP and GLOMAP (camera_model, max_num_features, triangulation thresholds and others, in ColmapSiftGlomapOptions); they are unsupported and may change. An unknown key or bad value is refused when /process dispatches the job: 400 invalid_input (Invalid sfm options: ...), and the charge is refunded (translatePydantic422 in api/src/lib/scenes.ts).

Camera poses you already have

There are two ways to supply poses. Only a COLMAP model skips SfM.

Path A: ARKit poses via API (recommended for iOS)

Pass camera poses directly in the POST /scenes/{id}/process request body. The API stores them server-side. No zip required — upload your video as normal. GLOMAP still solves the camera poses from the frames (_stage_sfm in pipeline/training/impl.py); the ARKit poses set which way is up and, for a video, which frames are cut: up to 150, at the pose timestamps (_extract_arkit_timestamp_frames).

# 1. Create a scene (standard video upload)
curl -X POST https://api.splat-3d.com/v1/scenes \
  -H "Authorization: Bearer s3d_..." \
  -H "Content-Type: application/json" \
  -d '{"title": "My scan", "preset": "standard", "content_type": "video/mp4"}'

# Response includes sceneId + uploadUrl

# 2. Upload video directly to the presigned URL
curl -X PUT "$UPLOAD_URL" \
  --upload-file capture.mp4 \
  -H "Content-Type: video/mp4"

# 3. Process with ARKit poses. poses.json is {"arkit_poses": [...]}, holding
#    5 to 1,000 poses shaped like the one below.
curl -X POST https://api.splat-3d.com/v1/scenes/$SCENE_ID/process \
  -H "Authorization: Bearer s3d_..." \
  -H "Content-Type: application/json" \
  -d @poses.json

One pose:

{
  "timestamp": 0.033,
  "file_path": "frame_000001.jpg",
  "transform": [1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,1],
  "intrinsics": [1674,0,960, 0,1674,720, 0,0,1],
  "width": 1920,
  "height": 1440
}

Pose format:

Field	Type	Description
`timestamp`	`number`	Capture time in seconds
`file_path`	`string`	Path in the zip of the image this pose corresponds to (the file name alone works when it is unique)
`transform`	`number[16]`	4×4 camera-to-world matrix, column-major (matches ARKit `simd_float4x4` layout), in a gravity-aligned world with +Y up
`intrinsics`	`number[9]`	3×3 intrinsics, row-major: `[fx, 0, cx, 0, fy, cy, 0, 0, 1]`
`width`	`integer`	Image width in pixels
`height`	`integer`	Image height in pixels

Minimum 5 poses, maximum 1000. You don’t set sfm.backend: the API sends the poses with the job, and the pipeline runs GLOMAP as for any upload. The scene’s orientation comes from these poses. The pipeline reads gravity off the world’s +Y axis, so record them in an ARKit session with worldAlignment set to .gravity (the default) or .gravityAndHeading, not .camera. It needs at least 30 frames or photos paired with a pose and a median tilt residual (each pair’s up against the fitted up) of 3° or less; otherwise it estimates up from the cameras, which lays portrait captures on their side (MIN_PAIRS, MAX_MEDIAN_TILT_DEG in pipeline/training/arkit_gravity.py). Photos are used as stored: EXIF orientation is ignored when arkit_poses is set, so each image’s pixels must be in the camera axes its transform describes, as ARKit’s capturedImage is. If you’re building an iOS app, the Splat3D Swift SDK handles all of this for you — ARKit capture, pose encoding, upload, and polling in one call.

Path B: COLMAP zip upload

If you’ve already run COLMAP (or Metashape, RealityCapture, or your own SfM code that exports COLMAP binaries), bundle your images and sparse model into a zip. The pipeline detects the sparse/ model, skips SfM entirely, and starts training immediately. This is the fastest path to a splat and gives you full control over the reconstruction. Zip structure:

your-upload.zip
├── images/
│   ├── frame_001.jpg
│   ├── frame_002.jpg
│   └── ...
└── sparse/
    └── 0/
        ├── cameras.bin
        ├── images.bin
        └── points3D.bin

A single wrapper folder is fine (my-scene/images/ + my-scene/sparse/0/) — zip the folder or its contents, whichever your tool produces. If several models are present, sparse/0 is used; among several sparse/0 folders, the one with images/ beside its sparse/. Two tool outputs also work zipped as they are:

Inria 3DGS convert.py — the undistorted sparse/0 and images/ are used; input/ and distorted/ are ignored.
nerfstudio ns-process-data — the model in colmap/sparse/0 is paired with the photos in the top-level images/; images_2/ and the other downscaled copies are ignored.

# 1. Create a scene — content_type marks it as a photo/zip upload
curl -X POST https://api.splat-3d.com/v1/scenes \
  -H "Authorization: Bearer s3d_..." \
  -H "Content-Type: application/json" \
  -d '{
    "title": "My scene",
    "preset": "standard",
    "content_type": "application/zip"
  }'

# 2. Upload zip to presigned URL
curl -X PUT "$UPLOAD_URL" \
  --upload-file my-scene.zip \
  -H "Content-Type: application/zip"

# 3. Trigger processing (no body needed — the model is detected in the zip)
curl -X POST https://api.splat-3d.com/v1/scenes/$SCENE_ID/process \
  -H "Authorization: Bearer s3d_..."

You do not need to set sfm.backend — detection is automatic. Uploading a zip with no sparse/ directory runs the normal GLOMAP path.

Requirements

Your model is validated before training starts, so a mismatch fails fast with a message instead of burning GPU minutes:

Requirement	Detail
Format	COLMAP binary (`.bin`) or text (`.txt`). A text model is converted to binaries before validation; binaries win when a folder has both.
Completeness	All three of `cameras.bin`, `images.bin`, `points3D.bin`, or of `cameras.txt`, `images.txt`, `points3D.txt`. An incomplete set of text files is ignored, and the photos go through SfM.
Filenames	Every name in `images.bin` must exist under `images/`, matched as a full relative path — `left/frame_001.jpg` in the model needs `images/left/frame_001.jpg` in the zip, not a flattened `frame_001.jpg`.
Camera models	`SIMPLE_PINHOLE`, `PINHOLE`, `SIMPLE_RADIAL`, `RADIAL`, `OPENCV`. Fisheye and other models are rejected — undistort to `PINHOLE` first with `colmap image_undistorter`, then zip its output folder: the undistorted `images/` together with its `sparse/` model. The original photos do not match an undistorted model.
Photo size	Each photo must be the width × height its camera has in `cameras.bin`, or a uniform resize of it. Photos are read as stored, without applying EXIF rotation, so zip the exact files the model was built from.
Registered images	Between 5 and 800. We cannot subsample a model for you — dropping an image would orphan its pose — so trim it yourself before uploading.
Sparse points	Between 1,000 and 5,000,000 in `points3D.bin`. Training starts from one Gaussian per point, with no random initialization, so a poses-only export is rejected — triangulate points first with `colmap point_triangulator`, which needs a `--database_path` holding features and matches for the same photos (`colmap feature_extractor`, then a matcher such as `colmap exhaustive_matcher`). Thin a denser cloud with `colmap point_filtering`.

Photos in the zip that the model never registered are ignored. Getting COLMAP binary output:

# Run COLMAP and export binary sparse reconstruction
colmap automatic_reconstructor \
  --workspace_path ./colmap_output \
  --image_path ./images

# Binary files are written to colmap_output/sparse/0/ by default

When to use pre-supplied poses

Scenario	Recommended path
iOS app with LiDAR (iPhone 12 Pro+)	Path A — ARKit poses via API
Existing COLMAP reconstruction	Path B — COLMAP zip
Custom SLAM / visual odometry	Path B — convert to COLMAP binary format
Re-training with different parameters	Path B — reuse existing sparse reconstruction

Matching strategies

Feature matching only happens when SfM actually runs, so a COLMAP model zip (prebuilt_colmap) ignores these settings. Otherwise you can control the matching strategy in sfm.options:

Matcher	Description	Speed	Best for
`exhaustive`	Compare every image pair	Slowest	Small-medium scenes (under 200 images)
`sequential`	Compare adjacent frames only	Fastest	Ordered video frames

{
  "sfm": {
    "options": {
      "matcher": "sequential"
    }
  }
}

For video input, sequential matching is usually both faster and more reliable since frames are naturally ordered, and it is what video gets when you set none. Photo sets get exhaustive.

Getting Started

Guides

SfM Backends

SfM Backends

Backend comparison

`colmap_sift_glomap` (default)

Options

Camera poses you already have

Path A: ARKit poses via API (recommended for iOS)

Path B: COLMAP zip upload

Requirements

When to use pre-supplied poses

Matching strategies

​SfM Backends

​Backend comparison

​colmap_sift_glomap (default)

​Options

​Camera poses you already have

​Path A: ARKit poses via API (recommended for iOS)

​Path B: COLMAP zip upload

​Requirements

​When to use pre-supplied poses

​Matching strategies

SfM Backends

Backend comparison

`colmap_sift_glomap` (default)

Options

Camera poses you already have

Path A: ARKit poses via API (recommended for iOS)

Path B: COLMAP zip upload

Requirements

When to use pre-supplied poses

Matching strategies