Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e71d827
Port CorbeauSplat to Windows / NVIDIA CUDA
claude Jun 26, 2026
e1548ee
Tidy imports in ported modules (ruff)
claude Jun 26, 2026
9d89d58
Auto-install COLMAP (CUDA) and FFmpeg into engines/
claude Jun 26, 2026
50d952c
Fix Windows runtime issues from first real-machine run
claude Jun 26, 2026
a29b730
Fix COLMAP GPU flag rejection and empty-input handling
claude Jun 26, 2026
edbe420
Auto-update run.bat from git on launch
claude Jun 26, 2026
d719a15
Make upscale optional and non-fatal when no model is present
claude Jun 26, 2026
d677ce5
Add automatic blur filtering of frames before COLMAP
claude Jun 26, 2026
d17cb9d
Add a dedicated splat-cleanup tab (load → clean → preview → save)
claude Jun 26, 2026
7a140e5
Add "Robust mode" for large scenes (anti-crash COLMAP) + tie in blur …
claude Jun 26, 2026
6301a32
Blur filter: show progress and run faster (downscale before Laplacian)
claude Jun 26, 2026
42665f6
Blur filter: parallelize sharpness analysis across CPU threads
claude Jun 26, 2026
84065f7
Speed up resolution check: read header dims only, in parallel
claude Jun 26, 2026
16b0cb9
Default COLMAP params to the recommended settings (GPU SIFT) + SIFT p…
claude Jun 26, 2026
bc1641f
Blur filter: cap removal at 20% (was 50%) and add restore hint
claude Jun 26, 2026
c8cb191
Robust mode no longer forces the blur filter (respect the checkbox)
claude Jun 26, 2026
4787656
Hardlink images into the project instead of copying (no disk duplicat…
claude Jun 26, 2026
75eb54f
Audit fixes: numpy.void PLY crash, cross-drive paths, streaming downl…
claude Jun 26, 2026
5e60723
COLMAP 4.1.0: GPU bundle adjustment + native 360, auto-upgrade installer
claude Jun 26, 2026
d35085d
Fix startup crash: restore resolve_project_root import in setup_depen…
claude Jun 27, 2026
2ddb94a
SOTA-fast defaults for video→splat + persist params on tab switch
claude Jun 27, 2026
3dcc5d9
Skip already-present images when preparing the dataset (no duplicates)
claude Jun 27, 2026
10c6626
Stop mouse wheel from changing combos/spin boxes; honor saved blur state
claude Jun 27, 2026
21260aa
README: credit the original CorbeauSplat repo first; fix stale SIFT f…
claude Jun 27, 2026
da7cb3b
Show progress during the silent DB-sort and matcher-startup phases
claude Jun 27, 2026
83e6ea8
Label "Linear solver failure" as a non-fatal warning (once)
claude Jun 27, 2026
e4645fd
Add actionable hint to the BA warning note (GPU BA / robust mode)
claude Jun 27, 2026
7b25ab3
Show registered-image count (num_reg_frames) in mapper status
claude Jun 27, 2026
e6be5ba
Perf: faster startup, image-prep, and COLMAP mapper (audit-driven)
claude Jun 27, 2026
afcdb82
Debug: fix 3 real bugs found reviewing the frame/resize path
claude Jun 27, 2026
5baf27a
Optimize: NVDEC GPU video decode with automatic CPU fallback
claude Jun 27, 2026
74aeadc
Fix mapper abort: gate every BA bound flag on COLMAP's advertised opt…
claude Jun 29, 2026
58964fa
Resume: skip extraction+matching when the COLMAP DB is valid for this…
claude Jun 29, 2026
ab5d0d9
Fix duplicate images on re-run; self-heal stale duplicates
claude Jun 29, 2026
2552f03
Speed up global BA: cap refinements + fix dead frequency control on 4…
claude Jun 29, 2026
e8f0a24
Brush: auto-retry on Vulkan when the burn-fusion backend panics
claude Jun 29, 2026
5422aaa
Train on the largest COLMAP sub-model (fix Brush getting an 11-image …
claude Jun 29, 2026
6cb39db
Brush: default to Vulkan on NVIDIA (DX12 compute limit crashes it)
claude Jun 29, 2026
4c1a5c0
Detect Ceres-without-CUDA so GPU BA isn't falsely reported
claude Jun 29, 2026
34fcb99
Switch GLOMAP to COLMAP's built-in global_mapper (no source build)
claude Jun 29, 2026
928a015
Add loop-closure detection to fix duplicated/ghost geometry
claude Jun 29, 2026
af9204f
Fix wrong 'Apple Silicon detected' label on Windows/CUDA
claude Jun 29, 2026
be8317b
Fix loop-detection vocab tree (FLANN→FAISS) and never abort matching
claude Jun 29, 2026
d0ca9bd
Proactive pipeline audit: photorealism + robustness fixes
claude Jun 29, 2026
757a318
Audit round 2: fix broken SPZ export, ASCII PLY parse, viewer server,…
claude Jun 30, 2026
cef05d2
Mapping + training audit: GLOMAP threading/options, matching coverage…
claude Jul 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,13 @@ refactor-plan.md

# Project Specific (Cleanup)
target/
.venv_sharp/
.venv_360/
.venv_4dgs/
.venv_upscale/

# upscayl-ncnn (downloaded at runtime, not committed)
bin/upscayl-bin
bin/upscayl-bin.exe
models/upscayl/*.bin
models/upscayl/*.param
*.pth
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Changelog

## [2.0.0-win] - Windows / CUDA fork

### 🪟 Platform port — macOS/Apple Silicon → Windows/NVIDIA CUDA
- **Device detection** (`app/core/system.py`): replaced Apple Silicon/MPS detection with CUDA detection via `nvidia-smi` (`get_device()` → `cuda`/`cpu`). Added Windows memory probing (`GlobalMemoryStatusEx`) and `.exe`/`COLMAP.bat` binary resolution, incl. auto-detection of `C:\COLMAP`.
- **GPU acceleration**: FFmpeg frame extraction now uses `-hwaccel cuda` (NVDEC); COLMAP SIFT extraction & matching pass `--SiftExtraction.use_gpu` / `--SiftMatching.use_gpu` when CUDA is present; Brush runs on wgpu with `WGPU_BACKEND=dx12`.
- **Zero-touch engine install**: COLMAP (CUDA) and FFmpeg now auto-install on first launch — `ColmapEngineDep` downloads the `colmap-x64-windows-cuda.zip` release into `engines/colmap`, and `FfmpegEngineDep` installs FFmpeg (winget, else a static build into `engines/ffmpeg`). `resolve_binary` finds them and `run_command` adds COLMAP's `bin`/`lib` to PATH so bundled DLLs load. No GPU → non-CUDA COLMAP build is selected automatically.
- **Windows launcher**: new `run.bat` replaces the macOS `run.command`.
- **Installers**: dropped Homebrew/Xcode; system dependencies (FFmpeg) install via `winget`, COLMAP CUDA build is detected/guided, Node/CMake/Ninja via winget, Rust via `rustup-init.exe`. Removed the macOS-only `pyobjc` dependency.
- **upscayl-bin**: Windows asset selection and `upscayl-bin.exe` discovery.

### 🗑 Removed
- **Apple ML Sharp** (image/video → 3D) — MLX/Apple-Silicon only with no Windows equivalent. Removed the engine, installer, GUI tab, CLI `sharp` subcommand and related tests. The pipeline now focuses on **video → frames → COLMAP → Brush splat training → view**.

### 🧪 Testing
- Test suite updated for the new platform and trimmed of macOS-only cases — **189/189 pass**.

## [1.0.1] - 2026-06-18

### 🐞 Bug Fixes
Expand Down
148 changes: 60 additions & 88 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,115 +1,87 @@
# CorbeauSplat v1.0.0
# CorbeauSplat — Windows / CUDA Edition

**CorbeauSplat** is an all-in-one Gaussian Splatting automation tool designed specifically for **macOS Silicon**. It streamlines the entire workflow from raw video/images to a fully trained and viewable 3D scene (Gaussian Splat).
> 🙏 **Fork of the original [CorbeauSplat](https://github.com/freddewitt/CorbeauSplat) by [@freddewitt](https://github.com/freddewitt)** (a macOS / Apple-Silicon project). All credit for the original concept and pipeline goes to the upstream author — this edition ports it to **Windows + NVIDIA CUDA** and is focused on **video → frames → COLMAP → Brush splat training → view**.

> 🎉 **v1.0.0 — First Stable Release**
> After extensive security hardening, architectural refactoring, and 200+ unit tests.

<div align="center">

[![Buy Me A Coffee](https://www.buymeacoffee.com/assets/img/custom_images/yellow_img.png)](https://www.buymeacoffee.com/freddewitt)

</div>
**CorbeauSplat (Windows / CUDA Edition)** is an all-in-one Gaussian Splatting automation tool for **Windows with NVIDIA CUDA GPUs**. It streamlines the whole workflow from raw video/images to a fully trained and viewable 3D scene (Gaussian Splat).

![CorbeauSplat Interface](assets/interface.webp)

## 🚀 What it does

This application provides a unified Graphical User Interface (GUI) to orchestrate the following steps:
1. **Project Management**: Automatically organizes your outputs into structured project folders with images, sparse data, and checkpoints.
2. **Sparse Reconstruction**: Automates **COLMAP** feature extraction, matching, and mapping. Supports **Glomap** as a modern alternative mapper.
3. **Undistortion**: Automatically undistorts images for optimal training quality.
4. **AI Upscaling**: Optionally enhances input images before reconstruction using **upscayl-ncnn** — a fast NCNN-based upscaler with 6 curated models (Real-ESRGAN x4+, 4xLSDIR, 4xNomos8kSC, and more). Installed automatically at first launch.
5. **Training**: Integrates **Brush** to train Gaussian Splats directly on your Mac.
6. **Visualization**: Includes a built-in tab running **SuperSplat** for immediate local viewing and editing of your PLY files.
7. **ML Sharp (Image/Video to 3D)**: Uses **Apple ML Sharp** to generate a 3D model from a single image or a sequence of 3D models directly from a video.
8. **4DGS Preparation (Experimental)**: A new module to prepare 4D Gaussian Splatting datasets (Multi-camera video -> Nerfstudio format).
9. **360 Extractor (Experimental)**: Converts equirectangular 360° videos into optimal planar image sets (Cube Map, Ring, etc.) for photogrammetry, with AI operator masking.

It is designed to be "click-and-run", handling dependency checks, process management, and **session persistence** for you.
It also includes built-in full localization support for **French, English, German, Italian, Spanish, Arabic, Russian, Chinese, and Japanese**.

## ✍️ A Note from the Author

> This program was realized through **"vibecoding"** with the help of **Gemini 3 Pro**.
>
> It was originally created to facilitate the technical workflow for a documentary film titled **"Le Corbeau"**. I am not a professional developer; I simply needed to automate a complex process by gathering the tools I use daily: COLMAP, the Brush app, and SuperSplat.
>
> I share this code in all humility. I didn't originally plan to release it, but I thought that perhaps someone, somewhere on this earth, might find it useful.
>
> As this software was built via "vibecoding" (AI-assisted coding), it is provided "as is" with no guarantees.
A unified GUI (and CLI) to orchestrate:
1. **Project Management**: Organises outputs into structured project folders (images, sparse data, checkpoints).
2. **Frame Extraction**: Pulls frames from a video with **FFmpeg**, using **NVIDIA NVDEC (`-hwaccel cuda`)** when a GPU is present.
3. **Sparse Reconstruction**: Automates **COLMAP** feature extraction, matching and mapping with **GPU-accelerated SIFT** (the CUDA build uses the GPU automatically). Defaults are tuned for video: **sequential** matching and optional **GPU bundle adjustment** (COLMAP 4.1.0). Supports **Glomap** as an alternative mapper.
4. **Undistortion**: Optionally undistorts images for optimal training quality.
5. **AI Upscaling** *(optional)*: Enhances input images with **upscayl-ncnn** before reconstruction.
6. **Training**: Integrates **Brush** (Rust/wgpu, DirectX 12 / Vulkan) to train Gaussian Splats on your GPU.
7. **Visualization**: Built-in **SuperSplat** tab for immediate local viewing/editing of `.ply` files.
8. **Splat Cleanup**: A **Nettoyage / Cleaner** tab that loads a `.ply` and automatically removes sky, "floaters", near-transparent and isolated splats (variance/opacity/scale + outlier filtering), with a SuperSplat preview before saving. Also available via `python main.py clean -i in.ply -o out.ply`.
8. **4DGS Preparation (Experimental)**: Prepares 4D Gaussian Splatting datasets (multi-camera video → Nerfstudio format).
9. **360 Extractor (Experimental)**: Converts equirectangular 360° videos into planar image sets.

It is designed to be "click-and-run", with dependency checks, process management and session persistence.
Includes full localization for **French, English, German, Italian, Spanish, Arabic, Russian, Chinese and Japanese**.

## 🛠 Prerequisites & Installation

### Requirements
- **macOS** (Apple Silicon recommended)
- **Python 3.13+** (Recommended for JIT/Performance) or Python 3.11 (Supported)
- **Xcode Command Line Tools** (Required for compiling custom engines like Glomap or Brush)
- **Homebrew** (for installing system dependencies like COLMAP and FFmpeg)
- **Git**
- **Windows 10/11 (x64)**
- **NVIDIA GPU** with up-to-date drivers (for CUDA acceleration; CPU fallback works but is slow)
- **Python 3.11+** ([python.org](https://www.python.org/downloads/) — tick *"Add Python to PATH"*)
- **Git** (optional, for updates and source builds)

That's it — **COLMAP (CUDA) and FFmpeg are installed automatically** on first launch (see below). No manual download/PATH setup required.

### Installation
1. Clone this repository:
```bash
git clone https://github.com/freddewitt/CorbeauSplat.git
cd CorbeauSplat
```

2. Run the launcher:
```bash
./run.command
```
*The script will automatically detect missing dependencies (Python packages, Brush, SuperSplat, Rust, Node.js, etc.) and attempt to install them for you.*
1. Clone or download this repository.
2. Double-click **`run.bat`** (or run it from a terminal).

On first launch it automatically:
- creates a Python virtual environment and installs the Python dependencies;
- **downloads the CUDA COLMAP build** (`colmap-x64-windows-cuda.zip`) from GitHub releases into `engines\colmap\`;
- **installs FFmpeg** (via `winget` if available, otherwise a static build into `engines\ffmpeg\`);
- downloads/builds the other engines (Brush, SuperSplat, upscayl);
- launches the app.

> All auto-downloaded engines live under `engines\` (git-ignored). To re-download everything cleanly, run `run.bat --clean`.
>
> If you already have COLMAP installed system-wide, CorbeauSplat will use that instead (it checks PATH and `C:\COLMAP`). No GPU? It falls back to the non-CUDA COLMAP build and CPU execution automatically.

Optional flags:
```bat
run.bat --clean :: wipe venvs + engines + config and start fresh
```

## 📖 How to Use

1. **Configuration Tab**:
- Select your input (Video or Folder of images).
- Define a **Project Name** (your files will be saved in `[Output Folder]/[Project Name]`).
- Click **"Create COLMAP Dataset"**.
2. **Params Tab**: (Optional) Tweak advanced COLMAP settings or enable **Glomap**.
3. **Upscale Tab**: (Optional)
- Enable **"Enable Upscale"** in the Training tab to apply upscaling during dataset creation.
- `upscayl-bin` is automatically downloaded and installed on first launch — no manual setup required.
- Choose a model (e.g., Real-ESRGAN x4+ for photos, 4xLSDIR for ultra fidelity) and configure scale, format, and tile size.
- Download additional models directly from the tab (4xLSDIR, 4xNomos8kSC, NMKD-Siax).
4. **Brush Tab**:
- **Auto-Refine**: Choose "Refine" mode to resume training from the latest checkpoint.
- **Presets**: Use specific densification strategies (e.g., "Aggressive Densification").
- Click **"Start Brush Training"**.
5. **SuperSplat Tab**:
- Load your trained `.ply` file.
- Click **"Start Servers"** to launch the viewer locally.
6. **4DGS Tab (Experimental)**:
- Check **"Activate"** to install the required dependencies (Nerfstudio).
- Select a folder containing your synced camera videos.
- Click **"Start Process"** to generate a dataset ready for 4DGS training.
7. **360 Extractor Tab (Experimental)**:
- **Activate**: Install the dedicated environment (PySide6, YOLOv8).
- **Convert**: Extract images from 360° videos with advanced layouts (Ring, Cube Map, Fibonacci).
- **AI Masking**: Automatically mask the operator.
8. **Apple Sharp Tab (Bonus)**:
- Select a single source image or a **Video**.
- Click **"Predict 3D Model"** or **"Start Conversion"** to generate a mesh sequence using machine learning.
1. **Configuration Tab**: choose your input (a video or a folder of images), set a **Project Name**, then click **"Create COLMAP Dataset"**.
2. **Params Tab**: *(optional)* tweak COLMAP settings or enable **Glomap**.
3. **Upscale Tab**: *(optional)* enable upscaling and pick a model.
4. **Brush Tab**: choose a preset (e.g. "Aggressive Densification"), set **Device** to `cuda`, then **"Start Brush Training"**.
5. **SuperSplat Tab**: load your `.ply` and **"Start Servers"** to view it locally.

### ⌨️ Command Line Interface (CLI)

CorbeauSplat exposes all its features via the command line.
```bat
python main.py pipeline -i video.mp4 -o C:\projects --type video --preset dense
python main.py colmap -i C:\photos -o C:\projects
python main.py brush -i C:\projects\scene -o C:\projects\scene --device cuda --preset dense
python main.py view -i scene.ply
python main.py upscale -i image.png -o C:\out --scale 4
```

� **[See CLI.md for full command line documentation](CLI.md)**
Run `python main.py <command> --help` for per-command options. No arguments launches the GUI.

## 👏 Acknowledgments & Credits

This project stands on the shoulders of giants. A huge thank you to the creators of the core technologies used here:

* **COLMAP**: Structure-from-Motion and Multi-View Stereo. [GitHub](https://github.com/colmap/colmap)
* **Brush**: An efficient Gaussian Splatting trainer for macOS. [GitHub](https://github.com/ArthurBrussee/brush)
* **SuperSplat**: An amazing web-based Splat editor by PlayCanvas. [GitHub](https://github.com/playcanvas/supersplat)
* **360Extractor**: Advanced 360° video extraction tool. [GitHub](https://github.com/nicolasdiolez/360Extractor)
* **Apple ML Sharp**: Machine Learning tools for Swift. [GitHub](https://github.com/apple/ml-sharp)
* **Nerfstudio**: The modular NeRF and Splatting framework (used for 4DGS data prep). [GitHub](https://github.com/nerfstudio-project/nerfstudio)
* **upscayl-ncnn**: High-performance AI image upscaling using NCNN. Powers the Upscale tab. [GitHub](https://github.com/upscayl/upscayl-ncnn)
* **Brush**: An efficient cross-platform Gaussian Splatting trainer (Rust/wgpu). [GitHub](https://github.com/ArthurBrussee/brush)
* **SuperSplat**: Web-based Splat editor by PlayCanvas. [GitHub](https://github.com/playcanvas/supersplat)
* **Glomap**: Global Structure-from-Motion. [GitHub](https://github.com/colmap/glomap)
* **Nerfstudio**: NeRF and Splatting framework (used for 4DGS data prep). [GitHub](https://github.com/nerfstudio-project/nerfstudio)
* **upscayl-ncnn**: High-performance AI image upscaling (NCNN). [GitHub](https://github.com/upscayl/upscayl-ncnn)

## 📄 License

This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details. This is the most permissive open-source license, allowing you to use, modify, and distribute this software freely.
MIT see [LICENSE](LICENSE). Originally created for the documentary *"Le Corbeau"*; ported to Windows/CUDA.
25 changes: 9 additions & 16 deletions app/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,20 @@
"""CorbeauSplat CLI package — command-line interface and GUI launcher."""
import sys

from .parser import get_parser
from .commands import (
run_colmap,
run_brush,
run_sharp,
run_supersplat,
run_upscale,
run_4dgs,
run_extract360,
run_pipeline,
DISPATCH,
BRUSH_DEFAULTS,
BRUSH_PRESETS,
)
from .launcher import _launch_gui
from app.core.system import check_dependencies

from .launcher import _launch_gui
from .parser import get_parser


def main():
parser = get_parser()
args = parser.parse_args()

# No subcommand + no --gui → GUI par défaut
# No subcommand + no --gui → GUI par défaut.
# The GUI is the default path, so the heavy CLI engine stack (.commands pulls
# in ColmapEngine/BrushEngine/… and app.core.engine) is imported lazily in the
# CLI branch below — a GUI launch must not pay for importing it.
if not args.command and not args.gui:
_launch_gui()
return
Expand All @@ -33,6 +24,8 @@ def main():
_launch_gui()
return

from .commands import DISPATCH

missing_deps = check_dependencies()
if missing_deps:
print(f"Attention : dépendances manquantes : {', '.join(missing_deps)}")
Expand Down
Loading