diff --git a/.gitignore b/.gitignore index dea6ce1..6f68f83 100644 --- a/.gitignore +++ b/.gitignore @@ -51,12 +51,13 @@ refactor-plan.md # Project Specific (Cleanup) target/ -.venv_sharp/ .venv_360/ +.venv_4dgs/ .venv_upscale/ # upscayl-ncnn (downloaded at runtime, not committed) bin/upscayl-bin +bin/upscayl-bin.exe models/upscayl/*.bin models/upscayl/*.param *.pth diff --git a/CHANGELOG.md b/CHANGELOG.md index 1aaa146..910d4c0 100755 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,21 @@ # Changelog +## [2.0.0-win] - Windows / CUDA fork + +### 🪟 Platform port — macOS/Apple Silicon → Windows/NVIDIA CUDA +- **Device detection** (`app/core/system.py`): replaced Apple Silicon/MPS detection with CUDA detection via `nvidia-smi` (`get_device()` → `cuda`/`cpu`). Added Windows memory probing (`GlobalMemoryStatusEx`) and `.exe`/`COLMAP.bat` binary resolution, incl. auto-detection of `C:\COLMAP`. +- **GPU acceleration**: FFmpeg frame extraction now uses `-hwaccel cuda` (NVDEC); COLMAP SIFT extraction & matching pass `--SiftExtraction.use_gpu` / `--SiftMatching.use_gpu` when CUDA is present; Brush runs on wgpu with `WGPU_BACKEND=dx12`. +- **Zero-touch engine install**: COLMAP (CUDA) and FFmpeg now auto-install on first launch — `ColmapEngineDep` downloads the `colmap-x64-windows-cuda.zip` release into `engines/colmap`, and `FfmpegEngineDep` installs FFmpeg (winget, else a static build into `engines/ffmpeg`). `resolve_binary` finds them and `run_command` adds COLMAP's `bin`/`lib` to PATH so bundled DLLs load. No GPU → non-CUDA COLMAP build is selected automatically. +- **Windows launcher**: new `run.bat` replaces the macOS `run.command`. +- **Installers**: dropped Homebrew/Xcode; system dependencies (FFmpeg) install via `winget`, COLMAP CUDA build is detected/guided, Node/CMake/Ninja via winget, Rust via `rustup-init.exe`. Removed the macOS-only `pyobjc` dependency. +- **upscayl-bin**: Windows asset selection and `upscayl-bin.exe` discovery. + +### 🗑 Removed +- **Apple ML Sharp** (image/video → 3D) — MLX/Apple-Silicon only with no Windows equivalent. Removed the engine, installer, GUI tab, CLI `sharp` subcommand and related tests. The pipeline now focuses on **video → frames → COLMAP → Brush splat training → view**. + +### 🧪 Testing +- Test suite updated for the new platform and trimmed of macOS-only cases — **189/189 pass**. + ## [1.0.1] - 2026-06-18 ### 🐞 Bug Fixes diff --git a/README.md b/README.md index 36567ca..afe68f6 100644 --- a/README.md +++ b/README.md @@ -1,115 +1,87 @@ -# CorbeauSplat v1.0.0 +# CorbeauSplat — Windows / CUDA Edition -**CorbeauSplat** is an all-in-one Gaussian Splatting automation tool designed specifically for **macOS Silicon**. It streamlines the entire workflow from raw video/images to a fully trained and viewable 3D scene (Gaussian Splat). +> 🙏 **Fork of the original [CorbeauSplat](https://github.com/freddewitt/CorbeauSplat) by [@freddewitt](https://github.com/freddewitt)** (a macOS / Apple-Silicon project). All credit for the original concept and pipeline goes to the upstream author — this edition ports it to **Windows + NVIDIA CUDA** and is focused on **video → frames → COLMAP → Brush splat training → view**. -> 🎉 **v1.0.0 — First Stable Release** -> After extensive security hardening, architectural refactoring, and 200+ unit tests. - -
- -[![Buy Me A Coffee](https://www.buymeacoffee.com/assets/img/custom_images/yellow_img.png)](https://www.buymeacoffee.com/freddewitt) - -
+**CorbeauSplat (Windows / CUDA Edition)** is an all-in-one Gaussian Splatting automation tool for **Windows with NVIDIA CUDA GPUs**. It streamlines the whole workflow from raw video/images to a fully trained and viewable 3D scene (Gaussian Splat). ![CorbeauSplat Interface](assets/interface.webp) ## 🚀 What it does -This application provides a unified Graphical User Interface (GUI) to orchestrate the following steps: -1. **Project Management**: Automatically organizes your outputs into structured project folders with images, sparse data, and checkpoints. -2. **Sparse Reconstruction**: Automates **COLMAP** feature extraction, matching, and mapping. Supports **Glomap** as a modern alternative mapper. -3. **Undistortion**: Automatically undistorts images for optimal training quality. -4. **AI Upscaling**: Optionally enhances input images before reconstruction using **upscayl-ncnn** — a fast NCNN-based upscaler with 6 curated models (Real-ESRGAN x4+, 4xLSDIR, 4xNomos8kSC, and more). Installed automatically at first launch. -5. **Training**: Integrates **Brush** to train Gaussian Splats directly on your Mac. -6. **Visualization**: Includes a built-in tab running **SuperSplat** for immediate local viewing and editing of your PLY files. -7. **ML Sharp (Image/Video to 3D)**: Uses **Apple ML Sharp** to generate a 3D model from a single image or a sequence of 3D models directly from a video. -8. **4DGS Preparation (Experimental)**: A new module to prepare 4D Gaussian Splatting datasets (Multi-camera video -> Nerfstudio format). -9. **360 Extractor (Experimental)**: Converts equirectangular 360° videos into optimal planar image sets (Cube Map, Ring, etc.) for photogrammetry, with AI operator masking. - -It is designed to be "click-and-run", handling dependency checks, process management, and **session persistence** for you. -It also includes built-in full localization support for **French, English, German, Italian, Spanish, Arabic, Russian, Chinese, and Japanese**. - -## ✍ A Note from the Author - -> This program was realized through **"vibecoding"** with the help of **Gemini 3 Pro**. -> -> It was originally created to facilitate the technical workflow for a documentary film titled **"Le Corbeau"**. I am not a professional developer; I simply needed to automate a complex process by gathering the tools I use daily: COLMAP, the Brush app, and SuperSplat. -> -> I share this code in all humility. I didn't originally plan to release it, but I thought that perhaps someone, somewhere on this earth, might find it useful. -> -> As this software was built via "vibecoding" (AI-assisted coding), it is provided "as is" with no guarantees. +A unified GUI (and CLI) to orchestrate: +1. **Project Management**: Organises outputs into structured project folders (images, sparse data, checkpoints). +2. **Frame Extraction**: Pulls frames from a video with **FFmpeg**, using **NVIDIA NVDEC (`-hwaccel cuda`)** when a GPU is present. +3. **Sparse Reconstruction**: Automates **COLMAP** feature extraction, matching and mapping with **GPU-accelerated SIFT** (the CUDA build uses the GPU automatically). Defaults are tuned for video: **sequential** matching and optional **GPU bundle adjustment** (COLMAP 4.1.0). Supports **Glomap** as an alternative mapper. +4. **Undistortion**: Optionally undistorts images for optimal training quality. +5. **AI Upscaling** *(optional)*: Enhances input images with **upscayl-ncnn** before reconstruction. +6. **Training**: Integrates **Brush** (Rust/wgpu, DirectX 12 / Vulkan) to train Gaussian Splats on your GPU. +7. **Visualization**: Built-in **SuperSplat** tab for immediate local viewing/editing of `.ply` files. +8. **Splat Cleanup**: A **Nettoyage / Cleaner** tab that loads a `.ply` and automatically removes sky, "floaters", near-transparent and isolated splats (variance/opacity/scale + outlier filtering), with a SuperSplat preview before saving. Also available via `python main.py clean -i in.ply -o out.ply`. +8. **4DGS Preparation (Experimental)**: Prepares 4D Gaussian Splatting datasets (multi-camera video → Nerfstudio format). +9. **360 Extractor (Experimental)**: Converts equirectangular 360° videos into planar image sets. + +It is designed to be "click-and-run", with dependency checks, process management and session persistence. +Includes full localization for **French, English, German, Italian, Spanish, Arabic, Russian, Chinese and Japanese**. ## 🛠 Prerequisites & Installation ### Requirements -- **macOS** (Apple Silicon recommended) -- **Python 3.13+** (Recommended for JIT/Performance) or Python 3.11 (Supported) -- **Xcode Command Line Tools** (Required for compiling custom engines like Glomap or Brush) -- **Homebrew** (for installing system dependencies like COLMAP and FFmpeg) -- **Git** +- **Windows 10/11 (x64)** +- **NVIDIA GPU** with up-to-date drivers (for CUDA acceleration; CPU fallback works but is slow) +- **Python 3.11+** ([python.org](https://www.python.org/downloads/) — tick *"Add Python to PATH"*) +- **Git** (optional, for updates and source builds) + +That's it — **COLMAP (CUDA) and FFmpeg are installed automatically** on first launch (see below). No manual download/PATH setup required. ### Installation -1. Clone this repository: - ```bash - git clone https://github.com/freddewitt/CorbeauSplat.git - cd CorbeauSplat - ``` - -2. Run the launcher: - ```bash - ./run.command - ``` - *The script will automatically detect missing dependencies (Python packages, Brush, SuperSplat, Rust, Node.js, etc.) and attempt to install them for you.* +1. Clone or download this repository. +2. Double-click **`run.bat`** (or run it from a terminal). + + On first launch it automatically: + - creates a Python virtual environment and installs the Python dependencies; + - **downloads the CUDA COLMAP build** (`colmap-x64-windows-cuda.zip`) from GitHub releases into `engines\colmap\`; + - **installs FFmpeg** (via `winget` if available, otherwise a static build into `engines\ffmpeg\`); + - downloads/builds the other engines (Brush, SuperSplat, upscayl); + - launches the app. + +> All auto-downloaded engines live under `engines\` (git-ignored). To re-download everything cleanly, run `run.bat --clean`. +> +> If you already have COLMAP installed system-wide, CorbeauSplat will use that instead (it checks PATH and `C:\COLMAP`). No GPU? It falls back to the non-CUDA COLMAP build and CPU execution automatically. + +Optional flags: +```bat +run.bat --clean :: wipe venvs + engines + config and start fresh +``` ## 📖 How to Use -1. **Configuration Tab**: - - Select your input (Video or Folder of images). - - Define a **Project Name** (your files will be saved in `[Output Folder]/[Project Name]`). - - Click **"Create COLMAP Dataset"**. -2. **Params Tab**: (Optional) Tweak advanced COLMAP settings or enable **Glomap**. -3. **Upscale Tab**: (Optional) - - Enable **"Enable Upscale"** in the Training tab to apply upscaling during dataset creation. - - `upscayl-bin` is automatically downloaded and installed on first launch — no manual setup required. - - Choose a model (e.g., Real-ESRGAN x4+ for photos, 4xLSDIR for ultra fidelity) and configure scale, format, and tile size. - - Download additional models directly from the tab (4xLSDIR, 4xNomos8kSC, NMKD-Siax). -4. **Brush Tab**: - - **Auto-Refine**: Choose "Refine" mode to resume training from the latest checkpoint. - - **Presets**: Use specific densification strategies (e.g., "Aggressive Densification"). - - Click **"Start Brush Training"**. -5. **SuperSplat Tab**: - - Load your trained `.ply` file. - - Click **"Start Servers"** to launch the viewer locally. -6. **4DGS Tab (Experimental)**: - - Check **"Activate"** to install the required dependencies (Nerfstudio). - - Select a folder containing your synced camera videos. - - Click **"Start Process"** to generate a dataset ready for 4DGS training. -7. **360 Extractor Tab (Experimental)**: - - **Activate**: Install the dedicated environment (PySide6, YOLOv8). - - **Convert**: Extract images from 360° videos with advanced layouts (Ring, Cube Map, Fibonacci). - - **AI Masking**: Automatically mask the operator. -8. **Apple Sharp Tab (Bonus)**: - - Select a single source image or a **Video**. - - Click **"Predict 3D Model"** or **"Start Conversion"** to generate a mesh sequence using machine learning. +1. **Configuration Tab**: choose your input (a video or a folder of images), set a **Project Name**, then click **"Create COLMAP Dataset"**. +2. **Params Tab**: *(optional)* tweak COLMAP settings or enable **Glomap**. +3. **Upscale Tab**: *(optional)* enable upscaling and pick a model. +4. **Brush Tab**: choose a preset (e.g. "Aggressive Densification"), set **Device** to `cuda`, then **"Start Brush Training"**. +5. **SuperSplat Tab**: load your `.ply` and **"Start Servers"** to view it locally. ### ⌚ Command Line Interface (CLI) -CorbeauSplat exposes all its features via the command line. +```bat +python main.py pipeline -i video.mp4 -o C:\projects --type video --preset dense +python main.py colmap -i C:\photos -o C:\projects +python main.py brush -i C:\projects\scene -o C:\projects\scene --device cuda --preset dense +python main.py view -i scene.ply +python main.py upscale -i image.png -o C:\out --scale 4 +``` -ï¿œ **[See CLI.md for full command line documentation](CLI.md)** +Run `python main.py --help` for per-command options. No arguments launches the GUI. ## 👏 Acknowledgments & Credits -This project stands on the shoulders of giants. A huge thank you to the creators of the core technologies used here: - * **COLMAP**: Structure-from-Motion and Multi-View Stereo. [GitHub](https://github.com/colmap/colmap) -* **Brush**: An efficient Gaussian Splatting trainer for macOS. [GitHub](https://github.com/ArthurBrussee/brush) -* **SuperSplat**: An amazing web-based Splat editor by PlayCanvas. [GitHub](https://github.com/playcanvas/supersplat) -* **360Extractor**: Advanced 360° video extraction tool. [GitHub](https://github.com/nicolasdiolez/360Extractor) -* **Apple ML Sharp**: Machine Learning tools for Swift. [GitHub](https://github.com/apple/ml-sharp) -* **Nerfstudio**: The modular NeRF and Splatting framework (used for 4DGS data prep). [GitHub](https://github.com/nerfstudio-project/nerfstudio) -* **upscayl-ncnn**: High-performance AI image upscaling using NCNN. Powers the Upscale tab. [GitHub](https://github.com/upscayl/upscayl-ncnn) +* **Brush**: An efficient cross-platform Gaussian Splatting trainer (Rust/wgpu). [GitHub](https://github.com/ArthurBrussee/brush) +* **SuperSplat**: Web-based Splat editor by PlayCanvas. [GitHub](https://github.com/playcanvas/supersplat) +* **Glomap**: Global Structure-from-Motion. [GitHub](https://github.com/colmap/glomap) +* **Nerfstudio**: NeRF and Splatting framework (used for 4DGS data prep). [GitHub](https://github.com/nerfstudio-project/nerfstudio) +* **upscayl-ncnn**: High-performance AI image upscaling (NCNN). [GitHub](https://github.com/upscayl/upscayl-ncnn) ## 📄 License -This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details. This is the most permissive open-source license, allowing you to use, modify, and distribute this software freely. +MIT — see [LICENSE](LICENSE). Originally created for the documentary *"Le Corbeau"*; ported to Windows/CUDA. diff --git a/app/cli/__init__.py b/app/cli/__init__.py index 5358105..84dad18 100644 --- a/app/cli/__init__.py +++ b/app/cli/__init__.py @@ -2,29 +2,20 @@ """CorbeauSplat CLI package — command-line interface and GUI launcher.""" import sys -from .parser import get_parser -from .commands import ( - run_colmap, - run_brush, - run_sharp, - run_supersplat, - run_upscale, - run_4dgs, - run_extract360, - run_pipeline, - DISPATCH, - BRUSH_DEFAULTS, - BRUSH_PRESETS, -) -from .launcher import _launch_gui from app.core.system import check_dependencies +from .launcher import _launch_gui +from .parser import get_parser + def main(): parser = get_parser() args = parser.parse_args() - # No subcommand + no --gui → GUI par défaut + # No subcommand + no --gui → GUI par défaut. + # The GUI is the default path, so the heavy CLI engine stack (.commands pulls + # in ColmapEngine/BrushEngine/
 and app.core.engine) is imported lazily in the + # CLI branch below — a GUI launch must not pay for importing it. if not args.command and not args.gui: _launch_gui() return @@ -33,6 +24,8 @@ def main(): _launch_gui() return + from .commands import DISPATCH + missing_deps = check_dependencies() if missing_deps: print(f"Attention : dépendances manquantes : {', '.join(missing_deps)}") diff --git a/app/cli/commands.py b/app/cli/commands.py index 2ffb9e5..aaf9ee9 100644 --- a/app/cli/commands.py +++ b/app/cli/commands.py @@ -1,19 +1,17 @@ #!/usr/bin/env python3 """CLI command handlers for CorbeauSplat.""" -import sys import os +import sys import time from pathlib import Path as _Path +from app.core.brush_engine import BrushEngine, rename_latest_ply +from app.core.engine import ColmapEngine from app.core.i18n import tr from app.core.params import ColmapParams -from app.core.engine import ColmapEngine -from app.core.brush_engine import BrushEngine -from app.core.sharp_engine import SharpEngine from app.core.superplat_engine import SuperSplatEngine from app.core.system import get_brush_build_mode - # ───────────────────────────────────────────────────────────────────────────── # Brush defaults and presets # ───────────────────────────────────────────────────────────────────────────── @@ -33,6 +31,19 @@ "refine_mode": False, } +_BLUR_FACTORS = {"light": 0.5, "medium": 0.7, "strong": 0.85} + + +def _apply_robust(params): + """Stabilise COLMAP bundle adjustment on large scenes (anti-crash).""" + params.camera_model = "PINHOLE" + params.ba_refine_extra_params = False + params.ba_refine_principal_point = False + params.multiple_models = True + params.filter_blurry = True + return params + + BRUSH_PRESETS = { "fast": { "total_steps": 7000, "refine_every": 100, @@ -63,7 +74,7 @@ def run_colmap(args): max_image_size=args.max_image_size, max_num_features=args.max_num_features, estimate_affine_shape=args.estimate_affine_shape, - domain_size_pooling=not args.no_domain_size_pooling, + domain_size_pooling=args.domain_size_pooling, max_ratio=args.max_ratio, max_distance=args.max_distance, cross_check=not args.no_cross_check, @@ -74,9 +85,15 @@ def run_colmap(args): ba_refine_extra_params=not args.no_refine_extra, min_num_matches=args.min_num_matches, matcher_type=args.matcher_type, + sequential_overlap=args.sequential_overlap, + loop_detection=not args.no_loop_detection, undistort_images=args.undistort, use_glomap=args.use_glomap, + filter_blurry=args.filter_blur, + blur_factor=_BLUR_FACTORS.get(args.blur_strength, 0.7), ) + if getattr(args, "robust", False): + _apply_robust(params) print(tr("cli_start_colmap")) print(tr("cli_input", args.input)) @@ -137,6 +154,8 @@ def run_brush(args): try: returncode = engine.train(args.input, args.output, params=params) if returncode == 0: + # CLI parity with the GUI: honour --ply_name on the exported splat. + rename_latest_ply(args.output, params.get("ply_name"), log=print) print(tr("msg_success")) else: print(tr("msg_error")) @@ -146,64 +165,18 @@ def run_brush(args): engine.stop() -def run_sharp(args): - engine = SharpEngine(logger_callback=print) - - params = { - "checkpoint": args.checkpoint, - "device": args.device, - "verbose": args.verbose, - } - - if args.mode == "image": - print(tr("cli_start_sharp")) - print(tr("cli_input", args.input)) - print(tr("cli_output", args.output)) - - try: - returncode = engine.predict(args.input, args.output, params=params) - if returncode == 0: - print(tr("msg_success")) - else: - print(tr("msg_error")) - sys.exit(1) - except KeyboardInterrupt: - print(tr("cli_stopping")) - engine.stop() - - else: # video mode - _run_sharp_video(args, engine, params) - - -def _run_sharp_video(args, engine, params): - """CLI handler for Sharp video mode — delegates to shared SharpEngine.process_video_frames().""" - video_path = _Path(args.input) - output_dir = _Path(args.output) - skip = max(1, args.skip_frames) - - print(f"Sharp vidéo : {video_path.name} (1 frame / {skip})") - print(tr("cli_output", args.output)) - - params["skip_frames"] = skip +def run_clean(args): + from app.core.ply_cleaner import clean_ply + print(f"Nettoyage du splat : {args.input} (sévérité: {args.strength})") try: - success_count = engine.process_video_frames( - video_path=str(video_path), - output_dir=str(output_dir), - params=params, - log_callback=print, - status_callback=lambda s: print(f" {s}"), - progress_callback=None, - cancel_check=None, - ) - except KeyboardInterrupt: - print(tr("cli_stopping")) - engine.stop() + stats = clean_ply(args.input, args.output, strength=args.strength, log=print) + except Exception as e: + print(f"{tr('msg_error')}: {e}") sys.exit(1) - print(f"Terminé : {success_count} frames converties.") - if success_count == 0: - sys.exit(1) + print(f"✅ {stats['kept']}/{stats['total']} splats conservés " + f"({stats['removed']} retirés) → {args.output}") def run_supersplat(args): @@ -393,10 +366,16 @@ def run_pipeline(args): colmap_params = ColmapParams( camera_model=args.camera_model, matcher_type=args.matcher_type, + sequential_overlap=args.sequential_overlap, + loop_detection=not args.no_loop_detection, max_image_size=args.max_image_size, undistort_images=args.undistort, use_glomap=args.use_glomap, + filter_blurry=args.filter_blur, + blur_factor=_BLUR_FACTORS.get(args.blur_strength, 0.7), ) + if getattr(args, "robust", False): + _apply_robust(colmap_params) colmap_engine = ColmapEngine( colmap_params, args.input, args.output, args.type, args.fps, @@ -417,6 +396,13 @@ def run_pipeline(args): sys.exit(1) dataset_path = _Path(args.output) / args.project_name + # Train on the undistorted dense/ dataset when it exists — otherwise the + # --undistort step would be dead weight (Brush would silently read the + # original distorted images/ + sparse/0 from the project root). + dense_dir = dataset_path / "dense" + if args.undistort and (dense_dir / "images").is_dir() and (dense_dir / "sparse").is_dir(): + dataset_path = dense_dir + print("Images non-distordues détectées : entraînement sur dense/.") print(f"\nDataset prêt : {dataset_path}") # ── Étape 2 : Brush ─────────────────────────────────────────────────────── @@ -453,6 +439,7 @@ def run_pipeline(args): sys.exit(0) if returncode == 0: + rename_latest_ply(str(dataset_path), brush_params.get("ply_name"), log=print) print(f"\nPipeline terminé. Splat disponible dans : {dataset_path}") else: print(f"\nBrush a retourné une erreur (code {returncode}).") @@ -467,7 +454,7 @@ def run_pipeline(args): "pipeline": run_pipeline, "colmap": run_colmap, "brush": run_brush, - "sharp": run_sharp, + "clean": run_clean, "view": run_supersplat, "upscale": run_upscale, "4dgs": run_4dgs, diff --git a/app/cli/launcher.py b/app/cli/launcher.py index 3fe79b5..b7b15a2 100644 --- a/app/cli/launcher.py +++ b/app/cli/launcher.py @@ -1,38 +1,24 @@ #!/usr/bin/env python3 -"""GUI launcher helpers for CorbeauSplat.""" +"""GUI launcher helpers for CorbeauSplat (Windows).""" import sys from pathlib import Path as _Path -def _set_macos_dock_icon(icon_path: _Path): - try: - from AppKit import NSApplication, NSImage - ns_image = NSImage.alloc().initWithContentsOfFile_(str(icon_path)) - if ns_image: - NSApplication.sharedApplication().setApplicationIconImage_(ns_image) - except Exception: - pass - - def _launch_gui(): - from PyQt6.QtWidgets import QApplication from PyQt6.QtGui import QIcon - from PyQt6.QtCore import QTimer + from PyQt6.QtWidgets import QApplication + from app.gui.main_window import ColmapGUI app = QApplication(sys.argv) assets = _Path(__file__).resolve().parent.parent.parent / "assets" - png_path = assets / "icon.png" - icns_path = assets / "icon.icns" - icon_file = png_path if png_path.exists() else icns_path + png_path = assets / "icon.png" + ico_path = assets / "icon.ico" + icon_file = ico_path if ico_path.exists() else png_path if icon_file.exists(): app.setWindowIcon(QIcon(str(icon_file))) - dock_src = icns_path if icns_path.exists() else png_path - if dock_src.exists(): - QTimer.singleShot(0, lambda: _set_macos_dock_icon(dock_src)) - window = ColmapGUI() window.show() sys.exit(app.exec()) diff --git a/app/cli/parser.py b/app/cli/parser.py index a84cfca..0ce7b46 100644 --- a/app/cli/parser.py +++ b/app/cli/parser.py @@ -15,7 +15,6 @@ def get_parser(): " python3 main.py pipeline -i video.mp4 -o ~/projets --type video --preset dense\n" " python3 main.py colmap -i video.mp4 -o ~/projets\n" " python3 main.py brush -i ~/projets/scene -o ~/projets/scene --preset dense\n" - " python3 main.py sharp -i photo.jpg -o ~/out\n" " python3 main.py view -i splat.ply\n" " python3 main.py upscale -i image.png -o ~/out --scale 4\n" " python3 main.py 4dgs -i ~/videos -o ~/out\n" @@ -53,8 +52,12 @@ def get_parser(): help="ModÚle de caméra COLMAP (défaut: SIMPLE_RADIAL)") p.add_argument("--undistort", action="store_true", help="Undistortion aprÚs reconstruction") p.add_argument("--use_glomap", action="store_true", help="Utiliser Glomap au lieu du mapper COLMAP") - p.add_argument("--matcher_type", choices=["exhaustive","sequential","vocab_tree"], default="exhaustive", - help="Stratégie de matching (défaut: exhaustive)") + p.add_argument("--matcher_type", choices=["exhaustive","sequential","vocab_tree"], default="sequential", + help="Stratégie de matching (défaut: sequential — rapide et correct pour vidéos)") + p.add_argument("--sequential_overlap", type=int, default=30, + help="Nb d'images voisines comparées en matching séquentiel (défaut: 30)") + p.add_argument("--no_loop_detection", action="store_true", + help="Désactiver la détection de boucles (anti-duplication des lieux revisités)") p.add_argument("--max_image_size", type=int, default=3200, help="Résolution max des images pour COLMAP (défaut: 3200)") # Brush @@ -65,11 +68,16 @@ def get_parser(): p.add_argument("--sh_degree", type=int, default=None, choices=range(1,5), help="Degré Spherical Harmonics 1-4 (défaut: 3)") p.add_argument("--device", default="auto", - choices=["auto","mps","cuda","cpu"], help="Device Brush (défaut: auto)") + choices=["auto","cuda","cpu"], help="Device Brush (défaut: auto)") p.add_argument("--with_viewer", action="store_true", help="Ouvrir le viewer interactif aprÚs entraînement") p.add_argument("--max_resolution", type=int, default=None, help="Résolution max entraînement 0=auto (défaut: 0)") p.add_argument("--ply_name", default=None, help="Nom du fichier PLY de sortie") + p.add_argument("--filter_blur", action="store_true", help="Écarter les images floues avant COLMAP") + p.add_argument("--blur_strength", choices=["light", "medium", "strong"], default="medium", + help="Sévérité du filtre de flou (défaut: medium)") + p.add_argument("--robust", action="store_true", + help="Mode robuste grandes scÚnes (PINHOLE, anti-crash BA, filtre flou)") # ── colmap ──────────────────────────────────────────────────────────────── p = subs.add_parser("colmap", help="Pipeline COLMAP (vidéo/images → dataset)") @@ -89,10 +97,15 @@ def get_parser(): p.add_argument("--max_image_size", type=int, default=3200, help="Résolution max des images (défaut: 3200)") p.add_argument("--max_num_features", type=int, default=8192, help="Nb max de features par image (défaut: 8192)") p.add_argument("--estimate_affine_shape", action="store_true", help="Estimer la forme affine des features") - p.add_argument("--no_domain_size_pooling", action="store_true", help="Désactiver le domain size pooling") + p.add_argument("--domain_size_pooling", action="store_true", + help="Activer le domain size pooling (force le SIFT sur CPU — désactivé par défaut)") # Feature matching - p.add_argument("--matcher_type", choices=["exhaustive","sequential","vocab_tree"], default="exhaustive", - help="Stratégie de matching (défaut: exhaustive)") + p.add_argument("--matcher_type", choices=["exhaustive","sequential","vocab_tree"], default="sequential", + help="Stratégie de matching (défaut: sequential — rapide et correct pour vidéos)") + p.add_argument("--sequential_overlap", type=int, default=30, + help="Nb d'images voisines comparées en matching séquentiel (défaut: 30)") + p.add_argument("--no_loop_detection", action="store_true", + help="Désactiver la détection de boucles (anti-duplication des lieux revisités)") p.add_argument("--max_ratio", type=float, default=0.8, help="Ratio max Lowe (défaut: 0.8)") p.add_argument("--max_distance", type=float, default=0.7, help="Distance max (défaut: 0.7)") p.add_argument("--no_cross_check", action="store_true", help="Désactiver le cross-check") @@ -103,6 +116,12 @@ def get_parser(): p.add_argument("--no_refine_focal", action="store_true", help="Ne pas affiner la focale") p.add_argument("--refine_principal", action="store_true", help="Affiner le point principal") p.add_argument("--no_refine_extra", action="store_true", help="Ne pas affiner les params extra") + # Blur filtering + p.add_argument("--filter_blur", action="store_true", help="Écarter les images floues avant COLMAP") + p.add_argument("--blur_strength", choices=["light", "medium", "strong"], default="medium", + help="Sévérité du filtre de flou (défaut: medium)") + p.add_argument("--robust", action="store_true", + help="Mode robuste grandes scÚnes (PINHOLE, anti-crash BA, filtre flou)") # ── brush ───────────────────────────────────────────────────────────────── p = subs.add_parser("brush", help="Entraînement Gaussian Splat (Brush)") @@ -115,7 +134,7 @@ def get_parser(): p.add_argument("--sh_degree", type=int, default=None, choices=range(1,5), help="Degré Spherical Harmonics 1-4 (défaut: 3)") p.add_argument("--device", default="auto", - choices=["auto","mps","cuda","cpu"], help="Device (défaut: auto)") + choices=["auto","cuda","cpu"], help="Device (défaut: auto)") p.add_argument("--refine_mode", action="store_true", help="Mode Refine (reprend depuis dernier checkpoint)") p.add_argument("--with_viewer", action="store_true", help="Ouvrir le viewer interactif") p.add_argument("--ply_name", default=None, help="Nom du fichier PLY de sortie") @@ -130,21 +149,6 @@ def get_parser(): p.add_argument("--checkpoint_interval", type=int, default=None, help="Sauvegarder tous les N iters (défaut: 7000)") p.add_argument("--max_resolution", type=int, default=None, help="Résolution max entraînement 0=auto (défaut: 0)") - # ── sharp ───────────────────────────────────────────────────────────────── - p = subs.add_parser("sharp", help="Single Image/Vidéo → 3D Splat (ML-Sharp)") - p.add_argument("--input", "-i", required=True, help="Image, dossier d'images ou vidéo") - p.add_argument("--output", "-o", required=True, help="Dossier de sortie") - p.add_argument("--mode", choices=["image","video"], default="image", - help="Mode : image unique ou vidéo (défaut: image)") - p.add_argument("--checkpoint", "-c", default=None, help="Chemin vers un checkpoint .pt") - p.add_argument("--device", default="default", - choices=["default","mps","cpu","cuda"], help="Device (défaut: default)") - p.add_argument("--skip_frames", type=int, default=1, - help="[mode vidéo] Traiter 1 frame sur N (défaut: 1)") - p.add_argument("--upscale", action="store_true", - help="Upscaler les images avant prédiction (requiert upscayl-bin)") - p.add_argument("--verbose", action="store_true", help="Afficher la sortie détaillée de Sharp") - # ── view ────────────────────────────────────────────────────────────────── p = subs.add_parser("view", help="Visualiser un .ply dans SuperSplat") p.add_argument("--input", "-i", required=True, help="Fichier .ply ou dossier") @@ -170,6 +174,13 @@ def get_parser(): p.add_argument("--compression", type=int, default=0, help="Niveau de compression sortie 0-9 (défaut: 0)") + # ── clean ───────────────────────────────────────────────────────────────── + p = subs.add_parser("clean", help="Nettoyer un splat .ply (ciel/floaters/bruit)") + p.add_argument("--input", "-i", required=True, help="Fichier .ply à nettoyer") + p.add_argument("--output", "-o", required=True, help="Fichier .ply de sortie") + p.add_argument("--strength", choices=["light", "medium", "strong"], default="medium", + help="Sévérité du nettoyage (défaut: medium)") + # ── 4dgs ────────────────────────────────────────────────────────────────── p = subs.add_parser("4dgs", help="Préparation dataset 4D Gaussian Splatting (Nerfstudio)") p.add_argument("--input", "-i", required=True, diff --git a/app/core/base_engine.py b/app/core/base_engine.py index 2868703..f9a969b 100644 --- a/app/core/base_engine.py +++ b/app/core/base_engine.py @@ -1,10 +1,11 @@ +import logging import os -import sys import signal import subprocess -import logging +import sys +from collections.abc import Iterator from pathlib import Path -from typing import Iterator + from .system import get_device, resolve_project_root logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s') @@ -13,19 +14,19 @@ class IProcessRunner: """Interface abstraite pour l'exécution d'un processus systéme (DIP & Testabilité)""" def start(self, cmd: list, env: dict = None, **kwargs): raise NotImplementedError() - + def poll(self): raise NotImplementedError() - + def wait(self, timeout=None): raise NotImplementedError() - + def terminate(self): raise NotImplementedError() - + def stdout_iter(self) -> Iterator[str]: raise NotImplementedError() - + def get_returncode(self) -> int: raise NotImplementedError() @@ -33,47 +34,59 @@ class SubprocessRunner(IProcessRunner): """Implémentation concrÚte de l'OS via subprocess""" def __init__(self): self._process = None - + def start(self, cmd: list, env: dict = None, **kwargs): base_kwargs = { 'stdout': subprocess.PIPE, 'stderr': subprocess.STDOUT, 'text': True, + 'bufsize': 1, # line-buffered: flush COLMAP/bundle-adjustment progress + # promptly so the long global-BA phase doesn't look frozen } base_kwargs.update(kwargs) - + # Sécurisation du process group pour permettre de kill l'arbre process if sys.platform != "win32" and 'preexec_fn' not in base_kwargs: base_kwargs['preexec_fn'] = os.setsid - + self._process = subprocess.Popen(cmd, env=env, **base_kwargs) return self._process - + def poll(self): if self._process: return self._process.poll() return None - + def wait(self, timeout=None): if self._process: return self._process.wait(timeout) return None - + def terminate(self): if not self._process: return try: if sys.platform != "win32": os.killpg(os.getpgid(self._process.pid), signal.SIGTERM) + self._process.wait(timeout=5) else: - self._process.terminate() - self._process.wait(timeout=5) + # On Windows a `cmd /c npx ...` wrapper spawns child node/colmap + # processes that process.terminate() does NOT kill, leaving them + # orphaned (e.g. holding the SuperSplat port). Kill the whole tree. + subprocess.run( + ["taskkill", "/F", "/T", "/PID", str(self._process.pid)], + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + self._process.wait(timeout=5) except (ProcessLookupError, PermissionError, OSError, subprocess.TimeoutExpired): - self._process.kill() - self._process.wait() - + try: + self._process.kill() + self._process.wait(timeout=5) + except (OSError, subprocess.TimeoutExpired): + pass + def stdout_iter(self) -> Iterator[str]: if getattr(self._process, 'stdout', None): for line in self._process.stdout: yield line - + def get_returncode(self) -> int: if self._process: return self._process.returncode return -1 @@ -89,9 +102,9 @@ def __init__(self, name, logger_callback=None, process_runner: IProcessRunner = self.device = get_device() self.project_root = resolve_project_root() self.stop_requested = False - + self.logger = logging.getLogger(self.name) - + # SOLID-DIP : Injection abstraite pour tests (mockable) self.runner = process_runner or SubprocessRunner() self.process = None # Retro-compatibilité temporaire @@ -113,28 +126,34 @@ def _execute_command(self, cmd: list, env: dict = None, line_callback=None, **kw Retourne le returncode (0 si succÚs, -1 si annulé ou erreur). """ if self.stop_requested: return -1 - + self.log(f"Exec: {' '.join(map(str, cmd))}") try: self.runner.start(cmd, env=env, **kwargs) self.process = getattr(self.runner, '_process', None) # Legacy mapping - + for line in self.runner.stdout_iter(): if self.stop_requested: self.runner.terminate() return -1 - + stripped = line.strip() if stripped: if line_callback: line_callback(stripped) else: self.log(stripped) - + return self.runner.wait() except Exception as e: self.logger.error("Exception in _execute_command", exc_info=True) self.log(f"Exception: {e}", level=logging.ERROR) + # Never leave the child running if streaming/parsing threw — otherwise a + # COLMAP/Brush process would be orphaned (zombie holding the GPU/port). + try: + self.runner.terminate() + except Exception: + self.logger.debug("terminate() after exception failed", exc_info=True) return -1 def _kill_process(self, process): @@ -152,20 +171,22 @@ def _kill_process(self, process): process.wait() def validate_path(self, path): - """Resolves and validates a path to prevent traversal using resolved containment""" + """Resolves and validates a user-supplied path. + + This is a desktop tool that operates on the user's own files, so any + absolute location (including other drives, e.g. ``I:\\projects``) is + allowed — the previous home/project containment broke legitimate output + folders. We still reject ``..`` parent-traversal segments, which are the + real escape vector for crafted/relative inputs. + """ if not path: return None try: - p = Path(path).resolve() - allowed_bases = [self.project_root.resolve(), Path.home().resolve()] - for base in allowed_bases: - try: - p.relative_to(base) - return p - except ValueError: - pass - self.log(f"SECURITY WARNING: Path access outside allowed boundaries: {p}") - return None + raw = Path(path) + if ".." in raw.parts: + self.log(f"SECURITY WARNING: path traversal rejected: {path}") + return None + return raw.resolve() except (TypeError, ValueError, OSError) as e: self.log(f"ERROR: Invalid path attempt : {path} ({e})") return None diff --git a/app/core/brush_engine.py b/app/core/brush_engine.py index 7712bd3..f06bbab 100755 --- a/app/core/brush_engine.py +++ b/app/core/brush_engine.py @@ -1,25 +1,37 @@ import os -import logging -from pathlib import Path -from typing import Optional, Dict, Any, Callable, List, Tuple +from collections.abc import Callable +from typing import Any from .base_engine import BaseEngine from .system import resolve_binary + class BrushEngine(BaseEngine): """Engine for executing the Brush training pipeline. Provides path validation, secure command construction, and structured logging. """ + # Custom-args whitelist — every entry verified against the real Brush v0.3.0 + # CLI (clap). A flag Brush doesn't know makes it ABORT with "unexpected + # argument", so phantom flags here are booby traps: --refine-pose, + # --test-split, --save-iterations and --log-level do NOT exist in any Brush + # release (eval split is --eval-split-every; logging is the RUST_LOG env). ALLOWED_FLAGS = { - "--save-iterations", "--log-level", "--test-split", "--start-iter", "--refine-every", "--growth-grad-threshold", "--growth-select-fraction", "--growth-stop-iter", "--max-splats", - "--eval-every", "--export-every", "--max-resolution", "--refine-pose" + "--eval-every", "--eval-split-every", "--eval-save-to-disk", + "--export-every", "--export-name", "--max-resolution", + "--max-frames", "--subsample-frames", "--subsample-points", "--seed", + # quality levers (photorealism): perceptual + structural loss weights, + # opacity/scale regularisation, and the main learning rates + "--ssim-weight", "--lpips-loss-weight", "--opac-loss-weight", + "--scale-loss-weight", "--match-alpha-weight", "--mean-noise-weight", + "--lr-mean", "--lr-mean-end", "--lr-coeffs-dc", "--lr-coeffs-sh-scale", + "--lr-opac", "--lr-scale", "--lr-scale-end", "--lr-rotation", } - def __init__(self, logger_callback: Optional[Callable] = None) -> None: + def __init__(self, logger_callback: Callable | None = None) -> None: """Initialize the Brush engine. Parameters @@ -32,7 +44,8 @@ def __init__(self, logger_callback: Optional[Callable] = None) -> None: self.process = None def build_command(self, input_path: str, output_path: str, - params: Optional[Dict[str, Any]] = None) -> Tuple[List[str], Dict[str, str]]: + params: dict[str, Any] | None = None, + backend_override: str | None = None) -> tuple[list[str], dict[str, str]]: """Build the Brush command list and environment from parameters. Parameters @@ -63,11 +76,14 @@ def build_command(self, input_path: str, output_path: str, cmd.append("--with-viewer") env = os.environ.copy() device = params.get("device", self.device) - if device == "mps": - env["WGPU_BACKEND"] = "metal" - env["WGPU_POWER_PREF"] = "high_performance" - elif device == "cuda": - env["WGPU_BACKEND"] = "vulkan" + # Brush runs on wgpu; on Windows/NVIDIA the DX12 and Vulkan backends both + # target CUDA-class GPUs. We pin DX12 (most reliable on Windows) and let + # wgpu pick the high-performance (discrete) adapter. + if device in ("cuda", "auto"): + # Vulkan is the default on NVIDIA: the DX12 backend reports a compute + # workgroup limit (768) below what Brush's reduce kernels need (1024), + # which crashes it. DX12 stays as the automatic fallback (see train). + env["WGPU_BACKEND"] = backend_override or params.get("wgpu_backend") or "vulkan" env["WGPU_POWER_PREF"] = "high_performance" for param_name, flag in [ @@ -105,7 +121,7 @@ def build_command(self, input_path: str, output_path: str, cmd.append(str(input_path)) return cmd, env - def train(self, input_path: str, output_path: str, params: Optional[Dict[str, Any]] = None) -> int: + def train(self, input_path: str, output_path: str, params: dict[str, Any] | None = None) -> int: """Run the Brush training process. Parameters @@ -129,6 +145,85 @@ def train(self, input_path: str, output_path: str, params: Optional[Dict[str, An raise ValueError("Chemins invalides ou non sécurisés détectés.") if not self.brush_bin: raise RuntimeError("Exécutable 'brush' non trouvé.") - cmd, env = self.build_command(str(safe_input), str(safe_output), params) - self.log(f"Lancement Brush: {' '.join(cmd)}") - return self._execute_command(cmd, env=env) + + params = params or {} + device = params.get("device", self.device) + # Brush (wgpu + burn-fusion) can crash on a backend that reports compute + # limits below what its kernels need (DX12 caps workgroup invocations at + # 768 < the 1024 Brush wants → wgpu Validation Error → burn-fusion panic). + # Switching the wgpu backend dodges it, so we try Vulkan first on NVIDIA + # and fall back to DX12 if that attempt crashes. + if device in ("cuda", "auto"): + primary = params.get("wgpu_backend") or "vulkan" + backends = [primary, "dx12" if primary != "dx12" else "vulkan"] + else: + backends = [None] + + last_rc = -1 + for idx, backend in enumerate(backends): + if getattr(self, "stop_requested", False): + break + cmd, env = self.build_command(str(safe_input), str(safe_output), params, + backend_override=backend) + if idx > 0: + self.log(f"⚠ Brush a planté (bug burn-fusion). Nouvelle tentative avec " + f"le backend GPU « {backend} » ") + self.log(f"Lancement Brush: {' '.join(cmd)}") + panic = {"hit": False} + + def _capture(line, _p=panic): + low = line.lower() + # burn-fusion panic, or the underlying wgpu compute-limit error + # that triggers it — both mean "retry on the other backend". + if ("panicked" in low or "ordering is bigger than operations" in low + or "wgpu error" in low or "validation error" in low): + _p["hit"] = True + self.log(line) + + rc = self._execute_command(cmd, env=env, line_callback=_capture) + last_rc = rc + if rc == 0 or getattr(self, "stop_requested", False): + return rc + if not panic["hit"]: + return rc # a non-fusion failure → don't blindly retry + # fusion panic → loop to the alternate backend + return last_rc + + +def rename_latest_ply(output_path: str, ply_name: str | None, + log: Callable | None = None) -> bool: + """Rename Brush's freshly exported .ply to the user-chosen name. + + Brush always writes its own export filename; the GUI worker renames it + afterwards, but the CLI used to ignore --ply_name entirely. This gives the + CLI the same behaviour: find the most recently written .ply under + output_path and move it to . Best-effort and never fatal. + """ + if not ply_name: + return False + import shutil + from pathlib import Path + + out = Path(output_path) + name = Path(ply_name).name # strip any directory component + if not name.endswith(".ply"): + name += ".ply" + + candidates = [p for p in out.rglob("*.ply") + if p.is_file() and p.name != name] + if not candidates: + if log: + log("Attention : aucun fichier PLY trouvé à renommer.") + return False + + newest = max(candidates, key=lambda p: p.stat().st_mtime) + dest = out / name + try: + shutil.move(str(newest), str(dest)) + if log: + log(f"Fichier PLY renommé en : {name}") + return True + except OSError as e: + if log: + log(f"Erreur renommage PLY : {e}") + return False diff --git a/app/core/engine.py b/app/core/engine.py index d502b17..cc43212 100755 --- a/app/core/engine.py +++ b/app/core/engine.py @@ -1,33 +1,104 @@ -import os -import shutil -import send2trash -import platform import json -import subprocess import logging +import os +import platform +import re +import shutil import sqlite3 +from collections.abc import Callable +from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path -from typing import Tuple, Any, Optional, Callable +from typing import Any + from .base_engine import BaseEngine -from .system import is_apple_silicon, get_optimal_threads, resolve_binary from .i18n import tr +from .system import get_optimal_threads, has_cuda, resolve_binary _IMAGE_EXTS = {'.jpg', '.jpeg', '.png'} +_VIDEO_EXTS = {'.mp4', '.mov', '.avi', '.mkv'} + + +def _imread_unicode(path, flags): + """cv2.imread that tolerates non-ASCII paths on Windows (cv2 fails on them).""" + import cv2 + import numpy as np + try: + data = np.fromfile(str(path), dtype=np.uint8) + if data.size == 0: + return None + return cv2.imdecode(data, flags) + except (OSError, ValueError): + return None + + +def _imwrite_unicode(path, img) -> bool: + """cv2.imwrite that tolerates non-ASCII paths on Windows. Returns success. + + Encodes to an in-memory buffer first, then unlinks the destination before + writing a fresh file — this both breaks any hardlink (so a hardlinked + original is never modified) and avoids leaving the file deleted if encoding + fails. + """ + import cv2 + p = Path(path) + suffix = p.suffix or ".png" + params = [] + if suffix.lower() in (".jpg", ".jpeg"): + # JPEG can't store an alpha channel and defaults to lossy quality 95. + # Drop alpha (IMREAD_UNCHANGED may return 4 channels) and re-encode at + # max quality so an in-place resize doesn't silently degrade the frame + # or fail outright on a 4-channel image. + if img.ndim == 3 and img.shape[2] == 4: + img = cv2.cvtColor(img, cv2.COLOR_BGRA2BGR) + params = [cv2.IMWRITE_JPEG_QUALITY, 100] + ok, buf = cv2.imencode(suffix, img, params) + if not ok: + return False + try: + p.unlink(missing_ok=True) + buf.tofile(str(p)) + return True + except OSError: + return False def _first_available_model() -> str: try: - from app.upscayl_models import get_downloaded_models from app.upscayl_manager import get_models_dir + from app.upscayl_models import get_downloaded_models models = get_downloaded_models(get_models_dir()) return models[0].id if models else "" except Exception: return "" + +def select_blurry_files(scores: dict, factor: float, max_remove_frac: float = 0.2): + """Selects which files to discard as too blurry. + + A file is blurry if its sharpness score (variance of Laplacian) is below + ``factor × median(scores)``. To avoid gutting the dataset, never removes + more than ``max_remove_frac`` of the files (only the blurriest are kept as + candidates if the cap is exceeded). Default cap is 20% — on shaky video a + lower-than-median threshold can otherwise flag a large fraction of frames. + + Returns (rejected_files: list, threshold: float). + """ + import statistics + if not scores or factor <= 0: + return [], 0.0 + median = statistics.median(scores.values()) + threshold = median * factor + rejected = [f for f, s in scores.items() if s < threshold] + cap = int(len(scores) * max_remove_frac) + if len(rejected) > cap: + rejected = sorted(rejected, key=lambda f: scores[f])[:cap] + return rejected, threshold + + class ColmapEngine(BaseEngine): """Moteur d'exécution COLMAP indépendant de l'interface graphique""" - - def __init__(self, params: Any, input_path: str, output_path: str, input_type: str, fps: int, project_name: str = "Untitled", logger_callback: Optional[Callable] = None, progress_callback: Optional[Callable] = None, status_callback: Optional[Callable] = None, check_cancel_callback: Optional[Callable] = None): + + def __init__(self, params: Any, input_path: str, output_path: str, input_type: str, fps: int, project_name: str = "Untitled", logger_callback: Callable | None = None, progress_callback: Callable | None = None, status_callback: Callable | None = None, check_cancel_callback: Callable | None = None): """Initialise le moteur COLMAP avec les paramÚtres de configuration.""" super().__init__("COLMAP", logger_callback) self.params = params @@ -36,28 +107,30 @@ def __init__(self, params: Any, input_path: str, output_path: str, input_type: s self.input_type = input_type self.fps = fps self.project_name = project_name - self.is_silicon = is_apple_silicon() + self.has_cuda = has_cuda() self.num_threads = get_optimal_threads() self._current_process = None self.progress = progress_callback if progress_callback else lambda x: None self.status = status_callback if status_callback else lambda x: None self.check_cancel = check_cancel_callback if check_cancel_callback else lambda: False self.logger = logging.getLogger(__name__) - + # Resolve binaries self.ffmpeg_bin = resolve_binary('ffmpeg') or 'ffmpeg' self.colmap_bin = resolve_binary('colmap') or 'colmap' self.glomap_bin = resolve_binary('glomap') or 'glomap' - - # Pre-load cv2 on the main thread to avoid Bus Error (SIGBUS) + + # Pre-load cv2 on the main thread to avoid Bus Error (SIGBUS) try: import cv2 self._cv2_loaded = True except ImportError: self._cv2_loaded = False - - if self.is_silicon: - self.log(f"Apple Silicon détecté - {self.num_threads} threads optimisés") + + if self.has_cuda: + self.log(f"GPU NVIDIA CUDA détecté - {self.num_threads} threads, accélération GPU activée") + else: + self.log(f"Aucun GPU CUDA détecté - exécution CPU ({self.num_threads} threads)") self.log(f"Binaires: {self.colmap_bin}, {self.ffmpeg_bin}, {self.glomap_bin}") @property @@ -69,34 +142,34 @@ def is_cancelled(self) -> bool: """Vérifie si l'utilisateur a demandé l'annulation.""" return self.check_cancel() - def run(self) -> Tuple[bool, str]: + def run(self) -> tuple[bool, str]: """Exécute le pipeline complet de reconstruction.""" try: setup_result = self._validate_and_setup_paths() if not setup_result: return False, "Erreur de validation des chemins" project_dir, images_dir, checkpoints_dir = setup_result - + if not self._process_input(project_dir, images_dir): if self.is_cancelled(): return False, tr("USER_CANCELLED") return False, "Erreur lors de la preparation de l'entree" pipeline_result, msg = self._run_reconstruction_pipeline(project_dir, images_dir) return pipeline_result, msg - + except Exception as e: self.log(f"Erreur lors de l'exécution du pipeline: {e}") self.logger.error("Exception in pipeline", exc_info=True) if self.is_cancelled(): return False, "Arrete par l'utilisateur" return False, "Une erreur est survenue lors du traitement." - def _validate_and_setup_paths(self) -> Optional[Tuple[Path, Path, Path]]: + def _validate_and_setup_paths(self) -> tuple[Path, Path, Path] | None: """Valide les chemins d'entrée/sortie et prépare la structure des dossiers.""" safe_output = self.validate_path(str(self.output_path)) if not safe_output: self.log("Chemin de sortie non sécurisé") return None self.output_path = safe_output - + if ".." in self.project_name or "/" in self.project_name or "\\" in self.project_name: self.log("Nom de projet invalide") return None @@ -104,13 +177,13 @@ def _validate_and_setup_paths(self) -> Optional[Tuple[Path, Path, Path]]: project_dir = self.output_path / self.project_name images_dir = project_dir / "images" checkpoints_dir = project_dir / "checkpoints" - + project_dir.mkdir(parents=True, exist_ok=True) images_dir.mkdir(parents=True, exist_ok=True) checkpoints_dir.mkdir(parents=True, exist_ok=True) - + self.log(f"Préparation du projet dans : {project_dir}") - + raw_input = str(self.input_path) if "|" in raw_input: self.log("Validation de multiples chemins d'entree...") @@ -118,7 +191,7 @@ def _validate_and_setup_paths(self) -> Optional[Tuple[Path, Path, Path]]: if not self.validate_path(p.strip()): self.log(f"Chemin d'entrée non sécurisé : {p}") return None - + first_path = Path(raw_input.split("|")[0].strip()) if not first_path.exists(): self.log(f"Entrée introuvable: {first_path}") @@ -138,7 +211,13 @@ def _process_input(self, project_dir: Path, images_dir: Path) -> bool: self.status(tr("status_prep_images", "Préparation des visuels...")) if not self._prepare_images(images_dir): return False - + + # Discard blurry frames before reconstruction (best on the raw frames, + # i.e. before upscaling). Optional and never fatal. + if getattr(self.params, 'filter_blurry', False): + self.status(tr("status_blur_filter", "Filtrage des images floues...")) + self._filter_blurry_images(images_dir, getattr(self.params, 'blur_factor', 0.7)) + upscale_conf = getattr(self, 'upscale_config', None) if upscale_conf and upscale_conf.get("active", False): self.status(tr("status_upscaling", "Upscaling des images...")) @@ -147,45 +226,165 @@ def _process_input(self, project_dir: Path, images_dir: Path) -> bool: if not self._check_and_normalize_resolution(images_dir): return False - + return True - def _run_reconstruction_pipeline(self, project_dir: Path, images_dir: Path) -> Tuple[bool, str]: + # Params that change the contents of database.db (features + matches). If any + # differ from a previous run, the DB must NOT be reused. + _DB_SIG_KEYS = ( + 'camera_model', 'single_camera', 'max_image_size', 'max_num_features', + 'estimate_affine_shape', 'domain_size_pooling', + 'matcher_type', 'max_ratio', 'max_distance', 'cross_check', + 'guided_matching', 'sequential_overlap', 'loop_detection', + ) + + def _db_sig_path(self, database_path: Path) -> Path: + return Path(str(database_path) + ".sig.json") + + def _db_signature(self, images_dir: Path) -> dict: + sig = {k: getattr(self.params, k, None) for k in self._DB_SIG_KEYS} + sig['image_count'] = len(self._current_image_names(images_dir)) + return sig + + def _current_image_names(self, images_dir: Path) -> set: + """Relative posix names of the images COLMAP would ingest (matches the + filter used by _write_sorted_image_list and stored in the DB).""" + root = Path(images_dir) + if not root.exists(): + return set() + return { + f.relative_to(root).as_posix() + for f in root.rglob('*') + if f.is_file() + and f.suffix.lower() in _IMAGE_EXTS + and not f.name.lower().endswith('.mask.png') + } + + def _write_db_signature(self, database_path: Path, images_dir: Path) -> None: + try: + self._db_sig_path(database_path).write_text( + json.dumps(self._db_signature(images_dir)), encoding='utf-8' + ) + except OSError as e: + self.log(f"(info) Impossible d'écrire la signature de reprise : {e}") + + def _database_is_reusable(self, database_path: Path, images_dir: Path) -> bool: + """True only if database.db can be safely resumed from: it has features + AND geometrically-verified matches for EXACTLY the current image set, and + the extraction/matching params are unchanged since it was built. + + Conservative by design — any missing data, mismatch, or error returns + False so the pipeline rebuilds from scratch (never reuse a stale DB). + """ + if not database_path.exists(): + return False + # 1. Params + image-count signature must match the stored one. + try: + stored = json.loads(self._db_sig_path(database_path).read_text(encoding='utf-8')) + except (OSError, ValueError): + return False + if stored != self._db_signature(images_dir): + return False + current = self._current_image_names(images_dir) + if not current: + return False + # 2. The DB must actually contain features + verified matches for exactly + # those images. + try: + con = sqlite3.connect(str(database_path)) + try: + tables = {r[0] for r in con.execute( + "SELECT name FROM sqlite_master WHERE type='table'")} + if not {'images', 'descriptors'} <= tables: + return False + db_names = {r[0] for r in con.execute("SELECT name FROM images")} + if db_names != current: + return False + n_images = con.execute("SELECT COUNT(*) FROM images").fetchone()[0] + n_desc = con.execute( + "SELECT COUNT(*) FROM descriptors WHERE rows > 0").fetchone()[0] + if n_desc < n_images: + return False + # verified matches (what the mapper consumes); fall back to raw + if 'two_view_geometries' in tables: + verified = con.execute( + "SELECT COUNT(*) FROM two_view_geometries WHERE rows > 0").fetchone()[0] + elif 'matches' in tables: + verified = con.execute( + "SELECT COUNT(*) FROM matches WHERE rows > 0").fetchone()[0] + else: + return False + return verified > 0 + finally: + con.close() + except sqlite3.Error: + return False + + def _run_reconstruction_pipeline(self, project_dir: Path, images_dir: Path) -> tuple[bool, str]: """Exécute les étapes de reconstruction COLMAP.""" database_path = project_dir / "database.db" sparse_dir = project_dir / "sparse" + # The sparse model is the mapper's output — always rebuilt fresh. if sparse_dir.exists(): shutil.rmtree(sparse_dir) self.log(f"Reconstruction sparse precedente supprimee : {sparse_dir.name}") sparse_dir.mkdir(exist_ok=True) - # Always start from a fresh database to avoid SQLite schema incompatibilities - # (especially between COLMAP and GLOMAP's bundled SQLite versions). - for db_file in [database_path, - database_path.with_suffix(".db-wal"), - database_path.with_suffix(".db-shm")]: - if db_file.exists(): - db_file.unlink(missing_ok=True) - self.log(f"Base de données précédente supprimée : {db_file.name}") + # Resume: if a previous run already extracted features AND matched them for + # EXACTLY these images with the SAME params, skip extraction + matching and + # go straight to the mapper. Guarded conservatively — any mismatch redoes + # everything, so a stale DB can never silently corrupt the result. + if self._database_is_reusable(database_path, images_dir): + self.log("♻ Base COLMAP existante valide pour ces images (mêmes paramÚtres) " + "— extraction et matching ignorés, reprise directe au mapper.") + self.status("Reprise : base COLMAP réutilisée") + self.progress(75) + else: + # Start from a fresh database to avoid SQLite schema incompatibilities + # (especially between COLMAP and GLOMAP's bundled SQLite versions) and + # stale features. + for db_file in [database_path, + database_path.with_suffix(".db-wal"), + database_path.with_suffix(".db-shm"), + self._db_sig_path(database_path)]: + if db_file.exists(): + db_file.unlink(missing_ok=True) + self.log(f"Base de données précédente supprimée : {db_file.name}") + + self.progress(25) + + if self.is_cancelled(): return False, tr("USER_CANCELLED") + self.status(tr("status_feature_extraction", "Analyse des images en cours...")) + if not self.feature_extraction(str(database_path), str(images_dir)): + return False, "Échec extraction features" + # Sort for sequential AND vocab_tree: the vocab-tree pass is followed + # by a sequential top-up pass (see feature_matching), whose adjacency + # relies on image IDs following temporal/filename order. The sort + # wipes matches, so it must happen before ANY matching. + if self.params.matcher_type in ('sequential', 'vocab_tree'): + self.log("Tri de la base de données COLMAP (ordre temporel des images)...") + self.status("Préparation du matching...") + self._sort_colmap_database_images(database_path) + self.log("Base triée. Démarrage du matching.") + + self.progress(50) + + if self.is_cancelled(): return False, tr("USER_CANCELLED") + self.status(tr("status_feature_matching", "Recherche des points communs...")) + # The matcher first loads every image's descriptors into RAM before it + # prints anything — on a large dataset that can be several minutes with + # no output. Warn so the run doesn't look frozen. + self.log("⏳ Matching en cours — COLMAP charge les descripteurs en mémoire. " + "Cette premiÚre phase peut durer plusieurs minutes sans affichage, " + "c'est normal, ne fermez pas le programme.") + if not self.feature_matching(str(database_path)): + return False, "Échec matching" + + # Stamp the DB so a later run can safely resume from it. + self._write_db_signature(database_path, images_dir) - self.progress(25) - - if self.is_cancelled(): return False, tr("USER_CANCELLED") - self.status(tr("status_feature_extraction", "Analyse des images en cours...")) - if not self.feature_extraction(str(database_path), str(images_dir)): - return False, "Échec extraction features" - if self.params.matcher_type == 'sequential': - self._sort_colmap_database_images(database_path) - - self.progress(50) - - if self.is_cancelled(): return False, tr("USER_CANCELLED") - self.status(tr("status_feature_matching", "Recherche des points communs...")) - if not self.feature_matching(str(database_path)): - return False, "Échec matching" - self.progress(75) - + if self.is_cancelled(): return False, tr("USER_CANCELLED") # GLOMAP's bundled SQLite does not support WAL journal mode created by @@ -197,9 +396,15 @@ def _run_reconstruction_pipeline(self, project_dir: Path, images_dir: Path) -> T self.status(tr("status_reconstruction", "Création de la scÚne 3D...")) if not self.mapper(str(database_path), str(images_dir), str(sparse_dir)): return False, "Échec reconstruction" - + + # With multiple_models on, COLMAP writes sub-models (sparse/0, sparse/1, 
) + # in no particular size order. Brush and the undistorter use sparse/0, so + # make sure sparse/0 is the model with the MOST registered images — + # otherwise training can land on a tiny fragment (e.g. 11 of 3000 images). + self._promote_largest_sparse_model(sparse_dir) + self.progress(90) - + if self.params.undistort_images: if self.is_cancelled(): return False, tr("USER_CANCELLED") dense_dir = project_dir / "dense" @@ -207,27 +412,46 @@ def _run_reconstruction_pipeline(self, project_dir: Path, images_dir: Path) -> T self.status(tr("status_undistorting", "Correction optique des images...")) if not self.image_undistorter(str(images_dir), str(sparse_dir), str(dense_dir)): return False, "Echec undistortion" - + self._normalize_dense_sparse_layout(dense_dir) + self.progress(95) - + if not self.is_cancelled(): self.status(tr("status_ready", "Traitement terminé !")) self.create_brush_config(project_dir, images_dir, sparse_dir) self.progress(100) return True, f"Dataset cree: {project_dir}" - + return False, "Arrete par l'utilisateur" + def _import_manifest_path(self, images_dir: Path) -> Path: + return images_dir.parent / "import_manifest.json" + + def _load_import_manifest(self, images_dir: Path) -> dict: + """Maps absolute source path → imported target filename (resize-proof).""" + try: + data = json.loads(self._import_manifest_path(images_dir).read_text(encoding='utf-8')) + return data if isinstance(data, dict) else {} + except (OSError, ValueError): + return {} + + def _save_import_manifest(self, images_dir: Path, manifest: dict) -> None: + try: + self._import_manifest_path(images_dir).write_text( + json.dumps(manifest), encoding='utf-8') + except OSError: + pass + def _prepare_images(self, images_dir: Path) -> bool: """GÚre l'extraction vidéo ou la copie d'images.""" if self.input_type == "video": if self.is_cancelled(): return False - + video_paths = [] if self.input_path.is_dir(): supported_exts = {'.mp4', '.mov', '.avi', '.mkv'} video_paths = [ - f for f in self.input_path.rglob('*') + f for f in self.input_path.rglob('*') if f.is_file() and f.suffix.lower() in supported_exts ] video_paths.sort() @@ -235,23 +459,27 @@ def _prepare_images(self, images_dir: Path) -> bool: video_paths = [Path(p.strip()) for p in str(self.input_path).split("|") if p.strip()] total_videos = len(video_paths) - + if total_videos == 0: self.log(f"Aucune vidéo trouvée dans: {self.input_path}") return False - + for i, video_path in enumerate(video_paths): if self.is_cancelled(): return False - + if not video_path.exists(): self.log(f"Attention: Video introuvable: {video_path}") continue - + base_name = video_path.stem - prefix = "".join([c for c in base_name if c.isalnum() or c in ('_', '-')]) - + sanitized = "".join([c for c in base_name if c.isalnum() or c in ('_', '-')]) + # Prefix with the enumeration index so two videos whose stems + # sanitize to the same string (or to an empty string) get distinct, + # non-empty prefixes and never overwrite each other's frames. + prefix = f"{i:03d}_{sanitized}" if sanitized else f"{i:03d}" + self.log(f"Extraction video ({i+1}/{total_videos}): {base_name}") - + if not self.extract_frames_from_video(str(video_path), images_dir, prefix=prefix): self.log(f"Echec extraction video: {base_name}") return False @@ -261,7 +489,7 @@ def _prepare_images(self, images_dir: Path) -> bool: try: raw_input = str(self.input_path) src_files = [] - + if "|" in raw_input: paths = [Path(p.strip()) for p in raw_input.split("|") if p.strip()] for p in paths: @@ -280,37 +508,130 @@ def _prepare_images(self, images_dir: Path) -> bool: and f.suffix.lower() in _IMAGE_EXTS and not f.name.lower().endswith('.mask.png') ] - + total_files = len(src_files) self.log(f"{total_files} images trouvées.") - + if total_files == 0: - return True - + # Allow re-runs where the working folder is already populated. + already_present = images_dir.exists() and any( + f.is_file() and f.suffix.lower() in _IMAGE_EXTS + for f in images_dir.iterdir() + ) + if already_present: + self.log("Aucune nouvelle image à copier — images déjà présentes, on continue.") + return True + self.log( + f"❌ Aucune image trouvée dans l'entrée : {self.input_path}\n" + f" Sélectionnez un dossier contenant des images (.jpg/.jpeg/.png), " + f"ou choisissez le mode Vidéo si l'entrée est une vidéo." + ) + return False + + # Map each source to a deterministic target by its absolute path + # (a manifest), so a source that was already imported — even after + # it was resized in place — is recognised and never re-imported + # under a "_1_" alias. Source content is NOT compared + # (the in-place resize changes it); provenance is the source path. + manifest = self._load_import_manifest(images_dir) + src_files = sorted(src_files, key=lambda p: (p.name.lower(), str(p))) + new_manifest = {} + used = set() # target names claimed this run + linked = skipped = 0 + logged_mode = False for i, file_path in enumerate(src_files): if self.is_cancelled(): return False - target_path = images_dir / file_path.name - if target_path.exists(): + key = str(file_path.resolve()) + # Prefer the name a prior run gave this source; else its + # basename. Disambiguate only against names claimed THIS run + # (two genuinely different sources sharing a basename). + target_name = manifest.get(key) or file_path.name + if target_name in used: counter = 1 while True: - target_path = images_dir / f"{file_path.parent.name}_{counter}_{file_path.name}" - if not target_path.exists(): + cand = f"{file_path.parent.name}_{counter}_{file_path.name}" + if cand not in used: + target_name = cand break counter += 1 - - shutil.copy2(file_path, target_path) - + target = images_dir / target_name + if target.exists(): + skipped += 1 # already imported (possibly resized) → adopt + else: + mode = self._link_or_copy(file_path, target) + linked += 1 + if not logged_mode: + self.log( + "Images liées (hardlink — pas de duplication sur le disque)." + if mode == "link" else + "Images copiées (entrée sur un autre volume — duplication inévitable)." + ) + logged_mode = True + new_manifest[key] = target_name + used.add(target_name) + if i % 10 == 0 or i == total_files - 1: p = 5 + int((i / total_files) * 15) self.progress(p) - self.status(f"Copie des images : {i+1} / {total_files}") - - self.log(f"✅ {total_files} images copiées vers {images_dir}") + self.status(f"Préparation des images : {i+1} / {total_files}") + + # Self-heal: drop stale top-level images no current source claims + # (e.g. "_1_" duplicates from earlier buggy runs, or + # frames of a source that was removed). Subfolders (images_src, + # blurry, 
) and mask files are left untouched. + removed = 0 + claimed = set(new_manifest.values()) + for f in images_dir.iterdir(): + if (f.is_file() and f.suffix.lower() in _IMAGE_EXTS + and not f.name.lower().endswith('.mask.png') + and f.name not in claimed): + try: + f.unlink() + removed += 1 + except OSError: + pass + + self._save_import_manifest(images_dir, new_manifest) + + msg = f"✅ {linked} images préparées" + if skipped: + msg += f", {skipped} déjà présentes" + if removed: + msg += f", {removed} doublons/obsolÚtes supprimés" + self.log(msg + ".") return True except Exception as e: self.log(f"Erreur copie images: {e}") return False + def _link_or_copy(self, src: Path, dst: Path) -> str: + """Hardlink src→dst to avoid duplicating the dataset on disk; fall back to + a copy when a hardlink can't be made (e.g. source on another volume). + + Hardlinks share the same data on disk, so the project's images/ folder + costs no extra space. In-place writers (resolution resize) break the + link first, so the user's originals are never modified. + """ + try: + os.link(str(src), str(dst)) + return "link" + except (OSError, NotImplementedError): + shutil.copy2(str(src), str(dst)) + return "copy" + + def _is_single_video_input(self) -> bool: + """True when the input is exactly one video file (not a dir, not a + '|'-joined list). Frames from one ffmpeg run share one resolution, so + callers can take resolution/format fast paths. Conservative: a folder of + videos returns False.""" + raw = str(self.input_path) + if "|" in raw: + return False + try: + return self.input_path.is_file() and self.input_path.suffix.lower() in _VIDEO_EXTS + except OSError: + return False + def _convert_db_journal_mode(self, database_path: Path): """Switch the COLMAP database from WAL to DELETE journal mode. @@ -319,9 +640,16 @@ def _convert_db_journal_mode(self, database_path: Path): rollback-journal mode before GLOMAP reads the file. """ try: - with sqlite3.connect(str(database_path)) as con: - con.execute("PRAGMA journal_mode=DELETE") + con = sqlite3.connect(str(database_path)) + try: con.execute("PRAGMA wal_checkpoint(TRUNCATE)") + con.execute("PRAGMA journal_mode=DELETE") + con.commit() + finally: + # sqlite3's context manager commits but does NOT close — leaving the + # connection open keeps the -wal/-shm files locked on Windows, so the + # unlink below would fail. Close explicitly before deleting them. + con.close() for wal_file in [database_path.parent / (database_path.name + "-wal"), database_path.parent / (database_path.name + "-shm")]: if wal_file.exists(): @@ -330,6 +658,84 @@ def _convert_db_journal_mode(self, database_path: Path): except Exception as e: self.log(f"Avertissement : conversion journal mode échouée : {e}") + def _filter_blurry_images(self, images_dir: Path, factor: float) -> None: + """Move blurry frames out of images_dir (sharpness = variance of Laplacian). + + Rejected frames are moved to a sibling ``images_blurry`` folder rather + than deleted, so they can be inspected/restored. Never fatal. + """ + self.log(f"\n{'='*60}\nFiltrage des images floues\n{'='*60}") + try: + import cv2 + except ImportError: + self.log("⚠ OpenCV non disponible — filtrage du flou ignoré " + "(installez opencv-python-headless pour l'activer).") + return + + files = sorted( + f for f in images_dir.iterdir() + if f.is_file() and f.suffix.lower() in _IMAGE_EXTS + and not f.name.lower().endswith('.mask.png') + ) + if len(files) < 10: + self.log(f"Trop peu d'images ({len(files)}) — filtrage du flou ignoré.") + return + + total = len(files) + self.log(f"Analyse de la netteté de {total} images sur {self.num_threads} threads...") + + def _score_one(f): + # cv2 releases the GIL during imread/resize/Laplacian, so threading + # scales well. Downscaling to ≀640px preserves the relative ranking. + img = _imread_unicode(f, cv2.IMREAD_GRAYSCALE) + if img is None: + return f, None + h, w = img.shape[:2] + longest = max(h, w) + if longest > 640: + s = 640.0 / longest + img = cv2.resize(img, (max(1, int(w * s)), max(1, int(h * s))), + interpolation=cv2.INTER_AREA) + return f, float(cv2.Laplacian(img, cv2.CV_64F).var()) + + scores = {} + done = 0 + with ThreadPoolExecutor(max_workers=max(2, self.num_threads)) as ex: + futures = [ex.submit(_score_one, f) for f in files] + for fut in as_completed(futures): + if self.is_cancelled(): + ex.shutdown(cancel_futures=True) + return + f, sc = fut.result() + if sc is not None: + scores[f] = sc + done += 1 + if done % 200 == 0 or done == total: + self.status(f"Analyse netteté : {done}/{total}") + self.progress(int(done / total * 100)) + if done % 1000 == 0 or done == total: + self.log(f" netteté analysée : {done}/{total}") + + rejected, threshold = select_blurry_files(scores, factor) + if not rejected: + self.log(f"Aucune image floue détectée (seuil de netteté ≈ {threshold:.0f}).") + return + + rejected_dir = images_dir.parent / "images_blurry" + rejected_dir.mkdir(parents=True, exist_ok=True) + moved = 0 + for f in rejected: + try: + shutil.move(str(f), str(rejected_dir / f.name)) + moved += 1 + except OSError as e: + self.log(f"⚠ Impossible de déplacer {f.name}: {e}") + self.log( + f"🔪 Filtre flou : {moved}/{len(files)} images écartées vers " + f"'images_blurry' (seuil ≈ {threshold:.0f}). Pour les réutiliser, " + f"remettez-les dans 'images' (ou désactivez le filtre)." + ) + def _run_upscale(self, project_dir: Path, images_dir: Path) -> bool: """GÚre l'upscaling via upscayl-bin.""" self.log(f"\n{'='*60}\nUpscaling (upscayl-ncnn)\n{'='*60}") @@ -340,57 +746,92 @@ def _run_upscale(self, project_dir: Path, images_dir: Path) -> bool: upscaler = UpscaleEngine(logger_callback=self.log) if not upscaler.is_installed(): - self.log("WARNING: upscayl-bin not found. Upscale skipped.") + self.log("⚠ upscayl-bin introuvable — upscale ignoré (les images originales sont conservées).") return True - images_sources_dir = project_dir / "images_src" - - if not images_sources_dir.exists(): - self.log(f"Moving originals to {images_sources_dir}...") - shutil.move(str(images_dir), str(images_sources_dir)) - images_dir.mkdir(parents=True, exist_ok=True) - - model_id = self.upscale_config.get("model_id") or _first_available_model() - scale = self.upscale_config.get("scale", 4) - out_format = self.upscale_config.get("format", "png") - tile = self.upscale_config.get("tile", 0) - tta = self.upscale_config.get("tta", False) - compression = self.upscale_config.get("compression", 0) - - self.log(f"Upscaling x{scale} with model '{model_id}'...") - success, msg = upscaler.upscale_folder( - input_dir=str(images_sources_dir), - output_dir=str(images_dir), - model_id=model_id, - scale=scale, - output_format=out_format, - tile=tile, - tta=tta, - compression=compression, - cancel_check=self.is_cancelled, + # Verify a model is actually available BEFORE touching the images. + # Upscale is an optional enhancement; a missing model must never + # abort dataset creation. + from app.upscayl_manager import get_models_dir + from app.upscayl_models import get_downloaded_models + downloaded = {m.id for m in get_downloaded_models(get_models_dir())} + model_id = self.upscale_config.get("model_id") + if model_id not in downloaded: + model_id = _first_available_model() # first downloaded model, or "" + if not model_id: + self.log( + "⚠ Aucun modÚle upscayl téléchargé — upscale ignoré. " + "Ouvrez l'onglet Upscale pour télécharger un modÚle " + "(l'upscale est optionnel et non requis pour le splatting)." ) - if not success: - self.log(f"Upscale failed: {msg}") - return False - self.log("Upscale complete.") - else: + return True + + images_sources_dir = project_dir / "images_src" + if images_sources_dir.exists(): self.log("'images_src' already exists — upscale already done.") + return True + + self.log(f"Moving originals to {images_sources_dir}...") + shutil.move(str(images_dir), str(images_sources_dir)) + images_dir.mkdir(parents=True, exist_ok=True) + + scale = self.upscale_config.get("scale", 4) + out_format = self.upscale_config.get("format", "png") + tile = self.upscale_config.get("tile", 0) + tta = self.upscale_config.get("tta", False) + compression = self.upscale_config.get("compression", 0) + self.log(f"Upscaling x{scale} with model '{model_id}'...") + success, msg = upscaler.upscale_folder( + input_dir=str(images_sources_dir), + output_dir=str(images_dir), + model_id=model_id, + scale=scale, + output_format=out_format, + tile=tile, + tta=tta, + compression=compression, + cancel_check=self.is_cancelled, + ) + if not success: + self.log(f"⚠ Upscale échoué ({msg}) — restauration des images originales.") + self._restore_originals(images_sources_dir, images_dir) + return True + self.log("Upscale complete.") return True - + except Exception as e: - self.log(f"Erreur Upscale: {e}") - return False + self.log(f"⚠ Erreur Upscale ({e}) — tentative de restauration des images originales.") + try: + src = project_dir / "images_src" + if src.exists() and not any(images_dir.iterdir()): + self._restore_originals(src, images_dir) + except OSError: + pass + return True # optional step: never abort the pipeline + + def _restore_originals(self, src_dir: Path, images_dir: Path) -> None: + """Move original images back from src_dir to images_dir after a failed upscale.""" + images_dir.mkdir(parents=True, exist_ok=True) + for f in src_dir.iterdir(): + if f.is_file(): + dest = images_dir / f.name + if not dest.exists(): + shutil.move(str(f), str(dest)) + shutil.rmtree(str(src_dir), ignore_errors=True) + self.log("Images originales restaurées — la reconstruction continue sans upscale.") def _check_and_normalize_resolution(self, images_dir: Path) -> bool: - """Vérifie et normalise la résolution des images.""" + """Vérifie et normalise la résolution des images. + + Reads only image dimensions from the file header (fast, no full decode) + in parallel. Frames from a single video are all the same size, so this + exits almost instantly. + """ self.log(f"\n{'='*60}\nVérification résolution images\n{'='*60}") - if not getattr(self, '_cv2_loaded', False): - self.log("⚠ OpenCV non disponible — vérification résolution ignorée.") + if not images_dir.exists(): return True - - import cv2 files = sorted([ f for f in images_dir.iterdir() @@ -400,18 +841,46 @@ def _check_and_normalize_resolution(self, images_dir: Path) -> bool: if len(files) < 2: return True - self.log(f"Analyse de {len(files)} images...") + from PIL import Image + + # Fast path: all frames from a single ffmpeg extraction share one + # resolution by construction. Sampling the first frame turns an O(n) + # header scan into O(1). Multi-video / image-folder inputs (which may mix + # resolutions) keep the full scan below. + if self._is_single_video_input(): + try: + with Image.open(files[0]) as im: + w, h = im.size + self.log(f"✅ Source vidéo unique — résolution uniforme {w}×{h} px (échantillon, scan complet ignoré)") + return True + except Exception: + pass # fall through to the full scan on any read error + + total = len(files) + self.log(f"Lecture des dimensions de {total} images (en-tête seulement, {self.num_threads} threads)...") + + def _size_one(f): + try: + with Image.open(f) as im: + return f, im.size # (w, h) read from the header, no decode + except Exception: + return f, None sizes = {} - for f in files: - if self.is_cancelled(): - return False - img = cv2.imread(str(f), cv2.IMREAD_UNCHANGED) - if img is None: - self.log(f"⚠ Lecture impossible: {f.name}") - continue - h, w = img.shape[:2] - sizes[f] = (w, h) + done = 0 + with ThreadPoolExecutor(max_workers=max(2, self.num_threads)) as ex: + futures = [ex.submit(_size_one, f) for f in files] + for fut in as_completed(futures): + if self.is_cancelled(): + ex.shutdown(cancel_futures=True) + return False + f, sz = fut.result() + if sz is not None: + sizes[f] = sz + done += 1 + if done % 1000 == 0 or done == total: + self.status(f"Dimensions : {done}/{total}") + self.progress(int(done / total * 100)) if not sizes: return True @@ -422,49 +891,62 @@ def _check_and_normalize_resolution(self, images_dir: Path) -> bool: self.log(f"✅ Résolution uniforme: {w}×{h} px") return True - min_w = min(s[0] for s in unique_sizes) - min_h = min(s[1] for s in unique_sizes) - to_resize = [f for f, s in sizes.items() if s != (min_w, min_h)] + if not getattr(self, '_cv2_loaded', False): + self.log("⚠ Résolutions différentes mais OpenCV indisponible — redimensionnement ignoré.") + return True + import cv2 + + # Unify to the LARGEST resolution: the sharpest cameras keep their full + # detail. Downscaling everything to the smallest (previous behaviour) + # permanently threw away detail from the best images, hurting photorealism. + target_w = max(s[0] for s in unique_sizes) + target_h = max(s[1] for s in unique_sizes) + to_resize = [f for f, s in sizes.items() if s != (target_w, target_h)] self.log(f"⚠ {len(unique_sizes)} résolutions différentes détectées.") - self.log(f"Redimensionnement de {len(to_resize)} images → {min_w}×{min_h} px") + self.log(f"Redimensionnement de {len(to_resize)} images → {target_w}×{target_h} px") - for i, f in enumerate(to_resize): - if self.is_cancelled(): - return False - img = cv2.imread(str(f), cv2.IMREAD_UNCHANGED) + def _resize_one(f): + img = _imread_unicode(f, cv2.IMREAD_UNCHANGED) if img is None: - self.log(f"⚠ Re-lecture impossible: {f.name}") - continue - resized = cv2.resize(img, (min_w, min_h), interpolation=cv2.INTER_AREA) - cv2.imwrite(str(f), resized) - del img, resized - if (i + 1) % 10 == 0 or (i + 1) == len(to_resize): - self.log(f"Redimensionnement: {i+1}/{len(to_resize)}") - self.status(f"Ajustement taille : {i+1} / {len(to_resize)}") - - self.log(f"✅ {len(to_resize)} images redimensionnées vers {min_w}×{min_h} px") + return f, None + h, w = img.shape[:2] + # INTER_AREA is best when shrinking, INTER_CUBIC when enlarging. + shrinking = w >= target_w and h >= target_h + interp = cv2.INTER_AREA if shrinking else cv2.INTER_CUBIC + resized = cv2.resize(img, (target_w, target_h), interpolation=interp) + return f, _imwrite_unicode(f, resized) + + # cv2 decode/resize/encode release the GIL → resize in parallel. + n = len(to_resize) + done = 0 + with ThreadPoolExecutor(max_workers=max(2, self.num_threads)) as ex: + futures = [ex.submit(_resize_one, f) for f in to_resize] + for fut in as_completed(futures): + if self.is_cancelled(): + ex.shutdown(cancel_futures=True) + return False + f, ok = fut.result() + if ok is None: + self.log(f"⚠ Re-lecture impossible: {f.name}") + elif not ok: + self.log(f"⚠ Écriture impossible: {f.name}") + done += 1 + if done % 10 == 0 or done == n: + self.status(f"Ajustement taille : {done} / {n}") + + self.log(f"✅ {n} images redimensionnées vers {target_w}×{target_h} px") return True - def extract_frames_from_video(self, video_path: str, images_dir: Path, prefix: Optional[str] = None) -> Optional[bool]: + def extract_frames_from_video(self, video_path: str, images_dir: Path, prefix: str | None = None) -> bool | None: """Extrait les frames d'une vidéo via FFmpeg.""" base_name = Path(video_path).stem self.log(f"\n{'='*60}\nExtraction frames: {Path(video_path).name}\n{'='*60}") images_dir.mkdir(parents=True, exist_ok=True) - - output_pattern = images_dir / (f'{prefix}_%04d.jpg' if prefix else 'frame_%04d.jpg') - - cmd = [self.ffmpeg_bin] - if self.is_silicon: - cmd.extend(['-hwaccel', 'videotoolbox']) - - cmd.extend([ - '-i', video_path, - '-vf', f'fps={self.fps}', - '-qscale:v', '2', - str(output_pattern) - ]) - + + frame_stem = f'{prefix}_' if prefix else 'frame_' + output_pattern = images_dir / f'{frame_stem}%04d.jpg' + def _ffmpeg_parser(line_str: str): if 'frame=' in line_str or 'error' in line_str.lower(): self.log(line_str) @@ -474,34 +956,189 @@ def _ffmpeg_parser(line_str: str): self.status(f"Extraction {base_name} : image {f_num}") except (IndexError, ValueError) as e: self.logger.debug("Failed to parse frame number: %s", e) - - try: - returncode = self._execute_command(cmd, line_callback=_ffmpeg_parser) - if self.is_cancelled(): return None - - if returncode == 0: - num_frames = len([f for f in images_dir.iterdir() if f.suffix == '.jpg']) + + def _build_cmd(decoder): + c = [self.ffmpeg_bin] + if self.has_cuda: + c.extend(['-hwaccel', 'cuda']) + if decoder: # NVDEC hardware decoder (cuvid) + c.extend(['-c:v', decoder]) + c.extend([ + '-i', video_path, + '-an', # no audio → faster, cleaner + '-vf', f'fps={self.fps}', + '-qscale:v', '1', # best JPEG quality (1=highest, 2≈95%) + str(output_pattern), + ]) + return c + + def _clear_frames(): + # Remove this prefix's frames before each attempt so a re-run (changed + # fps/source) or a failed GPU attempt never leaves stale frames behind. + for old in images_dir.glob(f'{frame_stem}*.jpg'): + try: + old.unlink() + except OSError: + pass + + def _count_frames(): + return len([ + f for f in images_dir.iterdir() + if f.suffix == '.jpg' and f.name.startswith(frame_stem) + ]) + + # Try NVDEC GPU decode first when a matching cuvid decoder is available, + # then fall back to standard decode. The CPU attempt is always present, so + # a missing/incompatible GPU decoder can never break extraction. + decoder = self._cuvid_decoder_for(video_path) if self.has_cuda else None + attempts = ([decoder] if decoder else []) + [None] + + for idx, dec in enumerate(attempts): + if self.is_cancelled(): + return None + if dec: + self.log(f"Décodage vidéo : NVDEC GPU ({dec})") + elif idx > 0: + self.log("⚠ Décodage GPU indisponible/échoué — bascule en décodage CPU.") + _clear_frames() + try: + returncode = self._execute_command(_build_cmd(dec), line_callback=_ffmpeg_parser) + except Exception as e: + self.log(f"Erreur: {str(e)}") + returncode = -1 + if self.is_cancelled(): + return None + num_frames = _count_frames() + if returncode == 0 and num_frames > 0: self.log(f"{num_frames} frames extraites") return True - else: - self.log(f"Erreur lors de l'extraction") - return None - except Exception as e: - self.log(f"Erreur: {str(e)}") - return False + # otherwise fall through to the next attempt (CPU) + + self.log("⚠ Aucune frame extraite de cette vidéo.") + return None - def run_command(self, cmd: list, description: str, status_prefix: Optional[str] = None) -> bool: - """Exécute une commande systÚme avec logging et callback de statut.""" + def _cuvid_decoder_for(self, video_path: str) -> str | None: + """Returns the NVDEC (cuvid) decoder name for the input video's codec, or + None if it can't be determined or isn't available in this ffmpeg build.""" + codec = self._probe_video_codec(video_path) + if not codec: + return None + cuvid_map = { + 'h264': 'h264_cuvid', 'hevc': 'hevc_cuvid', 'h265': 'hevc_cuvid', + 'mpeg1video': 'mpeg1_cuvid', 'mpeg2video': 'mpeg2_cuvid', + 'mpeg4': 'mpeg4_cuvid', 'vc1': 'vc1_cuvid', 'vp8': 'vp8_cuvid', + 'vp9': 'vp9_cuvid', 'av1': 'av1_cuvid', 'mjpeg': 'mjpeg_cuvid', + } + decoder = cuvid_map.get(codec.lower()) + if decoder and decoder in self._available_cuvid_decoders(): + return decoder + return None + + def _probe_video_codec(self, video_path: str) -> str | None: + """Reads the first video stream's codec name via ffprobe (best-effort).""" + ffprobe = self._ffprobe_bin() + if not ffprobe: + return None + try: + import subprocess + out = subprocess.run( + [ffprobe, '-v', 'error', '-select_streams', 'v:0', + '-show_entries', 'stream=codec_name', + '-of', 'default=nokey=1:noprint_wrappers=1', str(video_path)], + capture_output=True, text=True, timeout=15, + ) + name = out.stdout.strip().splitlines() + return name[0].strip() if name else None + except (OSError, subprocess.SubprocessError): + return None + + def _ffprobe_bin(self) -> str | None: + """ffprobe path, derived from ffmpeg's (they ship together).""" + if not getattr(self, 'ffmpeg_bin', None): + return None + p = Path(self.ffmpeg_bin) + cand = p.with_name(p.name.replace('ffmpeg', 'ffprobe')) + if cand.exists(): + return str(cand) + which = shutil.which('ffprobe') + return which + + def _available_cuvid_decoders(self) -> set: + """Set of cuvid decoders this ffmpeg build exposes (probed once, cached).""" + cached = getattr(self, '_cuvid_decoders', None) + if cached is not None: + return cached + decoders = set() + try: + import subprocess + out = subprocess.run( + [self.ffmpeg_bin, '-hide_banner', '-decoders'], + capture_output=True, text=True, timeout=15, + ) + for line in (out.stdout + out.stderr).splitlines(): + for tok in line.split(): + if tok.endswith('_cuvid'): + decoders.add(tok) + except (OSError, subprocess.SubprocessError): + pass + self._cuvid_decoders = decoders + return decoders + + def run_command(self, cmd: list, description: str, status_prefix: str | None = None, + omp_threads: int = 1) -> bool: + """Exécute une commande systÚme avec logging et callback de statut. + + omp_threads: value for OMP_NUM_THREADS. Defaults to 1 (see below) — + commands whose ONLY parallelism source is OpenMP (e.g. global_mapper / + GLOMAP, which has no --num_threads flag and derives its thread count + from OMP_NUM_THREADS) must pass omp_threads=self.num_threads, otherwise + they run essentially single-threaded. + """ self.log(f"\n{'='*60}\n{description}\n{'='*60}") - + env = os.environ.copy() - if self.is_silicon: - env['OMP_NUM_THREADS'] = str(self.num_threads) - env['VECLIB_MAXIMUM_THREADS'] = str(self.num_threads) - env['OPENBLAS_NUM_THREADS'] = str(self.num_threads) - + # Pin the inner BLAS/OpenMP pools to 1 thread. COLMAP already parallelizes + # at the task level via --*.num_threads (Ceres bundle-adjustment threads); + # letting BLAS *also* spawn N threads gives N×N oversubscription, which + # thrashes the CPU and worsens the "Linear solver failure" retries during + # global BA on large scenes. One source of parallelism, not nested. + env['OMP_NUM_THREADS'] = str(max(1, omp_threads)) + env['OPENBLAS_NUM_THREADS'] = '1' + env['MKL_NUM_THREADS'] = '1' + + # Windows: the bundled colmap.exe loads DLLs from its own folder and a + # sibling lib/ directory. Make both discoverable on PATH so we can call + # colmap.exe directly (instead of COLMAP.bat, which needs a shell). + if os.name == 'nt' and self.colmap_bin: + bin_dir = Path(self.colmap_bin).parent + dll_dirs = [bin_dir, bin_dir.parent / 'lib', bin_dir / 'lib', bin_dir.parent / 'bin'] + existing = [str(d) for d in dll_dirs if d.exists()] + if existing: + env['PATH'] = os.pathsep.join(existing) + os.pathsep + env.get('PATH', '') + def _colmap_parser(line_str: str): self.log(line_str) + # "Linear solver failure" is a non-fatal bundle-adjustment warning: + # one optimisation step failed, the solver retries and the mapper + # keeps registering images. Annotate it once so it doesn't look like + # a crash to the user. + if "Linear solver failure" in line_str and not getattr(self, "_ba_warn_noted", False): + self._ba_warn_noted = True + self.log("ℹ (info) « Linear solver failure » est un avertissement NON bloquant : " + "une étape d'optimisation a échoué, COLMAP réessaie automatiquement et " + "continue. Tant que les images continuent de s'enregistrer, tout va bien.") + # Some COLMAP builds advertise --Mapper.ba_use_gpu but ship a Ceres + # compiled without CUDA/cuDSS, so BA silently runs on the CPU. Surface + # this once — it's the real reason the mapper is slow. + if (("compiled without CUDA support" in line_str + or "compiled without cuDSS support" in line_str) + and not getattr(self, "_ceres_cpu_noted", False)): + self._ceres_cpu_noted = True + self.log("⚠ Important : ce build de COLMAP a Ceres SANS CUDA/cuDSS → le bundle " + "adjustment tourne en réalité sur le CPU (le flag GPU est sans effet ici). " + "C'est la principale cause de lenteur du mapper. Pour accélérer : utilisez " + "GLOMAP (qui évite le BA global répété) ou un COLMAP compilé avec " + "Ceres+CUDA+cuDSS.") if status_prefix: if "Processed file" in line_str: parts = line_str.split("Processed file") @@ -512,35 +1149,61 @@ def _colmap_parser(line_str: str): if len(parts) > 1: self.status(f"{status_prefix} : bloc {parts[1].strip()}") elif "Registering image" in line_str: - parts = line_str.split("Registering image") - if len(parts) > 1: - img_info = parts[1].split('(')[0].strip() - self.status(f"{status_prefix} : ajout image {img_info}") + # Show num_reg_frames (images successfully placed so far) — + # the real progress counter for the mapper — instead of the + # raw image id, so the user can see where they are. + m = re.search(r'num_reg_frames=(\d+)', line_str) + if m: + self.status(f"{status_prefix} : {m.group(1)} images placées") + else: + parts = line_str.split("Registering image") + if len(parts) > 1: + img_info = parts[1].split('(')[0].strip() + self.status(f"{status_prefix} : ajout image {img_info}") elif "Bundle adjustment report" in line_str: self.status(f"{status_prefix} : optimisation globale...") elif "Undistorting image" in line_str: parts = line_str.split("Undistorting image") if len(parts) > 1: self.status(f"{status_prefix} : image {parts[1].strip()}") - + try: returncode = self._execute_command(cmd, env=env, line_callback=_colmap_parser) if self.is_cancelled(): return False - + if returncode == 0: self.log(f"{description} termine") return True else: self.log(f"{description} echoue") return False - + except FileNotFoundError: - self.log(f"COLMAP non trouve. Installez avec: brew install colmap") + self.log("COLMAP introuvable. Installez une build CUDA depuis " + "https://github.com/colmap/colmap/releases (ex. colmap-x64-windows-cuda.zip) " + "et ajoutez-le au PATH.") return False def feature_extraction(self, database_path: str, images_dir: str) -> bool: """Exécute l'extraction des features SIFT.""" image_list_path = self._write_sorted_image_list(images_dir) + + # COLMAP's GPU SIFT is disabled when affine-shape or domain-size-pooling + # is on (it falls back to slow CPU extraction). Tell the user which path + # will run so a misconfiguration that loses the GPU is obvious. + cpu_only_opts = self.params.estimate_affine_shape or self.params.domain_size_pooling + if self.has_cuda and not cpu_only_opts: + self.log("SIFT : GPU (CUDA) ✅") + elif self.has_cuda and cpu_only_opts: + self.log("⚠ SIFT sur CPU : 'Affine Shape' ou 'Domain Pooling' est activé " + "→ désactivez-les pour utiliser le GPU.") + else: + self.log("SIFT : CPU (pas de GPU CUDA détecté)") + + if self.params.camera_model == 'EQUIRECTANGULAR': + self.log("Mode 360 natif : modÚle de caméra EQUIRECTANGULAR " + "(requiert COLMAP ≥ 4.1.0).") + cmd = [ self.colmap_bin, 'feature_extractor', '--database_path', database_path, @@ -553,11 +1216,14 @@ def feature_extraction(self, database_path: str, images_dir: str) -> bool: '--SiftExtraction.estimate_affine_shape', '1' if self.params.estimate_affine_shape else '0', '--SiftExtraction.domain_size_pooling', '1' if self.params.domain_size_pooling else '0', ] + # NOTE: a CUDA-enabled COLMAP uses the GPU for SIFT by default. We don't + # pass --SiftExtraction.use_gpu because some COLMAP builds (e.g. 4.x) + # reject that option name; the GPU is still used automatically. if image_list_path: cmd.extend(['--image_list_path', str(image_list_path)]) return self.run_command(cmd, "Extraction des features", status_prefix="Analyse") - def _write_sorted_image_list(self, images_dir: str) -> Optional[Path]: + def _write_sorted_image_list(self, images_dir: str) -> Path | None: """Write a deterministic COLMAP image list so sequential matching follows frame order.""" image_root = Path(images_dir) files = sorted( @@ -589,11 +1255,17 @@ def _sort_colmap_database_images(self, database_path: Path) -> None: ("pose_priors", "corr_data_id"), } + def _natural_key(name: str): + # frame_999.jpg must sort before frame_1000.jpg: split digit runs and + # compare them numerically. Plain SQL ORDER BY (lexicographic) breaks + # temporal adjacency past 9999 frames or with unpadded numbers. + return [int(tok) if tok.isdigit() else tok + for tok in re.split(r'(\d+)', name)] + try: with sqlite3.connect(str(database_path)) as con: - rows = con.execute( - "SELECT image_id, name FROM images ORDER BY name" - ).fetchall() + rows = con.execute("SELECT image_id, name FROM images").fetchall() + rows.sort(key=lambda r: _natural_key(r[1])) id_map = {old_id: new_id for new_id, (old_id, _) in enumerate(rows, start=1)} if all(old_id == new_id for old_id, new_id in id_map.items()): self.log("Ordre des images COLMAP deja trie.") @@ -664,61 +1336,404 @@ def _sort_colmap_database_images(self, database_path: Path) -> None: except Exception as e: self.log(f"Avertissement: tri de la base COLMAP echoue: {e}") + def _colmap_cmd_help(self, cmd_name: str) -> str: + """`colmap -h` output, cached per command (empty on failure). + Used to gate optional flags so we never pass an option a build rejects.""" + cache = getattr(self, "_cmd_help_cache", None) + if cache is None: + cache = {} + self._cmd_help_cache = cache + if cmd_name not in cache: + text = "" + try: + import subprocess + out = subprocess.run([self.colmap_bin, cmd_name, '-h'], + capture_output=True, text=True, timeout=15) + text = out.stdout + out.stderr + except (OSError, subprocess.SubprocessError): + text = "" + cache[cmd_name] = text + return cache[cmd_name] + + def _ensure_vocab_tree(self) -> Path | None: + """Path to a COLMAP vocabulary tree (for loop-closure / vocab-tree + matching), downloading it once into engines/ if needed. Returns None on + failure so matching can proceed without it. + + COLMAP switched its vocabulary-tree index from FLANN to FAISS in May + 2025. The classic demuc.de / 3.11.1 `vocab_tree_flickr100K_words32K.bin` + files are FLANN-format and make COLMAP 4.1.0 abort with + "Failed to read faiss index". This fork pins COLMAP >= 4.1.0, so we + fetch a FAISS-converted tree instead. If the download fails we return + None and the caller silently proceeds without loop detection.""" + dest = self.project_root / "engines" / "vocab_tree_flickr100K_words32K_faiss.bin" + if dest.exists() and dest.stat().st_size > 1_000_000: + return dest + # FAISS-format flickr100K/32K tree (converted from the official FLANN + # release); the legacy demuc.de .bin is FLANN and crashes COLMAP 4.1.0. + url = ("https://raw.githubusercontent.com/ZachMckennedyFWig/" + "ColmapFaissVocabTrees/main/vocab_tree_flickr100K_words32K.bin") + try: + import shutil + import urllib.request + dest.parent.mkdir(parents=True, exist_ok=True) + self.log("Téléchargement du vocabulaire COLMAP FAISS (détection de " + "boucles, ~10 Mo, une seule fois)...") + req = urllib.request.Request(url, headers={"User-Agent": "CorbeauSplat"}) + tmp = dest.with_suffix(".part") + with urllib.request.urlopen(req, timeout=180) as resp, open(tmp, "wb") as f: + shutil.copyfileobj(resp, f) + if tmp.stat().st_size < 1_000_000: + tmp.unlink(missing_ok=True) + raise OSError("fichier de vocabulaire trop petit (téléchargement incomplet)") + tmp.replace(dest) + return dest + except OSError as e: # URLError/HTTPError/timeouts all derive from OSError + self.log(f"(info) Vocabulaire COLMAP indisponible ({e}) — matching sans " + "détection de boucles.") + return None + + def _matching_common_opts(self) -> list: + """SIFT-matching flags shared by every matcher type.""" + return [ + '--FeatureMatching.num_threads', str(self.num_threads), + '--SiftMatching.max_ratio', str(self.params.max_ratio), + '--SiftMatching.max_distance', str(self.params.max_distance), + '--SiftMatching.cross_check', '1' if self.params.cross_check else '0', + '--FeatureMatching.guided_matching', '1' if self.params.guided_matching else '0', + ] + def feature_matching(self, database_path: str) -> bool: - """Exécute le matching des features.""" + """Exécute le matching des features. + + La détection de boucles et le matcher vocab-tree dépendent d'un arbre de + vocabulaire FAISS qui peut, selon le build COLMAP, échouer au chargement. + Le matching ne doit JAMAIS s'interrompre pour autant : en cas d'échec on + rejoue automatiquement une variante plus simple (sans détection de + boucles, puis exhaustive) afin de toujours produire des correspondances. + """ + common = self._matching_common_opts() + if self.params.matcher_type == 'sequential': - cmd = [ + base = [ self.colmap_bin, 'sequential_matcher', '--database_path', database_path, - '--FeatureMatching.num_threads', str(self.num_threads), - '--SiftMatching.max_ratio', str(self.params.max_ratio), - '--SiftMatching.max_distance', str(self.params.max_distance), - '--SiftMatching.cross_check', '1' if self.params.cross_check else '0', - '--FeatureMatching.guided_matching', '1' if self.params.guided_matching else '0', + *common, '--SequentialMatching.overlap', str(self.params.sequential_overlap), '--SequentialMatching.quadratic_overlap', '1', ] - description = "Matching Sequentiel" - else: - cmd = [ - self.colmap_bin, 'exhaustive_matcher', - '--database_path', database_path, - '--FeatureMatching.num_threads', str(self.num_threads), - '--SiftMatching.max_ratio', str(self.params.max_ratio), - '--SiftMatching.max_distance', str(self.params.max_distance), - '--SiftMatching.cross_check', '1' if self.params.cross_check else '0', - '--FeatureMatching.guided_matching', '1' if self.params.guided_matching else '0', - ] - description = "Matching Exhaustif" - - return self.run_command(cmd, description, status_prefix="Comparaison") + # Loop closure: detect revisited places so the same location isn't + # reconstructed twice. Needs a vocab tree; gated on the option existing. + loop_args = [] + if getattr(self.params, 'loop_detection', True) \ + and 'loop_detection' in self._colmap_cmd_help('sequential_matcher'): + vocab = self._ensure_vocab_tree() + if vocab: + loop_args = ['--SequentialMatching.loop_detection', '1', + '--SequentialMatching.vocab_tree_path', str(vocab)] + if loop_args: + self.log("Détection de boucles activée (anti-duplication des lieux revisités).") + if self.run_command(base + loop_args, "Matching Sequentiel", + status_prefix="Comparaison"): + return True + # Vocab tree incompatible / unreadable → never abort matching. + self.log("(repli) La détection de boucles a échoué (arbre de " + "vocabulaire incompatible ?) — nouvelle tentative SANS " + "détection de boucles.") + return self.run_command(base, "Matching Sequentiel", status_prefix="Comparaison") + + if self.params.matcher_type == 'vocab_tree': + # Real vocab-tree matcher (good for unordered/large collections). + vocab = self._ensure_vocab_tree() + if vocab: + cmd = [ + self.colmap_bin, 'vocab_tree_matcher', + '--database_path', database_path, + *common, + '--VocabTreeMatching.vocab_tree_path', str(vocab), + ] + # Retrieval breadth: the COLMAP default only matches each image + # against its 100 most-similar retrievals — on 2000+ image scenes + # weakly-covered views fall below that cutoff and never connect. + vt_help = self._colmap_cmd_help('vocab_tree_matcher') + if 'VocabTreeMatching.num_images' in vt_help: + cmd += ['--VocabTreeMatching.num_images', '150'] + if 'VocabTreeMatching.num_nearest_neighbors' in vt_help: + cmd += ['--VocabTreeMatching.num_nearest_neighbors', '8'] + if self.run_command(cmd, "Matching Vocab-Tree", status_prefix="Comparaison"): + # Top-up pass: video frames are temporally ordered, so a cheap + # sequential pass connects the neighbours the vocab-tree + # retrieval missed (matches ACCUMULATE in the same database). + # Best-effort — its failure never fails the matching stage. + seq_cmd = [ + self.colmap_bin, 'sequential_matcher', + '--database_path', database_path, + *common, + '--SequentialMatching.overlap', str(self.params.sequential_overlap), + '--SequentialMatching.quadratic_overlap', '1', + ] + if not self.run_command(seq_cmd, "Matching Séquentiel (complément)", + status_prefix="Comparaison"): + self.log("(info) Passe séquentielle complémentaire échouée — " + "les matchs vocab-tree suffisent, on continue.") + return True + self.log("(repli) Matching vocab-tree échoué (arbre incompatible ?) " + "— bascule sur le matching exhaustif.") + else: + self.log("(repli) Arbre de vocabulaire indisponible — bascule sur " + "le matching exhaustif.") + + cmd = [ + self.colmap_bin, 'exhaustive_matcher', + '--database_path', database_path, + *common, + ] + return self.run_command(cmd, "Matching Exhaustif", status_prefix="Comparaison") + + def _colmap_has_command(self, name: str) -> bool: + """True if the installed colmap exposes the subcommand `name` (probed via + `colmap help`, cached). Lets us use COLMAP 4.0+'s built-in global_mapper + without assuming it exists on older builds.""" + cached = getattr(self, "_colmap_help_cmds", None) + if cached is None: + cached = "" + try: + import subprocess + out = subprocess.run([self.colmap_bin, 'help'], + capture_output=True, text=True, timeout=15) + cached = out.stdout + out.stderr + except (OSError, subprocess.SubprocessError): + cached = "" + self._colmap_help_cmds = cached + return name in cached + + def _normalize_dense_sparse_layout(self, dense_dir: Path) -> None: + """Mirror the standard COLMAP layout inside dense/. + + `colmap image_undistorter` writes the model files directly into + dense/sparse/ (no /0 sub-model dir), but downstream tools — Brush's + COLMAP loader included — expect sparse/0. Move the model files into + dense/sparse/0 so dense/ can be used as a drop-in training root. + Best-effort: on failure the original layout is kept.""" + sparse = dense_dir / "sparse" + zero = sparse / "0" + try: + if not sparse.is_dir() or zero.exists(): + return + model_files = [f for f in sparse.iterdir() if f.is_file()] + if not model_files: + return + zero.mkdir() + for f in model_files: + f.rename(zero / f.name) + self.log("ModÚle non-distordu réorganisé en dense/sparse/0 (layout standard).") + except OSError as e: + self.log(f"(info) Réorganisation dense/sparse impossible : {e}") + + def _binary_cmd_help(self, binary: str, cmd_name: str) -> str: + """` -h` output, cached (empty on failure). Same idea + as _colmap_cmd_help but for a non-colmap binary (e.g. standalone glomap).""" + cache = getattr(self, "_bin_help_cache", None) + if cache is None: + cache = {} + self._bin_help_cache = cache + key = (str(binary), cmd_name) + if key not in cache: + text = "" + try: + import subprocess + out = subprocess.run([binary, cmd_name, '-h'], + capture_output=True, text=True, timeout=15) + text = out.stdout + out.stderr + except (OSError, subprocess.SubprocessError): + text = "" + cache[key] = text + return cache[key] + + def _global_mapper_opts(self, help_txt: str, ran_calibration: bool = False) -> list: + """Probe-gated quality options for global_mapper/GLOMAP. + + Unlike the incremental path, GLOMAP previously received ZERO options, so + the user's GUI settings (GPU BA, intrinsics refinement) were silently + dropped. Every flag is gated on the build's -h output so an unsupported + option is skipped instead of aborting. NB: we deliberately do NOT map + ba_global_max_num_iterations here — it is tuned low (30) for the + *repeated* incremental BA; the single global BA deserves GLOMAP's own + default iteration budget. + """ + opts: list = [] + + def add(flag: str, value) -> None: + if flag.lstrip('-') in help_txt: + opts.extend([flag, str(value)]) + + if self.params.ba_use_gpu: + # Ceres falls back to CPU with a warning if built without CUDA — + # harmless, and honours the user's setting on capable builds. + add('--BundleAdjustment.use_gpu', 1) + add('--GlobalPositioning.use_gpu', 1) + try: + gpu_index = int(getattr(self.params, 'ba_gpu_index', -1)) + except (TypeError, ValueError): + gpu_index = -1 + if gpu_index >= 0: + add('--BundleAdjustment.gpu_index', gpu_index) + add('--GlobalPositioning.gpu_index', gpu_index) + add('--BundleAdjustment.optimize_intrinsics', + 1 if self.params.ba_refine_focal_length else 0) + add('--BundleAdjustment.optimize_principal_point', + 1 if self.params.ba_refine_principal_point else 0) + if ran_calibration: + # We already ran view_graph_calibrator as a separate step — don't + # pay for it twice inside global_mapper. + add('--skip_view_graph_calibration', 1) + return opts + + def _global_mapper(self, database_path: str, images_dir: str, sparse_dir: Path) -> bool: + """Global SfM (a single global bundle adjustment instead of the repeated + per-image one) — much faster on large scenes. Prefers COLMAP 4.0+'s + built-in `global_mapper` (no extra install); falls back to a standalone + glomap binary, then to the incremental mapper.""" + if self._colmap_has_command('global_mapper'): + # Calibrating focal lengths from the view graph first markedly + # improves global-mapper quality (it relies on decent intrinsics). + ran_vgc = self._colmap_has_command('view_graph_calibrator') + if ran_vgc: + self.status("Calibrage du view-graph...") + self.run_command( + [self.colmap_bin, 'view_graph_calibrator', '--database_path', database_path], + "Calibrage des intrinsÚques (view-graph)", status_prefix="Calibrage", + omp_threads=self.num_threads) + self.log("Reconstruction GLOBALE (colmap global_mapper) : un seul bundle " + "adjustment global au lieu du BA incrémental répété → bien plus rapide.") + cmd = [self.colmap_bin, 'global_mapper', + '--database_path', database_path, + '--image_path', images_dir, + '--output_path', str(sparse_dir)] + cmd += self._global_mapper_opts(self._colmap_cmd_help('global_mapper'), + ran_calibration=ran_vgc) + # global_mapper has NO --num_threads flag: its thread count comes from + # OMP_NUM_THREADS, so it must NOT get the default 1-thread pinning. + return self.run_command(cmd, "Reconstruction 3D (globale)", + status_prefix="Reconstruction globale", + omp_threads=self.num_threads) + + if resolve_binary('glomap'): + self.log("Utilisation de GLOMAP (binaire séparé) pour la reconstruction globale...") + cmd = [self.glomap_bin, 'mapper', + '--database_path', database_path, + '--image_path', images_dir, + '--output_path', str(sparse_dir)] + cmd += self._global_mapper_opts(self._binary_cmd_help(self.glomap_bin, 'mapper')) + return self.run_command(cmd, "Reconstruction 3D (GLOMAP)", + status_prefix="Reconstruction GLOMAP", + omp_threads=self.num_threads) + + self.log("⚠ Reconstruction globale demandée mais indisponible (ce COLMAP n'a pas " + "'global_mapper' et glomap n'est pas installé) → bascule sur le mapper " + "incrémental classique.") + return self._incremental_mapper(database_path, images_dir, sparse_dir) def mapper(self, database_path: str, images_dir: str, sparse_dir: Path) -> bool: """Exécute la reconstruction 3D (Mapper).""" if self.params.use_glomap: - self.log("Utilisation de GLOMAP pour la reconstruction...") - cmd = [ - self.glomap_bin, 'mapper', - '--database_path', database_path, - '--image_path', images_dir, - '--output_path', str(sparse_dir) - ] - return self.run_command(cmd, "Reconstruction 3D (GLOMAP)", status_prefix="Reconstruction GLOMAP") - else: - cmd = [ - self.colmap_bin, 'mapper', - '--database_path', database_path, - '--image_path', images_dir, - '--output_path', str(sparse_dir), - '--Mapper.num_threads', str(self.num_threads), - '--Mapper.min_model_size', str(self.params.min_model_size), - '--Mapper.multiple_models', '1' if self.params.multiple_models else '0', - '--Mapper.ba_refine_focal_length', '1' if self.params.ba_refine_focal_length else '0', - '--Mapper.ba_refine_principal_point', '1' if self.params.ba_refine_principal_point else '0', - '--Mapper.ba_refine_extra_params', '1' if self.params.ba_refine_extra_params else '0', - '--Mapper.min_num_matches', str(self.params.min_num_matches), - ] - return self.run_command(cmd, "Reconstruction 3D (COLMAP)", status_prefix="Reconstruction 3D") + return self._global_mapper(database_path, images_dir, sparse_dir) + return self._incremental_mapper(database_path, images_dir, sparse_dir) + + def _incremental_mapper(self, database_path: str, images_dir: str, sparse_dir: Path) -> bool: + """COLMAP incremental SfM.""" + cmd = [ + self.colmap_bin, 'mapper', + '--database_path', database_path, + '--image_path', images_dir, + '--output_path', str(sparse_dir), + '--Mapper.num_threads', str(self.num_threads), + '--Mapper.min_model_size', str(self.params.min_model_size), + '--Mapper.multiple_models', '1' if self.params.multiple_models else '0', + '--Mapper.ba_refine_focal_length', '1' if self.params.ba_refine_focal_length else '0', + '--Mapper.ba_refine_principal_point', '1' if self.params.ba_refine_principal_point else '0', + '--Mapper.ba_refine_extra_params', '1' if self.params.ba_refine_extra_params else '0', + '--Mapper.min_num_matches', str(self.params.min_num_matches), + ] + # Bound bundle-adjustment cost (the dominant mapper time on large + # scenes) — faster than COLMAP defaults, safe for a 3DGS target. Each + # flag is gated on the installed COLMAP actually advertising it: COLMAP + # 4.1.0 renamed/removed some of these (e.g. ba_global_*_ratio), and an + # unrecognized option makes the whole mapper abort. + ba_bounds = [ + ('ba_global_max_num_iterations', self.params.ba_global_max_num_iterations), + ('ba_global_function_tolerance', self.params.ba_global_function_tolerance), + # Refinement caps — the biggest in-mapper lever on the global-BA step. + ('ba_global_max_refinements', self.params.ba_global_max_refinements), + ('ba_local_max_refinements', self.params.ba_local_max_refinements), + # How often global BA runs. COLMAP 4.1.0 RENAMED images_ratio → + # frames_ratio (rig/frame terminology); pass both — only the name + # the build advertises is used, so frequency control works on both + # 4.1.0 (frames) and 3.x (images). Without this it was silently + # dropped on 4.1.0, leaving global BA at the slow default cadence. + ('ba_global_frames_ratio', self.params.ba_global_images_ratio), + ('ba_global_images_ratio', self.params.ba_global_images_ratio), + ('ba_global_points_ratio', self.params.ba_global_points_ratio), + ('ba_local_max_num_iterations', self.params.ba_local_max_num_iterations), + ] + skipped = [] + for name, val in ba_bounds: + if self._mapper_supports(name): + cmd += [f'--Mapper.{name}', str(val)] + else: + skipped.append(name) + if skipped: + self.log(f"(info) Options BA non disponibles sur ce COLMAP, ignorées : {', '.join(skipped)}") + # GPU bundle adjustment (COLMAP 4.1.0 "Caspar"). Only add the flag if + # the installed COLMAP actually supports it — otherwise an older + # build would abort with "unrecognized option". This is exactly the + # step that throws "Linear solver failure" on the CPU for big scenes. + if self.params.ba_use_gpu: + if self._mapper_supports_gpu_ba(): + cmd += ['--Mapper.ba_use_gpu', '1'] + if self.params.ba_gpu_index is not None and self.params.ba_gpu_index >= 0: + cmd += ['--Mapper.ba_gpu_index', str(self.params.ba_gpu_index)] + self.log("Bundle adjustment GPU demandé (--ba_use_gpu) — effectif seulement " + "si ce build de COLMAP a Ceres compilé avec CUDA/cuDSS (vérifié au runtime).") + else: + self.log("⚠ GPU bundle adjustment demandé mais ce COLMAP ne le " + "supporte pas (requiert COLMAP ≥ 4.1.0) → fallback CPU. " + "Supprimez engines\\colmap et relancez run.bat pour mettre à jour.") + return self.run_command(cmd, "Reconstruction 3D (COLMAP)", status_prefix="Reconstruction 3D") + + def _mapper_help_text(self) -> str: + """`colmap mapper -h` output, fetched once and cached (empty on failure). + + Used to gate every optional --Mapper.* flag so we never pass an option a + given COLMAP build doesn't recognize (which aborts the whole mapper). + """ + cached = getattr(self, "_mapper_help", None) + if cached is not None: + return cached + text = "" + try: + import subprocess + out = subprocess.run( + [self.colmap_bin, 'mapper', '-h'], + capture_output=True, text=True, timeout=15, + ) + text = out.stdout + out.stderr + except (OSError, subprocess.SubprocessError): + text = "" + self._mapper_help = text + return text + + def _mapper_supports(self, option: str) -> bool: + """True if `colmap mapper` advertises --Mapper.