Auto pc_align with ICESat-2 in asp_plot report#117
Merged
Conversation
Relocates the runtime summary + command listing from the trailing page to page 2 (immediately after the DEM Summary on the title page) so the title page is followed by the processing context before the diagnostic figures. Adds a new AlignmentReportPage dataclass for the forthcoming ICESat-2 pc_align workflow. Each page can carry a small kwargs table, a single-row horizontal alignment-stats table (values rendered to two significant figures), a status paragraph, and an optional figure with caption below. compile_report now iterates a mixed list of ReportSection | AlignmentReportPage entries. Also factors shared figure rendering into _render_figure_with_caption and the parameters page into _add_processing_parameters_page.
Introduces an AlignmentResult dataclass and an Altimetry.align_and_evaluate method that wraps the existing alignment_report with a decision step: the aligned DEM is only retained when p50 (pc_align's unsigned error percentile) drops toward zero by more than a configurable percentage (default 5%). Non-successful runs delete any *_pc_align_translated.tif that was written so callers can treat the aligned DEM's presence as a signal that the alignment is worth using. The method and dataclass are kept free of any report/fpdf imports so notebooks can reuse them. Adds plot_aligned kwargs to histogram_by_landcover and plot_best_worst_segments. When set, histogram_by_landcover overlays the pre- and post-alignment distributions with two vertically stacked per- landcover text boxes whose outline colors match the bar colors (so color serves as the legend). plot_best_worst_segments keeps the same segment selection (based on unaligned dh, so segments are comparable across the two variants), overlays aligned DEM heights, and appends aligned Median/NMAD to each segment title. Both kwargs fall back gracefully with a warning if aligned_dem_fn or the aligned columns are missing.
After the existing four Earth ICESat-2 plots, calls
Altimetry.align_and_evaluate with a 5% p50 improvement threshold and
appends one of three outcomes to the report:
- insufficient_points: a single AlignmentReportPage explaining that
pc_align did not run (too few ATL06-SR points or no usable log).
- no_improvement: a single AlignmentReportPage with the parameters
and alignment-stats tables plus a status note that the aligned DEM
was removed.
- success: three pages. First, an AlignmentReportPage with
the parameters + stats + status text and a pre/post
histogram_by_landcover figure. Then two ReportSection pages with
the aligned-overlay profile and best/worst segment plots.
Planetary (moon/mars) path is untouched.
- align_and_evaluate: three monkeypatched tests covering the insufficient_points, no_improvement (aligned DEM deleted), and success (aligned DEM retained) branches. - histogram_by_landcover and plot_best_worst_segments: smoke tests for plot_aligned=True when no aligned DEM is available, asserting the fallback path does not raise. - compile_report: smoke test verifying the new Processing Parameters page 2 layout and that a mixed list of ReportSection plus AlignmentReportPage entries (success / no-improvement / insufficient cases) compiles without error. - _fmt_sig: unit checks for the compact 2-sig-fig numeric formatter used in the alignment stats row.
Adds an opt-out flag for the pc_align + ICESat-2 alignment workflow added earlier in this branch. Defaults to True so the alignment pages are still generated by default on Earth DEMs. Setting it to False (or disabling altimetry via --plot_altimetry False / --plot_icesat False) skips the pc_align step entirely, which is useful for offline runs or when the ICESat-2 coverage is known to be too sparse for a meaningful alignment. Updates docs/cli/asp_plot.md to document the new flag alongside the existing altimetry options.
When plot_atl06sr_dem_profile is called with plot_aligned=True and the aligned DEM columns are available, the bottom panel now shows the post-alignment dh (icesat_minus_aligned_dem) and its Med/NMAD, tagged "(Aligned DEM)" in the legend. The unaligned dh would be a duplicate of the pre-alignment profile on the same report, so this makes the aligned-variant page actually communicate something new about the aligned DEM's performance. Behavior is unchanged when plot_aligned=False or when the aligned DEM columns are missing.
Adds an explanatory 'description' field to AlignmentReportPage that documents what ASP's pc_align does and defines every column of the Alignment Parameters and Alignment Statistics tables directly above it. Populated by the CLI so all three alignment-status pages (insufficient_points / no_improvement / success) carry the same reference block. Also switches the description and outcome blocks to align='L' so the long lines no longer render with large justified word-spacing gaps, and promotes the outcome status_message to bold so it stands out as the per-run result rather than being buried in the reference text.
align_and_evaluate calls atl06sr_to_dem_dh on its success branch to interpolate the aligned DEM at each ATL06-SR point. The initial Earth altimetry block already ran atl06sr_to_dem_dh once (with the default n_sigma=3), so the filtered sample is already 3σ-clean against icesat_minus_dem. Passing n_sigma=None on the second call avoids a second outlier pass that would tighten the sample further (using the narrower mean/std of the already-filtered subset) and cause the aligned-DEM plots to operate on a slightly smaller sample than the unaligned plots on the same report. No ICESat-2 re-requesting here: atl06sr_to_dem_dh only interpolates the DEM at the existing geometries; it never calls SlideRule.
1. Share histogram bin edges across pre-/post-alignment distributions. histogram_by_landcover was calling ax.hist(bins=128) once per distribution, so matplotlib computed bin edges independently from each series' own range. The resulting bars had slightly different widths and offsets, which defeated the side-by-side visual comparison that the plot_aligned overlay is meant to support. Now computes a shared np.linspace(xmin, xmax, 129) and passes the same edges to every hist() call. 2. Preserve the 'All:' header label in the single-distribution case. The previous refactor changed the per-landcover stats box header from 'All: n=...' to 'DEM: n=...' even when plot_aligned=False, a cosmetic regression for the default pre-existing plot. The 'DEM' / 'Aligned DEM' labels are now only used when two distributions are being compared. 3. Downgrade success to no_improvement when pc_align refused to write the aligned DEM. alignment_report skips the aligned-DEM write when the translation magnitude is below min_translation_threshold * GSD, leaving self.aligned_dem_fn None. Previously align_and_evaluate could still take the success branch if p50 happened to drop > 5%, which would return an AlignmentResult claiming success with aligned_dem_fn=None and cause the downstream plot_aligned overlays to silently no-op with warnings. Now the success branch requires self.aligned_dem_fn to be non-None; otherwise it reports no_improvement with a message that cites the translation-threshold as the reason (so the user can raise min_translation_threshold or the improvement_threshold if they want to retry).
Previously the alignment-stats text and the histogram_by_landcover figure shared one AlignmentReportPage, which left the figure squeezed into whatever vertical space remained after the parameters table, the stats-row table, the multi-paragraph description, and the status message. On real reports (ASTER and Atlanta runs) the figure ended up too small to read. Splits the success-path into four pages: the alignment report (parameters + stats + description + status, no figure) followed by three full-page ReportSection figures (landcover histogram, profile, best/worst segments). The histogram now fills its page like every other diagnostic figure in the report.
Regenerates the ASTER, ASTER_mapproj, WorldView_Atlanta, and WorldView_UCSD example reports against the updated CLI. The new Processing Parameters page-2 layout and the pc_align + ICESat-2 alignment pages (parameters + stats + description followed by the landcover histogram, profile, and best/worst segments with the aligned DEM) show up in each PDF where ICESat-2 data is available.
New public surface area — Altimetry.align_and_evaluate, AlignmentResult, AlignmentReportPage, the plot_aligned kwargs on histogram_by_landcover and plot_best_worst_segments, and the --pc_align CLI flag — warrants a minor bump. No breaking changes; plot_aligned defaults to False so existing callers are unaffected.
Adds an "Automatic pc_align with ICESat-2" section to docs/cli/asp_plot.md covering the alignment report page, the success- path figures, the acceptance threshold (p50 reduction > 5% AND a non-trivial translation magnitude), and how to disable the step. Updates the "What it does" bullet on the docs landing page to mention the optional pc_align refinement alongside the ICESat-2 comparisons.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
asp_plotCLI report. Ifp50(pc_align's unsigned error percentile) drops toward 0 by more than 5%, the aligned DEM is retained and the land-cover histogram, profile, and best/worst segment plots are re-rendered against it. Otherwise the aligned DEM file is deleted and a status note is added explaining that no significant improvement was found.AlignmentReportPage(inasp_plot/report.py) for mixed layouts that combine a kwargs table, a 1-row horizontal stats table, a status paragraph, and an optional figure+caption.Altimetry.align_and_evaluate(...)returning a plainAlignmentResultdataclass (no fpdf/report imports), so notebooks can call this workflow and inspectstatus in {insufficient_points, no_improvement, success}without taking on the report stack.plot_alignedkwargs toAltimetry.histogram_by_landcover(vertically stacked pre/post stats boxes, outline colors matching bar colors — color = legend) andAltimetry.plot_best_worst_segments(overlays the aligned DEM, keeps the same segment selection so Median/NMAD are comparable).Planetary (Moon/Mars) path is unchanged.
Test plan
pytest— all 150 tests pass, including new tests covering:align_and_evaluate(insufficient_points / no_improvement / success)plot_aligned=Truefallback when no aligned DEM is availablecompile_reportwithAlignmentReportPage_fmt_sigcompact numeric formatterasp_plotend-to-end against a real processed stereo directory (e.g. one of the Atlanta/UCSD example outputs) and attach before/after PDFs to this PR.minimum_pointsthreshold (to forceinsufficient_points) and with a near-perfect DEM (to forceno_improvement).