Add likelihood-informed data processors

[ArneBouillon]

Implement the likelihood-informed data processor from our Overleaf

• Closes Add new utilities for the likelihood-informed DataProcessor #370
• Closes Add a machine learning tool for user-defined functions #380

This PR makes the following changes.

• We add a LikelihoodInformed data processor, which combines input data, output data and the actual inverse problem to find good reduced spaces.
• We add an (undocumented and inconvenient to access) option to construct a Decorrelator that uses a fixed reduced-space dimension, instead of calculating it based on a variance threshold. This helps primarily when testing or doing comparisons.
• We add a machine learning tool that simply calls a user-defined function on the decoded input and encodes the result again (closing Add a machine learning tool for user-defined functions #380)
  • This is currently in examples/DimensionReduction/emulate_sample_linlinexp.jl. I feel like we need to make it a bit more versatile and less hacky before extracting it from there.
  • @odunbar what do you think?
• We add an example comparing Decorrelator + LikelihoodInformed to just Decorrelator.
  • TODO: I'm working on more experiments.

Big TODO: The error estimate for output space reduction when α ≠ 0 is way off, which would result in high errors when using retain_KL as a criterion to find a reduced space. I'm still looking into why this is.

There are some test failures that I still have to look at, but I think this PR is ready to review.

[codecov]

Codecov Report

❌ Patch coverage is 7.76699% with 95 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.99%. Comparing base (b102f1f) to head (63829f4).

Files with missing lines	Patch %	Lines
src/Utilities/likelihood_informed.jl	0.00%	93 Missing ⚠️
src/Utilities.jl	50.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #376      +/-   ##
==========================================
- Coverage   93.98%   88.99%   -5.00%     
==========================================
  Files          10       11       +1     
  Lines        1630     1717      +87     
==========================================
- Hits         1532     1528       -4     
- Misses         98      189      +91     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.