-
Notifications
You must be signed in to change notification settings - Fork 1
Add trained NN models and clean up legacy weights #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Add trained TBNN models for channel and periodic hills flows - data/models/tbnn_channel_caseholdout/ (trained on McConkey channel data) - data/models/tbnn_phll_caseholdout/ (trained on McConkey periodic hills data) - Both models include weights, biases, normalization, and metadata - Remove all example/dummy weights - Deleted legacy root-level layer files from data/ - Removed example_scalar_nut/ and example_tbnn/ placeholder models - No more random/untrained weights in codebase - Update model loading to require explicit selection - Modified Config to require --nn_preset or --weights/--scaling - Updated default paths from 'data/' to empty strings - Added validation in Config::finalize() for NN models - Update all GPU CI tests to use trained models - test_nn_integration.cpp: Use tbnn_channel_caseholdout - test_gpu_execution.cpp: Use tbnn_channel_caseholdout - test_cpu_gpu_consistency.cpp: Use tbnn_channel_caseholdout - test_perturbed_channel.cpp: Dynamically select trained models - Update documentation - Rewrote data/README.md to direct users to data/models/ - Updated data/models/README.md with current model structure - Added USAGE.md files for each trained model - Update .gitignore to track trained models - Keep *_caseholdout/ directories in version control - Ignore transient training outputs (job-ID suffixes) - Add training infrastructure - scripts/train_realdata_local.sh for local training - Updated download_mcconkey_data.sh to use python -m kaggle All tests pass on GPU hardware (V100) with trained weights.
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||||
PR Code Suggestions ✨Latest suggestions up to 282ad8d
Previous suggestions✅ Suggestions up to commit ddc4146
✅ Suggestions up to commit ab96b34
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
/improve |
|
Persistent suggestions updated to latest commit c9f0ab7 |
|
/improve |
|
Persistent suggestions updated to latest commit ab96b34 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces production-ready trained TBNN models while cleaning up legacy example code and significantly enhancing the test infrastructure. The changes include two fully-trained neural network models (channel flow and periodic hills), comprehensive test coverage for turbulence features, improved GPU/CPU backend handling, and better configuration validation.
Key Changes:
- Added two trained TBNN models with complete weights, biases, and normalization parameters
- Fixed parameter order bug in GEP turbulence model initialization
- Enhanced configuration validation requiring explicit NN model specification
- Expanded test suite with analytic verification tests and comprehensive turbulence model validation
- Improved backend flexibility with graceful CPU fallback when GPU unavailable
Reviewed changes
Copilot reviewed 74 out of 75 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/turbulence_gep.cpp | Fixed parameter order from (nu_, u_ref_, delta_) to (nu_, delta_, u_ref_) in set_reference() calls |
| src/config.cpp | Added validation requiring explicit NN model specification with improved error messages |
| include/config.hpp | Changed default NN paths from "data/" to empty strings |
| tests/test_turbulence_features.cpp | New comprehensive test suite for EARSM, GEP, baseline models with analytic flow verification |
| tests/test_features.cpp | Replaced generic tests with analytic verification for pure shear, strain, and rotation flows |
| tests/test_perturbed_channel.cpp | Updated to use trained tbnn_channel_caseholdout model |
| tests/test_nn_integration.cpp | Updated to use trained tbnn_channel_caseholdout model |
| tests/test_cpu_gpu_consistency.cpp | Refactored for graceful CPU fallback when GPU unavailable |
| tests/test_backend_execution.cpp | Renamed from GPU-specific to backend-agnostic with CPU support |
| src/turbulence_nn_tbnn.cpp | Implemented full GPU pipeline with feature computation and postprocessing |
| src/turbulence_nn_mlp.cpp | Implemented full GPU pipeline with MLP-specific feature computation |
| src/gpu_kernels.cpp | Added GPU kernels for MLP features, TBNN features, and output postprocessing |
| include/gpu_kernels.hpp | Added declarations for new GPU kernel functions |
| data/models/tbnn_channel_caseholdout/* | CRITICAL ISSUE: Invalid normalization statistics with inf/extreme values |
| data/models/tbnn_phll_caseholdout/* | Added trained periodic hills model with valid normalization statistics |
| scripts/train_realdata_local.sh | New comprehensive training script for local TBNN/MLP training |
| scripts/download_mcconkey_data.sh | Improved Kaggle dataset download robustness |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/improve |
|
Persistent suggestions updated to latest commit ddc4146 |
|
/improve |
|
Persistent suggestions updated to latest commit 282ad8d |
PR Type
Enhancement, Bug fix, Tests, Documentation
Description
Added trained neural network models: Introduced two production-ready TBNN models (
tbnn_channel_caseholdoutandtbnn_phll_caseholdout) with complete weight matrices, biases, and normalization parameters for channel flow and periodic hills casesRemoved legacy example models: Cleaned up deprecated
example_tbnnandexample_scalar_nutmodels and their associated weight files from the repositoryEnhanced configuration validation: Implemented explicit NN model specification requirements in
config.cppwith improved error messages and automatic path mirroring, removing legacy fallback to defaultdata/directoryFixed GEP parameter order bug: Corrected parameter order in
set_reference()calls inturbulence_gep.cppfrom(nu_, u_ref_, delta_)to(nu_, delta_, u_ref_)Expanded test coverage: Added comprehensive test suites for feature computation (
test_features.cpp), turbulence model validation (test_turbulence_features.cpp), and generalized backend execution tests supporting both CPU and GPUImproved backend flexibility: Refactored GPU-specific tests to support graceful CPU fallback when GPU is unavailable, with clear backend status reporting
Updated all tests to use trained models: Migrated test suite from legacy example models to trained
tbnn_channel_caseholdoutweights across integration, backend, and perturbed channel testsEnhanced documentation: Created comprehensive usage guides and metadata files for trained models, including architecture details, training configuration, feature descriptions, and usage examples
Improved dataset handling: Enhanced McConkey dataset download robustness with better Kaggle CLI invocation and fallback unzip mechanisms; added local training script for TBNN/MLP models
Diagram Walkthrough
File Walkthrough
7 files
test_features.cpp
Expand feature tests with analytic velocity field verificationtests/test_features.cpp
tests for pure shear, pure strain, and solid body rotation flows
features, TBNN features, and tensor basis operations
conversion tests
compilation
test_turbulence_features.cpp
Add comprehensive turbulence model feature teststests/test_turbulence_features.cpp
nontrivial velocity gradients
and multiple EARSM variants (Wallin-Johansson, Gatski-Speziale, Pope)
verification, and solver backend execution
test_cpu_gpu_consistency.cpp
Improve GPU/CPU consistency tests with fallback pathstests/test_cpu_gpu_consistency.cpp
builds
tests
status
tbnn_channel_caseholdoutinsteadof legacy example models
test_backend_execution.cpp
Generalize backend execution tests for CPU and GPUtests/test_backend_execution.cpp
(
tbnn_channel_caseholdout)being tested
test_nn_integration.cpp
Update NN integration tests to use trained modelstests/test_nn_integration.cpp
example_scalar_nutandexample_tbnntotrained
tbnn_channel_caseholdoutNN integration tests
test_perturbed_channel.cpp
Update perturbed channel test with trained model pathstests/test_perturbed_channel.cpp
tbnn_channel_caseholdoutmodelbased on model type
CMakeLists.txt
Test infrastructure updates and new turbulence features testCMakeLists.txt
test_gpu_executiontotest_backend_executionforbackend-agnostic testing
GPUExecutionTesttoBackendExecutionTestCPUGPUConsistencyTesttoConsistencyTesttest_turbulence_featuresfor analyticvalidation of features, invariants, and model response
2 files
config.cpp
Add NN model configuration validation and error handlingsrc/config.cpp
weights/scaling paths)
options
provided
data/directoryconfig.hpp
Update config header with explicit NN model requirementsinclude/config.hpp
data/to empty strings (requireexplicit specification)
tbnn_channel_caseholdout)1 files
turbulence_gep.cpp
Fix parameter order in GEP feature computer initializationsrc/turbulence_gep.cpp
set_reference()calls from(nu_, u_ref_,delta_)to(nu_, delta_, u_ref_)ensure_initialized()andupdate()methods17 files
train_realdata_local.sh
Add local training script for real McConkey datasetscripts/train_realdata_local.sh
McConkey dataset
model export
learning rate, device)
credentials
download_mcconkey_data.sh
Improve McConkey dataset download robustnessscripts/download_mcconkey_data.sh
__main__support
layer0_W.txt
TBNN channel flow model input layer weightsdata/models/tbnn_channel_caseholdout/layer0_W.txt
model
neurons)
layer3_W.txt
TBNN periodic hills model output layer weightsdata/models/tbnn_phll_caseholdout/layer3_W.txt
coefficients)
layer3_W.txt
TBNN channel flow model output layer weightsdata/models/tbnn_channel_caseholdout/layer3_W.txt
coefficients)
layer0_b.txt
TBNN periodic hills model input layer biasesdata/models/tbnn_phll_caseholdout/layer0_b.txt
layer1_b.txt
TBNN periodic hills model first hidden layer biasesdata/models/tbnn_phll_caseholdout/layer1_b.txt
layer2_b.txt
TBNN periodic hills model second hidden layer biasesdata/models/tbnn_phll_caseholdout/layer2_b.txt
layer0_b.txt
TBNN channel flow model input layer biasesdata/models/tbnn_channel_caseholdout/layer0_b.txt
layer1_b.txt
TBNN channel flow model first hidden layer biasesdata/models/tbnn_channel_caseholdout/layer1_b.txt
layer2_b.txt
TBNN channel flow model second hidden layer biasesdata/models/tbnn_channel_caseholdout/layer2_b.txt
input_means.txt
Input normalization means for periodic hills TBNN modeldata/models/tbnn_phll_caseholdout/input_means.txt
input_stds.txt
Input normalization standard deviations for periodic hills TBNN modeldata/models/tbnn_phll_caseholdout/input_stds.txt
hills TBNN model
input_means.txt
Input normalization means for channel flow TBNN modeldata/models/tbnn_channel_caseholdout/input_means.txt
input_stds.txt
Input normalization standard deviations for channel flow TBNN modeldata/models/tbnn_channel_caseholdout/input_stds.txt
TBNN model
layer3_b.txt
TBNN periodic hills model output layer biasesdata/models/tbnn_phll_caseholdout/layer3_b.txt
layer3_b.txt
TBNN channel flow model output layer biasesdata/models/tbnn_channel_caseholdout/layer3_b.txt
1 files
layer0_W.txt
Add trained TBNN weights for periodic hills casedata/models/tbnn_phll_caseholdout/layer0_W.txt
units, 5 input features)
6 files
USAGE.md
Add usage guide for trained channel TBNN modeldata/models/tbnn_channel_caseholdout/USAGE.md
descriptions, and usage examples
original papers
README.md
Comprehensive documentation for trained neural network modelsdata/models/README.md
instead of legacy examples
tbnn_channel_caseholdoutandtbnn_phll_caseholdoutlosses, and usage examples
model-specific usage guidance
USAGE.md
Detailed usage guide for periodic hills TBNN modeldata/models/tbnn_phll_caseholdout/USAGE.md
case_1p0, test: case_1p2)
output tensor basis
structure reference
README.md
Simplified data directory documentation with model focusdata/README.md
data/models/README.md--nn_presetflag
defaults exist
metadata.json
Metadata documentation for periodic hills TBNN modeldata/models/tbnn_phll_caseholdout/metadata.json
and training
activations
normalization
reference
metadata.json
Metadata documentation for channel flow TBNN modeldata/models/tbnn_channel_caseholdout/metadata.json
and training
activations
normalization
reference
34 files