add option to exclude specific indices from the dataset #71
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we can restrict the number of samples/trajectories for each dataset using the
restrict_num_samplesorrestrict_num_trajectoriesparameters.If you agree, I'd like to enable an option to pass a list of indices to skip as a third option in the _build_restriction_set method.
That way, we can skip faulty or useless samples. On example are the steady-state solutions in the gray-scott dataset, which might distort training. Now we can just precompute/list the the indices to remove.