PSD noise ratio cuts #1406
PSD noise ratio cuts #1406
Conversation
msilvafe
left a comment
msilvafe
left a comment
There was a problem hiding this comment.
Main request is to change is trying to merge SplitFlags and AtomicInfo processes but otherwise this is a nice thing to add. Have you scraped the existing databases to make the r_det histograms? I'd be interested in comparing how different those are to the r_mean ones.
sotodlib/tod_ops/fft_ops.py
Outdated
| calc_aman.wrap("f_sel", np.array(f_sel)) | ||
| calc_aman.wrap("f_wn", np.array(f_wn)) |
There was a problem hiding this comment.
We have issues right now with wrapping scalars because we don't know how to merge them when loading back in. This is an issue for frequency cutoffs too. Since this info is stored in the config file and saved through the preprocess config check perhaps we can just omit this to avoid adding another case of this issue to fix.
There was a problem hiding this comment.
These should be len(2) lists/arrays but maybe I don't understand what you mean by scalars? Think it would be nice to have in the proc_aman but sure I'll take it out if it's a headache
There was a problem hiding this comment.
But I suppose this will also affect rmean which I'd like to keep..
Is this also why apparent scalars in get_splits_flags are wrapped as (dets) ?
There was a problem hiding this comment.
Has there been any update on this? Do I need to remove all scalars for this to work? @msilvafe
There was a problem hiding this comment.
Unfortunately no, we (mostly @mmccrackan) are working on this big infrastructure overhaul first: #1348 once that's done then we'll work on the new obsdbs for stats and "info" tracking as described here: #1324 and that work will enable stashing scalars.
msilvafe
left a comment
msilvafe
left a comment
There was a problem hiding this comment.
Thanks for addressing these changes. I'm not sure if we'll apply cuts from this just yet but perhaps for v4 we can run this to save the relevant ratio metrics to the preprocess archive. Would you mind adding an example config to iso-sat/v4/satp1 with such an entry?
earosenberg
force-pushed
the
er/noise_ratio
branch
from
1e4139d to
805a73e
Compare
Add cuts based on ratio of "signal band" to white noise in demod PSDs to preprocess.
Also add per-wafer-obs mean numbers to the atomic db so the cuts can be done at bundling level instead.