Skip to content

Conversation

@msilvafe
Copy link
Contributor

@msilvafe msilvafe commented Oct 7, 2025

Here I've merged into master #1396 and #1381 from @17-sugiyama to be able to set up a preprocessing flow and test running through 100 observations using this new processing collaboratively.

@17-sugiyama 17-sugiyama requested a review from satoru99 October 9, 2025 10:55
@satoru99
Copy link
Member

satoru99 commented Oct 9, 2025

Expected usage in preprocess config for my previous commit 8625498

process_pipe:
    # fit psd
    - name: "noise"
      skip_on_sim: False
      psd: "psdQ"
      fit: True
      calc:
        fwhite: [1, 2]
        f_max: 2
        merge_name: 'noise_fit_statsQ'
      save:
        wrap_name: "noise_fitQ"

    # apply counter 1/f
    - name: "fourier_filter"
      skip_on_sim: False
      signal_name: "demodQ"
      wrap_name: "demodQ"
      process:
        filt_function: "counter_1_over_f"
        noise_fit_array: "noise_fitQ"
        filter_params:
          each_detector: True

@17-sugiyama
Copy link
Contributor

I have tested the new T2P function, scan frequency cutoff and counter 1/f filter with this config file, and they ran successfully.
The residual noise ASD after the all processes is shown below.
The peak around 10mHz is the scan synchronous residual noise, which can be eliminated by the common mode subtraction.
image

Copy link
Contributor Author

@msilvafe msilvafe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments which @17-sugiyama and I talked about over zoom.

@17-sugiyama
Copy link
Contributor

This is the comparison of the transfer function between sat-iso v3 and this process.
(reference:/scratch/gpfs/SIMONSOBS/sat-iso/v3/spectra/satp1/south/full-dataset/science/transfer_functions/transfer_function_SATp1_f150_south_science_x_SATp1_f150_south_science.npz)
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants