oceanmodeling · tomsail · Mar 24, 2026 · Mar 26, 2026 · Mar 26, 2026 · Mar 30, 2026
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -20,8 +20,8 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: 3.x
-      - run: pip install zensical
-      - run: zensical build --clean 
+      - run: pip install zensical mkdocs-material mkdocstrings-python
+      - run: zensical build --clean
       - uses: actions/upload-pages-artifact@v4
         with:
           path: site

diff --git a/.github/workflows/run_tests.yml b/.github/workflows/run_tests.yml
@@ -14,7 +14,7 @@ jobs:
       fail-fast: false
       matrix:
         os: ["ubuntu-latest"]
-        python: ["3.11"]
+        python: ["3.11", "3.12", "3.13"]
     defaults:
       run:
         shell: "bash -eo pipefail {0}"

diff --git a/Makefile b/Makefile
@@ -30,3 +30,9 @@ deps:
 
 docs:
 	zensical build
+
+serve:
+	python -mpanel serve dashboard/cleanup_dashboard.py --port 60002
+
+poster:
+	python -mpanel serve scripts/poster.py --autoreload
diff --git a/README.md b/README.md
@@ -1,6 +1,8 @@
 # IOC Cleanup
 
-`ioc_cleanup` provides a 
+Check the online [documentation](https://oceanmodeling.github.io/ioc_cleanup/)
+
+`ioc_cleanup` provides a
  * reproducible
  * transparent,
  * and traceable
@@ -13,4 +15,13 @@ workflow for cleaning tide gauge (sea level) data from IOC (Intergovernmental Oc
 ## Use the dashboard
 ![demo](./docs/assets/dashboard_light.png)
 
-Check the [docs](https://tomsail.github.io/ioc_cleanup/)
+## How to cite
+
+```
+Saillour, T. and Mavrogiorgos, P.: Reproducible, transparent and traceable cleaning of IOC Tide Gauge Data, EGU General Assembly 2026, Vienna, Austria, 3–8 May 2026, EGU26-7777, https://doi.org/10.5194/egusphere-egu26-7777, 2026.
+```
+
+## Poster
+A reproducible poster has been created for the EGU 2026 conference (see DOI above).
+
+You can consult it directly in the [poster section](https://oceanmodeling.github.io/ioc_cleanup/poster/) in the docs.
diff --git a/docs/assets/cleaned_map.html b/docs/assets/cleaned_map.html
diff --git a/docs/assets/coverage_oceans.html b/docs/assets/coverage_oceans.html
diff --git a/docs/assets/data_availability_hist.html b/docs/assets/data_availability_hist.html
diff --git a/docs/assets/data_availability_map.html b/docs/assets/data_availability_map.html
diff --git a/docs/assets/data_removed_hist.html b/docs/assets/data_removed_hist.html
diff --git a/docs/assets/data_removed_map.html b/docs/assets/data_removed_map.html
diff --git a/docs/example.html → docs/assets/example.html b/docs/example.html → docs/assets/example.html
diff --git a/docs/assets/example_clean.html b/docs/assets/example_clean.html
diff --git a/docs/assets/flat.html b/docs/assets/flat.html
diff --git a/docs/assets/kamchatka_map.html b/docs/assets/kamchatka_map.html
diff --git a/docs/noise.html → docs/assets/noise.html b/docs/noise.html → docs/assets/noise.html
diff --git a/docs/assets/poster.html b/docs/assets/poster.html
diff --git a/docs/assets/poster.png b/docs/assets/poster.png
diff --git a/docs/seiche.html → docs/assets/seiche.html b/docs/seiche.html → docs/assets/seiche.html
diff --git a/docs/spikes.html → docs/assets/spikes.html b/docs/spikes.html → docs/assets/spikes.html
diff --git a/docs/step_long.html → docs/assets/step_long.html b/docs/step_long.html → docs/assets/step_long.html
diff --git a/docs/step_simple.html → docs/assets/step_simple.html b/docs/step_simple.html → docs/assets/step_simple.html
diff --git a/docs/assets/tonga_map.html b/docs/assets/tonga_map.html
diff --git a/docs/tsunami.html → docs/assets/tsunami.html b/docs/tsunami.html → docs/assets/tsunami.html
diff --git a/docs/tsunami_detided.html → docs/assets/tsunami_detided.html b/docs/tsunami_detided.html → docs/assets/tsunami_detided.html
diff --git a/docs/unknown.html → docs/assets/unknown.html b/docs/unknown.html → docs/assets/unknown.html
diff --git a/docs/cleaned_map.html b/docs/cleaned_map.html
diff --git a/docs/concept.md b/docs/concept.md
@@ -1,4 +1,4 @@
-# Motivation
+## Why `ioc_cleanup`?
 
 Cleaning tide gauge data is often:
 
@@ -57,3 +57,70 @@ declaring:
 - notes and metadata
 
 More details in the [JSON format](./reference/json-schema.md)
+
+
+## Dataset Details
+
+The following figures have been generated with the helper functions in `scripts/` folder:
+
+ 1. `download_ioc.py` to download IOC stations
+ 2. `generate_maps.py` to create maps and graphs for the online documentation
+ 3. `save_cleaning_scenarios.py` to create the time series graphs used in the online documentation
+
+Steps 2 and 3 require to have run step 1 for all cleaned IOC stations.
+
+The cleaned stations dataset can be retrieved with:
+
+```python
+import ioc_cleanup as C
+ioc = C.get_meta()
+stats = C.calc_statistics(ioc, stations_dir=C.TRANSFORMATIONS_DIR, pattern="*.json")
+```
+
+### Cleaned Stations
+<iframe
+  src="./assets/cleaned_map.html"
+  width="100%"
+  height="740"
+  style="border:none;">
+</iframe>
+
+### Coverage across oceans
+<iframe
+  src="./assets/coverage_oceans.html"
+  width="100%"
+  height="330"
+  style="border:none;">
+</iframe>
+
+### Data availability in the 2020 - 2025 period
+
+<iframe
+  src="./assets/data_availability_map.html"
+  width="100%"
+  height="590"
+  style="border:none;">
+</iframe>
+
+<iframe
+  src="./assets/data_availability_hist.html"
+  width="100%"
+  height="320"
+  style="border:none;">
+</iframe>
+
+### Ratio of data removed in the 2020 - 2025 period
+
+<iframe
+  src="./assets/data_removed_map.html"
+  width="100%"
+  height="590"
+  style="border:none;">
+</iframe>
+
+<iframe
+  src="./assets/data_removed_hist.html"
+  width="100%"
+  height="320"
+  style="border:none;">
+</iframe>
diff --git a/docs/contributing.md b/docs/contributing.md
@@ -5,12 +5,14 @@ Contributions are very welcome!
 ## How to contribute
 
  1. Fork the repository
- 2. Add or update a JSON transformation file
- 3. Use the dashboard to clean or flag data
+ 2. Add or update a [JSON transformation file](./reference/json-schema.md)
+ 3. Use the [dashboard](./workflows/dashboard.md) to clean or flag data
  4. Submit a pull request with a clear description of your changes
 
 ## Areas for improvement
 
- * Publication and doi for clean dataset (WIP)
+ * Presentation at [EGU 2026](https://doi.org/10.5194/egusphere-egu26-7777)
+ * Publication of the cleaned dataset
  * Add more IOC stations
  * Extend the cleaned time range (currently 2020–2025)
+ * Others areas of improvement? Please open an [issue](https://github.com/oceanmodeling/ioc_cleanup)!
diff --git a/docs/example_clean.html b/docs/example_clean.html
diff --git a/docs/guidelines.md b/docs/guidelines.md
@@ -15,7 +15,7 @@ This repository does NOT contain IOC data and does not manage data acquisition.
 In some cases, cleaning is easy and is just about removing spikes
 ### Spikes
 <iframe
-  src="../spikes.html"
+  src="./assets/spikes.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -26,15 +26,15 @@ In some cases, cleaning is easy and is just about removing spikes
 
     See details on the [JSON structure](reference/json-schema/)
 
-### Noise vs. Physical phenomena
+### Numerical vs. Physical phenomena
 
 It becomes more difficult when it comes to distinguishing noise (either numerical or physical e.g. boat wakes) from real physical events (like harbour seiches or tsunamis).
 
 
 #### Physical - Seiches
 Here an example of what seems to be a harbour seiche in `LA23` - Lampedusa station (IT):
 <iframe
-  src="../seiche.html"
+  src="./assets/seiche.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -47,7 +47,7 @@ Here an example of what seems to be a harbour seiche in `LA23` - Lampedusa stati
 Here is the 2025 Kamchatka Peninsula Tsunami captured by `cres` - Crescent City station (CA, USA):
 
 <iframe
-  src="../tsunami.html"
+  src="./assets/tsunami.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -57,7 +57,7 @@ Here is the 2025 Kamchatka Peninsula Tsunami captured by `cres` - Crescent City
 Same tsunami and station, detided:
 
 <iframe
-  src="../tsunami_detided.html"
+  src="./assets/tsunami_detided.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -68,11 +68,11 @@ Same tsunami and station, detided:
 
     See details on the [JSON structure](reference/json-schema/)
 
-#### Noise - Numerical
+#### Numerical - Noise
 In some case, numerical noise is easy to isolate like for this station:
 
 <iframe
-  src="../example.html"
+  src="./assets/example.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -83,7 +83,24 @@ In some case, numerical noise is easy to isolate like for this station:
 
     See details on the [JSON structure](reference/json-schema/)
 
-#### Noise - Unknown
+#### Numerical - Flat signal
+Some stations can have parts of flat signal.
+
+<iframe
+  src="./assets/flat.html"
+  width="100%"
+  height="710"
+  style="border:none;">
+</iframe>
+
+!!! tip "Advice for flat signal"
+    Remove flat parts from the data.
+
+    If the flat parts are long enough and easy to isolate, select the flat ranges and paste them in the `dropped_date_ranges`. If it is too complicated (like in this example), you can select multiple part of the data and paste in `dropped_timestamps`.
+
+    See details on the [JSON structure](reference/json-schema/)
+
+#### Numerical - Unknown
 In other case, the nature of the noise is difficult to identify. There could be lots of reasons:
 
  * physical induced noise:
@@ -96,15 +113,15 @@ In other case, the nature of the noise is difficult to identify. There could be
 
 ##### Example 1
 <iframe
-  src="../noise.html"
+  src="./assets/noise.html"
   width="100%"
   height="710"
   style="border:none;">
 </iframe>
 
 ##### Example 2
 <iframe
-  src="../unknown.html"
+  src="./assets/unknown.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -120,7 +137,7 @@ In other case, the nature of the noise is difficult to identify. There could be
 Some steps are easy to isolate and deal with. A recurrent error found on tidal stations occurs during DST (Daylight saving time) changes:
 
 <iframe
-  src="../step_simple.html"
+  src="./assets/step_simple.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -140,7 +157,7 @@ Some steps - or offsets - can be caused by mulitple reasons:
   * any ohter unkonwn reason
 
 <iframe
-  src="../step_long.html"
+  src="./assets/step_long.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -179,3 +196,13 @@ Some steps - or offsets - can be caused by mulitple reasons:
 !!! warning "Subjectivity"
      * Cleaning decisions are inherently subjective
      * Different operators may disagree on what should be discarded
+
+### De-tiding
+
+!!! warning "Chunks length for de-tiding"
+    Although `ioc_cleanup` does not directly tackle the de-tiding problem, it leverages de-tiding methods to better isolate and flag bad data on the tide gauges.
+
+    Here are some resources on this matter:
+
+    * Github discussion on [Utide](https://github.com/orgs/oceanmodeling/discussions/25)
+    * Detiding Theory and Practices, cited in the [Book of tides](https://www.researchgate.net/publication/280722791_De-tiding_Theory_and_practice)
diff --git a/docs/index.md b/docs/index.md
@@ -7,7 +7,7 @@ for cleaning tide gauge data from [IOC](https://www.ioc-sealevelmonitoring.org/l
 All stations with clean data between 1st of January 2020 and the 31st of december 2025.
 
 <iframe
-  src="cleaned_map.html"
+  src="assets/cleaned_map.html"
   width="100%"
   height="740"
   style="border:none;">
@@ -47,7 +47,7 @@ df_clean = C.transform(df_raw, trans)
 
 ### From raw signal...
 <iframe
-  src="example.html"
+  src="assets/example.html"
   width="100%"
   height="710"
   style="border:none;">
@@ -57,7 +57,7 @@ df_clean = C.transform(df_raw, trans)
 
 ### ... to clean signal
 <iframe
-  src="example_clean.html"
+  src="assets/example_clean.html"
   width="100%"
   height="710"
   style="border:none;">

diff --git a/docs/poster.md b/docs/poster.md
@@ -0,0 +1,11 @@
+A reproducible poster has been created for the EGU 2026 conference ([doi](https://doi.org/10.5194/egusphere-egu26-7777)).
+
+To reproduce the poster:
+```bash
+git clone https://github.com/oceanmodeling/ioc_cleanup.git
+cd ioc_cleanup
+pip install -r requirements/requirements.txt
+make poster
+```
+
+You can also consult it directly [here](https://tomsail.github.io/ioc_cleanup/assets/poster.html)
diff --git a/docs/tsunami.md b/docs/tsunami.md
@@ -0,0 +1,37 @@
+# Tsunami events
+
+Over the 2020 - 2026 period, major tsunamis have been registered and flagged in `ioc_cleanup`:
+
+## 2022 Tonga eruption
+[Wikipedia link](https://en.wikipedia.org/wiki/2022_Hunga_Tonga%E2%80%93Hunga_Ha%CA%BBapai_eruption_and_tsunami)
+
+<iframe
+  src="./assets/tonga_map.html"
+  width="100%"
+  height="430"
+  style="border:none;">
+</iframe>
+
+## 2025 Kamchatka earthquake
+[Wikipedia link](https://en.wikipedia.org/wiki/2025_Kamchatka_earthquake)
+
+
+Some X stations recorded the Kamchatka tsunami on the 30th July 2025.
+
+<iframe
+  src="./assets/kamchatka_map.html"
+  width="100%"
+  height="320"
+  style="border:none;">
+</iframe>
+
+Here is the distribution of waves heihgts recorded:
+
+and the highest wave (2.26m) recorded in Crescent city (California)
+
+<iframe
+  src="./assets/tsunami_detided.html"
+  width="100%"
+  height="710"
+  style="border:none;">
+</iframe>
diff --git a/docs/tsunami_map.html b/docs/tsunami_map.html