Merge remote-tracking branch 'origin/master' into fix_494

Author: mcarbonne

docker/example.hubspoke.docker-compose.yml

@@ -41,6 +41,9 @@ services:
     environment:
       COLLECTOR_API_ENDPOINT: 'http://web:8080'
       COLLECTOR_HOST_ID: 'scrutiny-collector-hostname'
+      # If true forces the collector to run on startup (cron will be started after the collector completes)
+      # see: https://github.com/AnalogJ/scrutiny/blob/master/docs/TROUBLESHOOTING_DEVICE_COLLECTOR.md#collector-trigger-on-startup
+      COLLECTOR_RUN_STARTUP: false
     depends_on:
       web:
         condition: service_healthy

docs/TESTERS.md

@@ -8,7 +8,7 @@ Thankfully the following users have been gracious enough to test/validate Scruti
 
 | Architecture Name | Binaries | Docker |
 | --- | --- | --- |
-| linux-amd64 | -- | @feroxy @rshxyz |
+| linux-amd64 | @TizzAmmazz | @feroxy @rshxyz |
 | linux-arm-5 | -- |  |
 | linux-arm-6 | -- |  |
 | linux-arm-7 | @Zorlin | @martini1992  |
@@ -17,4 +17,4 @@ Thankfully the following users have been gracious enough to test/validate Scruti
 | macos-amd64 | -- | -- |
 | macos-arm64 | -- | -- |
 | windows-amd64 | @gabrielv33 | -- |
-| windows-arm64 | -- | -- |
+| windows-arm64 | -- | -- |

docs/TROUBLESHOOTING_DEVICE_COLLECTOR.md

@@ -250,8 +250,9 @@ UPDATE devices SET device_status = null;
 
 ### Seagate Drives Failing
 
-As thoroughly discussed in [#255](https://github.com/AnalogJ/scrutiny/issues/255), Seagate (Ironwolf & others) drives are almost always marked as failed by Scrutiny. 
+As thoroughly discussed in [#255](https://github.com/AnalogJ/scrutiny/issues/255) and [#522](https://github.com/AnalogJ/scrutiny/issues/522), Seagate (Ironwolf & others) drives are almost always marked as failed by Scrutiny. 
 
+#### Seek Error Rate & Read Error Rate (#255)
 > The `Seek Error Rate` & `Read Error Rate` attribute raw values are typically very high, and the 
 > normalised values (Current / Worst / Threshold) are usually quite low. Despite this, the numbers in most cases are perfectly OK
 > 
@@ -286,6 +287,15 @@ other drives, please read the following:
 - http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html
 - https://www.truenas.com/community/threads/seagate-ironwolf-smart-test-raw_read_error_rate-seek_error_rate.68634/
 
+#### Seagate Raw Values are incorrect (#522)
+Basically Seagate drives are known to use a custom data format for a number of their SMART attributes. This causes issues with Scrutiny's threshold analysis. 
+
+- The workaround is to customize the smartctl command that Scrutiny uses for your drive by [creating a collector.yaml file](https://github.com/AnalogJ/scrutiny/issues/522#issuecomment-1766727988) and "fixing" each attribute
+- The **real "fix"** is to make sure your Seagate drive is correctly supported by smartmontools. See this [PR](https://github.com/smartmontools/smartmontools/pull/247)
+
+Sorry for the bad news, but this is a known issue and there's just not much we can do on the Scrutiny side. 
+
+
 ## Hub & Spoke model, with multiple Hosts.
 
 ![multiple-host-ids image](multiple-host-ids.png)

docs/TROUBLESHOOTING_DOCKER.md

@@ -17,3 +17,11 @@ So changing from `master-omnibus -> latest` will be the same thing for all inten
 > NOTE: Previously, there was a `automated cron build` that ran on the `master` and `beta` branches. 
 They used to trigger a `nightly` build, even if nothing has changed on the branch. This has a couple of benefits, but one is to 
 ensure that there's no broken external dependencies in our (unchanged) code. This `nightly` build no longer updates the `master-omnibus` tag. 
+
+# Running Docker `rootless`
+
+To avoid that the container(s) restart when you installed Docker as `rootless` you need to isssue the following commands to allow the session to stay alive even after you close your (SSH) sesssion:
+
+`sudo loginctl enable-linger $(whoami)`
+
+`systemctl --user enable docker`

docs/TROUBLESHOOTING_INFLUXDB.md

@@ -11,6 +11,32 @@ dependency. It's a dedicated timeseries database, as opposed to the general purp
 a bunch of testing and analysis before I made the change. With InfluxDB the memory footprint for Scrutiny (at idle) is ~
 100mb, which is still fairly reasonable.
 
+### Data Size
+
+It's surprisingly easy to reach extremely large database sizes, if you don't use downsampling, or you downsample incorrectly. 
+The growth rate is pretty unintuitive -- see https://github.com/AnalogJ/scrutiny/issues/650#issuecomment-2365174940 
+
+> Fasten stores the SMART metrics in a timeseries database (InfluxDB), and automatically downsamples the data on a schedule.
+>
+> The expectation was that cron would run daily, and there would be:
+>
+>    - 7 daily data points
+>    - 3 weekly data points
+>    - 11 monthly data points
+>    - and infinite yearly data points.
+>
+> These data points would be for each SMART metric, for each device.
+> eg. in one year, (7+3+11)*80ish SMART attributes = 1680 datapoints for one device
+>
+> If you're running cron every 15 minutes, your browser will instead be attempting to display:
+>
+>    - 96*7 daily data points
+>    - 3 weekly
+>    - 11 monthly
+>
+> so (96*7 + 3 + 11)*80 = 54,880 datapoints for each device 😭
+
+
 ## Installation
 
 InfluxDB is a required dependency for Scrutiny v0.4.0+.