Merge branch 'main' into dheath-boundary-ICU-18250

Author: Dan-Heath

content/hcp-docs/content/docs/vault-radar/get-started/add-data-sources/slack.mdx

@@ -0,0 +1,61 @@
+---
+page_title: Add a data source - Slack
+description: >-
+  Add Slack as a data source for Vault Radar.
+---
+
+# Onboard a Slack data source
+
+@include 'private-beta-feature.mdx'
+
+HCP Vault Radar allows you to connect to any workspace on [Slack
+](#onboard-slack) and scan for secrets in public channels.
+
+## Prerequisites
+
+- Permission in Slack to install third-party apps from the marketplace.
+
+   <Highlight title="Known limitations">
+
+   Vault Radar has a limit of 5000 channels per workspace. If you have more
+   than 5000 channels, Vault Radar selects the 5000 channels with the
+   most recent activity.
+
+   </Highlight>
+
+## Add a Slack Data source
+
+1. Click **Settings/Data Sources**.
+
+1. Select **HCP Vault Radar Scan**.
+
+1. Under **Team Collaboration**, click **Slack**.
+   ![Select Slack data source](/img/docs/vault-radar/slack-onboarding-flow.png)
+
+1. Click **Connect to Slack** to **install the Slack app**. This initiates
+   the OAuth flow with Slack.
+
+1. Select the **Slack workspace** from the dropdown list and click **Allow**
+   ![Install Slack App onto a workspace](/img/docs/vault-radar/install-slack-app-to-workspace.png)
+
+   <Note title="Permission required to install Slack app">
+
+   If you do not have permission to install apps into the selected workspace,
+   you will see a **Submit Request** button instead of the **Allow** button.
+
+   Coordinate with your Slack workspace administrator to approve the app
+   installation.
+
+   </Note>
+
+   HCP Vault Radar displays the message **Authenticated to Slack**.
+
+1. Click **Next**.
+
+1. Select either **All active channels** or **Select channels to monitor**.
+
+   The onboarding workflow joins the Slack app to the selected channels. If you
+   have a large number of channels, this process will take several minutes to
+   complete.
+
+1. Click **Finish** to start onboarding and scanning the selected channels.

content/hcp-docs/content/partials/private-beta-feature.mdx

@@ -0,0 +1,7 @@
+<Note title="Private beta feature">
+
+This feature is available as an _invitation only beta release_. Beta functionality is stable
+but possibly incomplete and subject to change. We strongly discourage using beta
+features in production.
+
+</Note>

content/hcp-docs/data/docs-nav-data.json

@@ -702,6 +702,10 @@
                     "path": "vault-radar/get-started/add-data-sources/jira/jira-data-center"
                   }
                 ]
+              },
+              {
+                "title": "Slack <sup>Beta</sup>",
+                "path": "vault-radar/get-started/add-data-sources/slack/"
               }
             ]
           },

content/hcp-docs/img/docs/vault-radar/install-slack-app-to-workspace.png

@@ -6,40 +6,76 @@ for versioned docs by comparing updates since a provided cutoff in the current
 
 The default cutoff date is the last run date for the provided product slug, if
 it exists. Otherwise, the script defaults to the creation date of the RC release
-branch. The script standardizes all timestamps to ISO for simplicity but takes
-the optional override date as a local time.
+branch, which it calculates as the date of the first commit associated with the
+branch that does not exist in main. The script standardizes all timestamps to
+UTC for simplicity but takes the optional override date as a local time.
 
 
 ## Assumptions
 
-- Your RC release branch use the following naming convention: `<product_slug>/<rc_version> <docTag>`.
+- Your RC folder uses the following naming convention: `<product_slug>/<rc_folder> <docTag>`.
 - You have the GitHub CLI (`gh`) installed. The CLI is required if you want the
   script to create a PR on your behalf. ==> DISABLED (THE PROCESS IS STILL BUGGY)
 
+## Usage
+
+```text
+node sync-ga-to-rc.mjs -slug <product> -ga <ga_folder> -rc <rc_folder> [<optional_flags>]
+```
 
 ## Flags
 
-Flag      | Type     | Default      | Description
---------- | -------- | ------------ | -----------
-`-slug`   | `string` | No, required | Product slug used for the root content folder
-`-ga`     | `string` | No, required | Version of the current docset
-`-rc`     | `string` | No, required | Version of the unreleased docset
-`-tag`    | `string` | ""           | String used to tag non-GA docsets (e.g., "(rc)")
-`-branch` | `string` | `main`       | Name of the GA branch
-`-date`   | `string` | null         | Local override date in "YYYY-MM-DD HH:MM:SS" format for the commit date cutoff
-`-update` | `bool`   | false        | Indicates whether to apply any safe changes locally
-`-pr`     | `bool`   | false        | Indicates whether to apply any safe changes locally and generate a PR if possible
-`-merged` | `bool`   | false        | Indicates that RC docs are merged to `-branch`
-`-help`   | `bool`   | false        | Print usage help text and exit
+Flag        | Type     | Default                      | Description
+----------- | -------- | ---------------------------- | -----------
+`-slug`     | `string` | None, required               | Product slug used for the root content folder
+`-ga`       | `string` | None, required               | GA folder; typically the GA version with `.x`
+`-rc`       | `string` | None, required               | Unreleased docset folder ; typically the RC version with `.x`
+`-tag`      | `string` | ""                           | String used to tag non-GA docsets (e.g., "(rc)")
+`-branch`   | `string` | `<product_slug>/<rc_folder>` | Name of the RC branch
+`-gaBranch` | `string` | `main`                       | Name of the GA branch
+`-date`     | `string` | null                         | Local override date in "YYYY-MM-DD HH:MM:SS" format for the commit date cutoff
+`-update`   | `bool`   | false                        | Indicates whether to apply any safe changes locally
+`-pr`       | `bool`   | false                        | Indicates whether to apply any safe changes locally and generate a PR if possible
+`-merged`   | `bool`   | false                        | Indicates that RC docs are merged to `-gaBranch`
+`-help`     | `bool`   | false                        | Print usage help text and exit
 
 
+## Adding exceptions
 
-## Usage
+If you have files you **know** the script should always ignore, you can add the
+relative path from your product root to the exclusion file `data/exclude.json`
+using your product slug.
 
-```text
-node sync-ga-to-rc.mjs -slug <product> -ga <ga_version> -rc <rc_version> [-tag <folder_tag>] [-branch <ga_branch>] [-date <override_date] [-update] [-pr]
+Expected schema:
+
+```json
+[
+  {
+    "<produc_slug>": [
+      "<relative_path_1>",
+      "<relative_path_1>",
+      ...
+      "<relative_path_N>",
+    ]
+  }
+]
+```
+
+For example:
+
+```json
+[
+  {
+    "vault": [
+      "/content/docs/updates/important-changes.mdx",
+      "/content/docs/updates/release-notes.mdx",
+      "/content/docs/updates/change-tracker.mdx"
+    ]
+  }
+]
 ```
 
+
 ## Examples
 
 ### Basic call
@@ -48,7 +84,13 @@ node sync-ga-to-rc.mjs -slug <product> -ga <ga_version> -rc <rc_version> [-tag <
 $ node sync-ga-to-rc.mjs -slug vault -ga 1.20.x -rc 1.21.x -tag rc
 ```
 
-### Provide an override date
+### Provide an explicit release branch name
+
+```shell-session
+$ node sync-ga-to-rc.mjs -slug boundary -ga 0.20.x -rc 0.21.x -branch boundary/0.21.0
+```
+
+### Provide an override date and explicit tag string
 
 Use `-tag` to provide a specific doc tag and set a custom override date with
 `-date`:
@@ -76,52 +118,16 @@ $ node sync-ga-to-rc.mjs  \
 
 ### Sync two published versions
 
-Use `-merged` and set `-tag` to "none" so the script compares folders for past
-versions in `main`:
+Use `-merged` so the script compares folders in `main`:
 
 ```shell-session
 $ node sync-ga-to-rc.mjs  \
     -slug vault           \
     -ga 1.19.x            \
     -rc 1.20.x            \
-    -merged               \
-    -tag none
-```
-
-## Adding exceptions
-
-If you have files you **know** the script should always ignore, you can add the
-relative path from your product root to the exclusion file `data/exclude.json`
-using your product slug.
-
-Expected schema:
-
-```json
-[
-  {
-    "<produc_slug>": [
-      "<relative_path_1>",
-      "<relative_path_1>",
-      ...
-      "<relative_path_N>",
-    ]
-  }
-]
+    -merged
 ```
 
-For example:
-
-```json
-[
-  {
-    "vault": [
-      "/content/docs/updates/important-changes.mdx",
-      "/content/docs/updates/release-notes.mdx",
-      "/content/docs/updates/change-tracker.mdx"
-    ]
-  }
-]
-```
 
 ## General workflow
 
@@ -133,27 +139,33 @@ Next, the script builds the following file sets:
 - exclusions  - a list of files the script should ignore during the sync
 - GAΔ         - files in the GA (current) docset with a last commit date later
                 than the provided cutoff date.
+                than the provided cutoff date.
 - RCΔ         - files in the RC (unreleased) docset with a last commit date
                 later than the provided cutoff date.
 - GA-only     - files in the GA (current) docset that do not exist in the RC
                 docset.
+- GAd         - files deleted from the GA (current) docset that still exist in
+                the RC (unreleased) docset
 
 The script determines what to do with the files based on the following rubric
 where GAu and RCu are the set of files unchanged since the cutoff in the GA and
 RC docsets:
 
-Set definition       | Implication        | Action
--------------------- | ------------------ | -------------------------
-file ∈ { RCu ∧ GAu } | file unchanged     | ignore 
-file ∈ { RCu ∧ GAΔ } | updated in GA only | safe to update in RC
-file ∈ { RCΔ ∧ GAu } | updated in RC only | ignore
-file ∈ { RCΔ ∧ GAΔ } | updated in both    | possible conflict; needs manual review
-file ∈ { RC  ∧ !GA } | new file for RC    | ignore 
-file ∈ { !RC ∧  GA } | new file for GA    | safe to update in RC
-
-If `-update` is `true`, the script slams files in the RC folder with files from
-the GA folder with any file deemed "safe", prints a note to review the
-information in the conflict file, and updates the last run date.
+Set definition       | Implication          | Action
+-------------------- | -------------------- | -------------------------
+file ∈ { RCu ∧ GAu } | file unchanged       | ignore 
+file ∈ { RCu ∧ GAΔ } | updated in GA only   | safe to update in RC
+file ∈ { RCΔ ∧ GAu } | updated in RC only   | ignore
+file ∈ { RCΔ ∧ GAΔ } | updated in both      | possible conflict; needs manual review
+file ∈ { RC  ∧ !GA } | new file for RC      | ignore 
+file ∈ { !RC ∧ GA }  | new file for GA      | safe to update in RC
+file ∈ { RC  ∧ GAd } | file deleted from GA | safe to delete in RC
+
+If `-update` is `true`, the script creates a working branch, changes to that
+branch, slams files in the RC folder with files from the GA folder with any file
+deemed "safe", deletes files in RC that show up as deleted in the gitlog for GA,
+prints a note to review the information in the conflict file, and updates the
+last run date.
 
 If `-update` is `false`, the script generates log files and exits.
 
@@ -171,6 +183,7 @@ but it also creates the following artifacts:
     File set            | Output file
     ------------------- | --------------------
     GAΔ                 | output/ga-delta.txt
+    GAd                 | output/delete-list.txt
     RCΔ                 | output/rc-delta.txt
     GA-only             | output/ga-only.txt
     updated files       | output/safe-list.txt

content/hcp-docs/img/docs/vault-radar/slack-onboarding-flow.png

@@ -20,6 +20,7 @@
 myDir=$(pwd)
 repoName="web-unified-docs"
 localReposDir=${myDir%"/${repoName}"*}
+outputDir="${myDir}/output"
 
 repoRoot="${localReposDir}/${repoName}"  # Local root directory of the repo
 docRoot="${repoRoot}/content/<PRODUCT>"  # Root directory of product docs
@@ -32,6 +33,30 @@ rcDocs=""   # Set in helper from command line arguments; for example, "${docRoot
 gaDocs=""   # Set in helper from command line arguments; for example, "${docRoot}/v1.20.x"
 
 jsonTemplate='{"file": "<FILENAME>", "shortname": "<SHORTNAME>", "commit": "<COMMIT>"}'
-prBranch="bot/<PRODUCT>-ga-to-rc-sync-$(date +%Y%m%d)"
+prBranch="bot/<PRODUCT>-ga-to-rc-sync-$(date +%Y%m%d-%H%M%S)"
 prTitle="<PRODUCT> GA to RC auto-sync"
 prBody="Draft PR created by \`sync-ga-to-rc.mjs\` to push recent GA updates to the RC release branch for <PRODUCT>"
+
+
+# Helper function to convert an ISO time string to UTC
+#
+function getUTCDate {
+ 
+  local dateString="${1}"
+  local myShell="${SHELL}"
+  local zBash="/bin/zsh"
+  local uBash="/bin/bash"
+  local unixTime
+
+  # Bail if any of the command line parameters were omitted
+  if [[ -z "${dateString}" ]] ; then return; fi
+
+  # The date command in zbash (standard shell for MacOS) is wildly different
+  # from standard bash, so we convert differently based on the shell
+  if [[ "${myShell}" == "${zBash}" ]] ; then
+    unixTime=$(date -j -f '%Y-%m-%d %H:%M:%S %z' "${dateString}" +'%s')
+    echo $(date -j -u -r ${unixTime} +'%Y-%m-%d %H:%M:%S')
+  else
+    echo $(date -u  +'%Y-%m-%d %H:%M:%S' -d "${dateString}")
+  fi
+}

scripts/sync-ga-to-rc/README.md

@@ -0,0 +1,44 @@
+# 
+# Copyright (c) HashiCorp, Inc.
+# SPDX-License-Identifier: BUSL-1.1
+# 
+# ------------------------------------------------------------------------------
+# Delete RC docs
+#
+# For every relative path in the input list, delete the RC version if it exists
+#
+# Expected usage: delete-rc-docs.sh <productKey> <gaFolder> <rcFolder> <deleteListFile>
+# Example:        delete-rc-docs.sh vault '1.20.x' '1.21.x (rc)' 'delete-list.txt'
+
+# Pull in the common variable definitions
+currDir="$(dirname "$0")"
+. "${currDir}/definitions.sh"
+
+# Set variables from command line argument
+productKey="${1}"  # root folder for product docs (product key)
+gaFolder="${2}"    # GA doc folder name
+rcFolder="${3}"    # RC doc folder name
+deleteList="${4}"  # file of GA paths we can overwrite in RC
+
+# Bail if any of the command line parameters were omitted
+if [[ -z "${productKey}" ]] ; then exit ; fi
+if [[ -z "${gaFolder}" ]]   ; then exit ; fi
+if [[ -z "${rcFolder}" ]]   ; then exit ; fi
+if [[ -z "${deleteList}" ]] ; then exit ; fi
+
+cd "${repoRoot}"
+
+while read line; do
+
+  # Grab the filename and generate the cooresponding RC path
+  gaPath=$(echo "${line}" | awk -F " " '{print $3}')
+  rcPath=${gaPath/${gaFolder}/${rcFolder}}
+
+  # Skip any file that may have ended up in the list from a different product
+  if [[ "${gaPath}" != *"/content/${productKey}/"* ]]; then continue ; fi
+  if [[ "${rcPath}" != *"/content/${productKey}/"* ]]; then continue ; fi
+
+  # If the file exists in the RC folder, delete it
+  if [[ -f "${rcPath}" ]] ; then  rm -r "${rcPath}" ; fi
+
+done < "${outputDir}/${deleteList}"