diff --git a/content/integrate/redis-data-integration/data-pipelines/deploy.md b/content/integrate/redis-data-integration/data-pipelines/deploy.md index ff33b5b12e..d126291b45 100644 --- a/content/integrate/redis-data-integration/data-pipelines/deploy.md +++ b/content/integrate/redis-data-integration/data-pipelines/deploy.md @@ -22,9 +22,9 @@ The sections below explain how to deploy a pipeline after you have created the r ## Set secrets Before you deploy your pipeline, you must set the authentication secrets for the -source and target databases. Each secret has a name that you can pass to the +source and target databases. Each secret has a name that you pass to the [`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) -command (VM deployment) or the `rdi-secret.sh` script (K8s deployment) to set the secret value. +command to set the secret value. You can then refer to these secrets in the `config.yaml` file using the syntax "`${SECRET_NAME}`" (the sample [config.yaml file]({{< relref "/integrate/redis-data-integration/data-pipelines/pipeline-config#example" >}}) @@ -53,10 +53,10 @@ secrets are only relevant for TLS/mTLS connections. {{< embed-md "rdi-tls-secrets.md" >}} {{< /note >}} -### Set secrets for VM deployment +### Set secrets with the CLI Use [`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) -to set secrets for a VM deployment. +to set secrets for any installation type (VM, Kubernetes, or Redis Cloud). The specific command lines for source secrets are as follows: @@ -92,62 +92,42 @@ redis-di set-secret TARGET_DB_KEY /path/to/myclient.key redis-di set-secret TARGET_DB_KEY_PASSWORD yourKeyPassword ``` -### Set secrets for K8s/Helm deployment using the rdi-secret.sh script +By default, `set-secret` waits for the pipeline to apply the change before returning. When you set +several secrets at once, set all but the last one with `--wait=false` to avoid a timeout while the +pipeline is only partially updated. See [Wait for changes to complete](#wait) below for details. -Use the `rdi-secret.sh` script to set secrets for a K8s/Helm deployment. To use this script, unzip the archive that contains the RDI Helm chart and navigate to the resulting folder. The `rdi-secret.sh` script is located in the `scripts` subfolder. The general pattern for using this script is: +### Manage secrets with the CLI -```bash -scripts/rdi-secret.sh set -``` - -The script also lets you retrieve a specific secret or list all the secrets that have been set: +Along with `set-secret`, the CLI has commands to list, inspect, and delete secrets. Because the API +never returns secret values, these commands show only the secret keys and whether they are set, not +the stored values. ```bash -# Get specific secret -scripts/rdi-secret.sh get - -# List all secrets -scripts/rdi-secret.sh list -``` +# List all the secrets of a pipeline and whether each one is set +redis-di list-secrets -The specific command lines for source secrets are as follows: +# Show a single secret and whether it is set +redis-di describe-secret SOURCE_DB_PASSWORD -```bash -# For username and password -scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername -scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword - -# With source TLS, in addition to the above -scripts/rdi-secret.sh set SOURCE_DB_CACERT /path/to/myca.crt - -# With source mTLS, in addition to the above -scripts/rdi-secret.sh set SOURCE_DB_CERT /path/to/myclient.crt -scripts/rdi-secret.sh set SOURCE_DB_KEY /path/to/myclient.key -# Use this only if SOURCE_DB_KEY is password-protected -scripts/rdi-secret.sh set SOURCE_DB_KEY_PASSWORD yourKeyPassword +# Delete a secret (prompts for confirmation unless you add --force) +redis-di delete-secret SOURCE_DB_CACERT ``` -The corresponding command lines for target secrets are: - -```bash -# For username and password -scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername -scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword - -# With target TLS, in addition to the above -scripts/rdi-secret.sh set TARGET_DB_CACERT /path/to/myca.crt - -# With target mTLS, in addition to the above -scripts/rdi-secret.sh set TARGET_DB_CERT /path/to/myclient.crt -scripts/rdi-secret.sh set TARGET_DB_KEY /path/to/myclient.key -# Use this only if TARGET_DB_KEY is password-protected -scripts/rdi-secret.sh set TARGET_DB_KEY_PASSWORD yourKeyPassword -``` +See the reference pages for +[`list-secrets`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-secrets" >}}), +[`get-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-secret" >}}), +[`describe-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-secret" >}}), +and [`delete-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete-secret" >}}) +for the full list of options. ### Set secrets for K8s/Helm deployment using Kubectl command -In some scenarios, you may prefer to use [`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) -to set secrets for a K8s/Helm deployment. The general pattern of the commands is: +{{< note >}}It is strongly recommended to manage secrets with the `redis-di` CLI rather than with +`kubectl` directly. The CLI applies the correct labels automatically, validates the secret keys, and +works the same way across all installation types.{{< /note >}} + +For a Kubernetes/Helm deployment, you can also use [`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/) +to set secrets instead of the CLI. The general pattern of the commands is: ```bash kubectl create secret generic \ @@ -247,16 +227,158 @@ kubectl create secret generic target-db-ssl --namespace=rdi \ Note that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets. +Secrets that you create directly with `kubectl` must also be labeled so that the RDI operator +discovers them as pipeline secrets. Each secret needs the following labels, where the +`app.kubernetes.io/instance` label is the pipeline name (`default` for the default pipeline): + +| Label | Value | +| :-- | :-- | +| `app.kubernetes.io/name` | `pipeline` | +| `app.kubernetes.io/instance` | `default` | +| `product` | `rdi` | + +Apply the labels to each secret with [`kubectl label`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_label/): + +```bash +kubectl label secret source-db --namespace=rdi --overwrite \ + app.kubernetes.io/name=pipeline \ + app.kubernetes.io/instance=default \ + product=rdi +kubectl label secret target-db --namespace=rdi --overwrite \ + app.kubernetes.io/name=pipeline \ + app.kubernetes.io/instance=default \ + product=rdi + +# With source TLS or mTLS +kubectl label secret source-db-ssl --namespace=rdi --overwrite \ + app.kubernetes.io/name=pipeline \ + app.kubernetes.io/instance=default \ + product=rdi + +# With target TLS or mTLS +kubectl label secret target-db-ssl --namespace=rdi --overwrite \ + app.kubernetes.io/name=pipeline \ + app.kubernetes.io/instance=default \ + product=rdi +``` + ## Deploy a pipeline When you have created your configuration, including the [jobs]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}), you are -ready to deploy. Use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) -to configure and deploy pipelines for both VM and K8s installations. - -For VM installations, you can also use the +ready to deploy. Use the [`redis-di deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}}) command to deploy a pipeline: ```bash redis-di deploy --dir -``` \ No newline at end of file +``` + +RDI first validates the configuration and then deploys it if it is correct. You can control the +validation and what happens after deployment with the following options: + +- `--dry-run`: Validate the configuration without deploying it. Off by default. +- `--validate-tables`: Validate the configuration against the source and target databases, for + example that the tables it references exist. On by default; pass `--validate-tables=false` to skip + this check, which is useful when the databases are not reachable at deploy time. +- `--validate-cdc`: Additionally validate that the source database is correctly configured for + [change data capture (CDC)]({{< relref "/integrate/redis-data-integration/architecture#overview" >}}). + Off by default; enable it with `--validate-cdc`. +- `--start`: Start the pipeline as soon as it is deployed. On by default; pass `--start=false` to + deploy the pipeline without starting it, then start it later with + [`redis-di start`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-start" >}}). + +See the [`redis-di deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}}) +reference page for the full list of options. + +You can also use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) +to configure and deploy pipelines for both VM and K8s installations. + +## Display the pipeline status + +Once a pipeline is deployed, use the +[`redis-di describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) +command (also available as `redis-di status`) to display its status. This combines the pipeline +configuration with its runtime status, showing its overall state, its sources and targets, its jobs +and components, and its per-stream statistics and performance metrics. + +```bash +redis-di describe +``` + +To watch the status update live, pair the command with `watch`: + +```bash +watch -n 1 redis-di describe +``` + +For a shorter overview, [`redis-di list`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list" >}}) +prints a one-line summary of the pipeline, and +[`redis-di get`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get" >}}) +does the same for a single pipeline. See the +[`redis-di describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) +reference page for details. + +## Start and stop a pipeline + +Use [`redis-di stop`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-stop" >}}) +to pause a running pipeline and +[`redis-di start`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-start" >}}) +to resume it. Stopping a pipeline halts data processing without deleting the pipeline or its +configuration, so you can start it again later from where it left off. + +```bash +redis-di stop +redis-di start +``` + +## Reset a pipeline + +Use [`redis-di reset`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-reset" >}}) +to return a pipeline to initial full-sync mode. This reloads a fresh +[snapshot]({{< relref "/integrate/redis-data-integration/architecture#overview" >}}) of the source +data and then resumes change data capture (CDC), which is useful when the source and target have +drifted out of sync. + +```bash +redis-di reset +``` + +## Undeploy a pipeline + +To remove a pipeline, use the +[`redis-di delete`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete" >}}) +command. This stops the pipeline and deletes it, along with its configuration and status, from RDI. +The secrets you set for the pipeline are not affected. + +```bash +redis-di delete +``` + +Because deleting a pipeline is destructive, the command asks for confirmation unless you add the +`--force` option. If you omit the pipeline name, the `default` pipeline is deleted. + +## Wait for changes to complete {#wait} + +The commands that change a pipeline's state, namely `deploy`, `delete`, `start`, `stop`, `reset`, +`set-secret`, and `delete-secret`, do not return as soon as the API accepts the request. By default, +they wait for the pipeline to finish transitioning to the expected state, polling its status until it +succeeds, reaches an error, or the `--timeout` (2 minutes by default) elapses. This is usually what +you want: the command reflects the real outcome, so a script can rely on the change having taken +effect and can fail fast if it did not. + +In some cases, though, a pipeline needs *several* changes before it can transition to a healthy state, +and waiting after each individual change would time out. The clearest example is rotating both the +username and the password of a database: if you set only the username with the default `--wait=true`, +the pipeline tries to reconnect with the new username and the old password, fails, and the command +times out after two minutes with the pipeline in a broken state. + +To avoid this, set all the related secrets, or at least all of them except the last, with +`--wait=false`, so the pipeline applies them together and only the final command waits for it to +become healthy: + +```bash +redis-di set-secret SOURCE_DB_USERNAME newUsername --wait=false +redis-di set-secret SOURCE_DB_PASSWORD newPassword +``` + +The same applies to any set of changes that are only valid together. \ No newline at end of file diff --git a/content/integrate/redis-data-integration/data-pipelines/rejected-records.md b/content/integrate/redis-data-integration/data-pipelines/rejected-records.md index a17eb440ce..73a8c50b92 100644 --- a/content/integrate/redis-data-integration/data-pipelines/rejected-records.md +++ b/content/integrate/redis-data-integration/data-pipelines/rejected-records.md @@ -108,8 +108,10 @@ for Redis Cloud, or use the appropriate self-managed RDI reset workflow. ## CLI and API access -For self-managed RDI, use the [`redis-di get-rejected`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-rejected" >}}) -command to inspect rejected records. +For self-managed RDI, use the [`redis-di list-dlqs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs" >}}) +command to see the dead-letter queues and the +[`redis-di list-dlq-records`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records" >}}) +command (also available as `redis-di get-rejected`) to inspect the rejected records of a queue. For Redis Cloud RDI, connect to the RDI database and inspect the corresponding DLQ streams directly when you need details that are not shown in the Redis Cloud diff --git a/content/integrate/redis-data-integration/faq.md b/content/integrate/redis-data-integration/faq.md index 4ef0779a5f..a7be7422bb 100644 --- a/content/integrate/redis-data-integration/faq.md +++ b/content/integrate/redis-data-integration/faq.md @@ -100,7 +100,7 @@ job then RDI can't transform the data. When this happens, RDI will store the ori in a "dead letter queue" along with a message to say why it was rejected. The dead letter queue is stored as a capped stream in the RDI staging database. You can see its contents with Redis Insight or with the -[`redis-di get-rejected`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-rejected" >}}) +[`redis-di list-dlq-records`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records" >}}) command from the CLI. See [Rejected records]({{< relref "/integrate/redis-data-integration/data-pipelines/rejected-records" >}}) for more information about DLQ. diff --git a/content/integrate/redis-data-integration/installation/install-k8s.md b/content/integrate/redis-data-integration/installation/install-k8s.md index a54ee5a3f7..64cc5fa11f 100644 --- a/content/integrate/redis-data-integration/installation/install-k8s.md +++ b/content/integrate/redis-data-integration/installation/install-k8s.md @@ -64,6 +64,8 @@ Complete the following steps before installing the RDI Helm chart: - If you want to use a private image registry, [prepare it with the RDI images](#using-a-private-image-registry). +- [Download the RDI CLI](#download-the-rdi-cli), which you use to deploy and manage pipelines. + ### Create the RDI database RDI uses a database on your Redis Enterprise cluster to store its state @@ -146,6 +148,41 @@ To pull images from a private image registry, you must provide the image pull se - [Google Kubernetes Engine (GKE)](https://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images) - [Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?tabs=azure-cli) +### Download the RDI CLI + +You manage RDI with the [`redis-di` CLI]({{< relref "/integrate/redis-data-integration/reference/cli" >}}), +which you use to deploy pipelines, set secrets, and inspect status. Unlike the VM installation, which +bundles the CLI, a Kubernetes installation requires you to download it separately from the Redis +download center. + +The CLI is currently built for the following platforms. Download the binary that matches the operating +system and architecture of the machine you will run it from. On Linux and macOS, if you are not sure +which to choose, run `uname -sm`: `Linux x86_64` is Linux amd64, `Linux aarch64` is Linux arm64, +`Darwin x86_64` is macOS on Intel, and `Darwin arm64` is macOS on Apple silicon. + +| Platform | Download | +| :-- | :-- | +| Linux, x86-64 (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/linux-amd64/redis-di) | +| Linux, ARM64 (aarch64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/linux-arm64/redis-di) | +| macOS, Intel (amd64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/darwin-amd64/redis-di) | +| macOS, Apple silicon (arm64) | [`redis-di`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/darwin-arm64/redis-di) | +| Windows, x86-64 (amd64) | [`redis-di.exe`](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/{{< rdi-version >}}/bin/windows-amd64/redis-di.exe) | + +For example, to download the CLI for Linux amd64, make it executable, and put it on your `PATH`: + +```bash +export RDI_VERSION={{< rdi-version >}} +wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/cli/$RDI_VERSION/bin/linux-amd64/redis-di +chmod +x redis-di +sudo mv redis-di /usr/local/bin/ +``` + +{{< note >}}The macOS and Windows binaries are not currently signed or notarized, so the operating +system may block them the first time you run them. On macOS, allow the binary to run in +**System Settings > Privacy & Security**, or remove the quarantine attribute with +`xattr -d com.apple.quarantine ./redis-di`, and then run it again. On Windows, if Microsoft Defender +SmartScreen blocks it, choose **More info > Run anyway**.{{< /note >}} + ## Supported versions of Kubernetes and OpenShift {{< embed-md "rdi-k8s-reqs.md" >}} @@ -350,12 +387,13 @@ kubectl get pod -n rdi NAME READY STATUS RESTARTS AGE collector-api- 1/1 Running 0 29m rdi-api- 1/1 Running 0 29m -rdi-metric-exporter- 1/1 Running 0 29m rdi-operator- 1/1 Running 0 29m rdi-reloader- 1/1 Running 0 29m ``` -You can verify that the RDI API works by adding a connection to the RDI API server to +You can verify that the RDI API works by running +[`redis-di info`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-info" >}}) +against it, or by adding a connection to the RDI API server to [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}). ## Using ingress controllers @@ -390,13 +428,13 @@ section to learn how to do this. ## Deploy a pipeline When the Helm installation is complete and you have prepared the source database for CDC, -you are ready to start using RDI. -Use [Redis Insight]({{< relref "/develop/tools/insight" >}}) to +you are ready to start using RDI. See the guides on how to [configure]({{< relref "/integrate/redis-data-integration/data-pipelines" >}}) and [deploy]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy" >}}) -your pipeline (see +RDI pipelines for more information. You can also configure and deploy a pipeline +using [Redis Insight]({{< relref "/develop/tools/insight" >}}). See [RDI in Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) -for full details on how to do this). +for full details on how to connect to RDI and deploy pipelines. ## Uninstall RDI diff --git a/content/integrate/redis-data-integration/installation/install-vm.md b/content/integrate/redis-data-integration/installation/install-vm.md index 8cbd20bef1..5ee2aed2b3 100644 --- a/content/integrate/redis-data-integration/installation/install-vm.md +++ b/content/integrate/redis-data-integration/installation/install-vm.md @@ -288,6 +288,12 @@ using [Redis Insight]({{< relref "/develop/tools/insight" >}}). See [RDI in Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) for full details on how to connect to RDI and deploy pipelines. +{{< note >}}The [`redis-di` CLI]({{< relref "/integrate/redis-data-integration/reference/cli" >}}) +is bundled with the VM installer, so it is already available on the VM where RDI is installed. If you +prefer to run it from your own laptop or desktop instead, you can +[download it separately]({{< relref "/integrate/redis-data-integration/installation/install-k8s#download-the-rdi-cli" >}}) +for your platform.{{< /note >}} + ## Uninstall RDI If you want to remove your RDI installation, go to the installation folder and run diff --git a/content/integrate/redis-data-integration/installation/upgrade.md b/content/integrate/redis-data-integration/installation/upgrade.md index 687c1467f4..8e8d8eca6d 100644 --- a/content/integrate/redis-data-integration/installation/upgrade.md +++ b/content/integrate/redis-data-integration/installation/upgrade.md @@ -51,7 +51,10 @@ If the previous version is v1.4.4 or later, go to the `rdi_install/}}If you don't use Prometheus or Grafana, you can still see RDI metrics with the RDI monitoring screen in Redis Insight or with the -[`redis-di status`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-status" >}}) +[`redis-di describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) command from the CLI.{{< /note >}} ## Accessing the metrics diff --git a/content/integrate/redis-data-integration/quick-start-guide.md b/content/integrate/redis-data-integration/quick-start-guide.md index 31b480da69..44134d605f 100644 --- a/content/integrate/redis-data-integration/quick-start-guide.md +++ b/content/integrate/redis-data-integration/quick-start-guide.md @@ -75,33 +75,32 @@ At this point, the pipeline is ready to deploy. To manage and inspect RDI, you can use the [`redis-di`]({{< relref "/integrate/redis-data-integration/reference/cli" >}}) -CLI command, which has several subcommands for different purposes. Most of these commands require you -to pass at least two options, `--rdi-host` and `--rdi-port`, to specify the host and port of your -RDI installation. You can avoid typing these options repeatedly by saving the -information in a *context*. +CLI tool, which has several commands for different purposes. Most of these commands connect to +the RDI API, which you specify with the `--api-url` option. You can avoid typing this and the other +connection options repeatedly by saving them in a *context*. -When you activate a context, the saved values of -`--rdi-host`, `--rdi-port`, and a few other options are passed automatically whenever +When you activate a context, its saved connection options are used automatically whenever you use `redis-di`. If you have more than one RDI installation, you can create a context for each of them and select the one you want to be active using its unique name. To create a context, use the -[`redis-di add-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-add-context" >}}) -command: +[`redis-di set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}) +command. For a VM installation, the API has the same hostname or IP address as your RDI VM and uses +the default HTTPS port 443: ```bash -redis-di add-context --rdi-host --rdi-port +redis-di set-context --api-url https:// --user ``` -These options are required but there are also a few others you can save, such as TLS credentials, if -you are using them (see the -[reference page]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-add-context" >}}) +You can save a few other options, such as a CA certificate (`--cacert`) if the API uses a private +certificate (see the +[reference page]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}) for details). When you have created a context, use -[`redis-di set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}) +[`redis-di use-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-use-context" >}}) to activate it: ```bash -redis-di set-context +redis-di use-context ``` There are also subcommands to @@ -111,19 +110,21 @@ contexts. ### Deploy the pipeline -You can use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) -to deploy the pipeline by adding a connection to the RDI API -endpoint (which has the same hostname or IP address as your RDI VM and uses the default HTTPS port 443) and then clicking the **Deploy** button. You can also deploy it with the following command: +You can deploy the pipeline with the following command: ```bash redis-di deploy --dir ``` where the path is the one you supplied earlier during the installation. (You may also need -to supply `--rdi-host` and `--rdi-port` options if you are not using a +to supply the `--api-url` option if you are not using a [context](#create-context) as described above.) RDI first validates your pipeline and then deploys it if the configuration is correct. +You can also use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}}) +to deploy the pipeline, by adding a connection to the RDI API +endpoint (which has the same hostname or IP address as your RDI VM and uses the default HTTPS port 443) and then clicking the **Deploy** button. + Once the pipeline is running, you can use Redis Insight to view the data flow using the pipeline metrics. You can also connect to your target database to see the keys that RDI has written there. @@ -146,6 +147,7 @@ To see the RDI pipeline working in CDC mode: (see [Generating load on the database](https://github.com/Redislabs-Solution-Architects/rdi-quickstart-postgres?tab=readme-ov-file#generating-load-on-the-database) to learn how to do this). - Run - [`redis-di status --live`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-status" >}}) - to see the flow of records. + [`redis-di describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) + to see the flow of records. To watch it update live, pair the command with `watch`, for example + `watch -n 1 redis-di describe`. - Use [Redis Insight]({{< relref "/develop/tools/insight" >}}) to look at the data in the target database. diff --git a/content/integrate/redis-data-integration/reference/cli/_index.md b/content/integrate/redis-data-integration/reference/cli/_index.md index e9554b86f1..242bc39c24 100644 --- a/content/integrate/redis-data-integration/reference/cli/_index.md +++ b/content/integrate/redis-data-integration/reference/cli/_index.md @@ -1,6 +1,11 @@ --- Title: CLI reference -aliases: /integrate/redis-data-integration/ingest/reference/cli/ +aliases: + - /integrate/redis-data-integration/ingest/reference/cli/ + - /integrate/redis-data-integration/reference/cli/redis-di-install/ + - /integrate/redis-data-integration/reference/cli/redis-di-upgrade/ + - /integrate/redis-data-integration/reference/cli/redis-di-monitor/ + - /integrate/redis-data-integration/reference/cli/redis-di-trace/ alwaysopen: false categories: - docs @@ -17,3 +22,106 @@ summary: type: integration weight: 60 --- + +`redis-di` is the command line tool that manages Redis Data Integration (RDI). +It is a thin client over the RDI REST API, so it works the same way for all +installation types: [VM]({{< relref "/integrate/redis-data-integration/installation/install-vm" >}}), +[Kubernetes]({{< relref "/integrate/redis-data-integration/installation/install-k8s" >}}), and Redis Cloud. +Use it to deploy pipelines, manage secrets, inspect status and metrics, and read rejected records. + +## Connecting to the API + +Most commands connect to the RDI API, which you specify with the `--api-url` option (or the +`RDI_API_URL` environment variable). Because the API is served over HTTPS, you can also supply +`--cacert` to trust a private or self-signed certificate, or `--insecure` to skip TLS verification. + +The CLI supports three authentication modes, selected by the credentials you provide: + +- **User authentication** (JWT): when you set a `--user`, the CLI logs in with that user and a + password from `--password`, the `RDI_PASSWORD` environment variable, or an interactive prompt. + This is the usual mode for VM and Kubernetes installations. +- **Redis Cloud authentication**: when you set an `--account-key`, the CLI authenticates to the + Redis Cloud API gateway with that account key and a user key from `--user-key`, the `RDI_USER_KEY` + environment variable, or an interactive prompt. This is the mode for RDI running in Redis Cloud. +- **No authentication**: when you set neither a user nor an account key, the CLI connects without + authenticating, which is the mode to use when authentication is disabled in the API. + +Setting both `--user` and `--account-key` is an error, as is setting both `--cacert` and `--insecure`. +Passwords and user keys are secrets and are never stored on disk. + +## Contexts + +Instead of passing the connection options on every command, you can save them in a *context*. +Contexts are stored in a `~/.redis-di` file that holds a map of named contexts and the active one, +similar to a `kubeconfig` file. Each context sets an `api-url`, an optional `user` or `account-key`, +and either a `cacert` or `insecure: true`. Secrets (the password and user key) are never stored, so +you still supply them per session. + +```yaml +# ~/.redis-di +current-context: prod +contexts: + prod: + api-url: https://rdi.example.com + user: default + cacert: /etc/rdi/ingress-ca.crt + dev: + api-url: https://localhost:8443 + insecure: true +``` + +Use the [`set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}) +and [`use-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-use-context" >}}) +commands to create and select contexts rather than editing the file by hand. + +## Commands + +Pipeline-scoped commands take the pipeline name as an optional positional argument that defaults to +`default`, for example `redis-di start [pipeline]`. Sub-resource commands (for a secret, DLQ, or job) +take their own key or name as the positional argument and target the pipeline with the `-p` / `--pipeline` +option, which also defaults to `default`. + +The commands group as follows: + +- **Information**: [`info`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-info" >}}). +- **Pipelines**: [`list`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list" >}}), + [`get`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get" >}}), + [`describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) (alias `status`), + [`deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}}) (alias `set`), + [`delete`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete" >}}), + [`start`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-start" >}}), + [`stop`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-stop" >}}), and + [`reset`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-reset" >}}). +- **Secrets**: [`list-secrets`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-secrets" >}}), + [`get-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-secret" >}}), + [`describe-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-secret" >}}), + [`set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}), and + [`delete-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete-secret" >}}). +- **Dead-letter queues**: [`list-dlqs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs" >}}), + [`get-dlq`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-dlq" >}}), and + [`list-dlq-records`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records" >}}) (alias `get-rejected`). +- **Jobs**: [`list-jobs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-jobs" >}}), + [`get-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-job" >}}), and + [`describe-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-job" >}}). +- **Metric collections**: [`list-metric-collections`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections" >}}) and + [`get-metric-collection`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-metric-collection" >}}). +- **Scaffolding**: [`scaffold`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-scaffold" >}}) +- **Contexts**: [`list-contexts`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-contexts" >}}), + [`describe-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-context" >}}), + [`set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}), + [`use-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-use-context" >}}), and + [`delete-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete-context" >}}). + +On VM installations, the CLI also exposes the +[`configure-rdi`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi" >}}) and +[`dump-support-package`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package" >}}) +administration commands. + +See the [`redis-di`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di" >}}) page for the +global options that apply to every command. + +## Output formats + +The `list` and `get` commands print an aligned, column-based table by default. Pass `-o` / `--output` +with `json` or `yaml` to emit the underlying data instead, which is useful for scripting and for tools +such as `jq`. The `describe` commands always print a human-readable, sectioned layout. diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-add-context.md b/content/integrate/redis-data-integration/reference/cli/redis-di-add-context.md deleted file mode 100644 index 74b84874c6..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-add-context.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -Title: redis-di add-context -linkTitle: redis-di add-context -description: Adds a new context -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di add-context [OPTIONS] CONTEXT_NAME -``` - -## Options - -- `context_name` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `context-name` - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di add-context [OPTIONS] CONTEXT_NAME - - Adds a new context - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-completion.md b/content/integrate/redis-data-integration/reference/cli/redis-di-completion.md new file mode 100644 index 0000000000..21e8086ffc --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-completion.md @@ -0,0 +1,28 @@ +--- +Title: redis-di completion +linkTitle: redis-di completion +description: Generates a shell autocompletion script +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Generates an autocompletion script for `redis-di` for the specified shell. Supported shells are +`bash`, `zsh`, `fish`, and `powershell`. + +## Usage + +``` +redis-di completion [bash|zsh|fish|powershell] +``` + +Run `redis-di completion --help` for the per-shell installation instructions. + +## Example + +To load completions into the current `bash` session: + +```bash +source <(redis-di completion bash) +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi.md b/content/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi.md index cf34ac5bf9..0f8fa58a45 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi.md @@ -1,126 +1,34 @@ --- Title: redis-di configure-rdi linkTitle: redis-di configure-rdi -description: Configures RDI db connection credentials +description: Configures the RDI database connection weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Configures the connection credentials for the RDI database. This is an administration command that is +available only on VM installations, where `redis-di` forwards it to the bundled `rdi-admin` tool. + ## Usage ``` -Usage: redis-di configure-rdi [OPTIONS] +redis-di configure-rdi [OPTIONS] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `rdi_log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `none` - - Usage: `--rdi-log-level` - - Log level for RDI components - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di configure-rdi [OPTIONS] - - Configures RDI db connection credentials - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --rdi-log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - Log level for RDI components - --help Show this message and exit. -``` +| Option | Description | +| :-- | :-- | +| `-l`, `--log-level` | Log level: `TRACE`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL` (default `INFO`). | +| `--rdi-namespace` | RDI Kubernetes namespace (default `rdi`). | +| `--rdi-host` | Host or IP of the RDI database (required). | +| `--rdi-port` | Port of the RDI database, `1`–`65535` (required). | +| `--rdi-user` | RDI database username. | +| `--rdi-password` | RDI database password. | +| `--rdi-key` | Private key file to authenticate with. | +| `--rdi-cert` | Client certificate file to authenticate with. | +| `--rdi-cacert` | CA certificate file to verify with. | +| `--rdi-key-password` | Password for unlocking an encrypted private key. | +| `--rdi-log-level` | Log level for the RDI components. | diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-delete-all-contexts.md b/content/integrate/redis-data-integration/reference/cli/redis-di-delete-all-contexts.md deleted file mode 100644 index 66b56f5d41..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-delete-all-contexts.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -Title: redis-di delete-all-contexts -linkTitle: redis-di delete-all-contexts -description: Deletes all contexts -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di delete-all-contexts [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `force`: - - Type: BOOL - - Default: `false` - - Usage: `--force --f` - - Force operation. Skips verification prompts - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di delete-all-contexts [OPTIONS] - - Deletes all contexts - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - -f, --force Force operation. Skips verification prompts - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-delete-context.md b/content/integrate/redis-data-integration/reference/cli/redis-di-delete-context.md index 8b92319d21..3215aa1381 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-delete-context.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-delete-context.md @@ -8,50 +8,26 @@ categories: ["redis-di"] aliases: --- +Deletes a context from the `~/.redis-di` context file. Because this is destructive, the command asks +for confirmation unless you pass `--force`. + ## Usage ``` -Usage: redis-di delete-context [OPTIONS] CONTEXT_NAME +redis-di delete-context [flags] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `context_name` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `context-name` - -- `force`: - - Type: BOOL - - Default: `false` - - Usage: `--force --f` - - Force operation. Skips verification prompts +| Option | Description | +| :-- | :-- | +| `--force` | Skip the confirmation prompt. | -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di delete-context [OPTIONS] CONTEXT_NAME +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Deletes a context +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - -f, --force Force operation. Skips verification prompts - --help Show this message and exit. +```bash +redis-di delete-context dev --force ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-delete-secret.md b/content/integrate/redis-data-integration/reference/cli/redis-di-delete-secret.md new file mode 100644 index 0000000000..e3d3700dd9 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-delete-secret.md @@ -0,0 +1,36 @@ +--- +Title: redis-di delete-secret +linkTitle: redis-di delete-secret +description: Deletes a secret of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Deletes a secret of a pipeline. Because this is destructive, the command asks for confirmation unless +you pass `--force`. + +## Usage + +``` +redis-di delete-secret [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `--force` | Skip the confirmation prompt. | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di delete-secret SOURCE_DB_CACERT --force +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-delete.md b/content/integrate/redis-data-integration/reference/cli/redis-di-delete.md new file mode 100644 index 0000000000..61e1195a00 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-delete.md @@ -0,0 +1,37 @@ +--- +Title: redis-di delete +linkTitle: redis-di delete +description: Deletes a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Deletes a pipeline. Because this is destructive, the command asks for confirmation unless you pass +`--force`. + +## Usage + +``` +redis-di delete [pipeline] [flags] +``` + +The pipeline name is an optional argument that defaults to `default`. + +## Options + +| Option | Description | +| :-- | :-- | +| `--force` | Skip the confirmation prompt. | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di delete my-pipeline --force +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-deploy.md b/content/integrate/redis-data-integration/reference/cli/redis-di-deploy.md index 908204eafe..ced2a2193f 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-deploy.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-deploy.md @@ -1,118 +1,47 @@ --- Title: redis-di deploy linkTitle: redis-di deploy -description: Deploys the RDI configurations including target +description: Deploys a pipeline with the specified configuration weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Deploys a pipeline, creating it or updating it from the configuration in the `--dir` directory. The +API validates the configuration and rejects an invalid one. By default, the command starts the +pipeline after deploying and waits for it to reach the expected state. `set` is an alias for this +command. + ## Usage ``` -Usage: redis-di deploy [OPTIONS] +redis-di deploy [pipeline] [flags] ``` -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` +The pipeline name is an optional argument that defaults to `default`. - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `directory`: - - Type: STRING - - Default: `.` - - Usage: `--dir` - - Directory containing RDI configuration - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` +## Options - Show this message and exit. +| Option | Description | +| :-- | :-- | +| `--dir` | Directory containing the pipeline configuration (default `.`). | +| `--dry-run` | Validate the configuration without deploying. | +| `--validate-tables` | Validate the configuration against the source and target databases (default `true`). | +| `--validate-cdc` | Validate the source database CDC configuration. | +| `--start` | Start the pipeline after deploying (default `true`). | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | -## CLI help +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). -``` -Usage: redis-di deploy [OPTIONS] +## Example - Deploys the RDI configurations including target +```bash +# Deploy the configuration in the current directory +redis-di deploy -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --dir TEXT Directory containing RDI configuration - [default: .] - --help Show this message and exit. +# Validate a configuration folder without deploying it +redis-di deploy --dir /opt/rdi/config --dry-run ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-describe-context.md b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-context.md new file mode 100644 index 0000000000..e3962de2d3 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-context.md @@ -0,0 +1,30 @@ +--- +Title: redis-di describe-context +linkTitle: redis-di describe-context +description: Describes a context +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Describes a single context from the `~/.redis-di` context file, showing its API connection details. +See the [CLI reference overview]({{< relref "/integrate/redis-data-integration/reference/cli#contexts" >}}) +for more about contexts. + +## Usage + +``` +redis-di describe-context [flags] +``` + +## Options + +This command takes only the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di describe-context prod +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-describe-job.md b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-job.md index 2984736d6d..9e2af371f7 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-describe-job.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-job.md @@ -1,114 +1,33 @@ --- Title: redis-di describe-job linkTitle: redis-di describe-job -description: Describes a transformation engine's job +description: Describes a job of a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Describes a single job of a pipeline, printing its source properties followed by tables that +summarize its transformations and outputs. + ## Usage ``` -Usage: redis-di describe-job [OPTIONS] JOB_NAME +redis-di describe-job [flags] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `job_name` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `job-name` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di describe-job [OPTIONS] JOB_NAME +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Describes a transformation engine's job +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --help Show this message and exit. +```bash +redis-di describe-job customers_hash_job ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-describe-secret.md b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-secret.md new file mode 100644 index 0000000000..7dd960deea --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-describe-secret.md @@ -0,0 +1,33 @@ +--- +Title: redis-di describe-secret +linkTitle: redis-di describe-secret +description: Describes a secret of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Describes a single secret of a pipeline. The API never returns secret values, so the output shows +only the key and whether it is set, not the stored value. + +## Usage + +``` +redis-di describe-secret [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di describe-secret TARGET_DB_PASSWORD +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-describe.md b/content/integrate/redis-data-integration/reference/cli/redis-di-describe.md new file mode 100644 index 0000000000..0ff5a63243 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-describe.md @@ -0,0 +1,80 @@ +--- +Title: redis-di describe +linkTitle: redis-di describe +description: Describes a pipeline with its status +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: + - /integrate/redis-data-integration/reference/cli/redis-di-status/ +--- + +Describes a pipeline, combining its configuration with its runtime status, components, errors, and +metrics in a human-readable, sectioned layout. `status` is an alias for this command. + +The RDI version is not shown here, because it is a property of the API connection rather than of the +pipeline; use [`info`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-info" >}}) +to see it. + +## Usage + +``` +redis-di describe [pipeline] [flags] +``` + +The pipeline name is an optional argument that defaults to `default`. + +## Options + +This command takes only the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di describe +redis-di status my-pipeline +``` + +To watch the status update live, pair the command with `watch`: + +```bash +watch -n 1 redis-di describe +``` + +The output has a section for each part of the pipeline, for example: + +``` +Name: default +Active: yes +Status: started +Current: yes + +Sources: + Name Type Db Type Connection Sync Mode Connected + ---- ---- ------- ---------- --------- --------- + mysql cdc mysql ${HOST_IP}:13000 streaming yes + +Targets: + Name Db Type Connection Connected + ---- ------- ---------- --------- + target redis ${HOST_IP}:12000 yes + +Jobs: + Name Source Transformations Outputs Connections + ---- ------ --------------- ------- ----------- + address_job inventory.addresses 1 1 target + customers_hash_job inventory.customers 0 1 target + +Components: + Name Type Version Status + ---- ---- ------- ------ + collector-api collector-api 0.0.0 started + collector-source debezium-collector ... started + processor processor 0.0.0 started + +Statistics: + Name Total Pending Inserted Updated Deleted Filtered Rejected Deduplicated Last Arrival + ---- ----- ------- -------- ------- ------- -------- -------- ------------ ------------ + {rdi}:inventory.customers 4 0 4 0 0 0 0 0 2026-06-18T13:42:44Z +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package.md b/content/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package.md index 05f34ba7cf..d0df406907 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package.md @@ -1,169 +1,38 @@ --- Title: redis-di dump-support-package linkTitle: redis-di dump-support-package -description: Dumps RDI support package +description: Dumps the RDI support package weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Dumps a comprehensive set of RDI forensics data that you can send to Redis support (see +[Dump support package]({{< relref "/integrate/redis-data-integration/troubleshooting#dump-support-package" >}})). +This is an administration command that is available only on VM installations, where `redis-di` +forwards it to the bundled `rdi-admin` tool. + ## Usage ``` -Usage: redis-di dump-support-package [OPTIONS] +redis-di dump-support-package [OPTIONS] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `directory`: - - Type: STRING - - Default: `.` - - Usage: `--dir` - - Directory where the support file should be generated - -- `dump_rejected`: - - Type: INT - - Default: `none` - - Usage: `--dump-rejected` - - Dumps rejected records - -- `trace_timeout`: - - Type: - - Default: `none` - - Usage: `--trace-timeout` - - Stops the trace after exceeding this timeout (in seconds) - -- `max_change_records`: - - Type: =1> - - Default: `10` - - Usage: `--max-change-records` - - Maximum traced change records - -- `trace_only_rejected`: - - Type: BOOL - - Default: `false` - - Usage: `--trace-only-rejected` - - Trace only rejected change records - -- `log_days`: - - Type: =0> - - Default: `2` - - Usage: `--log-days` - - Number of days to look back for log files - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di dump-support-package [OPTIONS] - - Dumps RDI support package - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --dir TEXT Directory where the support file should be - generated [default: .] - --dump-rejected INTEGER Dumps rejected records - --trace-timeout INTEGER RANGE Stops the trace after exceeding this timeout - (in seconds) [1<=x<=600] - --max-change-records INTEGER RANGE - Maximum traced change records [x>=1] - --trace-only-rejected Trace only rejected change records - --log-days INTEGER RANGE Number of days to look back for log files - [default: 2; x>=0] - --help Show this message and exit. -``` +| Option | Description | +| :-- | :-- | +| `-l`, `--log-level` | Log level: `TRACE`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL` (default `INFO`). | +| `--rdi-namespace` | RDI Kubernetes namespace (default `rdi`). | +| `--rdi-host` | Host or IP of the RDI database (required). | +| `--rdi-port` | Port of the RDI database, `1`–`65535` (required). | +| `--rdi-user` | RDI database username. | +| `--rdi-password` | RDI database password. | +| `--rdi-key` | Private key file to authenticate with. | +| `--rdi-cert` | Client certificate file to authenticate with. | +| `--rdi-cacert` | CA certificate file to verify with. | +| `--rdi-key-password` | Password for unlocking an encrypted private key. | +| `--dir` | Directory where the support file is generated (default `.`). | +| `--dump-rejected` | Dump rejected records. | +| `--log-days` | Number of days to look back for log files (default `2`). | diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get-dlq.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get-dlq.md new file mode 100644 index 0000000000..3122fb3ec1 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-get-dlq.md @@ -0,0 +1,34 @@ +--- +Title: redis-di get-dlq +linkTitle: redis-di get-dlq +description: Gets a dead-letter queue of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Gets a single dead-letter queue (DLQ) of a pipeline and prints it in the compact `list-dlqs` table +format. + +## Usage + +``` +redis-di get-dlq [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di get-dlq inventory.customers +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get-job.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get-job.md new file mode 100644 index 0000000000..d9cbffb14d --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-get-job.md @@ -0,0 +1,35 @@ +--- +Title: redis-di get-job +linkTitle: redis-di get-job +description: Gets a job of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Gets a single job of a pipeline and prints it in the compact `list-jobs` table format. Use +[`describe-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-job" >}}) +for the full job view with its transformations and outputs. + +## Usage + +``` +redis-di get-job [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di get-job customers_hash_job +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get-metric-collection.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get-metric-collection.md new file mode 100644 index 0000000000..bafe875c82 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-get-metric-collection.md @@ -0,0 +1,36 @@ +--- +Title: redis-di get-metric-collection +linkTitle: redis-di get-metric-collection +description: Gets a metric collection of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Gets a single metric collection of a pipeline, returning its raw metric data. This command is most +useful with `-o json` or `-o yaml` for scripting and for tools such as `jq`. Use +[`list-metric-collections`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections" >}}) +to see the available collections. + +## Usage + +``` +redis-di get-metric-collection [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di get-metric-collection processor -o json +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get-rejected.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get-rejected.md deleted file mode 100644 index 380b5a6923..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-get-rejected.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -Title: redis-di get-rejected -linkTitle: redis-di get-rejected -description: Returns all the stored rejected entries -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di get-rejected [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `max_records`: - - Type: =1> - - Default: `none` - - Usage: `--max-records` - - Maximum rejected records per DLQ - -- `oldest`: - - Type: BOOL - - Default: `false` - - Usage: `--oldest --o` - - Displays the oldest rejected records. If omitted, most recent records will be retrieved - -- `dlq_name`: - - Type: STRING - - Default: `none` - - Usage: `--dlq-name` - - Only prints the rejected records for the specified DLQ (Dead Letter Queue) name - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di get-rejected [OPTIONS] - - Returns all the stored rejected entries - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --max-records INTEGER RANGE Maximum rejected records per DLQ [x>=1] - -o, --oldest Displays the oldest rejected records. If - omitted, most recent records will be - retrieved - --dlq-name TEXT Only prints the rejected records for the - specified DLQ (Dead Letter Queue) name - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get-secret.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get-secret.md new file mode 100644 index 0000000000..0dd4b0cd8e --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-get-secret.md @@ -0,0 +1,35 @@ +--- +Title: redis-di get-secret +linkTitle: redis-di get-secret +description: Gets a secret of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Gets a single secret of a pipeline and prints it in the compact `list-secrets` table format. The API +never returns secret values, so the output shows only the key and whether it is set, not the stored +value. + +## Usage + +``` +redis-di get-secret [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di get-secret SOURCE_DB_USERNAME +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-get.md b/content/integrate/redis-data-integration/reference/cli/redis-di-get.md new file mode 100644 index 0000000000..994991e146 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-get.md @@ -0,0 +1,37 @@ +--- +Title: redis-di get +linkTitle: redis-di get +description: Gets a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Gets a single pipeline and prints it in the compact `list` table format. Use +[`describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) +for the full pipeline view with its status and metrics. + +## Usage + +``` +redis-di get [pipeline] [flags] +``` + +The pipeline name is an optional argument that defaults to `default`. + +## Options + +| Option | Description | +| :-- | :-- | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di get +redis-di get my-pipeline -o yaml +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-info.md b/content/integrate/redis-data-integration/reference/cli/redis-di-info.md new file mode 100644 index 0000000000..0babd497a8 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-info.md @@ -0,0 +1,22 @@ +--- +Title: redis-di info +linkTitle: redis-di info +description: Displays information about the RDI deployment +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Displays information about the RDI deployment, such as the RDI version reported by the API. + +## Usage + +``` +redis-di info [flags] +``` + +## Options + +This command takes only the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-install.md b/content/integrate/redis-data-integration/reference/cli/redis-di-install.md deleted file mode 100644 index b2bc2a5d2e..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-install.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -Title: redis-di install -linkTitle: redis-di install -description: Installs RDI -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di install [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `warning` - - Usage: `--log-level --l` - -- `file`: - - Type: - - Default: `none` - - Usage: `-f ---file` - - Path to a YAML configuration file for silent installation - -- `online`: - - Type: BOOL - - Default: `false` - - Usage: `--online` - - Run installer in online mode - -- `k3s_only`: - - Type: BOOL - - Default: `false` - - Usage: `--k3s-only` - - Install only k3s components - -- `https_port`: - - Type: INT - - Default: `443` - - Usage: `--https-port` - - HTTPS port for Traefik - -- `installation_dir`: - - Type: - - Default: `none` - - Usage: `--installation-dir` - - Custom installation directory - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di install [OPTIONS] - - Installs RDI - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: WARNING] - -f, --file FILE Path to a YAML configuration file for silent - installation - --installation-dir DIRECTORY Custom installation directory - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-contexts.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-contexts.md index 9986659503..e417ee1da1 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-list-contexts.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-contexts.md @@ -1,35 +1,30 @@ --- Title: redis-di list-contexts linkTitle: redis-di list-contexts -description: Lists all saved contexts +description: Lists all contexts weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Lists all contexts from the `~/.redis-di` context file and indicates which one is active. See the +[CLI reference overview]({{< relref "/integrate/redis-data-integration/reference/cli#contexts" >}}) +for more about contexts. + ## Usage ``` -Usage: redis-di list-contexts [OPTIONS] +redis-di list-contexts [flags] ``` ## Options -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di list-contexts [OPTIONS] +This command takes only the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Lists all saved contexts +## Example -Options: - --help Show this message and exit. +```bash +redis-di list-contexts ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records.md new file mode 100644 index 0000000000..bdf7f97280 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records.md @@ -0,0 +1,55 @@ +--- +Title: redis-di list-dlq-records +linkTitle: redis-di list-dlq-records +description: Lists the rejected records of a dead-letter queue +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: + - /integrate/redis-data-integration/reference/cli/redis-di-get-rejected/ +--- + +Lists the rejected records of a single dead-letter queue (DLQ), taking the queue name as an argument +and paging with `--limit`, `--offset`, and `--sort-order`. The operation code is shown by name +(create, update, delete, read). `get-rejected` is an alias for this command. + +Use [`list-dlqs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs" >}}) +to see all the pipeline's dead-letter queues and their record counts. + +## Usage + +``` +redis-di list-dlq-records [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `--limit` | Maximum number of records to return (default `20`). | +| `--offset` | Number of records to skip (default `0`). | +| `--sort-order` | Sort order: `asc` (oldest first) or `desc` (newest first, default). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +The following options are kept for backward compatibility with the `get-rejected` command and are +deprecated: + +| Deprecated option | Use instead | +| :-- | :-- | +| `--dlq-name` | Pass the DLQ name as an argument. | +| `--max-records` | `--limit` | +| `--oldest` | `--sort-order asc` | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +# Newest 20 rejected records of a queue +redis-di list-dlq-records inventory.customers + +# Oldest 100 records, as JSON +redis-di list-dlq-records inventory.customers --limit 100 --sort-order asc -o json +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs.md new file mode 100644 index 0000000000..c3bd7cc791 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs.md @@ -0,0 +1,36 @@ +--- +Title: redis-di list-dlqs +linkTitle: redis-di list-dlqs +description: Lists the dead-letter queues of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Lists the dead-letter queues (DLQs) of a pipeline with their record counts. A DLQ holds the records +that RDI rejected. Use +[`list-dlq-records`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records" >}}) +to read the records of a single queue. + +## Usage + +``` +redis-di list-dlqs [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di list-dlqs +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-jobs.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-jobs.md index 8526771cd0..85d6c064da 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-list-jobs.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-jobs.md @@ -1,109 +1,36 @@ --- Title: redis-di list-jobs linkTitle: redis-di list-jobs -description: Lists transformation engine's jobs +description: Lists the jobs of a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Lists the jobs of a pipeline, one row per job with its source, its transformation and output counts, +and the target connections of its outputs. Use +[`describe-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-job" >}}) +for the full view of a single job. + ## Usage ``` -Usage: redis-di list-jobs [OPTIONS] +redis-di list-jobs [flags] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di list-jobs [OPTIONS] +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Lists transformation engine's jobs +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --help Show this message and exit. +```bash +redis-di list-jobs ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections.md new file mode 100644 index 0000000000..89b9564543 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections.md @@ -0,0 +1,36 @@ +--- +Title: redis-di list-metric-collections +linkTitle: redis-di list-metric-collections +description: Lists the metric collections of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Lists the metric collections of a pipeline. Metric collections hold the raw component metrics that +the [`describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) +command summarizes in its Statistics and Performance sections. This command is most useful with +`-o json` or `-o yaml` for scripting and for tools such as `jq`. + +## Usage + +``` +redis-di list-metric-collections [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di list-metric-collections +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list-secrets.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list-secrets.md new file mode 100644 index 0000000000..caf31ff5f5 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list-secrets.md @@ -0,0 +1,35 @@ +--- +Title: redis-di list-secrets +linkTitle: redis-di list-secrets +description: Lists the secrets of a pipeline +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Lists the secrets of a pipeline. The API never returns secret values, so the output shows only the +secret keys and whether each one is set, not the stored values. + +## Usage + +``` +redis-di list-secrets [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di list-secrets +redis-di list-secrets -p my-pipeline +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-list.md b/content/integrate/redis-data-integration/reference/cli/redis-di-list.md new file mode 100644 index 0000000000..54550138cd --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-list.md @@ -0,0 +1,33 @@ +--- +Title: redis-di list +linkTitle: redis-di list +description: Lists all pipelines +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Lists all pipelines with their status in a compact table. + +## Usage + +``` +redis-di list [flags] +``` + +## Options + +| Option | Description | +| :-- | :-- | +| `-o`, `--output` | Output format: `table` (default), `json`, or `yaml`. | + +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di list +redis-di list -o json +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-monitor.md b/content/integrate/redis-data-integration/reference/cli/redis-di-monitor.md deleted file mode 100644 index 8f742601da..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-monitor.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -Title: redis-di monitor -linkTitle: redis-di monitor -description: Monitors RDI by collecting metrics and exporting to Prometheus -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di monitor [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `exporter_port`: - - Type: - - Default: `9121` - - Usage: `--exporter-port` - - HTTP port to start the exporter on - -- `collect_interval`: - - Type: - - Default: `5` - - Usage: `--collect-interval` - - Metrics collection interval (seconds) - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di monitor [OPTIONS] - - Monitors RDI by collecting metrics and exporting to Prometheus - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --exporter-port INTEGER RANGE HTTP port to start the exporter on - [1<=x<=65535] - --collect-interval INTEGER RANGE - Metrics collection interval (seconds) - [1<=x<=60] - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md b/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md index 519ea6f4c5..23cf473f25 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-reset.md @@ -1,127 +1,37 @@ --- Title: redis-di reset linkTitle: redis-di reset -description: Resets the pipeline into initial full sync mode +description: Resets a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Resets a pipeline into initial full-sync mode, so it reloads a snapshot of the source data before +resuming change data capture. By default, the command waits for the pipeline to reach a terminal +state before returning. + ## Usage ``` -Usage: redis-di reset [OPTIONS] +redis-di reset [pipeline] [flags] ``` -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with +The pipeline name is an optional argument that defaults to `default`. -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `force`: - - Type: BOOL - - Default: `false` - - Usage: `--force --f` - - Force operation. Skips verification prompts - -- `pause_for_confirmation`: - - Type: BOOL - - Default: `false` - - Usage: `--pause-for-confirmation` - - Pause for user confirmation if manual shutdown of collector required - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. +## Options -## CLI help +| Option | Description | +| :-- | :-- | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | -``` -Usage: redis-di reset [OPTIONS] +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Resets the pipeline into initial full sync mode +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - -f, --force Force operation. Skips verification prompts - --pause-for-confirmation Pause for user confirmation if manual - shutdown of collector required - --help Show this message and exit. +```bash +redis-di reset ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-scaffold.md b/content/integrate/redis-data-integration/reference/cli/redis-di-scaffold.md index e52cee8f74..2b4e61adb9 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-scaffold.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-scaffold.md @@ -1,82 +1,41 @@ --- Title: redis-di scaffold linkTitle: redis-di scaffold -description: Generates configuration files for RDI +description: Generates pipeline configuration files weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Generates a starter pipeline configuration for the given source database type. With `--dir`, the +command writes a `config.yaml` file into that directory, prompting before overwriting an existing +file unless `--force` is set. Without `--dir`, it prints the configuration to standard output. + ## Usage ``` -Usage: redis-di scaffold [OPTIONS] +redis-di scaffold [flags] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `db_type` (REQUIRED): - - Type: Choice([, , , , , , , , ]) - - Default: `none` - - Usage: `--db-type` - - DB type - -- `db_flavor`: - - Type: Choice([, , ]) - - Default: `none` - - Usage: `--db-flavor` - - DB flavor - - Output to directory or stdout - -- `directory`: - - Type: STRING - - Default: `none` - - Usage: `--dir` +| Option | Description | +| :-- | :-- | +| `--db-type` | Source database type (required): `mariadb`, `mongodb`, `mysql`, `oracle`, `postgresql`, `snowflake`, `sqlserver`, or `spanner`. | +| `--db-flavor` | Source database flavor: `mongodb-atlas`, `mongodb-replica-set`, or `mongodb-sharded-cluster`. | +| `--dir` | Directory to write `config.yaml` to; prints to standard output when omitted. | +| `--force` | Skip the confirmation prompt when overwriting an existing file. | - Directory containing RDI configuration +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). -- `preview`: - - Type: STRING - - Default: `none` - - Usage: `--preview` - - Print the content of the scaffolded config file to CLI output - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di scaffold [OPTIONS] +## Example - Generates configuration files for RDI +```bash +# Print a PostgreSQL configuration to stdout +redis-di scaffold --db-type postgresql -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --db-type [cassandra|mariadb|mongodb|mysql|oracle|postgresql|snowflake|sqlserver|spanner] - DB type [required] - --db-flavor [mongodb-atlas|mongodb-replica-set|mongodb-sharded-cluster] - DB flavor - Output formats: [mutually_exclusive, required] - Output to directory or stdout - --dir TEXT Directory containing RDI configuration - --preview TEXT Print the content of the scaffolded config - file to CLI output - --help Show this message and exit. +# Write a MySQL configuration into a directory +redis-di scaffold --db-type mysql --dir /opt/rdi/config ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-set-context.md b/content/integrate/redis-data-integration/reference/cli/redis-di-set-context.md index 0f9e23face..6f5ce84570 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-set-context.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-set-context.md @@ -1,48 +1,51 @@ --- Title: redis-di set-context linkTitle: redis-di set-context -description: Sets a context to be the active one +description: Creates or updates a context weight: 10 alwaysopen: false categories: ["redis-di"] aliases: + - /integrate/redis-data-integration/reference/cli/redis-di-add-context/ + - /integrate/redis-data-integration/reference/cli/redis-di-delete-all-contexts/ --- +Creates or updates a context in the `~/.redis-di` context file. A context stores an API connection +so you don't have to pass the connection options on every command. `set-context` merges only the +options you give on the command line, preserving the rest, and does not change which context is +active; use [`use-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-use-context" >}}) +for that. See the +[CLI reference overview]({{< relref "/integrate/redis-data-integration/reference/cli#contexts" >}}) +for more about contexts. + +Secrets (the password and the Redis Cloud user key) are never stored in a context, so `set-context` +rejects `--password` and `--user-key`. + ## Usage ``` -Usage: redis-di set-context [OPTIONS] CONTEXT_NAME +redis-di set-context [flags] ``` -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` +The connection is set from the global options `--api-url`, `--user`, `--account-key`, `--cacert`, +and `--insecure`. -- `context_name` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `context-name` - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` +## Options - Show this message and exit. +| Option | Description | +| :-- | :-- | +| `--unset-user` | Clear the stored user, so the context authenticates without a user. | +| `--unset-account-key` | Clear the stored account key, so the context authenticates without a Redis Cloud account key. | -## CLI help +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). -``` -Usage: redis-di set-context [OPTIONS] CONTEXT_NAME +## Example - Sets a context to be the active one +```bash +# Create or update a context for a VM or Kubernetes installation +redis-di set-context prod --api-url https://rdi.example.com --user default --cacert /etc/rdi/ingress-ca.crt -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --help Show this message and exit. +# Create or update a context that skips TLS verification +redis-di set-context dev --api-url https://localhost:8443 --insecure ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-set-secret.md b/content/integrate/redis-data-integration/reference/cli/redis-di-set-secret.md index 754e63ef3b..5b2a3193f2 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-set-secret.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-set-secret.md @@ -1,73 +1,51 @@ --- Title: redis-di set-secret linkTitle: redis-di set-secret -description: Creates a secret of a specified key +description: Creates or updates a secret of a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Creates or updates a secret of a pipeline. Secrets hold the credentials and certificates that the +pipeline uses to connect to the source and target databases (see +[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets" >}}) +for the list of secret names). You can then refer to a secret in the `config.yaml` file with the +syntax `${SECRET_NAME}`. + +The secret value comes from the `[value]` argument, the `--file` option, or the `--literal` option. +If you provide none of these on an interactive terminal, the command prompts for the value without +echoing it. + ## Usage ``` -Usage: redis-di set-secret [OPTIONS] {RDI_REDIS_USERNAME|RDI_REDIS_PASSWORD|RD - I_REDIS_CACERT|RDI_REDIS_CERT|RDI_REDIS_KEY|RDI_RED - IS_KEY_PASSPHRASE|SOURCE_DB_USERNAME|SOURCE_DB_PASS - WORD|SOURCE_DB_CACERT|SOURCE_DB_CERT|SOURCE_DB_KEY| - SOURCE_DB_KEY_PASSWORD|TARGET_DB_USERNAME|TARGET_DB - _PASSWORD|TARGET_DB_CACERT|TARGET_DB_CERT|TARGET_DB - _KEY|TARGET_DB_KEY_PASSWORD|JWT_SECRET_KEY} [VALUE] +redis-di set-secret [value] [flags] ``` ## Options -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace +| Option | Description | +| :-- | :-- | +| `-p`, `--pipeline` | Pipeline to target (default `default`). | +| `--file` | Read the secret value from the file at this path. | +| `--literal` | Use this literal string as the secret value. | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | -- `key` (REQUIRED): - - Type: Choice(['RDI_REDIS_USERNAME', 'RDI_REDIS_PASSWORD', 'RDI_REDIS_CACERT', 'RDI_REDIS_CERT', 'RDI_REDIS_KEY', 'RDI_REDIS_KEY_PASSPHRASE', 'SOURCE_DB_USERNAME', 'SOURCE_DB_PASSWORD', 'SOURCE_DB_CACERT', 'SOURCE_DB_CERT', 'SOURCE_DB_KEY', 'SOURCE_DB_KEY_PASSWORD', 'TARGET_DB_USERNAME', 'TARGET_DB_PASSWORD', 'TARGET_DB_CACERT', 'TARGET_DB_CERT', 'TARGET_DB_KEY', 'TARGET_DB_KEY_PASSWORD', 'JWT_SECRET_KEY']) - - Default: `none` - - Usage: `key` +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). -- `value`: - - Type: STRING - - Default: `none` - - Usage: `value` +## Example -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di set-secret [OPTIONS] {RDI_REDIS_USERNAME|RDI_REDIS_PASSWORD|RD - I_REDIS_CACERT|RDI_REDIS_CERT|RDI_REDIS_KEY|RDI_RED - IS_KEY_PASSPHRASE|SOURCE_DB_USERNAME|SOURCE_DB_PASS - WORD|SOURCE_DB_CACERT|SOURCE_DB_CERT|SOURCE_DB_KEY| - SOURCE_DB_KEY_PASSWORD|TARGET_DB_USERNAME|TARGET_DB - _PASSWORD|TARGET_DB_CACERT|TARGET_DB_CERT|TARGET_DB - _KEY|TARGET_DB_KEY_PASSWORD|JWT_SECRET_KEY} [VALUE] +```bash +# Value from an argument +redis-di set-secret SOURCE_DB_USERNAME myuser - Creates a secret of a specified key +# Value from a file (for example, a certificate) +redis-di set-secret SOURCE_DB_CACERT --file /path/to/myca.crt -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --help Show this message and exit. +# Value read from an interactive prompt +redis-di set-secret SOURCE_DB_PASSWORD ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-start.md b/content/integrate/redis-data-integration/reference/cli/redis-di-start.md index 17a8c01997..8eb9c633f1 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-start.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-start.md @@ -1,109 +1,37 @@ --- Title: redis-di start linkTitle: redis-di start -description: Starts the pipeline +description: Starts a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Starts a pipeline. By default, the command waits for the pipeline to reach the `started` state before +returning. + ## Usage ``` -Usage: redis-di start [OPTIONS] +redis-di start [pipeline] [flags] ``` -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password +The pipeline name is an optional argument that defaults to `default`. -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. +## Options -## CLI help +| Option | Description | +| :-- | :-- | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | -``` -Usage: redis-di start [OPTIONS] +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Starts the pipeline +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --help Show this message and exit. +```bash +redis-di start +redis-di start my-pipeline --wait=false ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-status.md b/content/integrate/redis-data-integration/reference/cli/redis-di-status.md deleted file mode 100644 index 5166de21fe..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-status.md +++ /dev/null @@ -1,155 +0,0 @@ ---- -Title: redis-di status -linkTitle: redis-di status -description: Displays the status of the pipeline end to end -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di status [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `live`: - - Type: BOOL - - Default: `false` - - Usage: `--live --l` - - Live data flow monitoring - -- `page_number`: - - Type: =1> - - Default: `none` - - Usage: `--page-number --p` - - Set the page number (live mode only) - -- `page_size`: - - Type: =1> - - Default: `none` - - Usage: `--page-size --s` - - Set the page size (live mode only) - -- `ingested_only`: - - Type: BOOL - - Default: `false` - - Usage: `--ingested-only --i` - - Display ingested data streams (live mode only) - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di status [OPTIONS] - - Displays the status of the pipeline end to end - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - -l, --live Live data flow monitoring - -p, --page-number INTEGER RANGE - Set the page number (live mode only) [x>=1] - -s, --page-size INTEGER RANGE Set the page size (live mode only) [x>=1] - -i, --ingested-only Display ingested data streams (live mode - only) - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-stop.md b/content/integrate/redis-data-integration/reference/cli/redis-di-stop.md index f901148a5b..4bbc9995d9 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-stop.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-stop.md @@ -1,109 +1,36 @@ --- Title: redis-di stop linkTitle: redis-di stop -description: Stops the pipeline +description: Stops a pipeline weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +Stops a pipeline. By default, the command waits for the pipeline to reach the `stopped` state before +returning. + ## Usage ``` -Usage: redis-di stop [OPTIONS] +redis-di stop [pipeline] [flags] ``` -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password +The pipeline name is an optional argument that defaults to `default`. -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. +## Options -## CLI help +| Option | Description | +| :-- | :-- | +| `--wait` | Wait for the pipeline to reach the expected state (default `true`). | +| `--timeout` | Maximum time to wait for the pipeline to reach the expected state (default `2m`). | -``` -Usage: redis-di stop [OPTIONS] +This command also accepts the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). - Stops the pipeline +## Example -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --help Show this message and exit. +```bash +redis-di stop ``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-trace.md b/content/integrate/redis-data-integration/reference/cli/redis-di-trace.md deleted file mode 100644 index b4e30493f0..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-trace.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -Title: redis-di trace -linkTitle: redis-di trace -description: Starts a trace session for troubleshooting data transformation -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di trace [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `max_change_records`: - - Type: =1> - - Default: `10` - - Usage: `--max-change-records` - - Maximum traced change records - -- `timeout` (REQUIRED): - - Type: - - Default: `20` - - Usage: `--timeout` - - Stops the trace after exceeding this timeout (in seconds) - -- `trace_only_rejected`: - - Type: BOOL - - Default: `false` - - Usage: `--trace-only-rejected` - - Trace only rejected change records - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di trace [OPTIONS] - - Starts a trace session for troubleshooting data transformation - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - --max-change-records INTEGER RANGE - Maximum traced change records [x>=1] - --timeout INTEGER RANGE Stops the trace after exceeding this timeout - (in seconds) [default: 20; 1<=x<=600; - required] - --trace-only-rejected Trace only rejected change records - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-upgrade.md b/content/integrate/redis-data-integration/reference/cli/redis-di-upgrade.md deleted file mode 100644 index 430b4274ad..0000000000 --- a/content/integrate/redis-data-integration/reference/cli/redis-di-upgrade.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -Title: redis-di upgrade -linkTitle: redis-di upgrade -description: Upgrades RDI without losing data or downtime -weight: 10 -alwaysopen: false -categories: ["redis-di"] -aliases: ---- - -## Usage - -``` -Usage: redis-di upgrade [OPTIONS] -``` - -## Options - -- `log_level`: - - Type: Choice(['TRACE', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']) - - Default: `info` - - Usage: `--log-level --l` - -- `rdi_namespace`: - - Type: STRING - - Default: `rdi` - - Usage: `--rdi-namespace` - - RDI Kubernetes namespace - -- `rdi_host` (REQUIRED): - - Type: STRING - - Default: `none` - - Usage: `--rdi-host` - - Host/IP of RDI Database - -- `rdi_port` (REQUIRED): - - Type: - - Default: `none` - - Usage: `--rdi-port` - - Port of RDI Database - -- `rdi_user`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-user` - - RDI Database Username - -- `rdi_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-password` - - RDI Database Password - -- `rdi_key`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key` - - Private key file to authenticate with - -- `rdi_cert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cert` - - Client certificate file to authenticate with - -- `rdi_cacert`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-cacert` - - CA certificate file to verify with - -- `rdi_key_password`: - - Type: STRING - - Default: `none` - - Usage: `--rdi-key-password` - - Password for unlocking an encrypted private key - -- `force`: - - Type: BOOL - - Default: `false` - - Usage: `--force --f` - - Force operation. Skips verification prompts - -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` - - Show this message and exit. - -## CLI help - -``` -Usage: redis-di upgrade [OPTIONS] - - Upgrades RDI without losing data or downtime - -Options: - -l, --log-level [TRACE|DEBUG|INFO|WARNING|ERROR|CRITICAL] - [default: INFO] - --rdi-namespace TEXT RDI Kubernetes namespace [default: rdi] - --rdi-host TEXT Host/IP of RDI Database [required] - --rdi-port INTEGER RANGE Port of RDI Database [1<=x<=65535; - required] - --rdi-user TEXT RDI Database Username - --rdi-password TEXT RDI Database Password - --rdi-key TEXT Private key file to authenticate with - --rdi-cert TEXT Client certificate file to authenticate with - --rdi-cacert TEXT CA certificate file to verify with - --rdi-key-password TEXT Password for unlocking an encrypted private - key - -f, --force Force operation. Skips verification prompts - --help Show this message and exit. -``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di-use-context.md b/content/integrate/redis-data-integration/reference/cli/redis-di-use-context.md new file mode 100644 index 0000000000..277a54dda2 --- /dev/null +++ b/content/integrate/redis-data-integration/reference/cli/redis-di-use-context.md @@ -0,0 +1,30 @@ +--- +Title: redis-di use-context +linkTitle: redis-di use-context +description: Sets a context to be the active one +weight: 10 +alwaysopen: false +categories: ["redis-di"] +aliases: +--- + +Sets a context in the `~/.redis-di` context file to be the active one, so its connection details are +used by subsequent commands. Create or update a context with +[`set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}). + +## Usage + +``` +redis-di use-context [flags] +``` + +## Options + +This command takes only the +[global options]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di#global-options" >}}). + +## Example + +```bash +redis-di use-context prod +``` diff --git a/content/integrate/redis-data-integration/reference/cli/redis-di.md b/content/integrate/redis-data-integration/reference/cli/redis-di.md index 72966f6ed0..4745e4b3c4 100644 --- a/content/integrate/redis-data-integration/reference/cli/redis-di.md +++ b/content/integrate/redis-data-integration/reference/cli/redis-di.md @@ -1,66 +1,86 @@ --- Title: redis-di linkTitle: redis-di -description: A command line tool to manage & configure Redis Data Integration +description: Command line tool to manage Redis Data Integration weight: 10 alwaysopen: false categories: ["redis-di"] aliases: --- +`redis-di` is the command line tool that manages Redis Data Integration (RDI). It is a thin client +over the RDI API and works the same way for VM, Kubernetes, and Redis Cloud installations. See the +[CLI reference overview]({{< relref "/integrate/redis-data-integration/reference/cli" >}}) for an +introduction to connecting, authentication, and contexts. + ## Usage ``` -Usage: redis-di [OPTIONS] COMMAND [ARGS]... +redis-di [command] ``` -## Options - -- `version`: - - Type: BOOL - - Default: `false` - - Usage: `--version` +Run `redis-di help` (or `redis-di --help`) to list every command, and `redis-di help ` +(or `redis-di --help`) to print the usage, flags, and arguments for a single command. - Show the version and exit. +## Global options -- `help`: - - Type: BOOL - - Default: `false` - - Usage: `--help` +These options apply to every command. Each one can also be set through an `RDI_`-prefixed environment +variable, for example `RDI_API_URL`, `RDI_USER`, or `RDI_PASSWORD`. Setting a secret such as the +password through an environment variable keeps it out of your shell history. - Show this message and exit. +| Option | Environment variable | Description | +| :-- | :-- | :-- | +| `--api-url` | `RDI_API_URL` | RDI API base URL. | +| `--user` | `RDI_USER` | User for API (JWT) authentication. | +| `--password` | `RDI_PASSWORD` | Password for API (JWT) authentication. Prompted for if a user is set and no password is supplied. | +| `--account-key` | `RDI_ACCOUNT_KEY` | Redis Cloud account key for API authentication. | +| `--user-key` | `RDI_USER_KEY` | Redis Cloud user key for API authentication. Prompted for if an account key is set and no user key is supplied. | +| `--cacert` | `RDI_CACERT` | CA certificate that verifies the API ingress. | +| `--insecure` | `RDI_INSECURE` | Skip TLS verification of the API ingress (insecure). Mutually exclusive with `--cacert`. | +| `--context` | `RDI_CONTEXT` | Context to use instead of the active one. | +| `--log-level` | `RDI_LOG_LEVEL` | Log level: `TRACE`, `DEBUG`, `INFO`, `WARNING`, or `ERROR` (default `INFO`). | +| `-v`, `--verbose` | | Enable verbose logging, equivalent to `--log-level DEBUG`. | +| `--version` | | Print the version and build metadata and exit. | +| `-h`, `--help` | | Print help for the CLI or a command. | -## CLI help +{{< note >}}Setting both `--user` and `--account-key` is an error, because they select mutually exclusive +authentication modes. Setting both `--cacert` and `--insecure` is also an error.{{< /note >}} -``` -Usage: redis-di [OPTIONS] COMMAND [ARGS]... +## Commands - A command line tool to manage & configure Redis Data Integration +| Command | Description | +| :-- | :-- | +| [`info`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-info" >}}) | Displays information about the RDI deployment | +| [`list`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list" >}}) | Lists all pipelines | +| [`get`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get" >}}) | Gets a pipeline | +| [`describe`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe" >}}) | Describes a pipeline with its status (alias `status`) | +| [`deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}}) | Deploys a pipeline with the specified configuration (alias `set`) | +| [`delete`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete" >}}) | Deletes a pipeline | +| [`start`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-start" >}}) | Starts a pipeline | +| [`stop`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-stop" >}}) | Stops a pipeline | +| [`reset`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-reset" >}}) | Resets a pipeline | +| [`list-secrets`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-secrets" >}}) | Lists the secrets of a pipeline | +| [`get-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-secret" >}}) | Gets a secret of a pipeline | +| [`describe-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-secret" >}}) | Describes a secret of a pipeline | +| [`set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}}) | Creates or updates a secret of a pipeline | +| [`delete-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete-secret" >}}) | Deletes a secret of a pipeline | +| [`list-dlqs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlqs" >}}) | Lists the dead-letter queues of a pipeline | +| [`get-dlq`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-dlq" >}}) | Gets a dead-letter queue of a pipeline | +| [`list-dlq-records`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-dlq-records" >}}) | Lists the rejected records of a dead-letter queue (alias `get-rejected`) | +| [`list-jobs`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-jobs" >}}) | Lists the jobs of a pipeline | +| [`get-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-job" >}}) | Gets a job of a pipeline | +| [`describe-job`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-job" >}}) | Describes a job of a pipeline | +| [`list-metric-collections`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-metric-collections" >}}) | Lists the metric collections of a pipeline | +| [`get-metric-collection`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-get-metric-collection" >}}) | Gets a metric collection of a pipeline | +| [`scaffold`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-scaffold" >}}) | Generates pipeline configuration files | +| [`list-contexts`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-list-contexts" >}}) | Lists all contexts | +| [`describe-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-describe-context" >}}) | Describes a context | +| [`set-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-context" >}}) | Creates or updates a context | +| [`use-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-use-context" >}}) | Sets a context to be the active one | +| [`delete-context`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-delete-context" >}}) | Deletes a context | +| [`completion`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-completion" >}}) | Generates a shell autocompletion script | -Options: - --version Show the version and exit. - --help Show this message and exit. - -Commands: - add-context Adds a new context - configure-rdi Configures RDI db connection credentials - delete-all-contexts Deletes all contexts - delete-context Deletes a context - deploy Deploys the RDI configurations including target - describe-job Describes a transformation engine's job - dump-support-package Dumps RDI support package - get-rejected Returns all the stored rejected entries - install Installs RDI - list-contexts Lists all saved contexts - list-jobs Lists transformation engine's jobs - monitor Monitors RDI by collecting metrics and exporting... - reset Resets the pipeline into initial full sync mode - scaffold Generates configuration files for RDI - set-context Sets a context to be the active one - set-secret Creates a secret of a specified key - start Starts the pipeline - status Displays the status of the pipeline end to end - stop Stops the pipeline - trace Starts a trace session for troubleshooting data... - upgrade Upgrades RDI without losing data or downtime -``` +On VM installations, the CLI also exposes the +[`configure-rdi`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-configure-rdi" >}}), +[`dump-support-package`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-dump-support-package" >}}), +and `admin` administration commands. diff --git a/content/integrate/redis-data-integration/troubleshooting.md b/content/integrate/redis-data-integration/troubleshooting.md index a5cfe43ae5..306d93f153 100644 --- a/content/integrate/redis-data-integration/troubleshooting.md +++ b/content/integrate/redis-data-integration/troubleshooting.md @@ -27,8 +27,7 @@ If the installer fails with an error, then try installing again with the log level set to `DEBUG`: ```bash -./install.sh -l DEBUG # Installer script -redis-di install -l DEBUG # Install command +./install.sh --log-level DEBUG ``` This gives you more detail about the installation steps and can often @@ -72,8 +71,6 @@ This command gathers the following data: - List of secret names used by RDI components (but not the secrets themselves) - RDI logs - RDI component versions -- Output from the [`redis-di status`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-status" >}}) command - Text of the `config.yaml` file - Text of the Job configuration files -- [optional] RDI DLQ streams content - Rejected records along with the reason for their rejection (should not exist in production)