Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions content/commands/ft.aggregate.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,8 @@ if set, overrides the timeout parameter of the module.
defines one or more value parameters. Each parameter has a name and a value.

You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set `DIALECT` to `2` or greater than `2`.

**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
</details>

<details open>
Expand Down
12 changes: 12 additions & 0 deletions content/commands/ft.search.md
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,8 @@ defines one or more value parameters. Each parameter has a name and a value.
You can reference parameters in the `query` by a `$`, followed by the parameter name, for example, `$user`. Each such reference in the search query to a parameter name is substituted by the corresponding parameter value. For example, with parameter definition `PARAMS 4 lon 29.69465 lat 34.95126`, the expression `@loc:[$lon $lat 10 km]` is evaluated to `@loc:[29.69465 34.95126 10 km]`. You cannot reference parameters in the query string where concrete values are not allowed, such as in field names, for example, `@loc`. To use `PARAMS`, set
[`DIALECT`]({{< relref "/develop/ai/search-and-query/advanced-concepts/dialects#dialect-2" >}})
to `2` or greater than `2` (this requires [RediSearch v2.4](https://github.com/RediSearch/RediSearch/releases/tag/v2.4.3) or above).

**Query attributes**: You can also use `PARAMS` to pass values to [query attributes]({{< relref "/develop/ai/search-and-query/advanced-concepts/query_syntax#query-attributes" >}}) in vector search queries. For example, `$SHARD_K_RATIO` controls cluster optimization for vector KNN queries by setting the ratio of results each shard retrieves relative to the requested `top_k`. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for details.
</details>

<details open>
Expand Down Expand Up @@ -679,6 +681,16 @@ Search for books with semantically similar title to _Planet Earth_. Return top 1
{{< / highlight >}}
</details>

<details open>
<summary><b>Vector search with cluster optimization</b></summary>

Search for books with semantically similar title to _Planet Earth_ using cluster optimization. Each shard retrieves 60% of the requested results for improved performance in Redis cluster environments.

{{< highlight bash >}}
127.0.0.1:6379> FT.SEARCH books-idx "*=>[KNN 100 @title_embedding $query_vec]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: title_score}" PARAMS 2 query_vec <"Planet Earth" embedding BLOB> SORTBY title_score DIALECT 2
{{< / highlight >}}
</details>

<details open>
<summary><b>Search for a phrase using SLOP</b></summary>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,8 @@ The supported attributes are:

As of v2.6.1, the query attributes syntax supports these additional attributes:

* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).
* **$yield_distance_as**: specifies the distance field name, used for later sorting and/or returning, for clauses that yield some distance metric. It is currently supported for vector queries only (both KNN and range).
* **$shard_k_ratio**: controls how many results each shard retrieves relative to the requested top_k in cluster setups. Value range: 0.1 - 1.0 (default: 1.0). Only applicable to vector KNN queries in Redis cluster environments. See [cluster-specific query parameters]({{< relref "develop/ai/search-and-query/vectors#cluster-specific-query-parameters" >}}) for detailed information.
* **vector query params**: pass optional parameters for [vector queries]({{< relref "develop/ai/search-and-query/vectors#querying-vector-fields" >}}) in key-value format.

## A few query examples
Expand Down Expand Up @@ -458,6 +459,10 @@ As of v2.6.1, the query attributes syntax supports these additional attributes:

@age:[(18 +inf]

* Vector search with cluster optimization - retrieve 100 nearest neighbors with each shard providing 50% of results:

*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}

## Mapping common SQL predicates to Redis Query Engine

| SQL Condition | Redis Query Engine Equivalent | Comments |
Expand Down
26 changes: 25 additions & 1 deletion content/develop/ai/search-and-query/query/vector-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,28 @@ query = (
.return_fields('vector_dist', 'description')
.dialect(2)
)
</!-->
</!-->

## Cluster optimization

In Redis cluster environments, you can optimize vector search performance using the `$SHARD_K_RATIO` query attribute. This parameter controls how many results each shard retrieves relative to the requested `top_k`, creating a tunable trade-off between accuracy and performance.

### Basic cluster optimization

Retrieve 100 nearest neighbors with each shard providing 60% of the requested results:

{{< clients-example query_vector vector3 >}}
FT.SEARCH idx:bikes_vss "(*)=>[KNN 100 @vector $query_vector]=>{$SHARD_K_RATIO: 0.6; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY vector_distance ASC RETURN 2 "vector_distance" "description" DIALECT 2
{{< /clients-example >}}

### Combined with filtering

You can combine `$SHARD_K_RATIO` with pre-filtering to optimize searches on specific subsets of data:

{{< clients-example query_vector vector4 >}}
FT.SEARCH idx:bikes_vss "(@brand:trek)=>[KNN 50 @vector $query_vector]=>{$SHARD_K_RATIO: 0.4; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 "query_vector" "Z\xf8\x15:\xf23\xa1\xbfZ\x1dI>\r\xca9..." SORTBY similarity ASC RETURN 2 "similarity" "description" DIALECT 2
{{< /clients-example >}}

{{% alert title="Note" color="warning" %}}
The `$SHARD_K_RATIO` parameter is only applicable in Redis cluster environments and has no effect in standalone Redis instances.
{{% /alert %}}
75 changes: 75 additions & 0 deletions content/develop/ai/search-and-query/vectors/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,24 @@ By default, Redis selects the best filter mode to optimize query execution. You
| `HYBRID_POLICY` | Specifies the filter mode to use during vector search with filters (hybrid). | `BATCHES` or `ADHOC_BF` |
| `BATCH_SIZE` | A fixed batch size to use in every iteration when the `BATCHES` policy is auto-selected or requested. | Positive integer. |

### Cluster-specific query parameters

**$SHARD_K_RATIO**

The `$SHARD_K_RATIO` parameter controls how many results each shard retrieves relative to the requested `top_k` in Redis cluster environments. This creates a tunable trade-off between accuracy and performance.

| Parameter | Description | Value Range | Default |
|:-----------------|:------------|:------------|:--------|
| `$SHARD_K_RATIO` | Ratio of `top_k` results to fetch per shard in cluster setups. Calculation: `shard_results = top_k × ratio`. | 0.1 - 1.0 (up to 2 decimal places) | 1.0 |

**Important considerations:**

- Only applicable to vector KNN queries in Redis cluster environments
- Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results
- If the user-defined ratio is lower than `top_k / #shards`, the server ignores it and uses the minimum required ratio
- Unbalanced shards returning fewer results than required may lead to fewer total results than requested in `top_k`
- Invalid ratios return descriptive error messages


### Index-specific query parameters

Expand Down Expand Up @@ -523,6 +541,63 @@ FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB]=>{$HYBRID_

To explore additional Python vector search examples, review recipes for the [`Redis Python`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) client library and the [`Redis Vector Library`](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb).

### Cluster optimization examples

The following examples demonstrate using `$SHARD_K_RATIO` to optimize vector search performance in Redis cluster environments.

Return the top 100 nearest neighbors with each shard providing 50% of the requested results (50 results per shard):

```
FT.SEARCH documents "*=>[KNN 100 @doc_embedding $BLOB]=>{$SHARD_K_RATIO: 0.5; $YIELD_DISTANCE_AS: vector_distance}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY vector_distance DIALECT 2
```

Combine cluster optimization with filtering and other query attributes. Among movies with `action` category, return top 50 nearest neighbors with 30% shard ratio and custom EF_RUNTIME:

```
FT.SEARCH movies "(@category:{action})=>[KNN 50 @movie_embedding $BLOB]=>{$SHARD_K_RATIO: 0.3; $EF_RUNTIME: 150; $YIELD_DISTANCE_AS: movie_distance}" PARAMS 4 BLOB "\x12\xa9\xf5\x6c" EF 150 SORTBY movie_distance DIALECT 2
```

Use a higher ratio for better accuracy when precision is more important than performance:

```
FT.SEARCH products "*=>[KNN 20 @product_embedding $BLOB]=>{$SHARD_K_RATIO: 0.8; $YIELD_DISTANCE_AS: similarity}" PARAMS 2 BLOB "\x12\xa9\xf5\x6c" SORTBY similarity DIALECT 2
```

### Cluster considerations and best practices

When using `$SHARD_K_RATIO` in Redis cluster environments, consider the following behavioral characteristics and best practices:

#### Minimum results guarantee

Each shard must retrieve at least `max(top_k / #shards, ceil(top_k × ratio))` results to ensure the cluster can return the requested `top_k` results. If your specified ratio would result in fewer results per shard than this minimum, Redis automatically adjusts to the minimum required ratio.

**Example scenarios:**
- `top_k=100`, 4 shards, `ratio=0.5` → 50 results per shard (valid)
- `top_k=100`, 4 shards, `ratio=0.2` → 25 results per shard (minimum: 25, so valid)
- `top_k=100`, 8 shards, `ratio=0.1` → 10 results per shard (minimum: 13, so Redis uses 13)

#### Handling unbalanced shards

In cases where some shards contain fewer documents than required to fulfill the `top_k` results, using a lower ratio may lead to fewer total results than requested. To mitigate this:

1. **Monitor shard distribution**: Ensure your data is relatively balanced across shards
2. **Adjust ratio dynamically**: If you consistently get fewer results than expected, increase the ratio
3. **Use higher ratios for critical queries**: When result completeness is more important than performance

#### Performance vs accuracy trade-offs

| Ratio Range | Use Case | Performance | Accuracy |
|:------------|:---------|:------------|:---------|
| 0.1 - 0.3 | High-performance, approximate results | Excellent | Good |
| 0.4 - 0.6 | Balanced performance and accuracy | Good | Very Good |
| 0.7 - 1.0 | High-accuracy, precision-critical | Fair | Excellent |

#### Error handling

Redis returns descriptive error messages for invalid `$SHARD_K_RATIO` values:
- Values outside the 0.1 - 1.0 range
- Values with more than 2 decimal places
- Non-numeric values

### Range query examples

Expand Down