Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6357,7 +6357,7 @@ menu:
url: observability_pipelines/destinations/datadog_archives/
parent: observability_pipelines_destinations
weight: 406
- name: Datadog CloudPrem
- name: BYOC Logs
url: observability_pipelines/destinations/cloudprem/
parent: observability_pipelines_destinations
identifier: observability_pipelines_datadog_cloudprem
Expand Down Expand Up @@ -6948,7 +6948,7 @@ menu:
parent: log_troubleshooting
identifier: log_troubleshooting_live_tail
weight: 1101
- name: CloudPrem
- name: BYOC Logs
pre: upload-wui
url: cloudprem/
parent: log_management_heading
Expand Down
2 changes: 1 addition & 1 deletion content/en/all_guides.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Guides in the Datadog documentation are pages that provide background knowledge,
{{< whatsnext desc="Log Management:">}}
{{< nextlink href="/logs/guide" >}}<i class="icon-log" style="font-size:25px;color:#7c3eb9;vertical-align:middle"></i>&nbsp;&nbsp;&nbsp;&nbsp;Log Management{{< /nextlink >}}
{{< nextlink href="/observability_pipelines/guide" >}}<i class="icon-pipelines" style="font-size:25px;color:#7c3eb9;vertical-align:middle"></i>&nbsp;&nbsp;&nbsp;&nbsp;Observability Pipelines{{< /nextlink >}}
{{< nextlink href="/cloudprem/guides/" >}}<i class="icon-upload-wui" style="font-size:25px;color:#7c3eb9;vertical-align:middle"></i>&nbsp;&nbsp;&nbsp;&nbsp;CloudPrem{{< /nextlink >}}
{{< nextlink href="/cloudprem/guides/" >}}<i class="icon-upload-wui" style="font-size:25px;color:#7c3eb9;vertical-align:middle"></i>&nbsp;&nbsp;&nbsp;&nbsp;BYOC Logs{{< /nextlink >}}
{{< /whatsnext >}}

{{< whatsnext desc="Administration:">}}
Expand Down
24 changes: 12 additions & 12 deletions content/en/cloudprem/_index.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
---
title: CloudPrem
description: Learn how to deploy and manage Datadog CloudPrem, a self-hosted log management solution for cost-effective log ingestion, processing, indexing, and search capabilities
title: BYOC Logs
description: Learn how to deploy and manage Datadog BYOC Logs, a self-hosted log management solution for cost-effective log ingestion, processing, indexing, and search capabilities
further_reading:
- link: "https://www.datadoghq.com/blog/introducing-datadog-cloudprem/"
tag: "Blog"
text: "Store and search logs at petabyte scale in your own infrastructure with Datadog CloudPrem"
text: "Store and search logs at petabyte scale in your own infrastructure with Datadog BYOC Logs"
---

{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="CloudPrem is in Preview" >}}
Join the CloudPrem Preview to access new self-hosted log management features.
{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="BYOC Logs is in Preview" >}}
Join the BYOC Logs Preview to access new self-hosted log management features.
{{< /callout >}}

CloudPrem is Datadog BYOC log management solution which offers a cost-effective solution for organizations that want to keep their data in their own cloud accounts and control costs over time using their existing cloud provider discounts. It is built to meet stringent security, data residency, regulatory and high-volume requirements.
BYOC Logs is Datadog's log management solution for organizations that want to keep their data in their own cloud accounts and control costs using their existing cloud provider discounts. It is built to meet stringent security, data residency, regulatory, and high-volume requirements.


## Get started

{{< whatsnext desc="Follow our guides to get CloudPrem up and running:">}}
{{< nextlink href="/cloudprem/introduction/" >}}What is CloudPrem?{{< /nextlink >}}
{{< nextlink href="/cloudprem/quickstart/" >}}Quickstart: Run CloudPrem locally in 5 minutes{{< /nextlink >}}
{{< nextlink href="/cloudprem/install/" >}}Installation - Deploy CloudPrem on AWS, Azure, or custom Kubernetes{{< /nextlink >}}
{{< nextlink href="/cloudprem/ingest/agent/" >}}Ingest Logs - Configure the Datadog Agent to send logs to CloudPrem{{< /nextlink >}}
{{< whatsnext desc="Follow our guides to get BYOC Logs up and running:">}}
{{< nextlink href="/cloudprem/introduction/" >}}What is BYOC Logs?{{< /nextlink >}}
{{< nextlink href="/cloudprem/quickstart/" >}}Quickstart: Run BYOC Logs locally in 5 minutes{{< /nextlink >}}
{{< nextlink href="/cloudprem/install/" >}}Installation - Deploy BYOC Logs on AWS, Azure, or custom Kubernetes{{< /nextlink >}}
{{< nextlink href="/cloudprem/ingest/agent/" >}}Ingest Logs - Configure the Datadog Agent to send logs to BYOC Logs{{< /nextlink >}}
{{< nextlink href="/cloudprem/operate/search_logs" >}}Search Logs - Explore your logs in the Datadog Log Explorer{{< /nextlink >}}
{{< nextlink href="/cloudprem/guides/" >}}Guides - Step-by-step guides for CloudPrem features and integrations{{< /nextlink >}}
{{< nextlink href="/cloudprem/guides/" >}}Guides - Step-by-step guides for BYOC Logs features and integrations{{< /nextlink >}}
{{< /whatsnext >}}

## Further reading
Expand Down
16 changes: 8 additions & 8 deletions content/en/cloudprem/configure/_index.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
---
title: Configure CloudPrem
description: Learn how to configure and customize your CloudPrem deployment for optimal performance and security
title: Configure BYOC Logs
description: Learn how to configure and customize your BYOC Logs deployment for optimal performance and security
further_reading:
- link: "/cloudprem/install/"
tag: "Documentation"
text: "Install CloudPrem"
text: "Install BYOC Logs"
- link: "/cloudprem/operate/sizing/"
tag: "Documentation"
text: "Size your cluster"
---

{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="CloudPrem is in Preview" >}}
Join the CloudPrem Preview to access new self-hosted log management features.
{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="BYOC Logs is in Preview" >}}
Join the BYOC Logs Preview to access new self-hosted log management features.
{{< /callout >}}

## Overview

After installing CloudPrem, you can configure your deployment to meet your environment, security, and performance requirements. Key configuration areas include account integration, cloud resource setup, cluster sizing, ingress, and processing options. These settings allow you to tailor CloudPrem for your specific needs.
After installing BYOC Logs, you can configure your deployment to meet your environment, security, and performance requirements. Key configuration areas include account integration, cloud resource setup, cluster sizing, ingress, and processing options. These settings allow you to tailor BYOC Logs for your specific needs.

If you don't see the CloudPrem entry in the Logs menu, it means CloudPrem is not activated on your account. Join the [CloudPrem Preview][1] to activate CloudPrem on your account.
If you don't see the BYOC Logs entry in the Logs menu, it means BYOC Logs is not activated on your account. Join the [BYOC Logs Preview][1] to activate BYOC Logs on your account.

{{< whatsnext desc="Customize your CloudPrem deployment:">}}
{{< whatsnext desc="Customize your BYOC Logs deployment:">}}
{{< nextlink href="/cloudprem/configure/indexes/" >}}Configure indexes{{< /nextlink >}}
{{< nextlink href="/cloudprem/configure/ingress/" >}}Configure Ingress{{< /nextlink >}}
{{< nextlink href="/cloudprem/configure/lambda/" >}}Lambda Search Offloading{{< /nextlink >}}
Expand Down
20 changes: 10 additions & 10 deletions content/en/cloudprem/configure/indexes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Indexes
description: Learn how to configure multiple indexes with custom filters and retention policies for your CloudPrem cluster
description: Learn how to configure multiple indexes with custom filters and retention policies for your BYOC Logs cluster
aliases:
- /cloudprem/configure/retention/
further_reading:
Expand All @@ -12,19 +12,19 @@ further_reading:
text: "Search Logs"
---

{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="CloudPrem is in Preview" >}}
Join the CloudPrem Preview to access new self-hosted log management features.
{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="BYOC Logs is in Preview" >}}
Join the BYOC Logs Preview to access new self-hosted log management features.
{{< /callout >}}

By default, a CloudPrem cluster stores all logs in a single index with a single retention policy. With multiple indexes, you can segment logs by defining filter queries and assigning a different retention period to each index. For example, you can retain audit logs for 1 year while keeping debug logs for only 3 days.
By default, a BYOC Logs cluster stores all logs in a single index with a single retention policy. With multiple indexes, you can segment logs by defining filter queries and assigning a different retention period to each index. For example, you can retain audit logs for 1 year while keeping debug logs for only 3 days.

To view and manage your CloudPrem indexes, navigate to the [CloudPrem page][1] in Datadog. Select a cluster and click {{< ui >}}View Indexes{{< /ui >}} to access the index configuration.
To view and manage your BYOC Logs indexes, navigate to the [BYOC Logs page][1] in Datadog. Select a cluster and click {{< ui >}}View Indexes{{< /ui >}} to access the index configuration.

{{< img src="cloudprem/indexes/indexes_configuration.png" alt="CloudPrem indexes configuration page showing three indexes with filters and retention periods" style="width:80%;">}}
{{< img src="cloudprem/indexes/indexes_configuration.png" alt="BYOC Logs indexes configuration page showing three indexes with filters and retention periods" style="width:80%;">}}

## Indexes filters

When a log is ingested, CloudPrem evaluates each index's filter from top to bottom and routes the log to the **first matching index**. This means index order matters:
When a log is ingested, BYOC Logs evaluates each index's filter from top to bottom and routes the log to the **first matching index**. This means index order matters:

- Place indexes with more specific filters above indexes with broader filters. For example, `source:security env:production` should appear above `source:security`.
- A catch-all index with a `*` filter at the bottom ensures that no logs are dropped.
Expand All @@ -36,13 +36,13 @@ You can reorder indexes at any time by dragging rows or using the {{< ui >}}Move

Each index has its own retention period, which determines how long logs are stored before automatic deletion.

<div class="alert alert-info">If you are upgrading from a previous CloudPrem version, the index retention configured through the Datadog UI takes precedence over the global retention set in the Helm chart values.</div>
<div class="alert alert-info">If you are upgrading from a previous BYOC Logs version, the index retention configured through the Datadog UI takes precedence over the global retention set in the Helm chart values.</div>

## Searching across indexes

To query logs stored in CloudPrem, select one or more CloudPrem indexes in the [Log Explorer][2]. You can select a specific index to narrow your search, or select all indexes in a cluster to search across them. From the index configuration page, use {{< ui >}}View in Log Explorer{{< /ui >}} to open a filtered view for a given index.
To query logs stored in BYOC Logs, select one or more BYOC Logs indexes in the [Log Explorer][2]. You can select a specific index to narrow your search, or select all indexes in a cluster to search across them. From the index configuration page, use {{< ui >}}View in Log Explorer{{< /ui >}} to open a filtered view for a given index.

For more information, see [Search CloudPrem Logs][3].
For more information, see [Search BYOC Logs][3].

## Further reading

Expand Down
30 changes: 15 additions & 15 deletions content/en/cloudprem/configure/ingress.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
---
title: CloudPrem Ingress Configuration
description: Learn how to configure and manage ingress controllers for your CloudPrem deployment
title: BYOC Logs Ingress Configuration
description: Learn how to configure and manage ingress controllers for your BYOC Logs deployment
further_reading:
- link: "/cloudprem/ingest/"
tag: "Documentation"
text: "Set up Log Ingestion"
- link: "/cloudprem/operate/monitoring/"
tag: "Documentation"
text: "Monitor CloudPrem"
text: "Monitor BYOC Logs"
---

{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="CloudPrem is in Preview" >}}
Join the CloudPrem Preview to access new self-hosted log management features.
{{< callout url="https://www.datadoghq.com/product-preview/cloudprem/" btn_hidden="false" header="BYOC Logs is in Preview" >}}
Join the BYOC Logs Preview to access new self-hosted log management features.
{{< /callout >}}

## Overview

Ingress is a critical component of your CloudPrem deployment. The Helm chart automatically creates two ingress configurations called public ingress and internal ingress. If the AWS Load Balancer Controller is installed on the cluster, it provisions one ALB per ingress configuration. Each load balancer can be further configured using ingress annotations.
Ingress is a critical component of your BYOC Logs deployment. The Helm chart automatically creates two ingress configurations called public ingress and internal ingress. If the AWS Load Balancer Controller is installed on the cluster, it provisions one ALB per ingress configuration. Each load balancer can be further configured using ingress annotations.

## Public ingress

<div class="alert alert-danger">Only the CloudPrem gRPC API endpoints (paths starting with <code>/cloudprem</code>) perform mutual TLS authentication. Exposing any other endpoints through the public ingress introduces a security risk, as those endpoints would be accessible over the internet without authentication. Always restrict non-gRPC endpoints to the internal ingress. </div>
<div class="alert alert-danger">Only the BYOC Logs gRPC API endpoints (paths starting with <code>/cloudprem</code>) perform mutual TLS authentication. Exposing any other endpoints through the public ingress introduces a security risk, as those endpoints would be accessible over the internet without authentication. Always restrict non-gRPC endpoints to the internal ingress. </div>

The public ingress is essential for enabling Datadog's control plane and query service to manage and query CloudPrem clusters over the public internet. It provides secure access to the CloudPrem gRPC API through the following mechanisms:
The public ingress is essential for enabling Datadog's control plane and query service to manage and query BYOC Logs clusters over the public internet. It provides secure access to the BYOC Logs gRPC API through the following mechanisms:
- Creates an internet-facing AWS Application Load Balancer (ALB) that accepts traffic from Datadog services
- Implements TLS encryption with termination at the load balancer level
- Uses HTTP/2 (gRPC) for communication between the ALB and CloudPrem cluster
- Uses HTTP/2 (gRPC) for communication between the ALB and BYOC Logs cluster
- Requires mutual TLS (mTLS) authentication where Datadog services must present valid client certificates
- Configures the ALB in TLS passthrough mode to forward client certificates to CloudPrem pods with the `X-Amzn-Mtls-Clientcert` header
- Configures the ALB in TLS passthrough mode to forward client certificates to BYOC Logs pods with the `X-Amzn-Mtls-Clientcert` header
- Rejects requests that are missing valid client certificates or the certificate header

This setup ensures that only authenticated Datadog services can access the CloudPrem cluster while maintaining secure encrypted communication end-to-end.
This setup ensures that only authenticated Datadog services can access the BYOC Logs cluster while maintaining secure encrypted communication end-to-end.

{{< img src="/cloudprem/ingress/cloudprem_public_ingress1.png" alt="Diagram showing CloudPrem public ingress architecture with Datadog services connecting through an internet-facing AWS ALB using mTLS authentication to access the CloudPrem gRPC API" style="width:100%;" >}}
{{< img src="/cloudprem/ingress/cloudprem_public_ingress1.png" alt="Diagram showing BYOC Logs public ingress architecture with Datadog services connecting through an internet-facing AWS ALB using mTLS authentication to access the BYOC Logs gRPC API" style="width:100%;" >}}

### IP Allowlisting

The Datadog control plane and query services connect to CloudPrem clusters using a set of fixed IP ranges, which can be retrieved for each Datadog site from the Datadog [IP Ranges API][1], specifically under the "webhooks" section. For example, to fetch the IP ranges for the datadoghq.eu site, you can run:
The Datadog control plane and query services connect to BYOC Logs clusters using a set of fixed IP ranges, which can be retrieved for each Datadog site from the Datadog [IP Ranges API][1], specifically under the "webhooks" section. For example, to fetch the IP ranges for the datadoghq.eu site, you can run:
```
curl -X GET "https://ip-ranges.datadoghq.eu/" \
-H "Accept: application/json" |
Expand All @@ -49,7 +49,7 @@ The internal ingress enables log ingestion from Datadog Agents and other log col

{{< img src="/cloudprem/ingress/internal_ingress.png" alt=" Internal ingress with ALB provisioned by Helm chart" style="width:100%;" >}}

By default, the chart creates an internal AWS Application Load Balancer (ALB) to route HTTP traffic to the appropriate CloudPrem services based on the requested API endpoint path. However, if you prefer to use your own ingress controller (such as HAProxy, NGINX, or Traefik), you can disable the default internal ALB and configure your controller with the following routing rules:
By default, the chart creates an internal AWS Application Load Balancer (ALB) to route HTTP traffic to the appropriate BYOC Logs services based on the requested API endpoint path. However, if you prefer to use your own ingress controller (such as HAProxy, NGINX, or Traefik), you can disable the default internal ALB and configure your controller with the following routing rules:

```
rules:
Expand Down Expand Up @@ -103,7 +103,7 @@ rules:

```

{{< img src="/cloudprem/ingress/internal_ingress_nginx_controller.png" alt="CloudPrem internal ingress configuration using NGINX ingress controller showing path routing to indexer, metastore, and searcher services" style="width:100%;" >}}
{{< img src="/cloudprem/ingress/internal_ingress_nginx_controller.png" alt="BYOC Logs internal ingress configuration using NGINX ingress controller showing path routing to indexer, metastore, and searcher services" style="width:100%;" >}}

## Further reading

Expand Down
Loading
Loading