diff --git a/content/en/blog/2023/end-user-q-and-a-02.md b/content/en/blog/2023/end-user-q-and-a-02.md index 590e82b26832..651ab8da740b 100644 --- a/content/en/blog/2023/end-user-q-and-a-02.md +++ b/content/en/blog/2023/end-user-q-and-a-02.md @@ -198,7 +198,7 @@ Uplight currently has a few different Collector configurations: Doug’s ultimate goal is for any deployment in any environment to be able to easily send telemetry to an -[OTel Collector gateway](/docs/collector/deployment/gateway/). +[OTel Collector gateway](/docs/collector/deploy/gateway/). Collectors at Uplight are typically run and maintained by the infrastructure team, unless individual teams decide to take ownership of their own Collectors. diff --git a/content/en/blog/2023/end-user-q-and-a-03.md b/content/en/blog/2023/end-user-q-and-a-03.md index 906d0230974e..8893f83fd9c3 100644 --- a/content/en/blog/2023/end-user-q-and-a-03.md +++ b/content/en/blog/2023/end-user-q-and-a-03.md @@ -253,14 +253,14 @@ Iris and team have not played around with this beta feature. Because there are so many Kubernetes clusters, having a single OTel Collector would be a bottleneck in terms of load and single point of failure. The team currently has one -[OpenTelemetry Collector agent](/docs/collector/deployment/agent/) per +[OpenTelemetry Collector agent](/docs/collector/deploy/agent/) per Kubernetes cluster. The end goal is to replace those agents with the [OTel Operator](/docs/platforms/kubernetes/operator/) instead, which allows you to deploy and configure the OTel Collector and inject and configure auto-instrumentation. Everything is then sent to a central OTel Collector (i.e. an -[OTel Collector gateway](/docs/collector/deployment/gateway/)) per data center, +[OTel Collector gateway](/docs/collector/deploy/gateway/)) per data center, where data masking (using the [transform processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor), or diff --git a/content/en/blog/2023/end-user-q-and-a-04.md b/content/en/blog/2023/end-user-q-and-a-04.md index 08c4dabc75ea..61da78c7403f 100644 --- a/content/en/blog/2023/end-user-q-and-a-04.md +++ b/content/en/blog/2023/end-user-q-and-a-04.md @@ -427,7 +427,7 @@ That's why OTel now has a really good Prometheus exporter, so it can do both." ### The centralized OTel Collector gateway -There is a [centralized gateway](/docs/collector/deployment/gateway/) in-flight, +There is a [centralized gateway](/docs/collector/deploy/gateway/) in-flight, which is part of the Collector chain Jacob mentioned earlier. The effort is centered around [Arrow](https://arrow.apache.org/). Lightstep has done some work around improving "the processing speed and ingress costs of OTel data by using diff --git a/content/en/blog/2024/otel-collector-anti-patterns/index.md b/content/en/blog/2024/otel-collector-anti-patterns/index.md index 1119dbc1edff..4ff8112334d9 100644 --- a/content/en/blog/2024/otel-collector-anti-patterns/index.md +++ b/content/en/blog/2024/otel-collector-anti-patterns/index.md @@ -34,12 +34,12 @@ Because one is often not enough. There are two deployment modes for Collectors: agent mode and gateway mode, and both are needed. -In [agent mode](/docs/collector/deployment/agent/), the Collector sits next to +In [agent mode](/docs/collector/deploy/agent/), the Collector sits next to the application or on the same host as the application. ![OTel Collector Agent Mode](otel-collector-agent.png) -In [gateway mode](/docs/collector/deployment/gateway/), telemetry data is sent +In [gateway mode](/docs/collector/deploy/gateway/), telemetry data is sent to a load balancer, which then determines how to distribute the load amongst a pool of Collectors. Because you have a pool of Collectors, should one Collector in that pool fail, one of the other Collectors in the pool can take over. This @@ -136,7 +136,7 @@ features, bug fixes, performance improvements, and security fixes. OpenTelemetry allows you to send telemetry signals from your application to an observability backend in one of two ways: -- [Directly from the application](/docs/collector/deployment/no-collector/) +- [Directly from the application](/docs/collector/deploy/other/no-collector/) - [Via the OpenTelemetry Collector](/docs/collector/) Sending telemetry data “direct from application” for non-production systems is diff --git a/content/en/blog/2024/scaling-collectors.md b/content/en/blog/2024/scaling-collectors.md index 41210f47b890..0c530077c27a 100644 --- a/content/en/blog/2024/scaling-collectors.md +++ b/content/en/blog/2024/scaling-collectors.md @@ -7,10 +7,10 @@ cSpell:ignore: ansible associated Ishan ishanjainn Jain --- You can scale the deployment of -[OpenTelemetry Collector](/docs/collector/deployment/) across multiple Linux +[OpenTelemetry Collector](/docs/collector/deploy/) across multiple Linux hosts through [Ansible](https://www.ansible.com/), to function both as -[gateways](/docs/collector/deployment/gateway/) and -[agents](/docs/collector/deployment/agent/) within your observability +[gateways](/docs/collector/deploy/gateway/) and +[agents](/docs/collector/deploy/agent/) within your observability architecture. Using the OpenTelemetry Collector in this dual capacity enables a robust collection and forwarding of metrics, traces, and logs to analysis and visualization platforms. diff --git a/content/en/docs/collector/deploy/_index.md b/content/en/docs/collector/deploy/_index.md new file mode 100644 index 000000000000..35e999e51374 --- /dev/null +++ b/content/en/docs/collector/deploy/_index.md @@ -0,0 +1,17 @@ +--- +title: Deployment +description: Patterns you can apply to deploy the OpenTelemetry collector +weight: 3 +--- + +The OpenTelemetry Collector is a single binary that you can deploy in different ways for different use cases. This section describes deployment patterns, their use cases, pros and cons, and best practices for Collector configurations across environments and backends. For deployment security considerations, see [Collector hosting best practices][security]. + +## Resources + +- KubeCon NA 2021 talk on [OpenTelemetry Collector deployment patterns][y-patterns] +- [Deployment patterns][gh-patterns] accompanying the talk + +[security]: /docs/security/hosting-best-practices/ +[gh-patterns]: + https://github.com/jpkrohling/opentelemetry-collector-deployment-patterns/ +[y-patterns]: https://www.youtube.com/watch?v=WhRrwSHDBFs diff --git a/content/en/docs/collector/deployment/agent.md b/content/en/docs/collector/deploy/agent.md similarity index 95% rename from content/en/docs/collector/deployment/agent.md rename to content/en/docs/collector/deploy/agent.md index 97aa64d2b2b3..16a142d28a7c 100644 --- a/content/en/docs/collector/deployment/agent.md +++ b/content/en/docs/collector/deploy/agent.md @@ -6,7 +6,7 @@ weight: 2 cSpell:ignore: prometheusremotewrite --- -The agent collector deployment pattern consists of applications — +The agent Collector deployment pattern consists of applications — [instrumented][instrumentation] with an OpenTelemetry SDK using [OpenTelemetry protocol (OTLP)][otlp] — or other collectors (using the OTLP exporter) that send telemetry signals to a [collector][] instance running with the application or on @@ -27,7 +27,7 @@ follows: you manually instrument, say, a [Java application to export metrics][instrument-java-metrics] using the OpenTelemetry Java SDK. In the context of the app, you would set the `OTEL_METRICS_EXPORTER` to `otlp` (which is the default value) and configure the [OTLP exporter][otlp-exporter] with the -address of your collector, for example (in Bash or `zsh` shell): +address of your Collector; for example, in Bash or zsh: ```shell export OTEL_EXPORTER_OTLP_ENDPOINT=http://collector.example.com:4318 @@ -106,7 +106,7 @@ service: If you want to try it out for yourself, you can have a look at the end-to-end [Java][java-otlp-example] or [Python][py-otlp-example] examples. -## Tradeoffs +## Trade-offs Pros: diff --git a/content/en/docs/collector/deployment/gateway/index.md b/content/en/docs/collector/deploy/gateway/index.md similarity index 99% rename from content/en/docs/collector/deployment/gateway/index.md rename to content/en/docs/collector/deploy/gateway/index.md index 0027fd889778..31f166145394 100644 --- a/content/en/docs/collector/deployment/gateway/index.md +++ b/content/en/docs/collector/deploy/gateway/index.md @@ -204,7 +204,7 @@ collector. ## Combined deployment of Collectors as agents and gateways Often a deployment of multiple OpenTelemetry collectors involves running both -Collector as gateways and as [agents](/docs/collector/deployment/agent/). +Collector as gateways and as [agents](/docs/collector/deploy/agent/). The following diagram shows an architecture for such a combined deployment: diff --git a/content/en/docs/collector/deployment/gateway/otel-gateway-arch.svg b/content/en/docs/collector/deploy/gateway/otel-gateway-arch.svg similarity index 100% rename from content/en/docs/collector/deployment/gateway/otel-gateway-arch.svg rename to content/en/docs/collector/deploy/gateway/otel-gateway-arch.svg diff --git a/content/en/docs/collector/deploy/no-collector.md b/content/en/docs/collector/deploy/no-collector.md new file mode 100644 index 000000000000..c63087691a27 --- /dev/null +++ b/content/en/docs/collector/deploy/no-collector.md @@ -0,0 +1,28 @@ +--- +title: No Collector +description: Why and how to send signals directly from applications to backends +weight: 1 +--- + +The simplest pattern is to not use a Collector. In this pattern, applications [instrumented][instrumentation] with an OpenTelemetry SDK export telemetry signals (traces, metrics, logs) directly to a backend: + +![No collector deployment concept](../../img/otel-sdk.svg) + +## Example + +See the [code instrumentation for programming languages][instrumentation] for end-to-end examples of exporting signals from your application directly to a backend. + +## Trade-offs + +Pros: + +- Simple to use (especially in development and test environments) +- No additional moving parts to operate + +Cons: + +- Requires code changes when collection, processing, or ingestion requirements change +- Strong coupling between application code and the backend +- Limited exporters per language implementation + +[instrumentation]: /docs/languages/ diff --git a/content/en/docs/collector/deployment/_index.md b/content/en/docs/collector/deployment/_index.md deleted file mode 100644 index decd4ebd80ca..000000000000 --- a/content/en/docs/collector/deployment/_index.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Deployment -description: Patterns you can apply to deploy the OpenTelemetry collector -weight: 3 ---- - -The OpenTelemetry Collector consists of a single binary which you can use in -different ways, for different use cases. This section describes deployment -patterns, their use cases along with pros and cons and best practices for -collector configurations for cross-environment and multi-backend deployments. -For deployment security considerations, see [Collector hosting best -practices][security]. - -## Resources - -- KubeCon NA 2021 Talk on [OpenTelemetry Collector Deployment - Patterns][y-patterns] -- [Deployment Patterns][gh-patterns] accompanying the talk - -[security]: /docs/security/hosting-best-practices/ -[gh-patterns]: - https://github.com/jpkrohling/opentelemetry-collector-deployment-patterns/ -[y-patterns]: https://www.youtube.com/watch?v=WhRrwSHDBFs diff --git a/content/en/docs/collector/deployment/no-collector.md b/content/en/docs/collector/deployment/no-collector.md deleted file mode 100644 index 725826e51965..000000000000 --- a/content/en/docs/collector/deployment/no-collector.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: No Collector -description: Why and how to send signals directly from app to backends -weight: 1 ---- - -The simplest pattern is not to use a collector at all. This pattern consists of -applications [instrumented][instrumentation] with an OpenTelemetry SDK that -export telemetry signals (traces, metrics, logs) directly into a backend: - -![No collector deployment concept](../../img/otel-sdk.svg) - -## Example - -See the [code instrumentation for programming languages][instrumentation] for -concrete end-to-end examples for how to export signals from your app directly -into a backend. - -## Tradeoffs - -Pros: - -- Simple to use (especially in a dev/test environment) -- No additional moving parts to operate (in production environments) - -Cons: - -- Requires code changes if collection, processing, or ingestion changes -- Strong coupling between the application code and the backend -- There are limited number of exporters per language implementation - -[instrumentation]: /docs/languages/ diff --git a/content/en/docs/languages/php/exporters.md b/content/en/docs/languages/php/exporters.md index ca2efdf54a87..f8f03616cd41 100644 --- a/content/en/docs/languages/php/exporters.md +++ b/content/en/docs/languages/php/exporters.md @@ -195,5 +195,5 @@ will not hold up request processing. To minimize the impact of slow transport of telemetry data, particularly for external or cloud-based backends, you should consider using the [OpenTelemetry Collector](/docs/collector/) as an -[agent](/docs/collector/deployment/agent/). The agent can quickly accept, then +[agent](/docs/collector/deploy/agent/). The agent can quickly accept, then batch send telemetry data to the backend. diff --git a/content/en/docs/platforms/kubernetes/getting-started.md b/content/en/docs/platforms/kubernetes/getting-started.md index 5fe36190372e..f437e7b9412f 100644 --- a/content/en/docs/platforms/kubernetes/getting-started.md +++ b/content/en/docs/platforms/kubernetes/getting-started.md @@ -34,8 +34,8 @@ tools at its disposal which allow it to efficiently collect all this data and enhance it in meaningful ways. To collect all the data, we'll need two installations of the collector, one as a -[Daemonset](/docs/collector/deployment/agent/) and one as a -[Deployment](/docs/collector/deployment/gateway/). The Daemonset installation of +[Daemonset](/docs/collector/deploy/agent/) and one as a +[Deployment](/docs/collector/deploy/gateway/). The Daemonset installation of the collector will be used to collect telemetry emitted by services, logs, and metrics for nodes, pods, and containers. The deployment installation of the collector will be used to collect metrics for the cluster and events.