Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 4 additions & 86 deletions docs/develop/go/workers/serverless-workers/cloud-run.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ For general Worker setup and options that are not specific to Cloud Run, see [Ru

## Configure the Temporal connection {/* #configure-connection */}

The `envconfig` package loads Temporal Client configuration from environment variables and an optional TOML config file, so the Worker code carries no Namespace or credentials.
The `envconfig` package loads [Temporal Client](/develop/go/client/temporal-client) configuration from environment variables and an optional TOML config file, so the Worker code carries no Namespace or credentials.
Set the non-secret values as environment variables on the Worker Pool, and mount the Temporal Cloud API key or TLS material from Secret Manager.
For the full list of supported variables, the config file format, and profiles, see [Environment configuration](/develop/environment-configuration).

Expand All @@ -120,89 +120,7 @@ func MyActivity(ctx context.Context, input MyInput) (string, error) {

For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle).

## Add observability with OpenTelemetry {/* #add-observability */}
## Add observability {/* #add-observability */}

The `go.temporal.io/sdk/contrib/gcp` package provides an OpenTelemetry plugin with defaults suited to a Cloud Run worker pool.
By default, the plugin configures:

- OTLP export to `http://localhost:4317`, the collector sidecar endpoint.
- A replay-safe OpenTelemetry tracer provider.
- Temporal Core metrics with a 60-second export interval.

The underlying metrics and traces are the same ones the Go SDK emits in any environment.
For general observability concepts and the full list of available metrics, see [Observability - Go SDK](/develop/go/platform/observability) and the [SDK metrics reference](/references/sdk-metrics).

Create the plugin and pass it in `client.Options.Plugins`. Client plugins that also implement `worker.Plugin` are applied automatically to Workers created from that Client:

<!--SNIPSTART go-cloud-run-otel-plugin-->
[contrib/gcp/e2e/main.go](https://github.com/temporalio/sdk-go/blob/main/contrib/gcp/e2e/main.go)
```go
otelPlugin, err := gcp.NewOpenTelemetryPlugin(ctx, gcp.OpenTelemetryPluginOptions{})
if err != nil {
return fmt.Errorf("creating GCP OpenTelemetry plugin: %w", err)
}
log.Printf(
"plugin_ready service_name=%q endpoint=%q",
otelPlugin.ServiceName(),
otelPlugin.Endpoint(),
)

temporalClient, err := client.DialContext(ctx, client.Options{
HostPort: cfg.address,
Namespace: cfg.namespace,
Credentials: client.NewAPIKeyStaticCredentials(cfg.apiKey),
Plugins: []client.Plugin{otelPlugin},
})
```
<!--SNIPEND-->

Do not install another OpenTelemetry metrics handler or tracing interceptor on the same Client. The plugin configures both through `go.temporal.io/sdk/contrib/opentelemetry`.

The OTLP endpoint resolves in this order:

1. `OpenTelemetryPluginOptions.Endpoint`.
2. The `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable.
3. `http://localhost:4317`.

### Flush telemetry on shutdown {/* #flush-on-shutdown */}

Stop the Worker and close the Client before shutting the plugin down, so telemetry buffered during the run is exported before the process exits.
Cloud Run stops an instance on scale-in, so reserve time for the flush inside the termination window:

<!--SNIPSTART go-cloud-run-otel-shutdown-->
[contrib/gcp/e2e/main.go](https://github.com/temporalio/sdk-go/blob/main/contrib/gcp/e2e/main.go)
```go
temporalWorker.Stop()
log.Print("worker_stopped")
temporalClient.Close()
log.Print("temporal_client_closed")

flushCtx, cancelFlush := context.WithTimeout(context.Background(), 3*time.Second)
flushErr := otelPlugin.ForceFlush(flushCtx)
cancelFlush()
if flushErr != nil {
log.Printf("plugin_force_flush_failed error=%q", flushErr)
} else {
log.Print("plugin_force_flush_complete")
}

shutdownCtx, cancelShutdown := context.WithTimeout(context.Background(), 3*time.Second)
shutdownErr := otelPlugin.Shutdown(shutdownCtx)
cancelShutdown()
if shutdownErr != nil {
log.Printf("plugin_shutdown_failed error=%q", shutdownErr)
} else {
log.Print("plugin_shutdown_complete")
}
```
<!--SNIPEND-->

`Shutdown` flushes and closes the metric and trace providers the plugin created. Use `ForceFlush` instead when the plugin-owned providers must stay usable afterward.

### Run the collector as a sidecar {/* #collector-sidecar */}

The plugin exports to a collector rather than directly to Google Cloud. Run the collector as a second container in the Worker Pool.
Without a collector at the configured endpoint, telemetry is not delivered.

Use the [Google-Built OpenTelemetry Collector](https://cloud.google.com/stackdriver/docs/instrumentation/google-built-otel), which detects the Cloud Run resource, authenticates through the Worker Pool's service account, and exports metrics to Google Managed Service for Prometheus and traces through the Google Cloud Telemetry API.
For a collector configuration that routes both pipelines, see [the collector config on the Python SDK page](/develop/python/workers/serverless-workers/cloud-run#collector-sidecar). The configuration is the same regardless of which SDK the Worker uses.
A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else.
For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - Go SDK](/develop/go/platform/observability) and the [SDK metrics reference](/references/sdk-metrics).
129 changes: 4 additions & 125 deletions docs/develop/python/workers/serverless-workers/cloud-run.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ keywords:
- cloud run
- gcp
- google cloud
- opentelemetry
- python sdk
- worker
- serverless worker
Expand All @@ -34,7 +33,6 @@ Register Workflows and Activities the same way you would with any other Python W

A Cloud Run Worker needs no Cloud Run-specific package.
The one addition to a standard Worker is Worker Versioning, which is required for Serverless Workers.
The `temporalio.contrib.gcp` package adds an OpenTelemetry plugin with defaults suited to a Cloud Run worker pool, covered below.

For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run).

Expand Down Expand Up @@ -103,7 +101,7 @@ For general Worker setup and options that are not specific to Cloud Run, see [Ru

## Configure the Temporal connection {/* #configure-connection */}

The `temporalio.envconfig` package loads Temporal Client configuration from environment variables and an optional TOML config file, so the Worker code carries no Namespace or credentials.
The `temporalio.envconfig` package loads [Temporal Client](/develop/python/client/temporal-client) configuration from environment variables and an optional TOML config file, so the Worker code carries no Namespace or credentials.
Set the non-secret values as environment variables on the Worker Pool, and mount the Temporal Cloud API key or TLS material from Secret Manager.
For the full list of supported variables, the config file format, and profiles, see [Environment configuration](/develop/environment-configuration).

Expand Down Expand Up @@ -139,126 +137,7 @@ async def my_activity(items: list[str]) -> str:

For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle).

## Add observability with OpenTelemetry {/* #add-observability */}

The `temporalio.contrib.gcp.OpenTelemetryPlugin` configures observability with defaults suited to a Cloud Run worker pool.
By default, the plugin configures:

- OTLP gRPC export to `localhost:4317`, the collector sidecar endpoint.
- `service.name` from the Cloud Run-provided `CLOUD_RUN_WORKER_POOL` environment variable.
- A replay-safe OpenTelemetry tracer provider.
- Temporal Core metrics with a 60-second export interval.

The underlying metrics and traces are the same ones the Python SDK emits in any environment.
For general observability concepts and the full list of available metrics, see [Observability - Python SDK](/develop/python/platform/observability) and the [SDK metrics reference](/references/sdk-metrics).

Pass the plugin to `Client.connect`. Opt into `add_temporal_spans=True` to trace named operations such as `RunWorkflow:GreetingWorkflow`:

<!--SNIPSTART python-cloud-run-otel-worker-->
[gcp_open_telemetry/worker.py](https://github.com/temporalio/samples-python/blob/main/gcp_open_telemetry/worker.py)
```py
# Endpoint, service name, Core metrics, and tracer provider all use the GCP
# plugin defaults. The opt-in adds named Temporal operation spans.
plugin = OpenTelemetryPlugin(add_temporal_spans=True)
client = await Client.connect(
settings.address,
namespace=settings.namespace,
api_key=settings.api_key,
tls=True,
plugins=[plugin],
)
worker = Worker(
client,
task_queue=settings.task_queue,
workflows=[GreetingWorkflow],
activities=[compose_greeting],
graceful_shutdown_timeout=WORKER_GRACEFUL_SHUTDOWN_TIMEOUT,
)
```
<!--SNIPEND-->

Named Temporal operation spans are opt-in. The endpoint, service name, tracer provider, and Core metrics use the plugin defaults, but you must set `add_temporal_spans=True` to emit spans for Workflow and Activity operations.

### Run the collector as a sidecar {/* #collector-sidecar */}

The plugin exports to a collector you run as a second container in the Worker Pool.
Use the [Google-Built OpenTelemetry Collector](https://cloud.google.com/stackdriver/docs/instrumentation/google-built-otel), which detects the Cloud Run resource, authenticates through the Worker Pool's service account, exports metrics to Google Managed Service for Prometheus, and exports traces through the Google Cloud Telemetry API.

Provide the following collector configuration. It receives OTLP on `localhost:4317`, detects the GCP resource, and routes metrics and traces to their respective pipelines:

<!--SNIPSTART python-cloud-run-otel-collector-config-->
[gcp_open_telemetry/collector-config.yaml](https://github.com/temporalio/samples-python/blob/main/gcp_open_telemetry/collector-config.yaml)
```yaml
receivers:
otlp:
protocols:
grpc:
endpoint: localhost:4317

processors:
batch/traces:
send_batch_max_size: 200
send_batch_size: 200
timeout: 5s
memory_limiter:
check_interval: 1s
limit_percentage: 65
spike_limit_percentage: 20
resource_detection:
detectors: [gcp]
timeout: 10s
transform/collision:
metric_statements:
- context: datapoint
statements:
- set(attributes["exported_location"], attributes["location"])
- delete_key(attributes, "location")
- set(attributes["exported_cluster"], attributes["cluster"])
- delete_key(attributes, "cluster")
- set(attributes["exported_namespace"], attributes["namespace"])
- delete_key(attributes, "namespace")
- set(attributes["exported_job"], attributes["job"])
- delete_key(attributes, "job")
- set(attributes["exported_instance"], attributes["instance"])
- delete_key(attributes, "instance")
- set(attributes["exported_project_id"], attributes["project_id"])
- delete_key(attributes, "project_id")
transform/set_project_id:
error_mode: ignore
trace_statements:
- set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil
- set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil

exporters:
googlemanagedprometheus:
otlp_grpc:
endpoint: telemetry.googleapis.com:443
compression: none
balancer_name: pick_first
auth:
authenticator: googleclientauth

extensions:
googleclientauth:
health_check:
endpoint: 0.0.0.0:13133

service:
extensions: [googleclientauth, health_check]
pipelines:
metrics:
receivers: [otlp]
processors: [memory_limiter, resource_detection, transform/collision]
exporters: [googlemanagedprometheus]
traces:
receivers: [otlp]
processors:
[memory_limiter, resource_detection, transform/set_project_id, batch/traces]
exporters: [otlp_grpc]
```
<!--SNIPEND-->

The metrics pipeline forwards each SDK export directly, without a batch processor, so a runtime shutdown-time export cannot collide with a periodic export on the same Managed Prometheus time series.
Traces use a dedicated five-second batch processor.
## Add observability {/* #add-observability */}

To verify telemetry after deploying, confirm that Trace Explorer contains `RunWorkflow:GreetingWorkflow` with `service.name` equal to the Worker Pool name, and that Metrics Explorer contains `prometheus.googleapis.com/temporal_workflow_completed_total/counter`.
A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else.
For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - Python SDK](/develop/python/platform/observability) and the [SDK metrics reference](/references/sdk-metrics).
2 changes: 1 addition & 1 deletion docs/develop/python/workers/serverless-workers/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ For the end-to-end deployment guide, see [Deploy a Serverless Worker](/productio
## Supported providers

- [**AWS Lambda**](/develop/python/workers/serverless-workers/aws-lambda) - Use the `lambda_worker` contrib package to run a Worker as a Lambda function. Covers setup, configuration, Lambda-tuned defaults, and observability.
- [**GCP Cloud Run**](/develop/python/workers/serverless-workers/cloud-run) - Run a standard Worker on a Cloud Run worker pool. Covers the versioned Worker setup, connection configuration, handling scale-in, and OpenTelemetry with the `temporalio.contrib.gcp` plugin.
- [**GCP Cloud Run**](/develop/python/workers/serverless-workers/cloud-run) - Run a standard Worker on a Cloud Run worker pool. Covers the versioned Worker setup, connection configuration, and handling scale-in.
32 changes: 23 additions & 9 deletions docs/develop/typescript/activities/standalone-activities.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import { ReleaseNoteHeader } from '@site/src/components';

Standalone Activities are Activities that run independently, without being orchestrated by a
Workflow. Instead of starting an Activity from within a Workflow Definition, you start a Standalone
Activity directly from a Temporal Client.
Activity directly from a [Temporal Client](/develop/typescript/client/temporal-client).

The way you write the Activity and register it with a Worker is identical to [Workflow
Activities](/develop/typescript/activities/basics). The only difference is that you execute a
Expand Down Expand Up @@ -126,9 +126,9 @@ The way you write a Standalone Activity is identical to how you write an Activit
by a Workflow. In fact, an Activity can be executed both as a Standalone Activity and as a Workflow
Activity.

<!--SNIPSTART typescript-standalone-activity-definition-->
[standalone-activity/src/activities.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/activities.ts)

```typescript
```ts
import { ApplicationFailure } from '@temporalio/activity';

export async function greet(name: string): Promise<string> {
Expand All @@ -138,6 +138,7 @@ export async function greet(name: string): Promise<string> {
return `Hello, ${name}!`;
}
```
<!--SNIPEND-->

## Run a Worker with the Activity registered {/* #run-worker */}

Expand All @@ -147,9 +148,9 @@ whether the Activity will be invoked from a Workflow or as a Standalone Activity
Worker](/develop/typescript/workers/run-worker-process#run-a-dev-worker) for more details on Worker setup and
configuration options.

<!--SNIPSTART typescript-standalone-activity-worker-->
[standalone-activity/src/worker.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/worker.ts)

```typescript
```ts
import { NativeConnection, Worker } from '@temporalio/worker';
import * as activities from './activities';
import { loadClientConnectConfig } from '@temporalio/envconfig';
Expand All @@ -175,6 +176,7 @@ run().catch((err) => {
process.exit(1);
});
```
<!--SNIPEND-->

Open a new terminal, navigate to the `samples-typescript/standalone-activity` directory, and run the Worker:

Expand Down Expand Up @@ -282,13 +284,16 @@ Use
[`start`](https://typescript.temporal.io/api/interfaces/client.TypedActivityClient#start) method of
the client or the typed interface to start your Standalone Activity and get a handle:

```typescript
<!--SNIPSTART typescript-standalone-activity-start-->
[standalone-activity/src/execute.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/execute.ts)
```ts
const handle = await activitiesClient.start('greet', {
...activityOptions,
id: activityId,
args: ['Temporal'],
});
```
<!--SNIPEND-->

Or use the Temporal CLI:

Expand All @@ -309,9 +314,12 @@ how the Activity was started, this method is not available in the typed interfac
takes an optional type argument to constrain the Activity result type, but correctness of this argument
is not verified.

```typescript
<!--SNIPSTART typescript-standalone-activity-get-handle-->
[standalone-activity/src/execute.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/execute.ts)
```ts
const newHandle = client.activity.getHandle<string>(activityId);
```
<!--SNIPEND-->

You can now use the handle to wait for the result, describe, cancel, or terminate the Activity.

Expand All @@ -322,9 +330,12 @@ is the same as calling [`start`](https://typescript.temporal.io/api/interfaces/c
to durably enqueue the Standalone Activity, and then calling `await handle.result()` to
wait for the Activity to be executed and fetch the result:

```typescript
<!--SNIPSTART typescript-standalone-activity-result-->
[standalone-activity/src/execute.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/execute.ts)
```ts
console.log(await handle.result()); // Hello, Temporal!
```
<!--SNIPEND-->

Or use the Temporal CLI to wait for a result by Activity Id:

Expand Down Expand Up @@ -381,10 +392,13 @@ the total count of executions (running, completed, failed, etc.) - not the numbe
works the same way as counting Workflow Executions. The same query will work for both listing and
counting.

```typescript
<!--SNIPSTART typescript-standalone-activity-count-->
[standalone-activity/src/list.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/list.ts)
```ts
const { count } = await client.activity.count(query);
console.log(`Total activities: ${count}`);
```
<!--SNIPEND-->

The sample file
[standalone-activity/src/list.ts](https://github.com/temporalio/samples-typescript/blob/main/standalone-activity/src/list.ts)
Expand Down
Loading