Skip to content

Commit 4f735f6

Browse files
docs: Update PII Scrubbing in Logs doc (#2156)
* docs: Update PII Scrubbing in Logs doc Signed-off-by: Jugal Kishore <[email protected]> * chore: Fix spellings Signed-off-by: Jugal Kishore <[email protected]> * chore: Fix spellings & code snippet Signed-off-by: Jugal Kishore <[email protected]> * chore: Update tag and date Signed-off-by: Jugal Kishore <[email protected]> --------- Signed-off-by: Jugal Kishore <[email protected]>
1 parent bec359a commit 4f735f6

File tree

1 file changed

+40
-15
lines changed

1 file changed

+40
-15
lines changed

data/docs/logs-management/guides/pii-scrubbing.mdx

Lines changed: 40 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
---
2-
date: 2024-07-05
2+
date: 2025-11-12
33
id: pii-scrubbing
4-
tags : [SigNoz Cloud]
4+
tags: [SigNoz Cloud, Self-Host]
55
title: Guide to perform PII Scrubbing using SigNoz
6+
description: Learn how to scrub PII (Personally Identifiable Information) from logs using OpenTelemetry Collector. This guide shows you how to remove sensitive data like emails, SSNs, and credit card numbers from application logs before they reach SigNoz to ensure GDPR and CCPA compliance.
67
---
78

89
import GetHelp from '@/components/shared/get-help.md'
@@ -12,29 +13,43 @@ import GetHelp from '@/components/shared/get-help.md'
1213
PII scrubbing removes sensitive personal information from data to protect privacy and comply with regulations like GDPR and CCPA. SigNoz simplifies this process for your applications using OpenTelemetry. This guide explains how to implement PII scrubbing in your data pipeline before sending information to SigNoz.
1314

1415
## Prerequisite
16+
1517
- [SigNoz Cloud](https://signoz.io/teams/) account
16-
- Logs in log file
18+
- Application sending logs to OpenTelemetry Collector (via OTLP or any other receivers)
1719

1820
## Send application logs to SigNoz
1921

20-
You need to configure OpenTelemetry Collector to send your logs in log files to SigNoz, checkout [this documentation](https://signoz.io/docs/userguide/collect_logs_from_file/)
21-
for detailed instructions on how to do this.
22+
You can send logs to SigNoz via OpenTelemetry Collector using various methods:
23+
24+
- **OTLP:** Direct instrumentation from your application
25+
- **File Logs:** Using filelog receiver, checkout [this documentation](https://signoz.io/docs/userguide/collect_logs_from_file/)
26+
- Other OpenTelemetry receivers and protocols
27+
28+
For more detailed guides on setting up log collection for your specific use case, refer to the [Send Logs to SigNoz documentation](https://signoz.io/docs/logs-management/send-logs-to-signoz/).
2229

2330
## Process of Scrubbing PII Data
2431

2532
OpenTelemetry Collector offers a powerful [transform processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/README.md) to filter and modify sensitive data before it reaches SigNoz. We'll use this processor to perform PII scrubbing on our logs.
2633

27-
### Sample log
34+
### Sample log
2835

2936
Let's use the following sample log line to demonstrate the PII scrubbing process:
3037

3138
```json
32-
{"timestamp": "2024-07-05T12:34:56Z", "severity": "INFO", "message": "User [email protected] (SSN: 123-45-6789) made a purchase with credit card 4111-1111-1111-1111", "user_id": "12345", "purchase_amount": 99.99}
39+
{
40+
"timestamp": "2024-07-05T12:34:56Z",
41+
"severity": "INFO",
42+
"message": "User [email protected] (SSN: 123-45-6789) made a purchase with credit card 4111-1111-1111-1111",
43+
"user_id": "12345",
44+
"purchase_amount": 99.99
45+
}
3346
```
3447

3548
This log contains several pieces of PII that we want to scrub: an email address, a social security number (SSN), and a credit card number inside the log body.
49+
3650
### Configuring the Transform Processor
37-
To scrub this PII, we'll use the transform processor in our OpenTelemetry Collector configuration. Trasform processor makes use of [Regular Expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions) for scrubbing your sensitive data. Here's how we can set it up:
51+
52+
To scrub this PII, we'll use the transform processor in our OpenTelemetry Collector configuration. Transform processor makes use of [Regular Expressions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions) for scrubbing your sensitive data. Here's how we can set it up:
3853

3954
```yaml
4055
processors:
@@ -58,7 +73,9 @@ Let's break down what each statement does:
5873
- The third statement identifies credit card number patterns and replaces all digits with asterisks.
5974
6075
### Applying the Processor
76+
6177
To apply this processor to your logs pipeline, add it to the processors list in your pipeline configuration:
78+
6279
```yaml
6380
service:
6481
pipelines:
@@ -68,9 +85,12 @@ service:
6885
processors: [batch, transform]
6986
exporters: [otlp]
7087
```
88+
7189
### Result
90+
7291
After applying this configuration and restarting your OpenTelemetry Collector, the sample log line would be transformed to:
73-
```yaml
92+
93+
```json
7494
{
7595
"timestamp": "2024-07-05T12:34:56Z",
7696
"severity": "INFO",
@@ -79,17 +99,22 @@ After applying this configuration and restarting your OpenTelemetry Collector, t
7999
"purchase_amount": 99.99
80100
}
81101
```
82-
<figure data-zoomable align='center'>
83-
<img src="/img/docs/logs-management/guides/sample-log-scrubbed-output.webp" alt="Sample log output with PII data scrubbed out"/>
84-
<figcaption><i>Sample log output with PII data scrubbed out</i></figcaption>
102+
103+
<figure data-zoomable align="center">
104+
<img
105+
src="/img/docs/logs-management/guides/sample-log-scrubbed-output.webp"
106+
alt="Sample log output with PII data scrubbed out"
107+
/>
108+
<figcaption>
109+
<i>Sample log output with PII data scrubbed out</i>
110+
</figcaption>
85111
</figure>
86112

87113
As you can see, the email address, SSN, and credit card number have been scrubbed, protecting this sensitive information before it reaches SigNoz.
88114
Remember to adjust the regular expressions in the `replace_pattern` statements as needed to match the specific formats of PII in your logs. You can add more statements to handle additional types of sensitive data that may appear in your logs.
89115

90-
91-
To know more about the capabilities of trasform processsor, checkout [this documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/README.md).
116+
To know more about the capabilities of transform processor, checkout [this documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/README.md).
92117

93118
## Get Help
94119

95-
<GetHelp />
120+
<GetHelp />

0 commit comments

Comments
 (0)