Observability data can become a security problem when it records more than engineers intended.
Logs, traces, and metric attributes often cross application, network, storage, and vendor boundaries. A single raw URL, database statement, authorization header, email address, session token, or payment identifier can turn useful telemetry into sensitive data with a long retention period and a wide audience.
The right goal is not to remove all context. It is to keep the context needed for reliable operations while preventing secrets and personal data from entering the telemetry path.
Start at the source
The strongest control is safe instrumentation. Applications should avoid recording credentials, access tokens, full request bodies, unrestricted headers, raw database values, and personal information unless there is a documented operational need and an approved protection model.
Once sensitive data reaches a Collector, it has already existed in application memory and crossed at least one boundary. Collector processing is valuable, but it is a second line of defence rather than permission to instrument carelessly.
Use an allowlist when the risk is high
The OpenTelemetry Collector redaction processor can retain only approved attribute keys and remove everything else. This fail-closed approach is safer than trying to predict every possible secret name.
A blocklist can still help catch known patterns such as keys containing token or api_key. It can also mask values matching controlled patterns. However, blocklists age quickly. New libraries, headers, and teams introduce fields that the original rules did not anticipate.
For critical payment, identity, health, and customer systems, begin with the minimum approved attribute set. Add new fields through review.
Choose deletion, masking, and hashing deliberately
- Delete data that has no operational value.
- Mask values when engineers need to know a field existed but do not need the original value.
- Hash only when stable correlation is required and the re-identification risk is understood.
Simple hashes are weak protection for low-entropy values such as predictable IP addresses or small identifier spaces. The redaction processor documentation recommends HMAC-based hashing for those cases. The secret key must be stored and rotated as a real credential.
Sanitize URLs and database operations
Raw URLs can contain user identifiers, search terms, tokens, and unique path segments. They also create high cardinality. Preserve route templates such as /accounts/{account_id}, not the full customer-specific path.
Database statements may expose customer data, credentials, and business-sensitive values. Preserve the query shape needed for performance analysis while removing literal values. The current Collector redaction processor supports URL and database sanitization, but teams must validate behavior against their actual protocols and instrumentation.
This also supports the discipline in our metric cardinality budget guide. Safe normalization can reduce both privacy exposure and telemetry growth.
Place controls before export
Processor order matters. Filtering, deletion, and redaction should happen before data is batched and exported. Apply controls to every relevant signal pipeline. A trace pipeline protected by redaction does not automatically protect logs or metrics.
Limit access to unprocessed telemetry. Secure Collector endpoints, encrypt traffic, authenticate senders and exporters, and keep management interfaces private. OpenTelemetry’s security guidance treats the Collector as production infrastructure that needs a hardened deployment.
Test the policy with seeded data
A redaction rule is not proven because the configuration loads. Build a test dataset containing synthetic tokens, email addresses, account identifiers, raw URLs, database literals, and safe operational attributes.
- Send the dataset through the same Collector distribution and configuration used in production.
- Inspect every exporter destination.
- Confirm blocked fields never arrive.
- Confirm required diagnostic attributes remain usable.
- Repeat the test after instrumentation, semantic convention, or Collector upgrades.
Never use real customer records or live credentials in the test.
Monitor redaction without creating a new leak
The redaction processor can add summary attributes describing its actions. Counts can help detect instrumentation changes. Detailed key names may reveal information about the data model, so use the least verbose summary suitable for the environment.
Track changes in redaction volume, rejected telemetry, configuration errors, and new attribute keys. A sudden rise may signal a new release, a library change, or an application that has begun exporting unsafe fields.
Create clear ownership
Security teams can define prohibited data classes, but platform and application teams must share responsibility for implementation. A practical policy should identify:
- approved telemetry attributes by service type;
- data that must never enter telemetry;
- retention and access rules;
- the owner of Collector redaction policy;
- the approval process for new attributes;
- the test required before deployment.
For organizations operating across countries, cloud regions, and service providers, data movement also requires legal and governance review. Technical redaction is not a substitute for determining which data may be collected, processed, retained, or transferred.
What to do this week
- Search telemetry schemas for credentials, personal information, raw URLs, and database statements.
- Remove unnecessary fields at the instrumentation source.
- Create an allowlist for one high-risk service.
- Add Collector redaction before export.
- Run a synthetic leakage test across every destination.
- Assign an owner and review date to the policy.
Good observability provides enough context to understand a system without collecting data that the operations team should never possess.
Organizations interested in supporting vendor-neutral education on secure telemetry and observability governance can explore sponsorship or workshop collaboration with Observability Africa. Sponsors do not control editorial conclusions.