observability-audit

Observability Audit

Use this skill to verify that a change can be operated after it ships.

Audit Questions

• Can an operator tell whether the workflow ran?
• Can failures be grouped by cause?
• Can a user report be traced to request, job, tenant, account, or external system IDs?
• Are retries, partial failures, and skipped records visible?
• Are secrets and personal data excluded from logs and evidence?
• Does the code preserve external request IDs or provider correlation IDs?

Required Evidence by Surface

HTTP APIs

• request or correlation ID
• route or operation name
• status code and duration
• authenticated actor or tenant ID when safe
• normalized error kind

Background Jobs and Syncs

• run ID
• source system
• checkpoint before and after
• counts for fetched, changed, skipped, failed
• retry count
• failure categories

External Adapters

• external system name
• operation name
• safe external identifier
• request ID returned by provider
• timeout/retry classification

Log Quality Checklist

• Logs are structured, not interpolated blobs.
• Levels match operational severity.
• Sensitive fields are redacted or omitted.
• Log lines include enough context to join related events.
• Expected validation failures are not logged as process errors.
• Exceptions preserve stack traces at the boundary that handles them.

Review Output

When auditing, report:

## Observability Findings

- [severity] [file:line] Finding and operational impact.

## Missing Evidence

- Request/job/correlation IDs:
- Counts/status summaries:
- External provider metadata:
- Redaction concerns:

## Recommended Fixes

- Concrete code or logging changes.

Common Mistakes

• Logging only at startup and final failure.
• Logging raw request bodies, tokens, headers, or presigned URLs.
• Recording "success" without counts or checkpoint evidence.
• Hiding provider request IDs.
• Treating observability as comments instead of executable code paths.