# Operations

## Required signals

Use structured logs carrying request, workspace, execution, action, and correlation IDs while redacting credentials and sensitive content. Monitor CPU, memory, disk, PostgreSQL, Redis, queues, worker count, failed jobs, browser health, provider latency/errors, and storage growth.

The production Core process runs a bounded maintenance pass at startup and every configured
interval. Alert on `maintenance_failed`, repeated `artifacts_pending_deletion`, unknown idempotency
records, expired controller leases, and reconciled write outcomes. Container JSON logs are bounded
locally, but production still needs centralized transport, retention, dashboards, and alert routing.

Minimum alerts cover authentication throttling spikes, repeated MFA/reset failures, 5xx rate,
database/Redis loss, browser worker loss, disk and inode pressure, certificate expiry, backup age or
failure, restore-drill failure, audit-chain verification, and emergency-stop activation. Every alert
needs a tested owner, severity, acknowledgement path, escalation, and runbook link.

## Incidents

Owner notifications are required for session expiry, queue stalls, disk pressure, backup failures, and repeated provider failures. Emergency stop prevents new side effects and fences active computer control. Uncertain actions are not automatically retried until their external state is reconciled.

Detailed dependency health and metrics are internal/authenticated. Public liveness reveals only whether the service process is alive.

The repository does not currently deploy a metrics collector, log backend, pager, or tracing system.
That target-environment work is a production release blocker.