Your AI agent touched production? How to build replayable security triage and a provable audit trail before the first incident

I’m Denis Shokhirev, an Agentic AI Systems Architect based in Freiburg im Breisgau, Germany, and at DennisCraft AI Studio I ship production AI systems for DACH B2B clients on Claude, Supabase, n8n, Doppler, and self-hosted Postgres. The real point of tension is not when an AI agent drafts code or proposes a workflow; it’s the first time it can actually touch production data, trigger an external side effect, or mutate a permission boundary.

If you do not have replayable security triage and a provable audit trail at that moment, you are already late. After the first disputed action, your team will need to answer four questions fast: what the agent saw, what it decided, what policy version evaluated that decision, and why the system allowed or blocked the action.

What “replayable triage” actually means

Most teams log outcomes and call it observability. That is not enough for an agent with production access. “Workflow executed” or “record updated” tells you an effect occurred, but it does not let you reconstruct whether the action was acceptable under the exact rules in force at that time.

Replayable triage means you can take an event from last week, feed the same normalized inputs through the same policy layer, and reproduce the decision path in a sandbox. Not approximately. Not from memory. From preserved inputs, preserved configuration, and preserved execution metadata.

The minimum artifacts I store per sensitive action

Notice what is missing: vague “model reasoning” dumps. I do not rely on free-form introspection for audit. I store structured decision records that are useful in incident response and acceptable in regulated environments.

Why prompts are not your control plane

Prompting helps, but prompts do not enforce boundaries. If an agent can call tools, change rows, create tickets, or hit a payment API, your real control plane is policy plus execution logging. OWASP’s Top 10 for LLM Applications 2025 explicitly calls out Excessive Agency and Insecure Output Handling as system risks, not abstract model risks: https://genai.owasp.org/.

Anthropic’s documentation for tool use follows the same direction: validate tool inputs, constrain available tools, and insert human review for sensitive operations. Source: Anthropic Docs, tool use guidance, https://docs.anthropic.com/.

A practical policy split

I usually divide agent actions into 3 classes:

This split is simple enough to operate and strong enough to survive the first real incident review.

A minimal audit trail design in Postgres

You do not need a giant platform to start. You need a stable event model and the discipline to record every attempted sensitive action before execution, including denied attempts. I usually keep this in Postgres because it gives me transactional guarantees and easy joins with app context.

create table agent_audit_events (
  id uuid primary key,
  created_at timestamptz not null default now(),
  tenant_id text not null,
  actor_type text not null,
  actor_id text not null,
  action_name text not null,
  input_hash text not null,
  policy_version text not null,
  risk_class text not null,
  decision text not null,
  requires_review boolean not null default false,
  execution_ref text,
  payload jsonb not null,
  outcome jsonb
);

The key field here is input_hash. Even if payloads are redacted later for privacy, the hash lets you prove the replay maps back to the original event. In regulated settings, that distinction matters.

Example: create the audit record before tool execution

import hashlib
import json
import uuid
from datetime import datetime, timezone

def build_audit_event(tenant_id, actor_id, action_name, payload, policy_version):
    raw = json.dumps(payload, sort_keys=True).encode()
    return {
        "id": str(uuid.uuid4()),
        "created_at": datetime.now(timezone.utc).isoformat(),
        "tenant_id": tenant_id,
        "actor_type": "ai_agent",
        "actor_id": actor_id,
        "action_name": action_name,
        "input_hash": hashlib.sha256(raw).hexdigest(),
        "policy_version": policy_version,
        "risk_class": "high",
        "decision": "pending",
        "payload": payload
    }

If your event is written after the tool call finishes, you still have an untracked execution window. For production agents, that is a design flaw.

How I build triage in front of real agent actions

I treat triage as a 3-stage path: pre-execution checks, execution-time gating, and post-execution verification.

1. Pre-execution checks

If the agent generates code, SQL, or configuration, I run static checks before anything reaches production. The exact stack depends on the artifact, but semgrep, bandit, and gitleaks cover a lot of failure modes with very little friction.

#!/usr/bin/env bash
set -euo pipefail

target_dir="${1:-./candidate}"
semgrep --config=p/owasp-top-ten "$target_dir"
bandit -r "$target_dir" -f txt
gitleaks dir "$target_dir" --no-banner

echo "static checks passed"

This is not academic hygiene. On real agent deployments, repeated unsafe patterns show up in generated SQL wrappers, secret handling, and permissive file operations.

2. Policy-gated execution

Every tool call passes through a policy layer that evaluates scope, risk class, and approval requirements. The agent does not call Postgres or an external API directly. It requests an action; the policy layer decides whether the request becomes execution.

3. Post-execution verification

After execution, I verify invariants: rows affected, tenant boundaries, resource type, and rollback viability. This stage catches orchestration bugs that look like “AI problems” but are really missing checks in the surrounding system.

Where teams usually fail

The last point matters more than people expect. Denied actions are often your best signal that the agent is repeatedly trying to cross a boundary you failed to model cleanly.

FAQ

Do I need to store full prompts?

Not always. If prompts contain sensitive data, store a redacted representation, structured context fields, and a hash of the original. That is often enough for replay and safer for compliance.

Is n8n execution history enough as an audit trail?

No. It is useful execution metadata, but it does not replace a separate policy decision record with hashes, risk classes, and review outcomes. I treat n8n as part of the trail, not the whole trail.

Should every write action require human approval?

No. Low-risk writes inside a narrow tenant scope can run automatically if your checks are strong. Privilege changes, destructive actions, and money-moving operations usually need a tighter gate.

What is the difference between logging and replay?

Logging tells you what happened. Replay lets you test whether the same decision would happen again under the same inputs and controls. For incident response, that is the difference between a story and evidence.

Is this only for large enterprises?

No. Smaller teams often need it more because they have less time for manual forensics and less margin for error with their first regulated customers.

If your AI agent can already read from or write to production, which stage in your pipeline would catch the next disputed action first: static analysis, runtime policy gating, or human review after the fact? I’d genuinely like to know.
I run a free 30-min stack audit for DACH founders building AI in regulated markets. DM me on LinkedIn or write to @ger_dennis_ai.