Your AI agent just got blocked: how to survive when external sites ban agent-to-agent traffic
I’m Denis Shokhirev, an Agentic AI Systems Architect in Freiburg im Breisgau, Germany, and at DennisCraft AI Studio I ship production AI systems for DACH B2B teams using Claude, Supabase, n8n, Doppler, and self-hosted Postgres. One of the ugliest production failures I see is not model quality but channel failure: an agent books appointments or submits orders through a third-party site on Monday, then gets a 403, CAPTCHA, or silent UI break on Tuesday and the business flow stalls. If your agent
I’m Denis Shokhirev, an Agentic AI Systems Architect in Freiburg im Breisgau, Germany, and at DennisCraft AI Studio I ship production AI systems for DACH B2B teams using Claude, Supabase, n8n, Doppler, and self-hosted Postgres. One of the ugliest production failures I see is not model quality but channel failure: an agent books appointments or submits orders through a third-party site on Monday, then gets a 403, CAPTCHA, or silent UI break on Tuesday and the business flow stalls.
If your agent touches external portals for booking, pricing, order entry, shipment tracking, or support actions, you do not control the final mile. As more companies separate human traffic from automated traffic, agent-to-agent access through public web interfaces will get harder, not easier. The real risk is not “the model made a bad decision.” The real risk is “your business process depends on a channel you do not own, and that channel can be withdrawn without notice.”
The failure mode is bigger than “the bot got blocked”
In production, external-site failure usually lands in 5 different buckets. If you treat all of them as one problem, your fallback plan will be weak.
| Failure type | Typical symptom | What breaks | Preferred response |
|---|---|---|---|
| Rate limiting | 429 after throughput increases | Batch actions and retries | Queueing, backoff, lower concurrency |
| Bot detection | CAPTCHA or JS challenge | Browser-driven UI automation | Move to API or human fallback |
| Policy change | Updated ToS or blocked endpoint | The integration model itself | Legal review and partner access |
| Markup drift | Selectors stop matching | Scraping and form fill | DOM checks and versioned selectors |
| Auth hardening | MFA, device binding, email challenge | Background workflows | Delegated access and manual checkpoint |
The dangerous version is not a clean outage. It is partial degradation. The agent still runs, but one step silently fails, retries the wrong action, or writes a misleading status to Postgres. That kind of failure contaminates downstream systems and is harder to detect than a crash. I’ve seen teams spend days debugging prompts when the real issue was a changed HTML structure or a newly enforced login challenge.
Design rule: the public website should be your last transport, not your first
If the first implementation path is browser automation against a public UI, you are building on the most fragile layer. A better channel order is usually: official API, partner file exchange, structured email ingestion, then human-assisted portal access as a fallback. For regulated workflows, this also improves auditability because API calls leave cleaner traces than UI clicks.
Separate intent from transport
Your agent should not “know” that booking means clicking a blue button on a portal. It should produce an intent object with validated fields and constraints. A transport layer then decides whether the action goes through an API, email, a queued file, or a manual task.
from dataclasses import dataclass
@dataclass
class BookingIntent:
customer_id: str
slot_id: str
date: str
reference: str
def choose_transport(intent: BookingIntent, capabilities: dict) -> str:
if capabilities.get("api_enabled"):
return "api"
if capabilities.get("file_exchange"):
return "edi"
if capabilities.get("email_fallback"):
return "email"
return "manual_review"This is boring architecture until the day a portal starts blocking you. Then it becomes the difference between degraded service and a full stop.
Keep channel permissions outside the prompt
Do not let the model guess whether a transport is available or allowed. Keep that in a capability registry in Postgres or Supabase. The LLM should receive the allowed action space, not invent it. That also makes incident response faster because you can disable a broken path with configuration instead of retraining behavior into prompts.
Operational controls: validate, observe, degrade safely
Anthropic’s tool use guidance explicitly recommends validating tool calls at the application layer rather than trusting model output to execute external actions unchecked: https://docs.anthropic.com/. That matters even more when the “tool” is a third-party site you do not own. The model should request an action; your system should decide whether, when, and how to perform it.
I usually put 3 safeguards around external channels.
1. Synthetic health checks
Do not wait for a live booking to discover a problem. Run lightweight checks against the login page, target form, and final confirmation state. Track them separately from business traffic so you can distinguish your own volume issue from partner-side changes.
#!/usr/bin/env bash
set -euo pipefail
URL="https://partner.example.com/login"
STATUS=$(curl -s -o /tmp/check.html -w "%{http_code}" "$URL")
if [ "$STATUS" -ne 200 ]; then
echo "partner_health status=$STATUS"
exit 1
fi
grep -q "Sign in" /tmp/check.html || {
echo "partner_health unexpected_markup=1"
exit 1
}2. Controlled degradation
When the external site becomes unstable, your agent should stop pretending it can finish the task. Use a queue, a circuit breaker, and a visible status such as “accepted, pending manual fulfillment.” That protects customer trust and prevents blind retry storms.
3. Human fallback with full context
A manual step is not a failure if it is designed well. The system should package customer data, attempted action, last error, and validated fields into a work item. The human operator should complete the exception, not rediscover the entire case.
fallback_task:
type: manual_review
priority: high
payload:
intent_id: "bk_48291"
partner: "carrier_portal"
last_error: "403_forbidden"
retry_allowed: false
customer_ref: "ORD-9912"The business and compliance angle matters more than the scraping angle
Most technical teams frame this as a bot-evasion problem. For B2B systems, that is usually the wrong frame. If a third-party portal is on your revenue path, you need a commercial path, not just a browser path. Partner access, written approval, quota discussions, and named contacts are less exciting than UI automation but they survive policy changes.
OWASP’s Automated Threats to Web Applications project explains why site owners increasingly defend against automated interactions: https://owasp.org/www-project-automated-threats-to-web-applications/. Their incentives are not aligned with yours by default. Once you accept that, the architecture choice gets clearer: assume conflict, design alternatives, and make manual escalation cheap.
I also use a simple concentration rule. If one external portal carries more than 15–20% of an operational flow, I treat that as concentration risk and ask for a second path. That is not a published benchmark; it is a practical threshold from real integrations where one UI workflow became a single point of failure for logistics or order processing.
What I would build now
If you already run agentic workflows against external websites, I would audit these 5 points first:
- Does every critical action have an alternative transport?
- Can you disable the website path with config or a feature flag?
- Do you distinguish 403, 429, CAPTCHA, auth failures, and markup drift in telemetry?
- Do you have queue-based manual fallback with enough context to act fast?
- Do you have a business plan to move from UI automation to official access if volume grows?
n8n is useful for orchestration, Doppler for secrets, and Supabase or Postgres for capability state and audit trails. But the tools are secondary. The core idea is simple: do not confuse agent intelligence with channel reliability. Your model can be good and your system can still be fragile if the final action depends on a third-party website that never agreed to be your API.
FAQ
Should I stop using browser automation entirely?
No. It is still useful for early market access, low-volume workflows, and exception handling. But it should not be the only path for a revenue-critical process.
Can technical workarounds solve blocking long term?
Sometimes they buy time, but they are rarely a stable B2B strategy. If the channel matters, work toward official access, negotiated usage, and a fallback that does not depend on evasion.
What matters more: observability or fallback?
If I must choose one, fallback saves the operation. But without observability, you will detect degradation too late and you will misdiagnose the layer that failed.
What telemetry should I collect at minimum?
Track HTTP status classes, per-step success rate, latency, retry counts, and manual escalation rate by cause. I also like to track transport mix: API, email, portal, and manual completion.
Is this overkill for a small team?
No, not if even one key workflow depends on a third-party portal. A clean intent/transport split and a manual fallback are often cheaper than repeated prompt tuning after incidents.
If a key external portal started banning agent-to-agent traffic tomorrow, which part of your flow would fail first: data retrieval, authentication, form submission, or result confirmation? I’d genuinely like to know where teams see the weakest link in production.
I run a free 30-min stack audit for DACH founders building AI in regulated markets. DM me on LinkedIn or write to @ger_dennis_ai.
I build agents like this for businesses in Germany and the EU.
Tell me in two sentences what eats your team’s time – reply within 24 hours.
Turn your process into an AI system
Production quality. DACH B2B focus.