Edge Gateway Architecture

The Andromity Edge Gateway (gateway-worker) is a globally distributed Cloudflare Worker acting as a stateless, high-availability proxy for LLM inference requests.

Located at gateway.agenticmarket.dev, the gateway provides real-time streaming, multi-provider failover, and dynamic rate limiting without ever persisting user code or prompts.

1. Zero-PII Streaming GuaranteeCopy Anchor Link

The gateway acts strictly as a transparent stream router:

• Unbuffered SSE: Incoming Server-Sent Events (text/event-stream) from upstream providers are piped directly through Cloudflare's unbuffered TransformStream to the client.
• No Disk or KV Prompt Persistence: Prompts, completion tokens, code snippets, and filenames are processed purely in ephemeral worker memory and discarded immediately upon stream completion.
• Zero Logging of Sensitive Headers: Authorization headers and client payloads are excluded from Cloudflare execution logs.

2. Multi-Provider Failover ChainsCopy Anchor Link

When an upstream LLM provider experiences an outage, high latency spike, or rate limit (429 Too Many Requests), the Edge Gateway automatically redirects the completion request across a prioritized failover chain:

[Primary Request: Gemini 2.5 Flash]
              │
         (HTTP 429 / 503)
              ▼
   [Fallback 1: Groq LPU]
              │
         (HTTP 429 / 503)
              ▼
   [Fallback 2: Cerebras]
              │
         (HTTP 429 / 503)
              ▼
[Fallback 3: Workers AI / DeepSeek]

Failover transitions occur transparently within milliseconds, preventing broken terminal sessions or IDE timeouts.

3. Dynamic Rate Limiting & Ring BufferCopy Anchor Link

To prevent abuse while ensuring high throughput for legitimate developers:

• Sliding-Window Rate Limiting: The gateway tracks requests per IP/token using a distributed KV sliding-window counter.
• Ring Buffer Quotas: Ephemeral client token quotas are evaluated in Cloudflare KV (GATEWAY_KV).
• Burst Protection: Transient bursts are permitted within safe concurrency bounds, while sustained over-limit traffic receives clear 429 responses with Retry-After headers.