Proposal: Refactor api-proxy with provider adapter abstraction

[lpcox]

Problem

containers/api-proxy/server.js (2,164 lines) handles 5 LLM providers (OpenAI, Anthropic, Copilot, Gemini, OpenCode) with no internal abstraction. Each provider's logic — auth injection, URL routing, request transformation, health checks, WebSocket handling — is inlined in separate http.createServer() callbacks (lines 1835–2146). Adding a new provider means copy-pasting ~50 lines of boilerplate and scattering provider-specific logic across multiple locations.

Evidence of duplication

Pattern	Occurrences	Lines
Per-port server creation (same structure)	5×	1837, 1886, 1943, 2001, 2077
Health endpoint handler	4×	1887, 1945, 2002, 2078
Rate limit + content-length parsing	5×	1839, 1893, 1951, 2008, 2092
Auth header construction	5× (10 sites incl. WS)	1843/1849, 1897/1925, 1974/1988, 2016/2024, dynamic
WebSocket upgrade handler registration	5×	1847, 1924, 1986, 2020, 2122

Impact

• Adding Gemini required touching 7+ locations — port allocation, env var reading, auth header format, URL transformation (query param stripping), health endpoint, WebSocket handler, startup validation
• OpenCode's dynamic routing (lines 291–316: resolveOpenCodeRoute()) is a one-off pattern that doesn't compose with the static provider structure
• Copilot's dual-auth model (OAuth vs API key, with /models special-casing) is embedded inline rather than encapsulated
• Anthropic's transforms (cache injection, ANSI stripping, tool dropping) are composed via composeBodyTransforms() but wired up manually
• No way to test providers in isolation — all logic is in the main module's startup block

Proposed architecture

Provider adapter interface

interface ProviderAdapter {
  /** Unique provider identifier */
  readonly name: string;

  /** Port this provider listens on (e.g., 10001 for Anthropic) */
  readonly port: number;

  /** Whether this provider is enabled (has required credentials) */
  isEnabled(): boolean;

  /** Upstream target hostname (e.g., "api.anthropic.com") */
  getTargetHost(): string;

  /** Optional base path prefix (e.g., "/serving-endpoints") */
  getBasePath(): string;

  /**
   * Build auth headers for a given request.
   * Allows request-specific logic (e.g., Copilot /models uses different token).
   */
  getAuthHeaders(req: http.IncomingMessage): Record<string, string>;

  /**
   * Optional request URL transform (e.g., Gemini strips ?key= params).
   * Returns modified URL or undefined to skip.
   */
  transformRequestUrl?(url: string): string;

  /**
   * Optional request body transform (e.g., model alias rewriting, Anthropic cache injection).
   * Returns transformed body buffer or null to pass through unchanged.
   */
  transformRequestBody?(body: Buffer, headers: Record<string, string>): Buffer | null;

  /**
   * Optional startup validation — probe the upstream API to verify credentials.
   */
  validateCredentials?(): Promise<{ ok: boolean; error?: string }>;

  /**
   * Optional model fetching for /reflect endpoint.
   */
  fetchModels?(): Promise<Array<{ id: string; [key: string]: unknown }>>;
}

Example: Anthropic adapter

class AnthropicAdapter implements ProviderAdapter {
  readonly name = "anthropic";
  readonly port = 10001;

  private apiKey: string;
  private target: string;
  private basePath: string;
  private bodyTransform: (body: Buffer) => Buffer | null;

  constructor(env: Record<string, string | undefined>) {
    this.apiKey = (env.ANTHROPIC_API_KEY || "").trim();
    this.target = normalizeApiTarget(env.ANTHROPIC_API_TARGET) || "api.anthropic.com";
    this.basePath = normalizeBasePath(env.ANTHROPIC_API_BASE_PATH);
    this.bodyTransform = composeBodyTransforms(
      makeModelBodyTransform("anthropic"),
      makeAnthropicTransform(/* header injection callback */)
    );
  }

  isEnabled() { return !!this.apiKey; }
  getTargetHost() { return this.target; }
  getBasePath() { return this.basePath; }

  getAuthHeaders(req) {
    const headers: Record<string, string> = { "x-api-key": this.apiKey };
    if (!req.headers["anthropic-version"]) {
      headers["anthropic-version"] = "2023-06-01";
    }
    return headers;
  }

  transformRequestBody(body, headers) {
    return this.bodyTransform(body);
  }

  async validateCredentials() {
    // POST to /v1/messages with empty body, expect 400 (not 401)
  }

  async fetchModels() {
    // GET /v1/models
  }
}

Example: OpenCode adapter (dynamic routing)

class OpenCodeAdapter implements ProviderAdapter {
  readonly name = "opencode";
  readonly port = 10004;

  private resolveRoute: () => { target: string; headers: Record<string, string>; basePath: string } | null;

  constructor(env: Record<string, string | undefined>) {
    // Capture env for dynamic resolution at request time
    this.resolveRoute = () => resolveOpenCodeRoute(env);
  }

  isEnabled() { return process.env.AWF_ENABLE_OPENCODE === "true" && !!this.resolveRoute(); }
  getTargetHost() { return this.resolveRoute()?.target || ""; }
  getBasePath() { return this.resolveRoute()?.basePath || ""; }

  getAuthHeaders() {
    const route = this.resolveRoute();
    return route?.headers || {};
  }
}

Generic server factory

The ~50 lines of duplicated per-provider server setup becomes a single factory:

function createProviderServer(adapter: ProviderAdapter): http.Server {
  const server = http.createServer((req, res) => {
    // 1. Health endpoint (generic)
    if (req.url === "/health" && req.method === "GET") {
      res.writeHead(200);
      res.end(JSON.stringify({ status: "ok", service: `awf-api-proxy-${adapter.name}` }));
      return;
    }

    // 2. Rate limiting (generic)
    const contentLength = parseInt(req.headers["content-length"] || "0", 10);
    if (checkRateLimit(req, res, adapter.name, contentLength)) return;

    // 3. URL transform (provider-specific, optional)
    if (adapter.transformRequestUrl) {
      req.url = adapter.transformRequestUrl(req.url);
    }

    // 4. Auth + proxy (generic plumbing, provider-specific headers)
    proxyRequest(req, res, adapter.getTargetHost(), adapter.getAuthHeaders(req),
      adapter.name, adapter.getBasePath(), adapter.transformRequestBody?.bind(adapter));
  });

  // WebSocket upgrade (generic plumbing, provider-specific headers)
  server.on("upgrade", (req, socket, head) => {
    if (adapter.transformRequestUrl) req.url = adapter.transformRequestUrl(req.url);
    proxyWebSocket(req, socket, head, adapter.getTargetHost(),
      adapter.getAuthHeaders(req), adapter.name, adapter.getBasePath());
  });

  return server;
}

Server startup becomes declarative

const adapters = [
  new OpenAIAdapter(process.env),
  new AnthropicAdapter(process.env),
  new CopilotAdapter(process.env),
  new GeminiAdapter(process.env),
  new OpenCodeAdapter(process.env),
];

for (const adapter of adapters) {
  if (adapter.isEnabled()) {
    const server = createProviderServer(adapter);
    server.listen(adapter.port, "0.0.0.0");
  }
}

Migration plan

Phase 1: Extract adapter interface (non-breaking)

1. Define ProviderAdapter interface
2. Create adapter classes for each provider, extracting logic from inline handlers
3. Create createProviderServer() factory
4. Replace inline handlers with factory calls
5. Behavior-preserving refactor — all existing tests must pass unchanged

Phase 2: Improve testability

1. Each adapter is independently unit-testable (auth headers, URL transforms, body transforms)
2. Factory function testable with mock adapters
3. Integration tests verify full request flow per provider

Phase 3: Simplify new provider onboarding

1. Adding a new provider = one new file implementing ProviderAdapter
2. Register in the adapters array
3. No need to touch server.js routing, health endpoints, WebSocket, rate limiting, etc.

Design inspiration

open-design (nexu-io's multi-agent orchestrator) uses a similar data-driven approach: agent definitions are config objects in an AGENT_DEFS array, each declaring its executable path, argument builder, stream parser, and capability probes. Adding a new agent means adding one config object — no framework code changes. AWF's api-proxy should follow a similar pattern for provider definitions.

Scope

In scope

• Refactoring the 5 existing providers into adapter classes
• Generic server factory for HTTP and WebSocket handling
• Health endpoint, rate limiting, and management endpoints as shared middleware
• Startup validation (validateApiKeys) using adapter's validateCredentials()
• Model fetching using adapter's fetchModels()

Out of scope (separate work)

• Changing the port allocation scheme (keep fixed ports for backward compatibility)
• Changing the proxy transport layer (proxyRequest/proxyWebSocket)
• Changing the NDJSON token tracking
• Adding new providers (that's the benefit of this refactor)

Benefits

Metric	Before	After
Lines to add a new provider	~80 (scattered across 7+ locations)	~40 (single adapter file)
Files to modify for new provider	1 (server.js, multiple locations)	2 (new adapter + registration)
Provider-specific test surface	None (all inline in startup block)	Per-adapter unit tests
Risk of copy-paste bugs	High (5 near-identical blocks)	Low (shared factory)
Understanding cost for new contributors	High (read all 2,164 lines)	Low (read interface + one adapter)