## I

I. The Business Pain Point: It's Not Technical Difficulty, It's Coordination Chaos

We're a second-hand trading platform team of about twenty people, with four groups—product, operations, algorithm, and risk control—all focused on the "AI-generated product image description" feature: a user uploads a few photos of an old phone or old furniture, and the system automatically generates a standardized product description, addressing the drop in transaction rates caused by sellers being too lazy to write descriptions.

Before the feature even launched, the problems erupted first:

1. Duplicate procurement: The algorithm team onboarded one vision model, the operations team onboarded another vendor for category review, and the risk control team onboarded a third for prohibited item detection. Three contracts, three billing systems, three security assessments—the manager spent a full day each month reconciling accounts.
2. Inconsistent quality standards: For the same second-hand handbag, the description generated by the algorithm team said "90% new," while the operations-side review model judged it as "condition description questionable." The two sides' standards weren't aligned, and customer service handled complaints every day.
3. No one owning the risks: The vision model would occasionally write private information from user photos (delivery slips or ID cards in the background) into descriptions. This item was scattered across three teams' backlogs, with everyone assuming someone else should handle it.
4. Vendor lock-in: When our primary vision model raised prices by 40%, we discovered during the migration attempt that the invocation code was scattered across four services—estimating the migration cost alone would take two weeks.

As the person managing this project, I realized the problem wasn't whether we picked a good model, but the lack of a unified governance layer. AI capabilities must be integrated, metered, and risk-controlled centrally like infrastructure, rather than each team pulling its own wire.

II. Architecture Design: One Gateway to Rule Them All

The refactored architecture has four layers:

• Access layer: All services (description generation, condition review, prohibited item detection) go through a unified AI API gateway. The gateway shields the differences between underlying models and exposes a unified OpenAI-compatible interface.
• Model routing layer: The gateway is configured with routing rules—product main-image descriptions go to a large vision model, batch reviews go to a lightweight specialized model, and cost-sensitive scenarios automatically fall back to a smaller model.
• Governance layer: Budgets and quotas are set per project, and usage reports are generated per team; sensitive content (private information, prohibited items) is filtered centrally at the gateway on the outbound side.
• Business layer: The description generation service only cares about prompt orchestration and business fallback logic—it doesn't care which model is behind it.

The biggest value of this architecture for managers: decision-making power is reclaimed. Switching models means changing one line of routing configuration on the gateway, not four teams each scheduling two weeks of migration tasks.

III. Key Implementation Steps

From project kickoff to launch, we broke the process into five steps, each with clear deliverables:

Step 1: Unified description specification (1 week)

Product and operations first aligned on the output format—a fixed structure: Category / Condition Observations / Accessories Notes / Suggested Title. All prompts are generated around this template, resolving the standards inconsistency at the source.

Step 2: Gateway integration and key consolidation (3 days)

We abolished the API keys scattered across various code repositories, consolidating everything under gateway-side management. Services now use access credentials issued by the gateway.

Step 3: Description generation pipeline (2 weeks)

The core process checklist:

用户上传图片
  ├─ 1. 基础校验（大小/格式/EXIF 隐私信息剥离）
  ├─ 2. 违禁品识别（专用小模型，网关路由）
  │     └─ 命中 → 转人工审核队列，流程终止
  ├─ 3. 商品要素识别（视觉模型：品类/品牌/成色线索）
  ├─ 4. 描述生成（视觉大模型 + 结构化 prompt 模板）
  ├─ 5. 出站内容过滤（网关侧：隐私信息/夸大用词/违禁词）
  ├─ 6. 质量抽检（低置信度样本回流人工标注）
  └─ 7. 写入商品库，附带模型版本与成本标记

Step 4: Canary release and risk control acceptance (1 week)

Roll out to 5% of traffic with daily reconciliation: generation success rate, manual rejection rate, and average cost per item. If the rejection rate exceeds the threshold, automatically roll back.

Step 5: Operational review mechanism (ongoing)

A weekly report on model usage and quality, with managers watching three numbers: cost per item, rejection rate, and the comparison curve after model switches.

A simplified call example (Go) showing how the business side stays "unaware of the underlying model":

// 业务侧只调网关，模型选择由路由配置决定
resp, err := client.CreateChatCompletion(ctx, ChatRequest{
    Model:    "vision-auto", // 网关路由别名，非真实模型名
    Messages: buildPrompt(images, categoryHint),
    Metadata: map[string]string{
        "team":     "algo",
        "project":  "listing-description",
        "trace_id": traceID,
    },
})
// 成本、token 用量、模型版本由网关自动记录，无需业务埋点

IV. Why a Unified AI API Gateway Significantly Reduces Maintenance Costs

This is the clearest calculation I've made as a project manager:

1. One integration codebase instead of N. Different vendors have different authentication methods, request formats, and rate-limiting policies. Direct connections mean rewriting the adaptation layer and re-running tests every time you switch models. With a unified gateway, business code only knows one interface, and the cost of switching models drops from "weeks" to "hours."
2. Key governance consolidated in one place. Keys used to be scattered across code repositories, configuration centers, and even individual local environments, making leak investigations extremely reactive. After consolidating to the gateway, permissions are isolated per project, and rotating a key requires changing only one place.
3. Costs are attributable and budgetable. The gateway keeps books per project/team, so there's no more manually stitching together three vendor invoices at month-end. If a feature's costs spike abnormally, you can pinpoint the specific call chain the same day.
4. Risk control policies iterate uniformly. Privacy filtering and prohibited word interception rules only need to be maintained once on the gateway side—no more policy drift where "the algorithm team updated but operations didn't."
5. A single point of observability for failures. If description generation suddenly slows down or errors out, check the gateway's call traces first—you can determine within five minutes whether it's a model-side or business-side problem, avoiding those meetings where three teams point fingers at each other.

By rough calculation, just the two items—"switching models without rewriting code" and "automated monthly reconciliation"—save us more than a full-time person-week of engineering and coordination time per quarter.

V. Manager's Retrospective

This project completed its full rollout after three weeks of canary release, with stable description adoption rates and controllable per-item costs. More importantly: for any new AI requirement going forward, the team defaults to going through the gateway first—this order is more valuable than the feature itself. My advice for small teams: even if you're only integrating one model, build the gateway before writing business code. The cost of governance is always more expensive when built late than when built early.

If you're also planning similar AI capability integration, you can start with a gateway that supports unified multi-model access and per-project metered billing—lay the architectural foundation first: https://api.thistoken.ai/register

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.