pool = exercise_db.filter(

equipment=profile.equipment,

level=profile.level,

exclude_conditions=profile.injuries

)

2. 检索相似成功模板作为few-shot

refs = template_store.search(profile.embedding, top_k=3)

3. LLM只做编排

raw = await llm.chat(

model=route_by_task("plan_compose"), # 网关智能路由

messages=build_prompt(profile, pool, refs),

response_format=PlanSchema # 强制结构化输出

)

4. 安全二次校验

return safety.audit(raw, profile)

## IV. Why a Unified AI Gateway Saved Our Maintenance Costs

The most valuable decision during the rebuild was routing all model calls through a unified gateway instead of hooking up each vendor's SDK directly in the code. The reasons are very concrete:

- **Swap models without changing code.** The gateway is OpenAI-protocol compatible, so the upper layer only needs one set of call logic. Later, we switched feedback classification from a large model to a small one—changing a single routing configuration with zero code changes cut per-call costs by about 70%.
- **Rate-limit fallback protection.** When the primary model gets rate-limited at peak times, the gateway automatically switches to a backup model—users barely notice. The "wait 40 seconds for a broken plan" scenario from version one never happened again.
- **Centralized key and billing management.** Three models, two environments—keys live only in the gateway config, never in the code repository. Bills are consolidated and viewable in one place. For a small team like ours, managing five platforms' consoles just wasn't feasible.
- **Unified logging.** Latency, token consumption, and failure reasons for all calls are visible in one place. Debugging "why did this plan generation fail" went from digging through three platforms' dashboards to looking up a single log entry.

For a small team, this effectively outsources "multi-model operations"—two people can support an architecture that looks like it has a dedicated AI infrastructure team behind it.

## V. Results and a Few Recommendations

The rewritten version: average plan generation time dropped to under 6 seconds, format error rates went to essentially zero, and week-two user retention improved noticeably (I won't go into specific numbers—user demographics vary widely). Looking back, the biggest lesson is this: **the last thing you should hand over to AI in an AI application is precisely the parts that can be nailed down with rules and Schemas.**

Three pieces of advice for indie developers planning similar features: build your data first, then plug in models; write hard constraints in code and leave soft decisions to the LLM; go through a unified gateway from day one—don't wait until you're switching to your second model to regret it.

If you're also planning model integration for an AI application, start with a unified API gateway. Registration is here: https://api.thistoken.ai/register

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.