## I

I. The Business Pain Point: Why This "Small" Summarization Task Dragged On for Six Months

We are a knowledge services company of about sixty people, with a large internal accumulation of contracts, research reports, meeting minutes, and client deliverables. Business departments repeatedly raised requests: Legal wants to quickly review contract key points, Sales wants to understand a client's background in one minute, and editors want to extract summaries from thirty-page research reports.

These requests were scattered across four departments' to-do lists, each finding their own tools: some pasted sensitive contracts into free web-based tools, some manually copied and pasted into LLM conversations, and some simply gave up. As the technical lead, I took stock, and the pain points actually fall into three layers:

1. Missing process: There was no unified entry point for who submits requests, who schedules them, and who accepts them. Four meetings over six months produced no conclusions.
2. Chaotic collaboration: Three developers each wrote their own calling scripts, with API keys scattered across personal computers and code comments.
3. Uncontrolled risk: Sensitive documents were being pasted into uncontrollable external websites. If something went wrong, no one would be held accountable.

Management's conclusion: set the rules first, then build the system. The goal was deliberately modest — launch an internal document summarization system within thirty days, covering PDF, Word, and Markdown formats, producing structured summaries, with a complete audit trail.

II. Architecture Design: Turning "Rule by Individuals" into "Process Governance"

We deliberately kept the architecture minimal, divided into four layers:

• Entry layer: An internal web application where all departments submit documents through the same entry point, eliminating private use of external tools.
• Task layer: After upload, documents enter a task queue for asynchronous processing, preventing long documents from blocking; each task is bound to the submitter, department, and document classification level.
• AI gateway layer: All model calls go through a unified AI API gateway. This layer is the key to cost control for the entire system, discussed separately below.
• Audit layer: For every call, the input summary hash, model, token usage, and latency are all persisted to the database, with monthly reports generated.

What management cares about most is risk control, so we set three red lines:

1. Documents classified as "high" sensitivity may only call designated models, and logs are retained for six months.
2. Overly long documents are automatically summarized in segments and then merged, preventing over-limit errors from causing silent task failures.
3. Each department has a monthly token budget threshold; exceeding 80% automatically notifies the department head.

III. Key Implementation Steps

The thirty-day schedule was roughly as follows (process checklist):

第 1 周：需求收敛与规则制定
  - 各部门提需求 → 合并为三档摘要模板（合同要点/调研摘要/会议纪要）
  - 确定文档密级分类标准与审批人

第 2 周：架构搭建
  - 部署内部 Web 应用（上传、队列、结果页）
  - 接入统一 AI API 网关，配置模型路由与密钥
  - FastAPI + Celery 异步任务，PostgreSQL 存任务与审计日志

第 3 周：摘要链路与模板
  - PDF/Word 解析 → 清洗 → 分段 → Map-Reduce 摘要 → 模板化输出
  - 普通文档用性价比模型，高密级路由到指定模型

第 4 周：验收与治理
  - 三部门各出 20 份真实文档做盲评（人工摘要 vs 系统摘要）
  - 上线审计报表与预算告警，写进运维手册

The core calling code is very thin, because the gateway shields all the differences:

async def summarize(text: str, level: str) -> str:
    model = MODEL_ROUTING[level]          # 按密级路由模型
    resp = await gateway.chat(
        model=model,
        messages=[
            {"role": "system", "content": TEMPLATES[level]},
            {"role": "user", "content": text},
        ],
        timeout=60,
    )
    audit_logger.log(model=model, usage=resp.usage, level=level)
    return resp.content

IV. Why a Unified AI API Gateway Reduces Maintenance Costs

Before launch, we did the math: if three developers each connected directly to different model vendors' APIs, maintenance costs would show up in four places:

1. One key change, three places to update: When vendors change API versions or adjust rate-limiting policies, every directly-connected script needs updating. With the gateway, only the gateway configuration changes — application-layer code stays untouched.
2. Unified billing and budgeting: All calls go through the same egress, so token usage can naturally be aggregated by department and by task, without manually reconciling three vendors' bills at month-end.
3. Single point of convergence for failures: When a model times out or gets rate-limited, the gateway handles retries and failover uniformly — business code doesn't need each developer writing their own retry logic, which was precisely the biggest hidden cost of the previous "three scripts" approach.
4. Centralized access control: Model whitelists and sensitivity routing rules are all configured on the gateway side. When someone leaves the company, there's no need to hunt down scattered API keys, and auditing has a single source of truth.

For a three-person team, this compresses "each developer maintaining their own integration" into "the team maintaining one configuration" — the time saved on manpower is enough for an extra template iteration every week.

V. Acceptance Results and Lessons Learned

In the blind evaluation, the adoption rate of summaries exceeded 80% across all three departments, with the editorial team reporting the most obvious time savings. But more important than the numbers is the governance effect: within three months, there were no further incidents of sensitive document leaks, and the monthly AI spending report became a standing item in management meetings.

Looking back, what managers should do in implementing AI applications isn't entirely about technology selection, but three things: funnel all requests into a single entry point, funnel all calls into a single gateway, and funnel all risks into a single report.

If your team is also preparing to build a similar AI application, you can start with an AI API gateway that supports unified multi-model access — lay the governance foundation first. Registration entry point is here: https://api.thistoken.ai/register

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.