## I

I. Business Pain Point: The "Input Friction" of Bookkeeping Products

We are a five-person team building a personal finance management assistant. Six months after launch, our data dashboard gave us a jarring number: seven-day user retention was under 30%, and 68% of churned users never successfully recorded a second transaction.

Reviewing user interviews, we found the problem wasn't analytical capability but the input step:

• Manually selecting a category, entering the amount, picking a date — recording one transaction takes an average of 7 screen taps;
• What users want is to "just say it casually," e.g., "Lunch cost 35, and 18 for bubble tea I split with a coworker";
• A single sentence may contain multiple expenses, missing time info, and ambiguity — traditional forms simply can't handle that.

The conclusion was clear: without conversational bookkeeping, the product wouldn't survive. But as the person in charge, I cared not just about "can we build it," but also three things management must watch:

1. Process: From the user speaking to the ledger being written to the database, can the pipeline be standardized and monitored?
2. Collaboration: How can the algorithm, backend, and product people develop in parallel without blocking each other?
3. Risk: What if a model hallucination records 35 yuan as 3500? Where does user privacy data flow? How do we control costs?

II. Architecture Design: Turning "Conversational Parsing" into a Pipeline

We didn't treat the LLM as a black box for free-form conversation; instead, we split it into four clearly bounded stages. This was a key management decision — each stage has a single responsibility, so when something goes wrong, it can be traced to a specific person.

用户消息 → ① 意图识别(轻量模型,便宜快)
        → ② 信息抽取(主力模型,结构化输出)
        → ③ 规则校验层(纯代码,不依赖AI)
        → ④ 确认卡片 → 用户点确认 → 落库

Responsibilities of each stage:

• Intent recognition: Determine whether it's bookkeeping, a query, or small talk. A small model suffices, with nearly negligible cost;
• Information extraction: The main model converts natural language into JSON (amount, category, time, note);
• Rule validation layer: This layer is one management must insist on. Amount limits (block and review anything over 5000 per transaction), date sanity checks (no future dates), category whitelist validation — all hardcoded in code. Don't trust the model's output; trust only the validated result;
• Confirmation card: The AI only drafts; writing to the database requires user confirmation. This step shifts responsibility for "wrong entries" from the system back to the human, and complaint rates dropped noticeably.

III. Key Implementation Steps (with a Process Checklist)

From project kickoff to gray release, we took three weeks. Here's a reusable checklist for teams of similar size:

Week 1: Define the contract

• [ ] Product and algorithm teams jointly freeze the JSON Schema (fields, enum values, required items)
• [ ] Prepare 200 pieces of anonymized test data covering colloquial speech, dialect-style expressions, and mixed multiple transactions
• [ ] Define an error-handling strategy: fall back to a form on parsing failure; don't force it

Week 2: Parallel development

• [ ] Backend develops the validation layer and database-write logic using mocked data per the Schema
• [ ] Algorithm side tunes prompts + few-shot, targeting ≥ 92% accuracy on the test set
• [ ] Integrate a unified AI API gateway, converging all model calls to a single base_url

Week 3: Gray release and risk control

• [ ] Route 10% of traffic to gray release; manually review 50 parsing results daily
• [ ] Configure per-user token usage alerts to prevent runaway loops from burning money
• [ ] Run sensitive fields (names, card numbers) through a desensitization middleware before sending requests to the model

Core prompt approach for the parsing stage (simplified):

SYSTEM_PROMPT = """你是记账信息抽取器。从用户消息中提取账目,
严格按以下 JSON 格式输出,无法确定的字段填 null,不要编造:
{
  "items": [
    {"amount": 数字, "category": 枚举值之一,
     "time_hint": 原文时间描述, "note": 备注}
  ],
  "ambiguity": 需要用户澄清的问题, 无则 null
}
分类枚举:餐饮/交通/购物/居住/医疗/娱乐/其他"""

IV. Why a Unified AI API Gateway Reduced Maintenance Costs

This is the point I feel most strongly about as a manager. Early in the project, the algorithm engineer called a vendor SDK directly. Later, when we needed to switch models for cost and quality reasons, it took three days of changes: SDK incompatibility, rewriting retry logic, all monitoring instrumentation broken — three days that could have been spent continuing to build the validation layer.

Afterward, we converged all model calls to a unified gateway, and things changed completely:

1. Switching models became a config change. Intent recognition uses a cheap model, information extraction uses a strong model — all routed at the gateway layer, with only one base_url in application code. When new models launch later, evaluation and switching no longer block business iteration;
2. Costs are attributable. The gateway tracks token consumption by the "feature × user" dimension. At month's end, I get a direct bill: how much each feature burned, which users had anomalous call volumes — budget governance is now evidence-based;
3. Risk control is centralized. Rate limiting, log desensitization, and sensitive-word interception are all done once at the gateway layer instead of each feature writing its own. For a five-person team, writing three fewer sets of duplicated code means maintaining three fewer modules that will rot in the future.

V. Management Retrospective: Three Numbers

Three months after the gray release: recording a transaction went from 7 taps to 1 sentence + 1 confirmation; the second-transaction rate rose from 32% to 71%; complaints from parsing errors stay in single digits per month, thanks to the confirmation card.

If I could do it over, I would build the gateway and validation layer from day one — they are the real moat for a small team; the model itself is actually the most easily replaceable component.

If you're also planning a similar AI feature and want to centralize model calls and keep costs and risk control in your own hands, you can register for a trial at https://api.thistoken.ai/register and get your first request running in ten minutes.

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.