## Why We Chose This Scenario

Why We Chose This Scenario

A friend of mine who runs a legal education channel asked me for help: he wanted to build a legal consultation assistant for ordinary people. When a user asks a question like "What should I do if my landlord won't return my deposit?", the system should provide reference advice based on real legal statutes. He's an independent developer with a limited budget — the whole team is just himself.

The core dilemma of this scenario is very typical: general-purpose large language models have only a shaky grasp of legal provisions and can easily fabricate statutes that don't exist (hallucination is fatal in legal scenarios); meanwhile, fine-tuning a model is too costly, time-consuming, and data-hungry for an independent developer. Retrieval-Augmented Generation (RAG) is practically the only realistic option — chunk and vectorize the text of statutes like the Civil Code, retrieve relevant chunks, and inject them into the prompt, so the model gets to take an "open-book exam."

Business Pain Points: Three Mountains — Time, Cost, and Risk

Before starting, we did the math, which also forms the efficiency-focused throughline of this post:

Pain point 1: Manual verification costs caused by hallucination risk. In early tests calling general-purpose models directly, about three out of ten answers cited incorrect statute numbers or fabricated "judicial interpretations." My friend had to verify every single answer manually, spending roughly 8 minutes of effective time per consultation. At an estimated 200 consultations a day, verification alone would consume nearly 27 hours of labor — completely unviable as a business.

Pain point 2: High trial-and-error costs across multiple models. Building a production RAG system requires repeatedly comparing models: retrieval reranking, answer generation, summary compression — different stages suit different models. If you have to apply for a separate API key and write separate adapter code for each model, onboarding each new model takes about half a day of development and debugging. And if a model raises prices or rate-limits you, switching takes another half day.

Pain point 3: Latency directly hurts user experience. Legal consultation questions tend to be long, and early end-to-end response times exceeded 25 seconds, with a visibly high bounce rate.

Architecture Design

The final architecture has five layers, all running on a single 2-core 4GB cloud server:

用户提问
   │
   ▼
┌─────────────┐
│ 意图澄清层   │ 小模型：判断问题类型、补全省略主语
└──────┬──────┘
       ▼
┌─────────────┐
│ 检索层       │ 向量库（法条切片，512 token/片）
└──────┬──────┘  + BM25混合检索 + 重排模型
       ▼
┌─────────────┐
│ 生成层       │ 大模型：只基于检索到的法条作答，
└──────┬──────┘  强制要求标注条文出处
       ▼
┌─────────────┐
│ 校验层       │ 规则引擎：校验引用的条文编号真实存在
└──────┬──────┘
       ▼
结构化回答（结论 + 法条依据 + 行动建议 + 免责声明）

Two key design decisions:

1. Statute chunks are split per "article," not by fixed length. The integrity of legal provisions takes priority over uniformity for vector retrieval, and this step measurably improved retrieval accuracy.
2. Decouple generation from validation. Citation validation uses a rules engine rather than another LLM call — checking whether a statute number exists in the statute database takes a single string match, eliminating unnecessary token overhead.

Key Implementation Steps

Step 1: Corpus preparation (~2 days). We collected provisions from the Civil Code covering high-frequency domains like labor, leasing, and consumer rights — about 12,000 articles in total — split them by article, and attached domain tags. During cleaning, we found 3% of the raw corpus was garbled or duplicated. Don't skip this step.

Step 2: Building the vector index (half a day). We used an embedding model for batch vectorization, with hybrid retrieval (vector + keyword) to address the problem that pure semantic retrieval is insensitive to exact statute numbers.

Step 3: Prompt and generation pipeline (1 day). Core prompt constraints:

PROMPT = """你是一个法律咨询助手。请严格遵循以下规则：
1. 只能基于下方提供的法条片段回答，禁止引用未出现在片段中的条文
2. 每个结论后用【条文：第X条第X款】标注出处
3. 如果检索到的法条不足以回答问题，明确说"根据现有资料暂无法回答"，并建议咨询律师
4. 结尾必须附上"本回答仅供参考，不构成法律意见"

法条片段：
{retrieved_chunks}

用户问题：{question}"""

Step 4: Unified AI API gateway access (the key efficiency decision). All model calls — the small model for intent clarification, the embedding model, the reranking model, the large generation model — go through a single AI API gateway. This step was the watershed for the project's maintenance costs:

• Configure once, apply everywhere. Application code references only the gateway address; switching models just means changing the routing configuration in the gateway console, with zero code changes. In practice, switching a generation model went from half a day to under 10 minutes.
• Unified billing and monitoring. All calls are visible in one dashboard — which stage is burning money is immediately obvious, no more logging into five or six platforms to reconcile bills.
• No more scattered API keys. The application server holds only one gateway key, dramatically shrinking the attack surface for key leakage.
• Zero-cost model comparison. For A/B testing, just change the model name parameter on the same request — no need to separately register accounts, top up balances, and write adapter layers for each candidate model.

Step 5: Latency and cost optimization (2 days). Before/after comparison:

Cost reduction came from three sources: intent clarification and summarization were routed to cheap small models (60% of call volume); retrieved chunks went through relevance reranking so only the top 5 most relevant were fed into the prompt, cutting context tokens nearly in half; and high-frequency questions (e.g., "quitting during probation") got a semantic cache — cache hits return immediately.

Pitfalls We Hit

First, a reranking model isn't always necessary — on a small database of 12,000 articles, pure hybrid retrieval plus simple scoring was sufficient, and adding a reranker just meant an extra network call and 0.8 seconds of latency. Second, the disclaimer must be forced into the output via the prompt — this is a compliance baseline in legal scenarios; don't count on the model doing it on its own.

Final Thoughts

From project kickoff to publicly usable service: one person, six working days. Looking back, the most cost-effective decision wasn't picking the most powerful model, but consolidating the model layer behind a unified AI API gateway — it turned every subsequent model replacement, cost experiment, and multi-model division of labor into a configuration change rather than a code change. For an independent developer, every "adapter code rewrite" saved is time you can spend building new features.

If you're also preparing to build your own RAG application, you can start by registering for a unified AI API gateway: https://api.thistoken.ai/register

---

Ready to try it yourself? Sign up at https://api.thistoken.ai/register to get your API key and start building.