## Two Failed Counterexamples First

Two Failed Counterexamples First

Counterexample One: Treating AI as a "Q&A Bot" Stuffed into the IDE

My first version was sloppy: I added a chat window to the editor sidebar, sent user questions directly to a large language model, and the model returned a chunk of code that developers had to copy and paste themselves. Two weeks after launch, almost no one on the team used it.

In the post-mortem, I found three fatal problems:

1. The context was fed incorrectly. When a developer asked "why is this function throwing an error," I only sent 200 characters around the cursor. The model knew nothing about the project structure, dependency versions, or the full call chain. The answers looked fluent but simply wouldn't run when placed into the project.
2. The answers weren't executable. The model returned narrative text like "you could modify it this way" plus code snippets, and developers had to figure out which file to change and which lines to change. For an indie developer racing a deadline, this "translation" cost was higher than not asking at all.
3. The model was single and hardcoded. I hardcoded the model name and API Key directly in the client. Whenever the model got upgraded or rate-limited, the whole plugin went down with it.

Counterexample Two: Letting AI Modify Files Directly, "Everything-at-Once" Rewrites

In the second version, I overcorrected: I sent the entire file to the model, had it return the complete modified file, and the client directly overwrote it to disk. The result was even worse—the model often "optimized" unrelated code along the way, renaming variables and adjusting whitespace, leaving the diff a total mess. Once it even deleted a developer's comments. Developers have zero tolerance for a loss of control, so it was only a matter of time before the second version was abandoned.

Three Principles Failure Taught Me

• AI's output must be a "patch," not an "answer." Answers require human translation; patches can be reviewed, applied, and rolled back directly.
• Context should be constructed on demand, not the more the better. Related files, cursor position, error stacks, dependency manifests—none can be missing, but unrelated code is just noise.
• Model integration must be decoupled from the business logic. The client shouldn't even know models exist.

Third Version Architecture Design

┌─────────────┐
│  IDE 插件    │  采集上下文 + 展示 diff + 应用补丁
└──────┬──────┘
       │ 统一协议（/v1/chat + 结构化输出）
┌──────▼──────────────────┐
│      AI API 网关         │  Key管理 / 路由 / 限流 / 降级
└──┬─────────┬─────────┬──┘
   │         │         │
模型A      模型B      模型C
(补丁生成) (上下文摘要) (意图识别)

Key division of labor:

• The plugin side is responsible for only three things: constructing context, rendering patch diffs, and one-click apply/undo.
• The gateway layer centrally holds all model API Keys, routes to different models by task type, and handles budget control and failover.
• The model side is tiered by task: a lightweight model for intent recognition, a mid-tier model for context summarization, and the strongest model for patch generation.

Key Implementation Steps

Step 1: Define the patch output protocol. Force the model to return a structured format rather than free text:

{
  "intent": "fix_bug",
  "patches": [
    {
      "file": "src/auth/login.ts",
      "explanation": "token 过期未刷新导致 401",
      "hunks": [
        {
          "start_line": 42,
          "old_lines": ["if (res.code === 401) throw new Error('unauthorized');"],
          "new_lines": ["if (res.code === 401) {",
                        "  await refreshToken();",
                        "  return retry(req);",
                        "}"]
        }
      ]
    }
  ],
  "confidence": 0.86
}

The plugin side renders this into a standard diff view; when confidence is below a threshold, it only suggests rather than auto-applies.

Step 2: Construct the task context. The checklist:

1. Read the current file + cursor position (including selection);
2. Use a lightweight model to classify the intent of the error message (bug fix / refactoring / adding tests);
3. Based on import relationships and symbol references, fetch up to 3 related files;
4. Use a summarization model to compress the related files within the token budget;
5. Assemble everything into a role-annotated prompt and send it through the gateway to the patch model.

Step 3: Patch validation and application. Before applying, run three checks: whether the line-number anchors still match (to guard against the file being changed while waiting), whether the syntax parses (validated with the project's parser), and whether unrelated regions are being modified out of bounds. After application, push to the undo stack for one-click rollback.

Step 4: Build a feedback loop. Every patch that gets applied, modified-then-applied, or rejected is fed back as a signal, used to tune prompt templates and routing strategy.

Why a Unified AI API Gateway Reduces Maintenance Costs

This is the most important lesson I learned from three rounds of rework. When every feature and every client holds its own model Keys and SDKs, the maintenance cost is O(number of features × number of models): a model upgrade requires N changes, Key rotation requires notifying everyone, and when one model gets rate-limited you're firefighting one by one. Converge it into a gateway, and it becomes O(1):

• Centralized Key management: rotate a Key once and all endpoints take effect immediately, with zero client changes;
• Models become swappable: switching the patch model from A to B only requires changing the gateway routing config, with the plugin protocol untouched;
• Unified monitoring and budgeting: all calls go through the same pipeline; billing, rate limiting, and degradation policies are configured centrally, and when something goes wrong, one log is all you need to check;
• Degradation with fallbacks: when the primary model times out, it automatically switches to a backup, invisibly to developers.

For a small team of two or three people, this architecture means: you can put all your energy into patch quality and the diff experience, instead of firefighting model issues every day.

Final Thoughts

The value of an AI coding assistant isn't that "AI can write code"—it's that it delivers patches that can be reviewed, applied, and rolled back directly. If you want your team to plug into multi-model capabilities faster without being dragged down by Keys and SDKs, start by registering with a unified AI API gateway—you can get access credentials for all mainstream models in just a few minutes: https://api.thistoken.ai/register

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.