A Typical Failed Opening

Last year I took on a small project for a chain store management system. After finishing the order module, I thought I'd use AI to add a set of integration tests. So I opened the chat window and typed what felt like a perfectly natural request at the time:

> "Help me write a set of integration test cases for the order module, using Python + pytest."

The AI was very cooperative, spitting out twenty-plus cases in seconds: create order, cancel order, modify shipping address, query order list... neatly formatted with complete assertions. I copied them into the project, ran them for three days, and the coverage report did look great.

Then we went live, and it blew up the same day.

The problem was in the "inventory locking" step: when placing an order, inventory must be locked first; if the lock fails, a rollback is needed; and if the user concurrently submits duplicate requests during the rollback, dirty data gets created. This was written in the requirements doc, it existed in my head—but the AI didn't know, because I never told it. All the cases it generated were "ideal paths + isolated exception branches"—not a single one covered the interconnected state transitions that happen in real business.

This failure taught me one thing: AI is not a test case generator; it's a context porter. Whatever business reality you feed it determines the quality of scenarios it can design; feed it nothing but a module name, and all it can give you is a textbook-style empty shell.

Three Common Misuses—You're Probably Making Them

While reworking my methodology, I reviewed how I and several indie developer friends had been using AI, and the failure patterns were highly concentrated:

Mistake 1: Treating AI as a test case generation button. Input "write tests for XX feature," and you get generic templates. The typical characteristics of these cases: lots of single-interface tests, few cross-module flows; many happy paths, few business exceptions. They run all green—green to the point of carrying zero information.

Mistake 2: Giving AI no system context. The essence of integration testing is verifying collaboration between modules, but many people just paste in an API definition. The AI doesn't know there's a locking relationship between orders and inventory, doesn't know payment callbacks are asynchronous—of course it can't design a scenario like "payment callback arrives late, but the order has already been canceled."

Mistake 3: Accepting whatever AI generates without review. The cases AI produces look professional, but they may include branches that don't exist in the business (for example, it designed a "guest checkout" case for the store system, which actually requires login). Adopting them as-is means maintaining a pile of tests for things that will never happen.

The Right Path: Upgrade AI from "Case Writer" to "Scenario Design Partner"

After changing my approach, I restructured the process into four steps, and the AI's role completely changed:

Step 1: Feed context first, don't mention "writing tests." I paste the key sections of the requirements doc, core table structures, and inter-module call relationships (even just a few lines of textual description) into the AI, and first have it paraphrase the system behavior back to me to confirm its understanding is correct.

Step 2: Have the AI ask questions instead of outputting directly. This is the most critical step. I ask the AI to play the role of a test designer and raise risk points it sees in the business flows. This time it asked: "Are inventory locking and order creation in the same transaction?" "How does the order state transition after a payment callback timeout?" "When a store's inventory is insufficient, is cross-store transfer allowed?" These questions forced me to write down all the implicit business rules in my head.

Step 3: Design a scenario matrix based on the confirmed rules. Have the AI output scenarios along four dimensions—"normal flows / exception branches / concurrency and timing / boundary data"—with each scenario annotated with the business rule numbers it covers. Cases generated this way are traceable, not imagined out of thin air.

Step 4: Human review—cut branches that don't exist, add pitfalls AI can't think of. AI doesn't know the real production environment. For example, I know a certain third-party logistics API occasionally times out—this kind of "local knowledge" has to be added by a human.

A Reusable Prompt Template

Here's the scenario design prompt I've refined through multiple iterations—just replace the bracketed content to use it:

你是一位资深测试架构师，帮我设计集成测试场景。请严格按以下流程执行，不要跳步：

【系统背景】
- 系统简介：[一段话描述系统是做什么的]
- 技术栈：[语言/框架/数据库/消息队列等]

【业务规则】
- 规则1：[例：下单时先锁库存，锁失败则订单创建失败]
- 规则2：[例：支付回调为异步消息，可能晚于用户取消操作到达]
- 规则3：[补充你所有的隐含业务规则，越多越好]

【模块与依赖关系】
- [例：订单模块 → 依赖库存模块（锁定/释放）、支付模块（回调）、通知模块（发短信）]

【你的任务】
第一步：用你自己的话复述上述业务规则和模块关系，
如果发现模糊、矛盾或缺失之处，先向我提问，等我回答后再继续。
第二步：等我确认后，输出集成测试场景矩阵，按四个维度组织：
1. 正常链路（跨模块完整流程）
2. 异常分支（每个依赖点失败时的行为）
3. 并发与时序（竞态、乱序、重复请求、超时）
4. 边界数据（数量、金额、状态枚举的临界值）
每个场景需包含：场景编号、前置条件、触发操作、
跨模块交互点、预期结果、对应的业务规则编号。
第三步：标注你认为风险最高、必须优先覆盖的5个场景，并说明理由。

The essence of this template lies in "ask questions first, then design" and "linking scenarios to business rule numbers"—these two points directly determine whether the output is a textbook or your actual system.

Before vs. After Using AI This Way: A Real, Tangible Difference

The time picture changed too: before, "generating cases" took only 5 minutes, but rework fixing bugs took two days; now, preparing context and back-and-forth Q&A takes 40 minutes upfront, but everything afterward is nearly smooth sailing. On the whole, what you save isn't time writing code—it's time troubleshooting production incidents.

Final Thoughts

With AI-assisted integration testing, the dividing line of value isn't in model capability, but in the quality of input you give it. It can't read the business rules in your head, and it can't see the pitfalls of your production environment—but it's extremely good at exhaustively enumerating combinations, finding gaps, and questioning ambiguities once you give it complete context. Treating AI as a challenger rather than a generator is the first step to using it well.

If you don't have a stable AI API to use yet, I recommend trying ThisToken (https://api.thistoken.ai/register), an API platform aggregating multiple mainstream models—register and use immediately, pay-as-you-go (see the official pricing page for details)—well suited for indie developers and small teams to get this "scenario co-creation" pipeline up and running.

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.