Let AI Read the Bad Reviews First: A Negative-Review Analysis Workflow for Indie Developers

First, a Counter-Example: My Old "Bad Review Handling Triple Play"

Last year I built a small tool-type product. The user base wasn't big, but there was no shortage of bad reviews. My handling process went something like this:

First move: emotional rebuttal. Upon seeing a harshly worded bad review, my first reaction was "the user doesn't know how to use it." I'd screenshot it, post it to the group chat, everyone would mock it, and we'd call it a day. The informational value of the review was completely swallowed up by emotion.

Second move: fixing things based on impressions. Once I genuinely resolved to improve the product, I asked the team, "What do you think users are most unhappy about?" Everyone said something different—because each person only remembered the few reviews they'd happened to see recently. In the end, we spent two weeks fixing what "I thought was the most frequent" problem, and after launch, the data showed no change whatsoever.

Third move: reading everything manually. After being burned, I swung to the other extreme: I printed out 300+ reviews and read them one by one. After two nights of reading, I was more confused than ever: one said slow, another said expensive, and another said the button color was ugly. When I finished, all I had left in my head was mush—no list of anything.

The common problem across all three stages: the value of bad reviews was never extracted in a structured way. Reviews are unstructured text, and human short-term memory and summarization ability simply can't handle hundreds of messy narratives. Indie developers and small teams suffer the most—no user research role, no data analysis role; the boss alone is the entire "user insights department."

A Different Approach: Let AI Do the Dirty Work in the Middle

Later, I changed the process to: humans define "what counts as a valid insight," and AI does the reading, categorizing, counting, and ranking.

The key to this division of labor: AI doesn't get annoyed by nasty reviews, doesn't get fatigued after reading three hundred of them, and doesn't form biases from the three most recent ones. What it's good at is exactly what humans are worst at—mechanically reading through all the text and categorizing it by a consistent standard.

The specific process has four steps:

1. Export the raw reviews: Export from app stores, customer service systems, and community feedback channels, keeping the timestamp, channel, and original text. Be careful to anonymize—strip out user IDs and contact information.
2. Run batch extraction: Feed the reviews into a large model API in batches (30–50 per batch, to avoid missed reads from overly long context), and have the AI output structured results along consistent dimensions.
3. Merge and aggregate: Merge the batch results once more, and have the AI do cross-batch deduplication and frequency counting, outputting a categorized problem table.
4. Humans make the call: The AI's list is sorted by "mention frequency × severity," and I only need to look at the top ten and decide what to do based on technical cost. The decision still rests with humans; AI is just responsible for giving the decision a basis.

After the first run, what impressed me most was this: the "most frequent problem" I'd guessed only ranked seventh. Ranked first was "missing onboarding for first launch"—something I'd never paid attention to, because the people complaining about it used mild wording, and when humans read reviews, their attention gets captured by the fiercest complaints. AI has no such bias.

Before and After Comparison

On cost: the API call cost for processing a few hundred reviews is very low—for exact pricing, refer to the official pricing page—for indie developers it's basically negligible.

A Reusable Prompt Template

你是一位产品分析师。我会提供一批用户差评，请严格按以下要求处理：

1. 逐条阅读，将每条评论反映的问题归入以下类别（不可归入的放"其他"并注明）：
   - 功能缺失
   - 性能/速度
   - 上手难度/引导
   - 稳定性/崩溃
   - 价格/付费
   - 交互设计
   - 客服/响应
2. 输出格式为JSON数组，每个元素包含：
   category（类别）、count（提及次数）、summary（该类问题的简要概括，50字内）、
   representative_quote（最有代表性的一条原文，保留用户原话）、severity（1-5，按对使用的影响程度）
3. 规则：
   - 一条评论提到多个问题时，分别计入对应类别
   - 纯情绪宣泄、无具体信息的评论标记为 noise，不计入统计
   - 不要发明评论中不存在的细节
4. 最后按 count × severity 降序排列，另附一段100字以内的总体结论。

以下是差评内容：
{comments}

Usage tip: plug this template into your workflow and replace {comments}. Small teams can also just paste it directly into a chat interface, but once you're past two or three hundred reviews, it's better to run it in batches via the API for better stability.

A Few Pitfall Warnings

• Sample and validate before running the full set. First feed 50 reviews to the AI for categorization, manually spot-check 20% to see if the categorization is reasonable; if not, adjust the prompt first, then run the full set. Garbage in, garbage out.
• Watch out for hallucinations. Explicitly require it to "not invent details that don't exist in the reviews," and keep the original-text quotes so you can trace back and verify.
• Run channels separately. App store reviews and support tickets differ greatly in tone and problem distribution; mixing them dilutes both. Separate tables per channel are more valuable.
• The list is not the end point. What AI gives you is a "problem distribution map"—what to do, how to do it, and how to schedule it still depends on your own understanding of the product.

Final Thoughts

Bad reviews are the cheapest, most genuine user research in the world—they just come in the least readable form. Once AI peels away that shell, indie developers can, for the first time, listen to users in a structured way just like big companies do—at a cost of just a few API calls.

If you're also looking for a stable large model API service to run this kind of text processing task, give https://api.thistoken.ai/register a try. Sign up and start integrating, and turn your pile of bad reviews into the next version's improvement list.

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.