A Counter-Example to Start With

Let me start with a pitfall I fell into myself.

Last year, I was troubleshooting a memory leak in a dependency library. The issue tracker had over two hundred related records. My approach: open them one by one, scan with my own eyes, and copy anything that "looked relevant" into a notes app. Three hours later, I had accumulated forty-some fragments of information, yet still couldn't answer the most critical questions—has this bug actually been fixed? Which PR fixed it? Is my version affected?

That's not diligence—that's simulating a database with manual labor. There are a few other similar failure modes:

• Only searching keywords while ignoring the timeline: Issues often span months from report to fix. If you only read the body and skip the comments, your conclusions may already be outdated.
• Dumping the full issue text to AI and asking it to "summarize": The AI will just give you a bland, safe summary that doesn't answer your specific questions.
• Asking AI to "analyze all issues": The context window can't fit everything, so the model starts confidently fabricating issue numbers and conclusions—and you have to verify each one by hand, which is even more exhausting than reading them yourself.

The problem isn't that AI is incapable. The problem is that you're treating it as a "faster search engine."

The Right Approach: Structured Extraction, Not Vague Summarization

The key mindset shift is: first define exactly which fields you need, then have the AI extract those fields from the noise.

The useful information in issue threads is actually highly structured: reproduction conditions, environment versions, error messages, maintainer attitudes, linked PRs, resolution status. What makes human scanning slow is that the text is littered with "me too," "+1," off-topic discussions, and venting. This is precisely the scenario AI excels at—extracting information from long texts in a fixed format.

My current workflow looks like this:

Step 1: Filtering and preprocessing. Use GitHub's search syntax (labels, status, keywords) to narrow the issue scope down to a few dozen, then export the bodies along with the comments. This step must be done by a human—AI can't judge "what counts as relevant" for you.

Step 2: Feed issues to AI in batches for structured extraction. About a dozen per batch, using a consistent field template. Require the AI to output a fixed structure for each issue, and to explicitly mark "not mentioned in the original text" rather than guessing.

Step 3: Have the AI aggregate and draw conclusions. Combine all the structured batch results and ask specific questions: Which versions does this bug affect? Is there a workaround? Are the maintainers planning to fix it, or is it marked wontfix?

Step 4: Manual spot-checking. For the AI's key conclusions (especially issue numbers and PR links), click back to the original text to verify. This step cannot be skipped—AI extraction accuracy is high, but not 100%.

Before and After

Before (purely manual): Three hours reading through two hundred issues, ending up with a pile of disordered fragments, unable to write a conclusion, and continuing to sift through them the next day.

After (AI-assisted workflow): Half an hour for filtering and export, ten minutes to run the extraction, ten minutes to aggregate, ten minutes to spot-check and verify. The output isn't "I vaguely remember someone mentioning this," but a table you can paste directly into your team docs: each issue's status, impact scope, fixing PR, and temporary workarounds, laid out clearly column by column.

As a side note, running this workflow repeatedly consumes a fair number of tokens, especially with dozens of long issues and full comment threads. If you want to control costs, you can opt for a pay-as-you-go API service that supports multiple models (see the official pricing page for rates), delegate the extraction tasks to a more cost-effective model, and switch to a stronger model for the aggregation step—this can compress costs considerably.

A Reusable Prompt Template

For batch extraction, I consistently use this template (placed before the content of each issue batch):

你是一个 GitHub issue 信息提取助手。我会给你若干条 issue 的完整内容（含评论）。
请对每条 issue 严格按以下格式输出，不要添加任何额外解释：

【Issue 编号】#xxx
【标题】一句话概括
【问题类型】bug / 功能请求 / 使用疑问 / 讨论 / 无法判断
【复现条件】具体环境、版本、触发步骤；原文未提及则写"未提及"
【关键报错信息】原文摘录；无则写"无"
【社区反馈】多少人报告同样问题（依据是评论数与内容）
【维护者态度】已确认 / 已忽略 / wontfix / 无回应 / 未提及
【关联 PR】有则列出编号，无则写"无"
【当前状态】已修复 / 修复中 / 未修复 / 已关闭未说明
【信息可信度】高 / 中 / 低（依据原文信息完整度判断，不要凭空推断）

注意：
1. 所有结论必须来自原文，禁止推测或编造编号
2. "我也是""+1"类评论只计入社区反馈数量，不作为问题证据
3. 如果一条 issue 在评论中状态发生变化（如后来被修复），以最新状态为准

In the aggregation phase, make your questions as specific as possible: "Based on the structured data above, list all issues related to memory leaks that remain unfixed, sorted by number of reporters, and describe the temporary workarounds for each." The more your question resembles SQL, the more the answer resembles a query result.

A Few Reminders

This method doesn't apply to every scenario: if you're only checking three to five issues, reading them manually is faster; if the issue bodies are already high-quality bug reports, reading the originals directly is more economical than running a whole workflow. AI's value lies in scale—only when the volume of information exceeds what a human can read does structured extraction truly save time and money.

Also, for decisions like "should I upgrade this dependency version," the AI-extracted results are just inputs. The final judgment still has to account for your own project's circumstances. AI can compress two hundred issues into one table, but the one reading the table and making the decision is still you.

If you want to try this workflow hands-on, you'll need a stable API channel that supports switching between multiple models—check this out: https://api.thistoken.ai/register

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.