## Business Pain Point: Logs Keep Growing, Information Ke...

Business Pain Point: Logs Keep Growing, Information Keeps Sinking

We're a SaaS team of about a dozen people with a standard backend operation log system: who, at what time, from which IP, changed which customer's configuration, or touched which data. Audits need it, compliance needs it, and incident investigations rely on it.

But the reality is: tens of thousands of log entries pile up every day, and nobody actually "reads" them. The security colleague samples them once a week; the operations lead only digs through them after customer complaints. The logs' value starts sinking the moment they're generated, and by the time you actually need them, the cost of manual searching is absurdly high.

From a manager's perspective, this is really three problems stacked together:

1. Process problem: Log review isn't part of any daily workflow—it's an "after-the-fact tool" rather than an "in-the-moment mechanism";
2. Collaboration problem: Engineering, operations, and security each look at the same logs differently, with inconsistent interpretations—during discussions you often hear "that entry you mentioned doesn't look like what I saw";
3. Risk problem: Real risk signals (an account repeatedly changing permissions in the middle of the night, a tenant's data being bulk-exported) drown in routine CRUD operations. The later they're discovered, the higher the remediation cost.

My judgment: this kind of problem isn't suited to throwing more people at it—it's suited to an AI layer that does "summarize + distill" preprocessing, so the team consumes conclusions rather than raw materials.

Architecture Design: AI Stays Out of the Critical Path, Consuming Only via a Side Path

I deliberately made one architectural decision: the AI summarization service is fully side-channeled and never touches the write path.

操作日志 ──写入──> 日志库(MySQL/ES)
                        │
                        ──异步拉取──> 摘要任务调度器
                                          │
                                          ──调用──> 统一AI API网关
                                          │              │
                                          │        多模型路由(长文用大模型，
                                          │          常规摘要用小模型)
                                          ↓
                                     摘要结果库
                                          │
                              ┌───────────┼───────────┐
                              ↓           ↓           ↓
                          每日风险简报   租户异常报告   周度趋势分析

A few key design points:

• Asynchronous batch processing, not real-time: The value of operation log summaries lies at the "hourly" and "daily" level. There's no need to chase real-time, and therefore no need to stuff AI into the business pipeline. If the AI service goes down, the log system is completely unharmed.
• Layered summarization: Individual logs are first structurally cleaned, then aggregated into chunks by "tenant + user + time window" before being sent to the model for summarization. Throwing tens of thousands of raw logs at a model directly would make both cost and quality spiral out of control.
• Model division of labor: A cheap small model is sufficient for routine daily summaries; high-risk segments like permission changes and bulk exports get routed to a large model for finer risk analysis.
• Human review as a safety net: AI-flagged "suspected anomalies" go into a pending-confirmation queue that the security colleague spends ten minutes confirming every day. The AI doesn't make decisions—it only does initial screening. This was the precondition for me being able to roll this mechanism out to the team.

Key Implementation Steps

During implementation, I broke it down into four steps, each with clear acceptance criteria:

第一步：日志结构化（1周）
  - [ ] 统一日志schema：actor / tenant / action / resource / result / ip / time
  - [ ] 按 租户+操作者+30分钟窗口 聚合成摘要单元
  验收：单日日志可压缩为500~800个摘要单元

第二步：提示词与摘要管道（1周）
  - [ ] 编写摘要prompt：要求输出"做了什么、有无异常、风险等级"
  - [ ] 强制JSON输出，便于入库和后续统计
  验收：抽样50个单元人工核对，事实性错误率为0

第三步：接统一AI网关（2~3天）
  - [ ] 所有模型调用走同一个网关入口
  - [ ] 配置路由规则：按任务类型分发大小模型
  验收：切换模型供应商时业务代码零改动

第四步：报告与流转（1周）
  - [ ] 每日9点生成风险简报，推送到协作群
  - [ ] 疑似异常自动建单，指定安全同事确认
  验收：高风险操作从发生到被人工确认 < 24小时

The whole project went live in three weeks with less than two thousand lines of core code. Most of the effort actually went into prompt polishing and internal process agreements.

Why a Unified AI Gateway Reduces Maintenance Costs

This project gave me a very concrete appreciation of the value of a "unified AI API gateway." Three points:

First, models become replaceable. For tasks like summarization, model iteration is fast—the model with the best cost-performance today may not be in six months. Once all calls go through the gateway, switching models is just changing one routing configuration, with zero changes to business code. Early on, we connected directly to a vendor's SDK; later they changed their API version, and the pipeline was down for half a day before anyone noticed.

Second, cost becomes observable. Summarizing tens of thousands of logs incurs real token costs. With everything going through the gateway, every task's call volume, token consumption, and error rate show up in unified reports. I can view costs specifically for the "log summarization" project, and I no longer have to manually patch together data for the budget report at the start of each month.

Third, keys and permissions get centralized management. Log data must be desensitized before leaving the network. Desensitization logic, outbound whitelisting, and key management all converge at the gateway layer—security only needs to review one egress point instead of reviewing the calling code every team member writes. For a small team, this matters far more than saving a few lines of code—it turns "the security of AI integration" from everyone's individual discipline into a single piece of inspectable infrastructure.

A Manager's Closing Thoughts

Three months after launch, the biggest change isn't how many person-days we saved, but that log review became part of the process for the first time: every morning at nine, the briefing appears automatically in the group chat; the security colleague glances at the confirmation queue, and the operations lead scans the tenant report. The average delay in risk discovery went from "after a customer complaint" to "the next morning."

For indie developers and small teams, my advice: start with a scenario where "the data already exists, it's just nobody consumes it"—operation log summarization is a classic example. Here, AI isn't making decisions for people; it's driving the cost of processing raw materials down to nearly zero, so people can focus their attention on judgment.

If you're also planning a similar AI side-channel application, a unified AI API gateway is worth setting up on day one—the downstream costs of model selection, cost accounting, and security audits will all be much lower. You can start here: https://api.thistoken.ai/register

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.