A Manager's Real Concerns

Last month, our four-person team decided to add an AI chat feature to our app. Two voices emerged during the discussion:

One was "build a proxy service on the backend, and Flutter just calls our own API." The reasoning: controllable and secure. The other was "use an OpenAI-compatible AI gateway directly from the client, saving us a backend module."

I ultimately chose the latter, but with a few governance rules in place. This article shares that decision-making and implementation process—especially how to implement streaming (SSE) chat on the Flutter side, and the three things you, as a team lead, need to think through in advance.

Why We Chose Direct Connection via AI Gateway

The traditional approach is: Flutter → self-built backend → model API. The problem with this chain is that, for a small team without dedicated ops, building your own proxy means:

• One more service to deploy, monitor, and scale
• Streaming forwarding (SSE passthrough) is actually more error-prone on the backend
• When switching models, both the backend and proxy layer need changes

OpenAI-compatible AI gateways like ThisToken.AI unify the protocol: the Flutter side sends requests in OpenAI format, and switching models only requires changing a single string. This is exactly the solution to the "writing the API three times" problem we discussed in another article, so we won't elaborate here.

Three things a manager needs to think through:

1. Key ownership: Who applies for the API Key, who keeps it, and how it gets rotated. I recommend the team shares a gateway account, with keys managed centrally by one person (for specific pricing, refer to the official pricing page).
2. Usage limits: Set call limits before launch to prevent a client bug from burning through the budget.
3. Fallback plan: What users see when the model is unavailable. Timeout and disconnection handling for streaming interfaces must be written into acceptance criteria.

Preparation: Registration and Getting a Key

1. Open https://api.thistoken.ai/register to register a team account (a company email is recommended for easier member collaboration and billing management later).
2. Go to the console and create an API Key. It's recommended to separate keys by environment: dev-staging, prod-app. When something goes wrong, you can disable one individually without affecting everything else.
3. Note the Base URL: https://api.thistoken.ai/v1. All requests go through this address.

For team collaboration, I recommend putting keys in environment variables or CI Secrets, leaving only placeholders in the code repository. Add this to your Code Review checklist.

Implementing Streaming Chat on the Flutter Side

For handling SSE streaming responses in Flutter, the simplest approach is the http package's StreamedRequest. The following code runs as-is:

import 'dart:async';
import 'dart:convert';
import 'package:http/http.dart' as http;

Future<void> chatStream(String userMessage) async {
  final request = http.Request(
    'POST',
    Uri.parse('https://api.thistoken.ai/v1/chat/completions'),
  );

  // Key从环境变量或安全存储读取，不要硬编码进仓库
  request.headers['Content-Type'] = 'application/json';
  request.headers['Authorization'] =
      'Bearer ${const String.fromEnvironment('THISTOKEN_API_KEY')}';

  request.body = jsonEncode({
    'model': 'gpt-4o-mini', // 换模型只改这一行
    'stream': true,
    'messages': [
      {'role': 'system', 'content': '你是一个简洁的助手'},
      {'role': 'user', 'content': userMessage},
    ],
  });

  final response = await http.Client().send(request);

  final lines = response.stream
      .transform(utf8.decoder)
      .transform(const LineSplitter());

  await for (final line in lines) {
    if (!line.startsWith('data: ')) continue;
    final payload = line.substring(6).trim();
    if (payload == '[DONE]') break;

    final delta = jsonDecode(payload)['choices'][0]['delta']['content'];
    if (delta != null) {
      stdout.write(delta); // 实际项目中用setState更新UI
    }
  }
}

void main() async {
  await chatStream('用一句话介绍流式输出');
}

To run it:

flutter run --dart-define=THISTOKEN_API_KEY=sk-你的密钥

A few details at the team standards level:

• stream: true is the switch for streaming. Forget to include it and the response comes back all at once, leaving users staring at a blank loading spinner.
• Consume with StreamBuilder at the UI layer. In the code above, just replace stdout.write with your text widget update.
• Error handling must be differentiated: network disconnection, invalid key, and model rate limiting should show different messages to users. This is the most commonly missed item during acceptance testing.

Risk Control Checklist

As the person in charge, I put together this checklist to tick off item by item before launch:

Summary

For small teams integrating AI capabilities, the biggest cost is often not the code, but the absence of collaboration rules. An AI gateway unifies the protocol; what remains is settling the three matters of keys, budget, and fallback on day one. With Flutter connecting directly to a streaming interface, a working demo runs in a dozen lines of code—I suggest you register an account right now, run the code above, and fill in the standards as you go.

Registration link: https://api.thistoken.ai/register

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.