Reflections on Shipping an AI Trip-Planning Feature: From Unpredictable Output to a Controlled Pipeline

I lead a five-person team building a trip-planning application. The core product is simple: users enter a destination, number of days, budget, and preferences; AI generates an editable itinerary, with booking entry points attached for transportation and tickets. It doesn't sound complicated, but once we actually launched, I found that half the difficulty of itinerary generation lies in technology, and the other half in team collaboration and risk control. This post is a retrospective from a manager's perspective on our approach, and I hope it's useful for indie developers and small teams who want to ship quickly.

1. Business Pain Points: Three Real Headaches with AI Features on the Product Side

Before launch, we sat down with operations and customer support for a requirements review. The pain points around itinerary generation clustered into three areas:

1. Uncontrollable output. The same input might produce an artsy route today and a checklist of tourist hotspots tomorrow. Users complained that "the restaurants recommended in the itinerary have per-capita costs that don't match my budget at all." With no structured constraints on AI output, the downstream booking modules simply couldn't parse it reliably.

2. Unpredictable cost and latency. A single itinerary generation required multiple model calls (destination understanding, itinerary orchestration, attraction verification). Early on, each module called its own API directly, with keys scattered across the configs of three services. At month-end reconciliation, nobody could say which step burned how many tokens.

3. Single-vendor risk. We had tied all of itinerary generation to a single model provider. When that vendor had a regional service disruption, itinerary generation went down entirely, and customer support was flooded with feedback overnight. For a product still in cold start, this kind of incident is nearly fatal.

2. Architecture Design: Unified AI Gateway + Structured Generation Pipeline

After the redesign, the architecture has four layers:

• Access layer: User requirement form + natural language supplements, with an initial intent normalization pass.
• AI gateway layer: All model calls converge into a single unified AI API gateway. Internally, the team maintains only one SDK, one authentication scheme, and one logging standard.
• Orchestration layer: Itinerary generation runs in three steps—constraint extraction, itinerary draft generation, and feasibility verification (opening hours, geographic distance, budget fit). Each step can independently swap models and run its own canary releases.
• Delivery layer: Output is forced into a JSON Schema, with every itinerary item carrying a type tag, consumed by the booking modules and the editor.

The key decision: model selection lives in gateway configuration, not in code. The itinerary draft uses a strong model to guarantee quality, while constraint extraction and format verification use lightweight models to control cost. Switching models is just a config change—no release required.

3. Key Implementation Steps (Process Checklist)

1. Define the itinerary Schema
   - Day/time slot/item (type, name, estimated cost, coordinates, duration)
   - All downstream modules consume via the Schema; rendering free-form text directly is forbidden

2. Connect the unified AI gateway
   - One key for the whole team, tagged by project/module
   - Configure model routing on the gateway side: extraction → lightweight model, orchestration → flagship model
   - Enable call logging and token usage stats by tag

3. Build the generation pipeline (three stages)
   - Constraint extraction: destination/days/budget/preferences/exclusions → structured constraint object
   - Itinerary draft: constraint object + destination knowledge base snippets → initial itinerary JSON
   - Feasibility verification: rule engine + model review (distance/duration/budget overrun detection)

4. Risk control and fallback
   - Primary model timeout of 8s auto-degrades to backup model (gateway routing rule)
   - One automatic retry on verification failure; if it still fails, return a template-based fallback itinerary
   - Rate limiting at peak + per-user daily generation caps

5. Pre-launch review
   - 50 sets of standard inputs as a regression suite; run on every prompt change/model swap
   - Pass criteria: 100% Schema validity rate, ≥ 85% of itinerary items bookable

4. Why a Unified AI Gateway Genuinely Reduces Maintenance Cost

This is the part I spent the most time on at the retrospective meeting. From a manager's perspective, let's do the math on three fronts:

First, the headcount math. Without a gateway, five services each integrated with model providers on their own. Every time a vendor upgraded an interface or changed a parameter format, we had to change code in five places and ship five releases. After converging on the gateway, this kind of adaptation happens once, at the gateway layer. For a small team like ours with no dedicated ops staff, what this saves isn't just a few work-hours—it's the on-call question of "who gets up in the middle of the night to fix it."

Second, the accounting math. Once the gateway tagged token usage by module, we could say in the very first review meeting that "itinerary orchestration accounts for 70% of total cost," and we decided to switch the verification step to a lightweight model—costs dropped noticeably that same month. Before unified accounting, these discussions were all guesswork.

Third, the risk math. With keys centrally managed, there's no longer the hidden risk of "a departed colleague still has production keys on their laptop." The gateway's routing rules turned primary/backup model switching into a configuration operation—if that service disruption had happened under the new architecture, users would barely have noticed. For a manager, the essence of risk control is turning unpredictable incidents into drillable processes, and the unified gateway is precisely what turns model calls from "everyone for themselves" into a controlled system with logging, quotas, and degradation paths.

5. Two Additional Points on Collaboration Mechanisms

Once the technology was sorted out, we also established two team rules. First, prompts and gateway configurations go into version control, with changes subject to review, to avoid disputes like "who changed the prompt and caused output format drift." Second, we review the gateway's cost and quality reports weekly, with three metrics pinned on the dashboard: Schema validity rate, average generation time, and degradation trigger count. A small team doesn't need heavy processes, but it does need a shared ledger everyone agrees on.

Looking back, for AI features like itinerary generation, model capability only determines the floor. What really determines whether it runs stably is the "not-so-AI" engineering: interface governance, structured constraints, and fallback plans. If you're about to lead a small team shipping a similar application, I recommend converging all model calls into a unified AI gateway from day one—the earlier you save, the more you benefit. You can register for a trial at https://api.thistoken.ai/register and get this pipeline up and running.

---

Every example in this post runs with a single API key — get yours at https://api.thistoken.ai/register and start in minutes.