A Real-World Failure Story

I know several indie developers who built a pet medical Q&A mini-app last year. Their first version was very "intuitive": pick the most powerful LLM, throw users' pet questions directly at it, and let it answer. Three days after launch, problems emerged:

Failure #1: Using a cannon to kill a mosquito—costs spiraled out of control. The most common user questions were actually high-frequency simple ones like "how often should a kitten be dewormed" or "my dog ate grapes, is that dangerous?" Every single call went to a flagship model, making the per-request cost absurd. The free quota burned through in three days, while paid conversions didn't follow.

Failure #2: Answers that missed the point. A user uploaded a photo of a skin condition on their pet and asked "what is this?"—a text-only model obviously couldn't handle it. When another user described "my cat is lethargic and hasn't eaten for three days," the model gently offered a list of possibilities without emphasizing the urgency of "see a vet as soon as possible."

Failure #3: Hallucinations crossing the line. The most dangerous incident: for the question "how long can a dog live with distemper-related seizure sequelae," the model gave specific medication dosage advice. The user followed it, and the pet's condition worsened. The developers took the answering feature offline that very night.

All three pitfalls share the same root cause: treating a complex business as a "single API call" problem. Pet medical Q&A is inherently a tiered problem—some questions can be answered from a knowledge base, some require reasoning, and some should never be answered by AI at all.

The Right Approach: Three-Tier Architecture + Double Fallbacks

Tier 1: Rules & Knowledge Base (Intercepts 60%+ of Traffic)

High-frequency, standardized questions don't need generative AI. Deworming schedules, vaccination timelines, common food hazards—these all have standard answers. The approach:

• Maintain a structured knowledge base (a few dozen to a few hundred FAQs is enough to start)
• Route user questions first through vector retrieval + keyword matching
• On a high-confidence hit → return the knowledge base answer directly, at near-zero cost

Tier 2: Lightweight Models (Handling Routine Consultations)

Routine questions that miss the knowledge base go to a cost-effective small model (e.g., 7B-class models or lightweight APIs from various providers), paired with a strict system prompt: only provide educational explanations, no diagnosis, no specific medication dosages.

Tier 3: Flagship Models (Complex Judgment & Multimodal)

Only route to a flagship model when the question involves image recognition (skin lesions, vomit), complex multi-turn reasoning, or when the user is highly emotional and requires carefully worded responses.

Fallback #1: Medical Safety Guardrails

Add a validation step after model output and before returning it to the user:

• Keyword scanning: if the output contains specific dosages, "guaranteed cure," "no need to see a vet," or similar phrasing → intercept and replace with standard safe copy
• Emergency signal detection: keywords like vomiting blood, ingestion of chocolate/lilies, seizures → strongly prompt "please seek veterinary care immediately," with an explanation of urgency
• Fallback copy: if any tier times out or errors, return a preset safe reply + guidance toward human consultation—never fail silently

Fallback #2: Degradation Chain

Flagship model timeout or error → automatically degrade to a lightweight model with more conservative prompts; if the lightweight model also fails → return the closest knowledge base entry. Users always get a "safe answer," even if it's not the smartest one.

Key Implementation Steps

def answer_pet_question(question, image=None, history=None):
    # 第一层：知识库
    hit = kb.retrieve(question, threshold=0.85)
    if hit and not image:
        return hit.answer

    # 安全信号前置检测（优先级最高）
    if detect_emergency(question):
        return EMERGENCY_TEMPLATE  # 强制就医提示

    # 路由决策
    need_premium = image is not None or is_complex(history)

    # 第二/三层调用，带降级
    try:
        raw = call_model("premium" if need_premium else "light",
                         question, image, timeout=8)
    except GatewayError:
        raw = call_model("light", question, None, timeout=8)

    # 输出护栏校验
    if violates_medical_safety(raw):
        return SAFE_FALLBACK
    return attach_disclaimer(raw)

Pre-launch checklist:

1. Knowledge base covers the Top 100 high-frequency questions, each manually reviewed
2. Emergency keyword list (poisoning, bleeding, seizures, etc.) cross-verified by two people
3. Prepare 50 boundary-case questions for regression testing (dosages, euthanasia, zoonotic diseases, etc.)
4. Set independent timeouts and degradation paths for each tier, and run a load test
5. Append a fixed disclaimer to every answer: cannot replace diagnosis by a licensed veterinarian

Why a Unified AI Gateway Reduces Maintenance Costs

In this tiered architecture, you need to integrate at least two or three models (lightweight, flagship, and possibly a different provider for embedding retrieval). If you integrate each provider's SDK separately, maintenance costs grow linearly with the number of models: different authentication methods, different error code formats, different streaming interfaces—and when a provider changes pricing or you swap models, you have to modify business code.

A unified AI API gateway consolidates this complexity:

• One authentication point, multiple model providers. Switching models means changing one parameter, no need to rewrite calling code;
• Unified error codes and timeout semantics. The degradation chain only needs to be written once, without adapting exception handling for each provider's API;
• Unified usage monitoring. Call volumes and costs for each tier are clear at a glance, giving cost governance real data to work with;
• Fast switching when a provider fails. For medical scenarios with fallback requirements, this is especially critical—you can't be scrambling to integrate a new SDK on the night of an outage.

For small teams, this effectively compresses "multi-model operations" into "editing a config," freeing up time to spend on knowledge base quality and guardrail rules—which is the real moat for a pet Q&A product.

Final Thoughts

The success of businesses like pet medical Q&A often depends not on how powerful the model is, but on whether the tiering is sound and the fallbacks are thorough. Figure out "which questions AI shouldn't answer" before chasing intelligence. If you're planning to build your own multi-model architecture, start with a unified gateway: https://api.thistoken.ai/register

---

Tired of juggling provider integrations? Register at https://api.thistoken.ai/register and call every model through one base_url.