Check whether they design the system around business risk
Ask how the team separates model behavior from deterministic business rules. Permissions, money movement, data access and high-impact actions should not depend on prompt wording.
A mature team will explain trust boundaries, policy layers, approval points and fallback behavior before discussing model brands.
Ask for their evaluation strategy
Production AI needs regression tests for model outputs, retrieval quality, tool calls and policy violations. Ask how representative tasks are collected and how releases are gated.
If the answer is only “we test prompts manually,” the system will be hard to improve safely.
Inspect the data architecture
For RAG or internal knowledge systems, ask about ingestion, metadata, permissions, freshness and deletion. For agentic systems, ask how tool data is normalized and audited.
Data boundaries usually determine reliability more than the final model choice.
Understand ownership and portability
Your product should not become hostage to one model provider or one agency-specific platform. Ask which components you own, how provider abstraction works and what happens if the vendor relationship ends.
Clear repositories, infrastructure ownership, documentation and deployment access reduce long-term switching risk.
Evaluate delivery, not just expertise claims
Ask how architecture, QA, security, observability and release gates are handled. A production product needs coordinated delivery across these layers.
The best partner should be able to explain how a feature moves from discovery to monitored production, including rollback and incident handling.
What a strong vendor proposal should contain
A serious proposal should translate the business goal into architecture boundaries, delivery phases, measurable risks and acceptance criteria. It should describe who owns repositories and infrastructure, how security and QA are handled, and how the team will prove that the AI behavior works on representative tasks.
Ask the vendor to separate discovery assumptions from committed scope. This reduces the chance of buying a large implementation before the highest-risk data, integration or model assumptions have been tested.