Setting up evaluations for an AI application is hard. You need to know what your users actually ask, what your AI is supposed to know, and what rules it has to follow. Then you need to turn all of that into evaluation criteria that catch real problems without flagging false ones. For most teams, this takes weeks of manual work and a lot of guessing. QuickStart removes the guessing. You hand over your real conversations, your knowledge base, and your policies. Avido reads through everything and builds an evaluation suite that reflects how your AI is actually used and what it’s supposed to do. You stay in control: every output is reviewed and approved by your team before it goes live.Documentation Index
Fetch the complete documentation index at: https://docs.avidoai.com/llms.txt
Use this file to discover all available pages before exploring further.
Why QuickStart exists
Most teams trying to evaluate an AI application run into the same wall. You sit down to write evaluations and quickly realise you don’t really know what tasks your users perform with the application. You think you do, but you’re guessing. You write evaluations based on what the AI is supposed to do, not what users actually ask it. The result is an evaluation suite that misses the real failure modes and over-flags safe responses. Even when you know the tasks, mapping them to your knowledge base is its own problem. Which articles support which questions? Are there contradictions between documents that would confuse the AI? Are there topics users ask about that nobody has written documentation for yet? You can’t answer these questions without reading through everything, and nobody has time for that. QuickStart solves both. It analyses your conversations to find the tasks that actually matter, checks your knowledge base for gaps and contradictions, turns your policies into evaluation criteria, and runs a first baseline so you have a starting point. What used to take weeks of manual work and produce a guess becomes a structured workflow with full traceability.What you get at the end
When QuickStart finishes, your application has:- A deduplicated set of tasks grouped into topics, derived from your real conversations
- Evaluations at three scopes (global, topic, task) tied back to the source material that produced them
- A style guide generated from your conversations and writing guidelines
- A baseline evaluation run with scores you can build from
- Sign-off from every reviewer on your team
- Optionally, a structured knowledge base if you asked QuickStart to build or restructure one
Three tiers of evaluation
Not every rule applies to every conversation. A rule about investment risk disclaimers shouldn’t fire on a question about branch opening hours. QuickStart organises evaluations into three tiers so each one runs in the right context.| Tier | Applies to | Example |
|---|---|---|
| Global | Every conversation | No emojis, no swearing, prompt injection protection |
| Topic | All tasks within a topic | Investment queries require a risk disclaimer |
| Task | A single task | ”What is the price of X?” should return an accurate price |
What you upload
QuickStart works with three categories of input. Bring whatever combination you have. The system adapts: you can run it with just conversations, just a knowledge base, just a website to scrape, or any mix of the above plus policies and style guides on top. The more you provide, the richer the output, but you don’t need a complete set to get started.Conversations
Real conversation data from the process you want to evaluate, for example customer support transcripts or chat logs. The dataset should be large enough that the same kind of question appears multiple times, so deduplication has something to work with. Upload via API, direct upload, or a data dump.Knowledge base
The articles, help-centre content, or internal documentation your AI relies on. Add it via website scraping or file upload. If you don’t have a structured knowledge base yet, QuickStart can build one for you (see below).Policies and guidelines
Policies, SOPs, training materials, brand voice guides: any “how we talk to our customers” documents. If rules aren’t explicitly written down, QuickStart can also infer them from patterns in your conversations. For example, if no emojis appear anywhere in your conversation data, that becomes “don’t use emojis.”Don’t have a knowledge base? QuickStart can build one
Plenty of teams don’t have a clean, structured knowledge base sitting ready to upload. Maybe the documentation lives across a public help site, a few internal wikis, and the heads of your support team. QuickStart handles this directly: you can ask it to build a knowledge base for you as part of the run. There are two ways it works, and you can use either or both. Build from conversations and scraped pages. Point QuickStart at your help-centre URL or any other public documentation, and optionally upload your conversations alongside. The system extracts the underlying facts and structures them into proper knowledge base articles, organised by topic, ready to be reviewed and approved. Restructure your existing knowledge base. If you do have documentation but it’s messy, scattered, or not organised the way you’d want it organised, QuickStart can ingest it and produce a cleaner, structured version. Same review and approval flow as everything else. Either way, the knowledge base QuickStart builds is yours. It lives in Documents, versioned and editable, and you can keep using it after QuickStart finishes.How QuickStart works
QuickStart runs as a sequence of stages that alternate between system processing and human review. You approve at each gate before the next stage begins, so you never review evaluations only to discover the underlying knowledge base had contradictions.The exact stages depend on what you uploaded. If you didn’t provide a knowledge base, the KB stages are skipped (or replaced with KB-building if you asked QuickStart to build one). If you didn’t provide policies, the policy stages are skipped. The flow adapts to your inputs.
Upload all materials
A single screen where you upload everything at once: conversations, knowledge base, and policies. Web scraping runs in the background while you continue.
Classification and routing
Avido classifies each uploaded item and routes it to the right downstream processing: conversations, knowledge documents, or policy documents.
Task and topic extraction
Each distinct question or request in your conversations becomes a task. Multi-question conversations get split into multiple tasks. Tasks and topics are then deduplicated within and across conversations. All examples are scrubbed for personally identifiable information before anyone reviews them.Worked example: 1,000 conversations might produce 1,100 raw tasks (some conversations contain multiple) which deduplicate down to roughly 800 unique tasks.
Task and topic review ✋
Your first review gate. You see:
- Every topic with the conversations that produced it
- Conversation count per task, so you can see which tasks come up most
- Topic volume distribution as a percentage
Knowledge base processing
Three checks run against your documentation in parallel:
- Contradictions and overlaps. Avido finds documents that say conflicting things, and documents that duplicate each other.
- Coverage. For each approved task, Avido checks whether your documentation can actually answer it. Tasks with no supporting article get flagged as gaps.
- Orphans. Documents that aren’t tied to any task get surfaced, so you can see what’s no longer relevant.
Knowledge base review ✋
Your knowledge base audit, surfaced in one place:
- Coverage percentage per topic, so you know which areas have solid backing and which don’t
- Contradictions between documents, with the conflicting passages displayed side by side. You pick which document is authoritative.
- Orphaned documents that aren’t tied to any task
- White spots: topics that come up in conversations but have no KB article
Policy processing
Avido maps the rules in your policy documents to the right scope (global, topic, or task) and produces evaluation definitions for each. A separate process extracts a style guide from your conversations and any explicit writing guidelines.If you uploaded conversations but no policies, Avido still infers a small set of global rules from conversation patterns.
Policy review ✋
Review the generated evaluation definitions, grouped by topic, each tagged with its scope. Review the generated style guide. Reviews can be assigned to specific team members the same way as the KB review.
Generate evaluations
Avido finalises evaluations across all three tiers and links each one back through its rule statement to the original sentence in the source document. This traceability chain is preserved permanently.
Run baseline
Avido runs the new evaluations against your uploaded conversations to produce a first set of scores. This is your starting point for everything that follows.
Validation and sign-off ✋
Sample good and bad results from the baseline. Confirm whether you agree with each verdict. QuickStart cannot close until every assigned reviewer has signed off. Once it does, you get a final report listing the evaluation definitions and the style guide, and your application is ready for ongoing evaluation.
Reviewing as a team
QuickStart is built around the assumption that different parts of the review need different domain experts. The workflow supports this directly. Assign reviews to specific team members. Each topic area can go to a different person. They review on their own schedule. Sign-off is tracked per assignee. QuickStart cannot close until every assigned reviewer has approved their area. Skip-with-warning. Any review step can be skipped, but the system warns you and audit-locks the decision: who skipped what, when, and how long they spent. Double opt-in on approvals. Every approval requires a confirmation step. Once locked, the decision is recorded. Full audit log. Track who reviewed what, how long they spent, how many warnings were skipped, and what was approved. This matters for regulated industries where you need to demonstrate due diligence later. For most teams setting up QuickStart for the first time, we recommend running 3 to 4 review workshops with the relevant stakeholders rather than reviewing in isolation. Live discussion catches things that asynchronous review misses, especially during the task and topic review stage.Traceability
Every evaluation produced by QuickStart carries its full origin story:Evaluation criterion → rule statement → original sentence or paragraph in the source document (with title)This matters because evaluations only work if your team trusts them. When an evaluation flags a response and someone asks “why does this rule exist?”, you can always trace it back to the policy document, the page, and the exact sentence. For the fact statements extracted during KB review, each fact stores the source document, the exact quote at the time of extraction (preserved even if the document is later edited), and whether it was generated by QuickStart or added manually by your team. This is what lets you trust hallucination findings later: when the AI says something the KB doesn’t support, you can see exactly what the KB did say at the moment your team signed off on it.
Scale and timing
A typical QuickStart run takes a few weeks end-to-end, mostly because of stakeholder coordination, not processing time. The processing itself is much faster.| Stage | Typical bank (~300 docs) | Large enterprise (~3,000 docs) |
|---|---|---|
| Knowledge base preprocessing | ~3 minutes | ~10 minutes |
| Verification (3 parallel checks) | ~10 minutes | ~30 minutes |
| Fact extraction | ~5 minutes | ~15 minutes |
| Total processing time | ~15 minutes | ~45 minutes |
Best practices
Upload more than you think you need. Deduplication only works if the same task shows up multiple times in your conversation data. Bias towards uploading too much, not too little. Don’t try to review alone. The first task and topic review especially benefits from a live workshop with people who actually handle the conversations. They will spot category errors that nobody else will. Treat the KB review as a free audit. The contradictions and white spots QuickStart surfaces are real problems in your knowledge base regardless of whether you ever ship an AI application. Fix them. Use your internal language. Rename auto-generated topics to match the labels your team already uses. This makes evaluations and reports immediately readable when you go live. Don’t skip review steps without a reason. They exist because the next stage builds on what you approve. If you have to skip one, leave a note explaining why.Getting started
Upload your materials
Drop in conversations, knowledge base files or URLs, and any policies or style guides you have.
Assign reviewers
Assign each review stage to the right person on your team. Different topics can go to different people.
Walk through each stage
Approve as you go. Avido handles the processing in between; you handle the judgement calls.
Documents
Manage the knowledge base content QuickStart surfaces and reviews
Evaluations
Understand how the evaluations QuickStart generates actually run