Analytics pages are easy to admire and hard to act on. The line goes up, you feel good, nothing changes. The useful question is narrower: which number, if it moved, would make you do something different?
For an assistant there are about four. This is how to find them, what a healthy reading looks like, and what to change when one of them moves.
Set the frame first
Two controls sit above everything and apply to the whole page.
- Assistant — defaults to All Assistants. If you run more than one, always narrow to a single assistant first. Averaged across a busy support assistant and a quiet one, every number is meaningless.
- Date range — Today, 7d, 30d, 90d, Year, Custom. The page opens on 30d. Custom reveals two date pickers and an Apply button; nothing changes until you click Apply.
Every figure with a change indicator compares your range against the equal-length period immediately before it. Volume metrics show a percentage change; rate metrics show a change in percentage points (pts). If either period had no activity, no indicator is shown at all.
Two weeks is the shortest window that tells you anything about quality. A single bad Tuesday is not a trend.
The five tiles at the top
| Tile | What it measures |
|---|---|
| Conversations | Every interaction in the range |
| Resolution | Share of graded conversations that met the assistant's goal, judged by an LLM |
| Self-served | Share of chats handled without handing off to a human agent |
| Satisfaction | Positive share of the replies visitors rated |
| Credits Spent | Credits used across your assistants |
Three come with a condition attached, and the conditions matter more than the numbers.
Resolution is blank until you set a goal
Resolution reads as a dash and Not measured yet until the assistant has a Goal — a plain-English sentence on the General tab describing what success looks like. Once set, every conversation is graded against it when it ends: achieved or not met, plus a grade from 1 to 5 and a short written reason.
Grading is not retroactive, so the tile fills in going forward. If Resolution is empty, that is the reason — not a data delay and not a bug.
Write a goal a reader could verify from a transcript. "Be helpful" cannot be graded. "The visitor gets a working answer to their billing question, or is handed to a human" can.
Satisfaction only counts what visitors rated
Satisfaction counts only the replies a visitor rated with the thumbs buttons in the widget, so it reads as a dash until somebody rates something. Treat it as a signal with a sample size, not a score. Ten thumbs-downs in a week is worth reading; 80% positive on five votes is worth nothing.
Self-served depends on how you hand off
Self-served counts a chat as escalated when it was handed to a live agent queue. Hand-offs that go out by email or to your own endpoint are not counted here — so on the normal email-and-webhook setup this tile sits near 100% permanently and tells you nothing. Watch how many escalations actually landed in your inbox instead, and whether they were real. Set Up Team Handoff covers that side.
The tab that actually changes your week
Open the Quality tab. Four cards, in the order you should read them.
Resolution — a donut split into Resolved and Not met, with the average grade out of 5 and how many conversations were graded. Without a goal it becomes an activation prompt with a Set a goal button.
Satisfaction — the positive share of rated messages, a split bar, and the like and dislike counts.
Needs review — the one to actually work. A feed of conversations where a visitor disliked a reply, or the assistant fell back on an "I don't know" answer. Each entry carries a Disliked or Fallback badge, the detected language, the visitor's last message and the flagged reply. Click one to open the transcript.
That feed is the fastest route to your next fix: a Fallback badge is nearly always a missing document, a Disliked badge nearly always a missing line in your instructions.
Languages — detected language of conversations, with counts. Worth a glance monthly: it is how you discover a market you did not know you had.
Overview and Usage & Cost
The Overview tab is for volume and shape rather than quality: total messages, average messages per chat, completion rate and active contacts, over a daily Channel Activity line, Peak Hours bucketed in your own time zone, Conversation Status (Completed / Active / Errored) and a per-assistant table.
Avg Messages / Chat is the most diagnostic number here, and it is bidirectional. Very short conversations mean visitors are bouncing before they get value. Very long ones usually mean the assistant is circling — asking again for something it was already told, or explaining without resolving. The worst pattern is long conversations that still end in a hand-off.
The Usage & Cost tab answers "where did the credits go": tokens, credits spent, session duration and success rate, plus per-day and per-assistant breakdowns. Read it whenever Credits Spent moves without conversation volume moving with it — that gap is a configuration change, not a traffic change.
What a healthy mix looks like
There is no universal benchmark, and anyone quoting one has not seen your traffic. What is stable across sites is the shape:
- Most conversations are short and resolved.
- A visible minority are longer and still resolved.
- A small tail hands off to a person, and those hand-offs are worth a person's time.
- Errored conversations are near zero.
- Satisfaction is positive on a sample large enough to argue about.
Deviations from that shape are more informative than any absolute figure. Zero hand-offs means the assistant is either brilliant or never offering.
What to change when a number moves
| The number | Likely cause | What to change |
|---|---|---|
| Resolution drops | Questions the knowledge base does not cover, or a goal that no longer matches reality | Read the Goal not met transcripts; the grader names where it stalled. Add documents, or rewrite the goal. |
| Resolution was never there | No goal set | Write one sentence on the General tab. It grades from then on. |
| Satisfaction drops | Tone, length, or confidently wrong answers | Filter Conversations by Disliked and read the comments — they usually say what the visitor wanted instead. |
| Fallbacks rise in Needs review | A knowledge gap, not a prompt problem | Upload the document that answers the question — see Building Your Knowledge Base. |
| Avg messages / chat rises | The assistant is circling or re-asking | Add an instruction against re-asking for details already given; cap answer length. |
| Errored conversations rise | Configuration or service, not wording | Open the transcripts and look for Failed turn markers. A run of them is systemic. |
| Credits Spent rises without volume | A model, memory or voice setting changed | Check the AI Model section and whether voice was enabled somewhere. |
| Escalations rise | A whole category the assistant cannot answer | Find the category in the transcripts before widening the assistant's scope. |
The discipline that makes this work is changing one thing at a time. Two changes in a week and you learn nothing from the next reading.
Read the transcripts, not just the charts
Analytics tells you where to look. Conversations tells you what to fix — every interaction, searchable by what was actually said. Three filters are worth a habit:
- Feedback → Disliked. The comment attached to a thumbs-down is usually a complete bug report.
- Failed turn markers. Turns the assistant did not answer, which visitors experienced as it being broken.
- Goal not met. Hover the resolution badge to read the grader's reasoning about where it stalled.
Twenty minutes reading transcripts beats two hours staring at a chart. Using Agent Analytics to Improve Your AI Agent sets out a weekly cadence for this loop, and Set Up Your Assistant's Persona and Tone covers where most of the fixes go.
The weekly ten minutes
Once a week, per assistant: set the range to 7d, read the Needs review feed top to bottom, pick the single most common cause, fix that one thing, note the date. Next week, compare.
That is the whole method. Not sophisticated, and better than any dashboard you can build.
Your assistants' numbers are on the Analytics page at hiroi.ai.