Docs
HeyRik DocsAcademyLesson 8 of 8
Lesson 8 of 8

Improve the agent from real call evidence

Review transcripts, outcomes, captured details, recordings when available, and tool behavior; identify the earliest divergence; change one responsible layer; and regression-test before increasing traffic.

12 minute lesson Outcome: A repeatable improvement and rollback process
In this lessonOutcome

You will turn one imperfect call into a small, evidence-backed correction. You will locate the earliest divergence, assign it to the responsible product layer, make one change, rerun the failed scenario and regression suite, and keep a rollback decision available.

What done looks like

You can demonstrate the lesson result in the dashboard and complete its practical checklist before continuing.

01

Open the complete call evidence

Dashboard → AI Agents → Property Enquiry Assistant → Recent Calls. Use Analytics for patterns only after understanding individual calls.

  1. Open Recent Calls on the agent

    Start with the first controlled live calls or the phone-test call that failed.

  2. Open one call record

    Read call state, direction, duration, outcome, transcript, summary, recording when available, and captured details.

  3. Read from the first turn

    Do not begin with the summary. Find the earliest sentence where actual behavior differs from the expected scenario.

  4. Compare outputs to the transcript

    A plausible output is still wrong if the caller corrected it or never confirmed it.

  5. Inspect tool behavior when relevant

    Use the call’s function/tool record to compare the arguments sent, response received, and what the agent told the caller.

Recording may not be immediately or always available

A recording URL can remain empty while upload finishes, and recording storage must be configured. Use the transcript and structured record without promising that every call will always have a playable recording.

02

Assign the first divergence to one layer

Fixing the wrong layer adds complexity without removing the cause.

Instructions

Wrong question order, duplicate question, weak correction, unsupported promise, or bad closing.

Edit Assistant Details
Knowledge

Wrong, stale, conflicting, missing, or over-broad factual answer.

Replace/repair source
Variables

Unresolved input, stale corrected value, vague output, or inconsistent result shape.

Edit Input/Output
Integration

Wrong arguments, failed lookup, missing result, false success claim, or unauthorized action.

Inspect tool invocation
Voice

Language mismatch, poor pronunciation, unclear clone, or unsuitable delivery.

Choose/record/test voice
Call Settings

Talking over callers, slow response, poor silence retry, failed transfer, or runaway duration.

Tune one call control
Channel

Wrong number assignment, inactive number, wrong direction, carrier-quality, or routing problem.

Verify Phone Numbers
Operations

Unstaffed handoff, obsolete source owner, missing opt-out handling, or no rollback response.

Change process/ownership
Divergence note template
Call ID:
Scenario:
Expected behavior:
First divergent turn (quote or timestamp):
Actual behavior:
Responsible layer:
Evidence for that layer:
Smallest proposed change:
Failed scenario to repeat:
Regression scenarios to repeat:
Rollback condition:
03

Complete improvement example

The caller corrected the budget, but the structured result retained the old amount. The earliest divergence and affected layer are identifiable.

Call call_1dd990f7
needs correction
ScenarioCorrection
Caller turn“Actually, make that ninety lakh, not eighty.”
Agent turn“Okay. Which location do you prefer?”
ExpectedAcknowledge, replace, repeat ₹90 lakh, ask confirmation
Output budget₹80 lakh
First divergenceAgent did not confirm the correction
Responsible layersInstruction recovery rule + output confirmation
Not responsibleVoice, knowledge, phone number, transfer
Before
If the caller corrects an answer, update it and continue.
Smallest correction
If the caller corrects an answer:
1. Acknowledge the correction.
2. Replace the earlier value.
3. Repeat the new value and ask whether it is correct.
4. Continue only after confirmation.
The extracted output must use the last confirmed value.
Why this is not a voice or knowledge fix

The transcript already recognized the correction correctly, and no external fact was needed. Changing the voice or uploading more documents would not repair the missing confirmation rule.

04

Change one cause and run regression tests

Keep the agent’s current traffic risk in mind while editing. A critical failure may require rollback before analysis is complete.

  1. Stop or limit traffic when needed

    For a critical safety, opt-out, privacy, or false-action issue, return the agent to Draft, unassign the number, or pause outbound traffic first.

  2. Open the responsible area

    For this example, edit the Recovery section in Assistant Details and make the Output budget description require the last confirmed amount.

  3. Make only the smallest causal change

    Do not simultaneously replace the voice, rewrite the whole prompt, and change silence settings.

  4. Save and rerun the failed scenario

    Use the same correction wording and verify both the spoken confirmation and output budget.

  5. Run the complete regression suite

    At minimum repeat happy path, ambiguity, missing fact, silence, human request, wrong person, refusal/opt-out, and closing.

  6. Restore or increase traffic gradually

    Only after the correction and regression suite pass under the final saved configuration.

Retest result
pass
Agent confirmation“Thanks for correcting that—₹90 lakh, right?”
Caller“Yes.”
Output budget₹90 lakh
Old value retainedNo
Happy-path regressionPass
Missing-fact regressionPass
Opt-out regressionPass
05

Verify the improvement process

A fixed call is useful; a repeatable operating loop is the real Academy outcome.

Complete lesson 8: Improve from calls
Complete this lessonLoading saved progress · Checking saved progress…
0 of 9
06

Common improvement mistakes

Uncontrolled iteration makes the agent harder to understand and can reintroduce defects that already passed.

Reading only the summary

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Use the full transcript and first divergence; a summary can omit the exact failure.

Changing several layers at once

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Make one causal change so the result is attributable and rollback is simple.

Fixing a fact in the prompt

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Repair the approved knowledge source when the problem is factual.

Fixing behavior with more documents

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Change the responsible instruction when the source was correct but the agent acted incorrectly.

Ignoring structured outputs

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Compare every downstream field to the caller’s last confirmed value.

Retesting only the failure

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Run the full regression suite so a local fix does not damage another branch.

Leaving dangerous traffic active

Why it fails

It leaves behavior, data, or ownership ambiguous.

Correct it

Rollback first when safety, consent, privacy, opt-out, or false-action behavior is involved.

API equivalent

Read the call and its tool invocations

GET /calls/{id} · GET /calls/{id}/functionsOpen reference

Fetch the transcript, recording URL when available, outcome, duration, cost, and call metadata with GET /calls/{id}. Fetch the custom APIs/tools invoked, their arguments, and returned results with GET /calls/{id}/functions. Use GET /analytics and GET /calls/stats for patterns after the underlying records and outcome meanings have been validated.