Improvement Agent

The Improvement Agent is an AI-powered tool that helps you systematically improve your chatbot's responses based on real customer feedback.

Overview

When a chat is marked as "needs improvement", the Improvement Agent analyzes:

• The original conversation
• Where the chatbot didn't respond optimally
• How the responses can be improved

Accessing the Improvement Agent

There are three ways to start an improvement session:

From the Chat Detail View

1. Go to Chats in the dashboard and open a chat
2. In the "AI Improvement Agent" section, you'll see the feedback status
3. If the chat is marked as "Needs Improvement", click Improve with AI

From the Chats List

Chats with a negative rating that haven't been improved yet show an inline Improve with AI button directly in the Chats list — click it to jump straight into the agent without opening the chat first.

From a Chat Review Email

If you receive a chat review email and click Needs improvement, you're taken directly into an Improvement Agent session for that chat. See Rating chats from email below.

The Improvement Process

1. Analyze the Original Conversation

The left panel shows the original chat conversation:

• Each message is numbered for easy reference
• User messages (purple) and bot responses (gray) are color-coded
• Bot messages have an "Improve" button to select them for improvement

Smart Message Selection:

• If the chat contains only one bot message, it is automatically pre-selected — the agent knows immediately which response you want to improve.
• If there are multiple bot messages, the agent's greeting will guide you to click "Improve" next to the specific message you want to fix. The selected message is highlighted in blue and an indicator bar appears near the chat input.

2. Chat with the Improvement Agent

The center panel is an interactive chat with the AI Improvement Agent:

• Describe what wasn't optimal about the chat
• Ask for improvement suggestions
• Discuss alternative response strategies
• Reference specific messages from the original chat

Example Prompts:

• "The answer to question 3 was too vague. How could the bot respond better?"
• "The bot didn't understand the product inquiry. What can we improve?"
• "Suggest a better fallback response when the bot is uncertain."

3. Review and Apply Suggestions

The right panel shows the generated improvement suggestions:

For each suggestion, you can:

• Simulate: Test how the bot would respond with the improvement (use Simulate All to run the simulation for every pending suggestion at once)
• Apply: Apply the improvement
• Reject: Discard the suggestion

Test Simulation

Before applying a suggestion, you can simulate it:

1. Click Simulate on a suggestion (or Simulate All to test all pending suggestions)
2. The simulation shows:
   • The original user question
   • The simulated response with the change applied
   • The original bot response remains visible in the left panel for comparison
3. Rate the simulated response:
   • Perfect!: The agent confirms the suggestion is ready — apply it with the Apply button
   • Almost good: The agent refines the suggestion based on what's still missing
   • Not good: The agent asks what was wrong with the simulated response so it can rework the suggestion

Completing a Session

Once you've dealt with every suggestion, click Complete Improvement in the suggestions panel:

• The button stays disabled until every suggestion has been applied or rejected
• On completion, you're returned to the Chats list
• The chat is then badged as Improved, and its detail view shows a View Improvement link back to the session

Feedback Workflow

Rating Chats

Before the Improvement Agent can be used, a chat must be rated:

1. Open a chat
2. In the "AI Improvement Agent" section:
   • Good (Thumbs up): Marks as positive example
   • Needs Work (Thumbs down): Marks the chat as needing improvement and recalculates the confidence score — an Improve with AI button then appears to start the agent
   • Reviewed (Check): Marks the chat as reviewed/dismissed without rating it good or bad (does not affect the confidence score and does not trigger emails or improvement sessions)

When you mark a chat as Needs Work, the AI automatically classifies how severe the issue was — this determines how much the rating affects your confidence score (see Confidence score below).

Ratings are not final: chats marked as Good or Reviewed can be reopened later with the Mark as needs improvement button on the chat detail page. The confidence score is recalculated and the Improvement Agent becomes available again.

Rating Chats from Email

You can also rate chats directly from the chat review emails you receive, without opening the dashboard. Each email shows the conversation and asks "Was this a good response?" with two buttons:

• Yes, good answer: Marks the chat as positive and takes you to a success page showing your updated confidence score — including how the score of the affected answer category changed
• Needs improvement: Marks the chat as negative and takes you straight into an Improvement Agent session for that chat

Using the Learning Phase

During the learning phase, you'll be automatically prompted to rate chats — rating requests arrive as chat review emails. This significantly accelerates the improvement of your chatbot.

Confidence Score

The confidence score measures how reliably your chatbot answers, based on your ratings. You'll find it in the Confidence tab of your chatbot's settings.

Per-Category Scoring

Conversations are grouped into answer categories, and each category gets its own confidence score alongside the global score:

• Categories are created automatically as chats happen
• You can add, rename, or delete categories in the Confidence tab (chats from a deleted category are moved to an automatic fallback category)

Severity of Negative Ratings

Each thumbs-down is automatically classified by AI into one of four severity levels that determine how much the rating lowers the confidence score:

• Minor issue
• Moderate issue
• Major issue
• Critical issue

The chat detail page shows the assigned severity, the applied score adjustment, and the reason. If you disagree with the classification, use the Override severity dropdown to change it — the confidence score is recalculated immediately.

Chat Review Emails

In the Confidence tab you control how often you receive chat review emails:

• The Chat Review Emails toggle enables or disables review emails entirely
• For each category, you can set its own email frequency: After every chat, Every X hours, Daily, Weekly, or Never

This lets you keep a close eye on weak categories while muting categories that already perform well.

Reset Session

If you want to start a new improvement session:

1. Click Restart in the header
2. Confirm the reset
3. All messages and pending suggestions are deleted — changes you've already applied are kept

Availability

Best Practices

1. Be specific: The more precisely you describe the problem, the better the suggestions
2. Provide context: Mention what response you would have expected
3. Simulate before applying: Always test suggestions first
4. Use regularly: Weekly improvement sessions keep your bot up to date

Next Steps

• FAQ Management →
• Knowledge Base →
• Chatbot Settings →