AI conversation quality scoring
Score completed conversations against configured criteria and let managers review the evidence and audit results.
What it is
Conversation Scoring assesses completed conversations against your scorecards. Each scorecard groups quality checks into weighted categories. Operators and managers can inspect the scores and supporting conversation evidence.
Who uses it and why
- Users with Settings and permission to manage AI, or Developer, define the quality criteria and which conversations they cover.
- Supervisors with Manager review evidence, re-score conversations and record manual audits.
- Operators review their own results and submit feedback when they disagree.
Where to find it
Configure a Conversation Scoring feature at Settings › AI › Features.
Open a conversation for review and expand AI Score to see categories and criteria. Manager actions include Score now, Re-score and Audit. Broader analysis is available in the Operator QA dashboard.
With the Settings or Manager role, configure operator roles and Allow AI Scoring at Settings › Account › Operators.
How it works
- After a conversation closes or is resolved, INSIDE checks its eligibility. The default minimum is four combined visitor and operator messages.
- It selects the first scorecard that matches the interaction type. A scorecard needs categories to be usable.
- AI assesses individual human participants separately. AI contributions are grouped into one AI participant.
- INSIDE calculates category scores from criterion weights, then calculates the overall weighted score.
- Expand a criterion to inspect the explanation and evidence. Selecting evidence can jump to the relevant message.
Managers can Re-score using an eligible configuration, or use Audit to enter a manual assessment. A criterion override keeps the AI score alongside the manual value and recalculates totals. Operators can use Disagree with this score? or the overall-score feedback control without changing the score.
Configure it
Use Settings with permission to manage AI, or Developer.
- Create a Conversation Scoring feature and select its Account.
- Set Min Message Count.
- Under Scorecards, use + Add scorecard, name it and select Interaction types this scorecard applies to.
- Add categories and criteria, or use Insert default criterion as a starting point.
- Set each criterion's question, rating mode, impact, weight and Applies to. Equal weights distributes weights evenly.
- Save and enable the feature.
The initial default scorecard covers human and mixed AI/human interactions. Additional scorecards start with no interaction types selected. New criteria default to Pass / Fail, Positive, enabled and applying to all agents.
Settings
| Setting | Where | Applies to | Default | Who can change it | What it does |
|---|---|---|---|---|---|
| Min Message Count | Settings › AI › Features | website | 4 messages | Settings role, Developer role | Skips automatic scoring when the combined visitor and operator message count is below this minimum. |
| Add scoring thoughts as system message | Settings › AI › Features | website | Off | Settings role, Developer role | Adds the scoring explanation as an internal system note on the conversation. |
| Interaction types this scorecard applies to | Settings › AI › Features | website | Human and mixed AI/human for the initial scorecard; none for additional scorecards | Settings role, Developer role | Selects the conversation interaction types eligible for this scorecard. |
| Rating mode | Settings › AI › Features | website | Pass / Fail | Settings role, Developer role | Selects Pass / Fail or Per Occurence Deductions. |
| Impact | Settings › AI › Features | website | Positive | Settings role, Developer role | Sets whether the criterion concerns positive or negative agent behaviour. |
| Weight (%) | Settings › AI › Features | website | 0%; treated as a weight of one when calculating scores | Settings role, Developer role | Sets the relative contribution of a category or criterion to the final score. |
| Deduction | Settings › AI › Features | website | 20 points per occurrence in Per Occurence Deductions mode | Settings role, Developer role | Sets the points removed per occurrence in Per Occurence Deductions mode. |
| Enabled | Settings › AI › Features | website | On | Settings role, Developer role | Includes or excludes an individual criterion from scoring. |
| Applies to | Settings › AI › Features | website | All Agents | Settings role, Developer role | Selects All Agents, First Agent only or Last Agent only for a criterion. |
| Scoring Prompt Override | Settings › AI › Features | website | Not set: uses the default scoring instructions | Settings role, Developer role | Replaces the default scoring instructions when supplied. |
Fields
| Field | Where | Required | What it is for |
|---|---|---|---|
| Scorecard name | Settings › AI › Features | No | Identifies the scorecard in configuration and review. |
| Category name | Settings › AI › Features | No | Names a group of scoring criteria. |
| Category description | Settings › AI › Features | No | Explains the category to the scoring AI. |
| Criterion name | Settings › AI › Features | No | Names an individual quality check. |
| Question | Settings › AI › Features | No | Describes what the AI should judge for this criterion. |
| Positive examples, one per line | Settings › AI › Features | No | Gives examples to guide the criterion assessment. |
Permissions
| Permission | Lets you |
|---|---|
| Chat | Access scoring views with Allow AI Scoring. |
| Allow AI Scoring | View your own participant scores; managers with this permission can view all participant scores. |
| Manager | Re-score conversations, create manual audits and override criterion scores. |
| Settings | Configure scorecards when AI is allowed under Settings Permissions. |
| AI | Allows AI configuration within Settings Permissions. |
| Developer | Configure Conversation Scoring features. |
Limits and good to know
- Pass / Fail converts an AI value of at least 50 to 100; lower values become zero.
- Per Occurence Deductions starts at 100 and subtracts the deduction per occurrence, with a minimum of zero. A Positive criterion with no occurrences scores zero.
- Zero or negative weights count as a weight of one. Disable a criterion to exclude it.
- Category and overall totals round to two decimal places.
- Manual criterion scores must be between 0 and 100.
- First Agent only and Last Agent only let opening and closing checks target the appropriate participant.
- Viewing needs Chat and Allow AI Scoring. Operators see their own participant results; managers with those permissions can see all results.
- Re-scoring and auditing require Manager. The Settings role alone does not grant those actions.
Depends on and works with
View all connections as a list
This feature uses
- Admin settings and feature switches · managed by PowerfrontLicence
Powerfront must enable AI before automatic conversation scoring can run.
- Roles and permissionsConfigured by
Chat, Allow AI Scoring and Manager control viewing and review actions.
- AI feature settings and change historyConfigured by
A matching enabled Conversation Scoring feature supplies the scorecards and account.
- AI instructions and templatesConfigured by
Scoring instructions can use the configured prompt and template handling.
- AI accounts and shared defaultsRequires
Automatic scoring uses an AI account.
- Conversation history and searchRelated
Conversation History lets reviewers open the messages used as scoring evidence.
- AI conversation scoring reportsRelated
AI scoring reports aggregate the recorded conversation scores.
- Operator QA and coaching dashboardRelated
Operator QA combines quality scores, review feedback and coaching detail.
Used by
- Stats Bar and My StatsEnhances
Configured conversation scoring provides the optional personal quality-score measures.
- Academy coaching and message feedbackEnhances
Configured AI scorecards make operator score feedback available to managers.
- AI conversation scoring reportsRequires
Conversation quality scoring supplies scores and manager overrides.
- Operator QA and coaching dashboardRequires
Conversation scoring, manager reviews and operator feedback supply the dashboard results.
FAQ and troubleshooting
Why does AI Score say Not scored yet?
The conversation may still be open, have too few messages, lack a matching scorecard or have no actual exchange. Read the reason shown in AI Score.
Why cannot I see another operator's score?
Ordinary operators see their own participant results. Wider viewing requires Manager, Chat and Allow AI Scoring.
Can an operator correct a score?
An operator can submit disagreement and an optional comment. A manager must make a scoring override or manual audit.
Why does a zero-weight criterion still affect the total?
This release treats non-positive weights as one. Turn off that criterion's Enabled control to exclude it.
Why did one scorecard take precedence?
The first scorecard matching the conversation's interaction type is selected. Check the scorecards' coverage.