# AI conversation quality scoring

Score completed conversations against configured criteria and let managers review the evidence and audit results.

*Release 3.3 · admin, operator, supervisor*

**Settings › AI › Features** · **Settings › Account › Operators**

## What it is

**Conversation Scoring** assesses completed conversations against your scorecards. Each scorecard groups quality checks into weighted categories. Operators and managers can inspect the scores and supporting conversation evidence.

## Who uses it and why

- Users with **Settings** and permission to manage **AI**, or **Developer**, define the quality criteria and which conversations they cover.
- Supervisors with **Manager** review evidence, re-score conversations and record manual audits.
- Operators review their own results and submit feedback when they disagree.

## Where to find it

Configure a **Conversation Scoring** feature at **Settings › AI › Features**.

Open a conversation for review and expand **AI Score** to see categories and criteria. Manager actions include **Score now**, **Re-score** and **Audit**. Broader analysis is available in the **Operator QA** dashboard.

With the **Settings** or **Manager** role, configure operator roles and **Allow AI Scoring** at **Settings › Account › Operators**.

## How it works

1. After a conversation closes or is resolved, INSIDE checks its eligibility. The default minimum is four combined visitor and operator messages.
2. It selects the first scorecard that matches the interaction type. A scorecard needs categories to be usable.
3. AI assesses individual human participants separately. AI contributions are grouped into one AI participant.
4. INSIDE calculates category scores from criterion weights, then calculates the overall weighted score.
5. Expand a criterion to inspect the explanation and evidence. Selecting evidence can jump to the relevant message.

Managers can **Re-score** using an eligible configuration, or use **Audit** to enter a manual assessment. A criterion override keeps the AI score alongside the manual value and recalculates totals. Operators can use **Disagree with this score?** or the overall-score feedback control without changing the score.

## Configure it

Use **Settings** with permission to manage **AI**, or **Developer**.

1. Create a **Conversation Scoring** feature and select its **Account**.
2. Set **Min Message Count**.
3. Under **Scorecards**, use **+ Add scorecard**, name it and select **Interaction types this scorecard applies to**.
4. Add categories and criteria, or use **Insert default criterion** as a starting point.
5. Set each criterion's question, rating mode, impact, weight and **Applies to**. **Equal weights** distributes weights evenly.
6. Save and enable the feature.

The initial default scorecard covers human and mixed AI/human interactions. Additional scorecards start with no interaction types selected. New criteria default to **Pass / Fail**, **Positive**, enabled and applying to all agents.

### Settings

| Setting | Where | Applies to | Default | Who can change it | What it does |
|---|---|---|---|---|---|
| Min Message Count | Settings › AI › Features | website | 4 messages | Settings role, Developer role | Skips automatic scoring when the combined visitor and operator message count is below this minimum. |
| Add scoring thoughts as system message | Settings › AI › Features | website | Off | Settings role, Developer role | Adds the scoring explanation as an internal system note on the conversation. |
| Interaction types this scorecard applies to | Settings › AI › Features | website | Human and mixed AI/human for the initial scorecard; none for additional scorecards | Settings role, Developer role | Selects the conversation interaction types eligible for this scorecard. |
| Rating mode | Settings › AI › Features | website | Pass / Fail | Settings role, Developer role | Selects Pass / Fail or Per Occurence Deductions. |
| Impact | Settings › AI › Features | website | Positive | Settings role, Developer role | Sets whether the criterion concerns positive or negative agent behaviour. |
| Weight (%) | Settings › AI › Features | website | 0%; treated as a weight of one when calculating scores | Settings role, Developer role | Sets the relative contribution of a category or criterion to the final score. |
| Deduction | Settings › AI › Features | website | 20 points per occurrence in Per Occurence Deductions mode | Settings role, Developer role | Sets the points removed per occurrence in Per Occurence Deductions mode. |
| Enabled | Settings › AI › Features | website | On | Settings role, Developer role | Includes or excludes an individual criterion from scoring. |
| Applies to | Settings › AI › Features | website | All Agents | Settings role, Developer role | Selects All Agents, First Agent only or Last Agent only for a criterion. |
| Scoring Prompt Override | Settings › AI › Features | website | Not set: uses the default scoring instructions | Settings role, Developer role | Replaces the default scoring instructions when supplied. |

### Fields

| Field | Where | Required | What it is for |
|---|---|---|---|
| Scorecard name | Settings › AI › Features | No | Identifies the scorecard in configuration and review. |
| Category name | Settings › AI › Features | No | Names a group of scoring criteria. |
| Category description | Settings › AI › Features | No | Explains the category to the scoring AI. |
| Criterion name | Settings › AI › Features | No | Names an individual quality check. |
| Question | Settings › AI › Features | No | Describes what the AI should judge for this criterion. |
| Positive examples, one per line | Settings › AI › Features | No | Gives examples to guide the criterion assessment. |

### Permissions

| Permission | Lets you |
|---|---|
| Chat | Access scoring views with Allow AI Scoring. |
| Allow AI Scoring | View your own participant scores; managers with this permission can view all participant scores. |
| Manager | Re-score conversations, create manual audits and override criterion scores. |
| Settings | Configure scorecards when AI is allowed under Settings Permissions. |
| AI | Allows AI configuration within Settings Permissions. |
| Developer | Configure Conversation Scoring features. |

## Limits and good to know

- **Pass / Fail** converts an AI value of at least 50 to 100; lower values become zero.
- **Per Occurence Deductions** starts at 100 and subtracts the deduction per occurrence, with a minimum of zero. A **Positive** criterion with no occurrences scores zero.
- Zero or negative weights count as a weight of one. Disable a criterion to exclude it.
- Category and overall totals round to two decimal places.
- Manual criterion scores must be between 0 and 100.
- **First Agent only** and **Last Agent only** let opening and closing checks target the appropriate participant.
- Viewing needs **Chat** and **Allow AI Scoring**. Operators see their own participant results; managers with those permissions can see all results.
- Re-scoring and auditing require **Manager**. The **Settings** role alone does not grant those actions.

## Depends on and works with

### This feature uses

- **Requires** [AI accounts and shared defaults](AIP-001.md): Automatic scoring uses an AI account.
- **Configured in** [Roles and permissions](ADM-008.md): Chat, Allow AI Scoring and Manager control viewing and review actions.
- **Configured in** [AI feature settings and change history](AI-001.md): A matching enabled Conversation Scoring feature supplies the scorecards and account.
- **Configured in** [AI instructions and templates](AI-002.md): Scoring instructions can use the configured prompt and template handling.
- **Related** [Conversation history and search](CHAT-024.md): Conversation History lets reviewers open the messages used as scoring evidence.
- **Related** [AI conversation scoring reports](REP-025.md): AI scoring reports aggregate the recorded conversation scores.
- **Related** [Operator QA and coaching dashboard](REP-031.md): Operator QA combines quality scores, review feedback and coaching detail.
- **Needs licence or module** Admin settings and feature switches (managed by Powerfront): Powerfront must enable AI before automatic conversation scoring can run.

### Used by

- **Required by** [AI conversation scoring reports](REP-025.md): Conversation quality scoring supplies scores and manager overrides.
- **Required by** [Operator QA and coaching dashboard](REP-031.md): Conversation scoring, manager reviews and operator feedback supply the dashboard results.
- **Enhances** [Stats Bar and My Stats](CON-024.md): Configured conversation scoring provides the optional personal quality-score measures.
- **Enhances** [Academy coaching and message feedback](CON-030.md): Configured AI scorecards make operator score feedback available to managers.

## FAQ and troubleshooting

### Why does AI Score say Not scored yet?

The conversation may still be open, have too few messages, lack a matching scorecard or have no actual exchange. Read the reason shown in **AI Score**.

### Why cannot I see another operator's score?

Ordinary operators see their own participant results. Wider viewing requires **Manager**, **Chat** and **Allow AI Scoring**.

### Can an operator correct a score?

An operator can submit disagreement and an optional comment. A manager must make a scoring override or manual audit.

### Why does a zero-weight criterion still affect the total?

This release treats non-positive weights as one. Turn off that criterion's **Enabled** control to exclude it.

### Why did one scorecard take precedence?

The first scorecard matching the conversation's interaction type is selected. Check the scorecards' coverage.