# AI response quality evaluators

Ask Powerfront to configure AI checks that score test responses against your chosen quality criteria.

*Release 3.3 · admin, developer, supervisor*

## What it is

An evaluator uses an AI model to judge another AI response. Its saved instructions define what to check and how to score it. You can use the results to compare test responses, with an explanation alongside the score.

## Who uses it and why

- Administrators and developers agree the checks and reference answers with Powerfront.
- Supervisors review scores and explanations to find responses that need attention.

## Where to find it

Powerfront manages **AI Evaluators** for your website. Ask Powerfront to create or change an evaluator and include it in an experiment.

## How it works

1. Define the quality question, such as whether an answer is correct or relevant.
2. Powerfront selects an **AI Account** and writes the **Evaluation Prompt**. The prompt can include the original input, generated output, expected output and context.
3. An experiment runs the selected, enabled evaluators after a test produces an output successfully.
4. Review the score, explanation and any evaluation error with the test result.

Example templates are **Correctness**, **Helpfulness**, **Relevance** and **Toxicity**. Read the prompt to understand the direction of the score. A higher toxicity score, for example, need not mean a better answer.

## Configure it

Ask Powerfront to set the **Name**, **Score Name**, **AI Account** and **Evaluation Prompt**. New evaluators have **Enabled** switched on and use **Numeric (0-1)**. **Boolean (0 or 1)** is also available in the editor.

Provide a few examples with agreed outcomes so Powerfront can check the evaluator before using its scores for comparisons. Changing the judging prompt changes what the score means.

### Settings

| Setting | Where | Applies to | Default | Who can change it | What it does |
|---|---|---|---|---|---|
| Score Type | AI Evaluators | website | Numeric (0-1) |  | Chooses numeric scoring from 0 to 1 or boolean scoring as 0 or 1. |
| AI Account | AI Evaluators | website | Not set |  | Selects the account that judges responses; an account must be selected before saving. |
| Evaluation Prompt | AI Evaluators | website | Not set |  | Defines the judging criteria and requested answer format; a prompt must be supplied or chosen through a template before saving. |
| Enabled | AI Evaluators | website | On |  | Makes the evaluator eligible for selected experiment runs. |

### Fields

| Field | Where | Required | What it is for |
|---|---|---|---|
| Template | AI Evaluators | No | Starts an evaluator with an example quality prompt. |
| Name | AI Evaluators | Yes | Identifies the evaluator. |
| Description | AI Evaluators | No | Explains what the evaluator measures. |
| Score Name | AI Evaluators | Yes | Names the measure recorded with the test result. |

### Permissions

| Permission | Lets you |
|---|---|
| Powerfront | Manage and test AI evaluators for your website. |

## Limits and good to know

- Scores are AI judgements and should be reviewed with their explanations.
- Parsed scores are restricted to 0 to 1. Scores above 1 up to 10 are divided by 10; scores above 10 up to 100 are divided by 100.
- If the response contains no recognised score, INSIDE records 0.5 for numeric scoring or 0 for boolean scoring. The evaluation can still be marked successful.
- Disabled or missing evaluators are skipped when an experiment requests them.
- Selecting an evaluator does not automatically apply it to every live conversation.

## Depends on and works with

### This feature uses

- **Requires** [AI accounts and shared defaults](AIP-001.md): A configured AI account runs the evaluator.
- **Related** [AI Experiments Lab for response testing](AI-032.md): Experiments can use selected evaluators to score successful test outputs.

### Used by

- **Enhances** [AI Experiments Lab for response testing](AI-032.md): Selected AI evaluators can judge successful test outputs.

## FAQ and troubleshooting

### Why do many results have a score of 0.5?

This can be the fallback when INSIDE cannot recognise a numeric score in the evaluator's reply. Ask Powerfront to inspect the explanation and tighten the requested answer format.

### Why is there no score for a test item?

The test may have failed, the evaluator may be disabled or unavailable, or the judging account may have returned an error. Ask Powerfront to check the item result.

### Can I change the evaluator myself?

Powerfront manages evaluator configuration in this release. Send Powerfront the criteria, reference answers and examples you want it to check.

### Does a successful evaluation prove the answer is correct?

No. It means the judging request completed. Review the explanation and remember that an unrecognised score can produce a fallback value.