Back to Help Centre
Evaluation

Setting up an AI interview

A complete guide to configuring AI interviews for jobs and screenings — questions, evaluation, voice, and more.

AI interviews are conversational voice interviews conducted by ScoutUp's AI agent. Candidates receive a link, complete an audio check, and then have a natural conversation where the AI asks your configured questions, probes for detail, and evaluates their responses.

Where AI interviews are used

AI interviews can be set up in two contexts:

  • Jobs — each job has one interview template. Go to the job's Settings page and click on the AI Interviews section
  • Screenings — each screening can have one or more interview requirements. Go to the screening's Settings > Requirements page and add an AI interview requirement

The configuration is identical in both cases. The settings pages described below apply to both jobs and screenings.

Interview questions

Questions are the foundation of the interview. Navigate to the Questions tab to manage them.

Adding questions

Click Add Question and enter the question text. Questions are asked in the order they appear — use the arrow buttons to reorder them.

Question settings

Each question has several options:

  • Knockout — mark a question as a knockout criterion. If the candidate fails a knockout question and the evaluation is configured to fail on knockout, the entire interview fails regardless of the overall score
  • Answer depth — controls how much the AI probes for detail:
    • Low — accepts brief answers and moves on quickly
    • Medium — asks one or two follow-up questions for more detail
    • High — probes thoroughly, asking multiple follow-ups to get comprehensive answers

Initial greeting and closing message

At the top of the questions page you can set:

  • Initial greeting — the first thing the AI says when the interview starts (e.g. "Hi, thanks for joining. I'll be asking you some questions about the role today.")
  • End of interview message — the message played when the interview concludes

Evaluation criteria

Navigate to the Evaluation tab to configure how candidates are scored.

Overall evaluation criteria

Write high-level instructions for how the AI should evaluate candidates. This is a free-text field where you describe what makes a good answer, what you're looking for, and any specific standards to apply. Be as detailed as possible — the AI uses this to guide its scoring.

Sub-criteria

Add individual evaluation criteria for more granular scoring. For example:

  • "Technical Knowledge" — does the candidate demonstrate relevant technical skills?
  • "Communication" — are answers clear, structured, and professional?
  • "Problem Solving" — can the candidate think through scenarios logically?

Each sub-criterion gets its own independent score, giving you a detailed breakdown alongside the overall score.

Voice selection

Navigate to the Voice tab to choose the AI interviewer's voice. ScoutUp offers several voice options with different accents and speaking styles. For each voice you can:

  • See the name, gender, and accent
  • Click the play button to hear a sample
  • Select it as your active voice

Interview settings

The Interview Settings tab has advanced options:

  • Allow candidates to ask questions — when enabled, the AI will answer candidate questions using the knowledge you provide
  • Knowledge for answering questions — if the above is enabled, enter information the AI should use when candidates ask about the role, company, or process (e.g. salary range, start date, team size)
  • Offer final comments — give candidates a chance to add anything at the end of the interview
  • Limit to single interview — prevent candidates from taking the interview more than once
  • Allow retake after completion — let candidates redo the interview even after completing it

Invitation templates

The Invitations tab (available for job interviews) lets you customise the messages sent when inviting candidates to interview.

SMS invitation

Customise the SMS message sent to candidates. Use template variables like {{first_name}} and {{interview_link}} to personalise the message.

Email invitation

Customise the email subject, body, sender name, and reply-to address. The same template variables are available.

Public application confirmations

If you're using the public application form, you can also configure the confirmation SMS and email sent after a candidate submits their application.

Pass criteria (screenings only)

When an AI interview is part of a screening, you can also configure pass criteria on the requirement:

  • Minimum score — the interview score needed to pass (e.g. 60 out of 100)
  • Knockout rule — choose Ignore (score only) or Fail on any (fail if any knockout question is failed, regardless of score)

The candidate experience

Here's what candidates experience when they receive an interview invitation:

  1. They receive an SMS or email with a unique interview link
  2. They click the link and land on the interview page
  3. They complete an audio check to make sure their microphone works
  4. The AI interview begins — a natural voice conversation
  5. The AI asks each question in order, probing for detail based on the answer depth setting
  6. If enabled, the candidate can ask questions and give final comments
  7. The interview ends with the closing message and a thank-you page
  8. The AI evaluates the interview against your criteria and generates a score