October 3, 2026

AI Interview Discussion Guide: Template and Follow-Up Rules

Design an ai interview discussion guide that actually works. Templates, follow-up rules, bias safeguards, and tips to integrate with Uxia.

AI Interview Discussion Guide: Template and Follow-Up Rules

A participant opens a fintech onboarding prototype and mentions that the billing email felt confusing. The AI moderator asks what was confusing, then asks whether billing emails are generally trustworthy, then asks about support, account recovery, and a previous product. Several minutes later, the participant has offered a handful of generic complaints, but the team still doesn't know whether the onboarding flow explains its value clearly.

That failure isn't caused by AI being curious. It happens because the moderator has no definition of a sufficient answer, no limit on probe depth, and no instruction for closing an exhausted thread. An effective AI interview discussion guide must control the conversation around evidence, not merely generate plausible questions.

AI-led interviewing has become common enough to affect candidate behavior and research operations. In a 2026 survey of 2,950 active job seekers across the U.S., U.K., Ireland, Germany, and Australia, 63% said they had been interviewed by AI, an increase of 13 percentage points in six months. The same survey found that 38% had abandoned a hiring process because it included an AI interview, while another 12% said they would do so (Greenhouse's survey findings). The implication for UX teams is practical: discussion guides need to protect both research quality and participant trust.

When an AI Moderator Stops Listening

The session described above felt conversational. That was the problem. The moderator acknowledged each answer politely and produced reasonable follow-ups, but it never recognized that the billing-email thread had stopped producing useful evidence. The conversation sounded active while the research objective disappeared.

Human moderators make similar mistakes, but experienced researchers usually notice the signals. The participant repeats a point, starts speaking in generalities, or answers a question that has already been resolved. A person can decide to say, “That's helpful. I'd like to return to the onboarding step.” An AI moderator needs that decision encoded in advance.

The three controls that prevent drift

A dependable guide defines probe depth, stop conditions, and an answer-received rule.

  • Probe depth limits how far the moderator can pursue one answer. A useful default is a small bank of follow-ups per primary question, used only when the answer is vague, incomplete, contradictory, or unusually revealing.

  • Stop conditions tell the moderator when to move on. These can include a clear answer, repetition, silence, “I don't know,” a direct refusal, or two unsuccessful probes.

  • Answer received describes the evidence the team needs, not whether the participant has said something interesting. For example, “The participant can describe the last time they completed onboarding and identify the step that caused hesitation.”

This distinction matters because AI systems often treat lexical richness as relevance. A long answer about billing can pull the moderator away from a short but decisive answer about first-use value. Without an explicit evidence target, the system follows the most available thread rather than the most important one.

The discussion guide as a constraint envelope

A strong guide contains more than questions. It specifies the research decision, the participant context, the intended sequence, acceptable probes, branching logic, scoring criteria, and exit rules. It also gives the AI permission to stop. That permission is essential because a moderator instructed to “dig deeper” without boundaries can create agreement loops, repeat itself, or turn a complaint into a suggestion.

Research on AI-led interviews supports treating the method as a designed system rather than a novelty. A University of Chicago Booth study reported that candidates interviewed by a voice-AI agent received 12% more job offers, 18% more job starters, and 16% higher 30-day retention than candidates processed through traditional methods (Chicago Booth's research summary). Related experiments found that finalists selected using AI-interview information passed final human interviews at rates approximately 17.5 to 20 percentage points higher than finalists screened from resumes alone. Those findings don't remove the need for discipline. They make disciplined design more valuable.

Core principle: A discussion guide isn't a list of questions. It's a constraint envelope that keeps the AI disciplined when curiosity runs ahead of evidence.

Designing the Guide Before You Write a Single Prompt

Start with the decision the team needs to make. “Understand onboarding” is a topic. “Decide whether the first value explanation should appear before the paywall” is a decision. The second version gives the moderator something concrete to investigate and gives the researcher a basis for judging whether the session produced usable evidence.

Move from decision to competency map

Write down what a participant must demonstrate for the team to make that decision. Keep the map focused. A useful competency map might include three to five capabilities, such as:

  1. Recall the product's pricing or access conditions.

  2. Describe the first moment that feels valuable.

  3. Explain the emotional response to a dismissal or paywall.

  4. Describe whether they would retry the flow.

  5. Identify a workaround they would use instead.

These aren't survey variables yet. They're evidence targets. The AI should learn what happened, what the participant understood, and what they did, rather than merely collect favorable or unfavorable language.

For each competency, write a primary question, a fallback question, and a success signal. If the participant can't answer the primary wording, the fallback should preserve the same objective while using more concrete language.

Competency

Primary question

Fallback question

Success signal

Pricing recall

“What did you understand about the price at this point?”

“What, if anything, do you remember being asked to pay?”

The participant states what they understood without guessing.

First value moment

“When did this product first seem useful?”

“Walk me through the last moment that felt helpful.”

The participant names a specific screen or action.

Retry behavior

“What would you do after dismissing this?”

“What did you do the last time a product interrupted you like this?”

The participant describes a likely next action or past behavior.

Keep each prompt answerable

A primary question should ask for one thing. “What did you understand about the pricing, how did it make you feel, and would you continue?” is efficient for the researcher but difficult for both participant and AI. The participant may answer the first part, while the system incorrectly marks the whole question complete.

The U.S. Office of Personnel Management's structured interview sequence supports starting with job analysis, identifying competencies, selecting question types, building rating scales, pilot-testing questions, and preparing the interviewer's guide. For UX research, the equivalent is a competency map that keeps generated prompts tied to measurable objectives instead of generic conversation.

A diagram outlining a process for creating an AI interview discussion guide before writing prompts.

Why Structured Beats Unstructured Every Time

Open-ended conversation can reveal unexpected detail, but openness without an evidence target creates inconsistent sessions. One participant may receive three probes about a payment concern, while another receives none because the AI notices a more descriptive answer elsewhere. The resulting transcripts are difficult to compare, and the researcher can't tell whether differences reflect participants or moderation paths.

A major meta-analysis reports validity of approximately r = .51 for structured interviews, compared with r = .38 for unstructured interviews, roughly a 34% improvement in predictive accuracy (structured and unstructured interview evidence). The practical lesson is straightforward: define the capability, ask a fixed question tied to it, apply the same rubric, and use anchored ratings rather than free-form impressions.

What structure gives an AI moderator

Structured design maps cleanly to prompt construction. Each question can carry its own:

  • Evidence target, the answer the researcher needs.

  • Probe bank, the permitted follow-ups.

  • Scoring rubric, with observable anchors.

  • Exit condition, which tells the AI to advance.

Consider a first-use confusion question. An unstructured note might say, “The participant seemed confused.” A structured rubric turns that impression into comparable evidence.

Criterion

Structured

Unstructured

Question

“What did you expect to happen after tapping Continue?”

“Tell me about your experience.”

Scoring

1: no expectation; 3: expectation partly matches the flow; 5: expectation clearly matches the next step

Researcher decides what “confused” means after the session

Probe

“What on the screen led you to that expectation?”

AI follows whichever phrase sounds most detailed

Exit rule

Move on after the expectation and screen cue are clear

Continue until the conversation feels complete

Comparison

Same target and scale for each participant

Different paths and inconsistent evidence

Structure doesn't mean the moderator must sound robotic. It means freedom to probe is bounded by a clear evidence target. For teams building a semi-structured approach, an example such as this semi structured interview sample guide can help distinguish fixed primary questions from conditional follow-ups.

Writing the AI Moderator's Operating Rules

The system prompt is where moderation either stays disciplined or spirals. I write it as four blocks, each answering a different operational question: who the moderator is, how turns work, which probes are allowed, and how much time each topic receives.

Define role and scope

Start with a narrow identity:

You are a neutral UX researcher running a concept test on [product]. You aren't a therapist, sales representative, product marketer, or support agent. Your job is to collect evidence about [research decision].

This prevents the model from reassuring the participant, defending the design, or proposing solutions. A useful guide to AI safety controls for business provides broader context for treating guardrails as operational controls rather than decorative language.

Set the turn-taking contract

Use explicit rules:

  • Ask one question per turn.

  • Wait for the participant's full response.

  • Never combine two questions.

  • Keep each probe to 15 words or fewer.

  • Don't praise an answer in a way that implies approval.

  • If the participant goes off-topic, acknowledge it in one sentence and return to the last unanswered question.

  • Don't summarize an interpretation as fact. Ask whether the interpretation is accurate.

These constraints protect against double-barreled prompts and sycophancy. “That makes sense, so you'd prefer a simpler flow?” leads the participant toward agreement. “What would you prefer?” preserves uncertainty.

Add probes and hard exits

List allowed probes verbatim. For each primary question, attach clarification, elaboration, contrast, and concrete-example probes. Use each only when the answer quality requires it, and stop after the permitted attempts.

A reusable rule block might look like this:

  • Ask the primary question first.

  • If the answer is vague, ask one clarification probe.

  • If it remains vague, ask one concrete-example probe.

  • If the participant repeats the answer, says “I don't know,” declines, or becomes silent, acknowledge and advance.

  • If a workaround appears, explore it briefly, then close the thread explicitly.

  • Never introduce a new topic while probing the current one.

The guide to AI-moderated interviews on Uxia is useful context for connecting branching questions with AI-led research workflows.

Paste-ready checklist

Before launching, verify that the prompt includes:

  • Role and research boundary.

  • One-question turn rule.

  • Neutral, non-leading tone.

  • Question-level probe banks.

  • Maximum probe use.

  • Silence and “I don't know” exits.

  • Topic length budgets.

  • Off-topic recovery language.

  • Transcript and escalation instructions.

An infographic titled Writing the AI Moderator's Operating Rules outlining roles, conversation structure, tone, and error handling guidelines.

Sample Questions and Follow-Up Rules That Actually Work

The following example uses a fintech onboarding study. The sequence moves from context to observed behavior, then pain, emotion, workaround, and reaction to a concept. The primary questions stay fixed. Probes activate only when the answer is vague, incomplete, contradictory, or especially useful.

Question

Intent

Primary probe

Contrast probe

Fallback

“Tell me about the last time you opened a financial app.”

Context

“What were you trying to do?”

“Compared to how you do it today, what was different?”

“Walk me through the last time you tried this.”

“What did you do first after opening it?”

Behavior

“What happened next?”

“Compared to your usual process, what changed?”

“What was the first action you remember?”

“Which part felt hardest?”

Pain

“What made it difficult?”

“Compared to other apps, was this easier or harder?”

“Where did you hesitate?”

“How did that moment make you feel?”

Emotion

“What caused that reaction?”

“Compared to how you expected to feel, was it different?”

“What were you thinking at the time?”

“Did you use another way to complete the task?”

Workaround

“Can you walk me through that?”

“Compared to the app, what worked better?”

“What did you do when the app didn't work?”

“What do you expect from this proposed flow?”

Reaction to concept

“What would you do first?”

“Compared to how you do it today, what would change?”

“If this appeared the next time you opened the app, what would you tap?”

Branch only when the evidence calls for it

If the participant mentions a workaround, the AI should explore it for up to two turns, then close the thread: “That helps me understand the alternative. I'll return to the app flow now.” The limit matters less than the explicit closure. Participants shouldn't have to infer why the moderator has suddenly changed topics.

Good probes clarify vague statements, expand short answers, explore contrasts, and use silence or an echo when the participant seems ready to continue. A practical topic-guide resource on follow-up prompts recommends preparing two to three probes under each primary question and using specific prompts such as “Can you walk me through that?”, “What happened next?”, and “Can you give a specific example?”.

Repair common AI failures

Double-barreled question:

  • Weak: “Did the verification step feel safe and easy?”

  • Better: “What did you think about the verification step?”

  • Follow-up: “What made it feel safe or unsafe?”

  • Separate behavior check: “What did you do next?”

Hypothetical future-state question:

  • Weak: “Would you use this feature every week?”

  • Better: “What would you do if this feature appeared during your next task?”

  • Fallback: “Think about the last similar task. What did you do then?”

For a fuller foundation on moderation mechanics, see this guide on how to conduct user interviews.

A worked exchange might look like this:

Participant: “I usually just use a spreadsheet because the app is confusing.”

Moderator rule: Ask one clarification probe, not “What is confusing and why do you prefer spreadsheets?”

AI: “Which step caused the confusion?”

Participant: “The account connection.”

Moderator rule: The evidence target is now specific. Ask for the last observed action, then close.

AI: “What did you expect to happen when you connected the account?”

Participant: “I expected a confirmation, but nothing happened.”

AI: “Thanks. I'll move to what you did next. What did you try after that?”

The edits are small. They prevent a generic complaint from becoming a long, unscored conversation.

Bias, Consent, and What Can Go Quietly Wrong

A calm interface can make AI moderation look neutral. It isn't. The system may lead through adjectives inherited from training data, agree with the participant's last statement, infer demographic traits from writing style, or analyze sensitive words without the participant understanding that a model is involved.

The most damaging errors are often subtle because the transcript remains fluent. A moderator can reframe “I couldn't tell what would happen” as “You wanted clearer guidance,” turning observed confusion into a feature request. The corrective instruction is direct: “Record complaints as complaints. Don't convert negative evidence into a proposed solution.”

A small failure catalogue

  • Leading prompt: “Was the reassuring confirmation helpful?” Fix it with: “What did you think of the confirmation?”

  • Sycophancy: “That sounds frustrating, so the design needs more explanation.” Fix it with: “What, specifically, caused the difficulty?”

  • Demographic assumption: “As someone familiar with investing...” Fix it with: “What experience, if any, did you have with investing?”

  • Consent gap: The participant assumes a person is reading live. Fix it by naming the AI moderator and explaining storage before the session.

The safeguards should be operational. Disclose AI moderation and retention before participation, obtain opt-in for transcript storage, provide a human escalation path, maintain a banned-token list for leading adjectives, and conduct a post-session audit of at least 20% of transcripts for leading behavior. That audit threshold is a governance choice, not a guarantee, but it creates an accountable review practice.

An infographic titled Bias, Consent, and What Can Go Quietly Wrong, listing four risks in AI-driven discussions.

Consent script: “This session is moderated by an AI system. Your responses may be transcribed and analyzed for this research. We won't store the transcript unless you opt in. You can skip any question, stop at any time, or request human assistance.”

Feeding Interview Insights Back Into Uxia

An interview transcript shouldn't end as a summary document. Treat it as a source for sharper test briefs. Extract the participant's theme, the observed pain point, the underlying job to be done, the condition that triggered the issue, and the evidence that would confirm or challenge it.

Suppose three interviews identify confusion at an account-connection step. Don't write “users dislike onboarding.” Create a structured hypothesis: participants may not understand what will happen after connection, so the next test should vary the explanation, the confirmation state, and the order of information.

Convert the finding into a test brief

A useful brief contains:

  • Persona variables: relevant experience, confidence, goals, and context.

  • Task scenario: the exact situation that leads into the flow.

  • Design variant: the copy, screen order, or interaction being tested.

  • Success criteria: completion, correct expectation, reported clarity, and recovery after hesitation.

  • Negative evidence: what would disconfirm the interview pattern.

Uxia can run synthetic testers through uploaded images or video prototypes, using defined missions and audiences. It captures interaction transcripts, flags friction in areas such as usability, navigation, copy, trust, and accessibility, and summarizes patterns into reports with metrics, heatmaps, and prioritized insights. Use the same rubric from the interview guide so the synthetic test measures the same construct rather than creating a second, incompatible definition.

A process flow diagram showing how interview transcripts are converted into structured input for Uxia.

Compare, challenge, and revise

Synthetic results should challenge interview findings, not replace them. Compare the simulated behavior and explanations with the original quotes. If synthetic testers complete the step but can't explain what will happen next, completion alone isn't evidence that the design is clear. If they fail in a way the interviews never revealed, add that negative case to the next discussion guide.

Keep a compact research log:

  1. Transcript: preserve the relevant quote and surrounding context.

  2. Theme: name the pattern without overstating prevalence.

  3. Hypothesis: state what might cause it.

  4. Test brief: define the scenario, variant, and rubric.

  5. Challenge result: record supporting and contradictory evidence.

  6. Guide revision: add, remove, or reword a probe based on what you learned.

This is the practical bridge described in research documentation from insight to action. The interview discovers and clarifies the issue. Uxia tests the refined interpretation at speed. The next interview then probes the uncertainty that remains, rather than repeating the original script.

A good cadence is iterative: interview, code, test, compare, revise. Don't turn synthetic findings into population claims without separate quantitative validation, and don't treat a fluent AI transcript as proof that the participant's meaning was captured. The feedback loop works when each method keeps its role and the researcher makes the handoff explicit.

Uxia helps teams run AI-moderated research and synthetic UX tests with structured missions, follow-up conversations, transcripts, and prioritized usability findings. Use it to turn a disciplined AI interview discussion guide into a repeatable loop from participant evidence to prototype testing, then visit Uxia to start building that workflow.

The future of UX research starts here

Validate every decision with AI and human testers before you ship.

Book a demoGet started

The future of UX research starts here

Validate every decision with AI and human testers before you ship.

Book a demoGet started

The future of UX research starts here

Validate every decision with AI and human testers before you ship.

Book a demoGet started