Gains:
- Ability to generate position-specific, competency-based and behavioral interview questions
- Ability to create a fair and comparable evaluation guide (rubric)
- Recognize AI's limitations in interview assessment and retain human judgment
The interview is the most decisive but most subjective stage of recruitment. Two different interviewers may evaluate the same candidate very differently; because everyone asks different questions, attaches importance to different things, and often bases their decisions on "feelings". Research shows that unstructured interviews are poor at predicting hiring success; however, it shows that structured interviews are one of the strongest predictors. Artificial intelligence (AI) is the most practical tool to reduce this subjectivity: position-specific questions produce a common assessment guide and a consistent structure. In this unit, we will learn how to build a fair, comparable and defensible interview process with AI.
Terms: A structured interview is an interview in which all candidates are asked the same competency-based questions and the answers are scored on a common scale. Rubric (evaluation guide) is a table that defines how many points each question will receive for which answer. The behavioral question asks what the candidate has actually done in the past (“Tell me about a time you resolved a conflict”) — because past behavior is the best indicator of future behavior. The situational question asks what you would do in a hypothetical scenario. STAR is a framework used to structure the candidate's answer: Situation, Task, Action, Result.
Step by Step Interview Design
- Identify competencies. Write down 4-6 critical competencies of the position (e.g. problem solving, teamwork, technical depth).
- Generate questions for each competency. Ask behavioral and situational questions from the AI.
- Create a rubric. For each question, define what a "weak/sufficient/strong" answer looks like.
- Prepare follow-up questions. If the candidate answers superficially, go deeper.
- Conduct a discrimination audit. Have the list of questions reviewed legally and ethically.
- Collect the scores and interpret them with a human. AI adjusts the score; The team decides.
Your role: HR specialist designing structured interview.Position: Senior Software DeveloperCritical competencies: problem solving, code quality, team communication,learning agility.For each competency:- 2 behavioral questions (based on past experience)- 1 situational question (hypothetical scenario)- 2 follow-up questions for each question Questions should not be discriminatory; Do not ask about personal life/religion/politics/child plan. Title the output according to competencies.
Tip: Explicitly instruct the AI to “ask about personal life, marital status, child plans, religion, politics, health.” Such questions are both discriminatory and risky in terms of Labor Law and equal treatment. AI can exceed this limit even with good intentions.
Rubric: Tool to Reduce Subjectivity
Without a rubric, scores vary depending on the interviewer's mood. The rubric gets everyone on the same page by predefining “what a strong answer looks like.”
Write a rubric on a scale of 1-5 for the following interview question. Question: "What did you do when a technical decision turned out to be wrong on a project?" Define the following levels in the rubric: 1 (weak): typical answer and missing signals3 (sufficient): typical answer and positive signals5 (strong): typical answer and strong signals Give examples of observable behavior for each level (such as taking responsibility, root cause analysis, learning).
When you apply the rubrics to all questions, the scores given by different interviewers converge; this is called inter-rater consistency. Collecting the scores in a common template makes the decision based on evidence rather than a single "I liked it" sentence.
Weak Prompt / Strong Prompt
Weak prompt:Give interview questions for software developer.
Conclusion: general questions you can find on the internet; No rubric, no competency matching, no guarantee of fairness.
Strong prompt: [Position + critical competencies + behavioral/situational question request + discriminatory question ban + 1-5 rubrics for each question + follow-up questions]
The result: a scoreable, defensible interview kit that is applicable to all candidates.
Answer Evaluation with AI: Limits
AI can help organize your interview notes by rubric; But a strong warning is needed here: AI cannot see the candidate's tone of voice, sincerity, real competence; It only reads the note you write. So AI's role is to structure your observations — not to make decisions.
Organize my interview notes below under competency headings according to the rubric I gave. For each competency: - summarize the evidence in the notes - show which level it corresponds to in the rubric - mark where evidence is missing/unclear Finalize the score for me; just edit the evidence.<notes>[your interview notes]</notes>
Appropriate role of AI
Inappropriate role of AI
Creating questions and rubrics
Scoring and eliminating the candidate alone
Organizing the interviewer's notes according to the rubric
Decision by video/audio "emotion analysis"
Marking inconsistency in notes
Judgments like "This candidate is lying"
Warning about a question/note containing bias
Finalizing the hiring decision
Caution: AI-based “candidate evaluation through facial expression/tone of voice analysis” tools are scientifically questionable and pose serious risk of discrimination. Do not give decision authority to such tools; Many countries and frameworks (e.g. the EU Artificial Intelligence Law) consider such systems used in recruitment to be high risk.
Three Mini Cases
Case 1 — Consistency gain. A retail chain was evaluating 8 store manager candidates with 3 different interviewers and the scores were inconsistent. A common rubric was created with AI. In the second round, the difference in scores between interviewers (to the same candidate) decreased from an average of 1.8 points to 0.6 points; decisions became more defensible.
Case 2 — Catching the discriminatory question. In the list of questions he prepared, a manager asked: "You have a small child, can you stay overtime?" He put the question. When the AI was told to "check questions for discrimination" this question was flagged and asked "This role requires occasional overtime; what is your suitability for this?" It has been replaced by a neutral version that anyone can ask. Thus, justice was maintained and the risk of a possible complaint was prevented.
Case 3 — Configuring notes. One interview panel had scattered notes. The AI organized the notes by rubric headings and showed which competencies were missing evidence. The panel held a second brief discussion on the missing competency and made a more solid decision. The AI did not decide; edited the information that fed the decision.
Common mistakes
- Interview without rubrics. Producing questions and not preparing a scoring guide preserves subjectivity.
- Asking different questions to candidates. Comparability is only possible through common questions.
- Let AI decide. AI does not have enough data to make a final judgment about the candidate.
- Not moderating discriminatory questions. Even well-intentioned interviewers can write legitimately risky questions; AI control is the safety net.
- Relying on “emotion/face analysis” tools. The scientific basis is weak and the risk of discrimination is high.
- Writing the note long after the meeting. Take notes while the observations are fresh; AI only works as well as you write it.
In summary
Fair interview; It means common questions, a clear rubric, and preservation of human judgment. AI is an excellent assistant in producing the first two of this trio: position-specific competency-based questions, defensible scoring guides, and discrimination monitoring. But the actual evaluation of the candidate and the hiring decision — belongs to human observation and responsibility.
Application task
Choose a position. (1) Identify 4 critical competencies. (2) Generate behavioral/situational questions and follow-up questions with the prompt above. (3) Create 1-5 rubrics for the 3 most critical questions. (4) Pass the list of questions through the "discrimination check" prompt. (5) Have AI edit a sample interview note according to the rubric and give the final score yourself.
checklist
- [ ] Are the questions mapped to critical competencies?
- [ ] Is it standardized so that all candidates are asked the same questions?
- [ ] Are there 1-5 rubrics for each critical question?
- [ ] Have the questions been checked for discrimination?
- [ ] Is the final evaluation and decision left to the human?
- [ ] Have risky tools such as “emotion/face analysis” been avoided?