Why pre-screen AI ethicists before the interview
This role fails in two directions. Someone with no technical grounding is dismissed by the engineering team within weeks, and someone who never blocks anything becomes a signature on a form. Both look identical on a curriculum vitae. Ethicists worth hiring can name a system that changed because they raised something. A short screen asks for that, and for what they would refuse to sign off.
What actually matters when screening AI Ethicist candidates
- 01
Theoretical command
Probe command of fairness metrics (demographic parity, equalised odds), value alignment literature, and instruments like the EU AI Act, NIST AI RMF, and OECD principles.
- 02
From theory to hardware or code
Ask what they built or shipped: model cards, red-team protocols, bias audits in Fairlearn or AIF360, dataset documentation, impact assessments tied to a real model release.
- 03
Research judgement
Test how they choose which harms to chase when evidence is thin: sampling a harm taxonomy, scoping red-teaming, deciding when a system should not be deployed.
- 04
Explaining it to non-specialists
Judge whether they can move engineers, legal counsel, and executives without jargon: policy memos, ethics review boards, disclosure language for users and regulators.
Pre-screening questions to ask AI Ethicist candidates
12 questions grouped by what they test. Ask the same set in every screen and score answers on a consistent scale, or send them as an async video screen and compare answers side by side.
Risks that led to change
3 questions01Can you describe a time when you identified an ethical risk in a project?
Listen forA specific risk found through examining the system, with the design or deployment change that followed.
Risks described in the abstract, or concerns raised that produced no change to anything.
02Can you give an example of how you would address bias in a system?
Listen forConcrete steps across data, model and threshold, with the trade-off against accuracy acknowledged.
Bias addressed only by adding more diverse data, or the accuracy trade-off not mentioned.
03What steps would you take to reduce the potential misuse of a technology?
Listen forMisuse anticipated at design stage, with access controls and release decisions treated as ethical choices.
Misuse treated as the user's responsibility, or release decisions considered outside their remit.
Frameworks applied
3 questions04What frameworks do you use to evaluate the ethical implications of a system?
Listen forFrameworks applied to a specific system with the conclusions they produced, not cited as a list.
Principles listed with no application, or frameworks that never produced a difficult conclusion.
05How would you evaluate a new application before it is built?
Listen forConsequence to affected people assessed first, with a route to recommend the project not proceed.
Evaluation limited to risk mitigation, or no option to conclude that something should not be built.
06What role does transparency play in your approach?
Listen forTransparency defined by what an affected person needs to know, not by publishing technical documentation.
Transparency described as publishing model cards, or the affected person not considered.
Held under pressure
3 questions07How would you handle business objectives conflicting with ethical guidance?
Listen forA real conflict with a workable alternative offered, and a line they would not cross stated clearly.
Every conflict resolved by accommodation, or no situation where they held a position against pressure.
08How do you advocate for these considerations in a technical team?
Listen forCredibility built through technical engagement, with concerns framed as engineering problems to solve.
Advocacy through escalation, or engineers described as resistant to ethical thinking.
09What would you propose to hold developers accountable for ethical failures?
Listen forAccountability tied to named owners and decision records, so responsibility can be traced afterwards.
Accountability described as culture, or no mechanism for establishing who decided what.
High stakes treated differently
3 questions10How do you approach deployment in high-stakes settings such as healthcare or policing?
Listen forHigher evidential standards demanded, with human review and appeal designed in rather than assumed.
High-stakes uses treated like any other, or oversight described without authority to overturn.
11What is your position on automated decisions in hiring or lending?
Listen forA considered position with the specific harms named, and the legal position understood accurately.
A position stated with no reasoning, or the applicable legal constraints not known.
12Can you describe your experience with regulatory compliance in this area?
Listen forSpecific obligations understood for the sector, with the difference between law and guidance clear.
Regulation described in headlines, or voluntary guidance presented as legal requirement.
How to score responses
Score every candidate on the same four criteria immediately after the screen. At this stage you are shortlisting for panel interviews, not making the final call.
Theoretical command
35%5Names specific fairness definitions, explains where they mathematically conflict, and cites concrete AI Act risk tiers or RMF functions accurately.
From theory to hardware or code
30%5Shows artefacts they authored, such as a completed FRIA, eval suite, or audit that changed a launch decision or model card.
Research judgement
20%5Prioritises harms by severity and reversibility, admits open questions, and describes a case where they recommended halting or restricting deployment.
Explaining it to non-specialists
15%5Translates alignment or bias findings into decisions and trade-offs a product lead grasps, with evidence a recommendation was adopted.
Without technical grounding they are ignored; without a line they are a signature. A one-way video screen tests both.
Try it on HirevireScreening FAQ
Process basics
How long should a pre-screening round for this role take?
Fifteen minutes across eight to ten questions, answered async. Enough to establish risks they caught, test how they apply frameworks in practice, and hear how they handle pressure.
How technical does an ethicist need to be?
Technical enough to read an evaluation and ask a useful question about it. They do not need to build models, but they cannot be dependent on the team's own summary of how a system behaves.
Evaluating answers
What is the strongest signal when screening this role?
A system that changed because of them. Ethicists with influence name it. Anyone whose contribution is principles, frameworks and workshops has raised awareness rather than changed anything.
How do I judge their independence?
Ask what they would refuse to sign off. Real answers name a specific application and the reason. Anyone who can find an accommodation for everything will not stop a bad launch.
Go deeper on this role
Sanat has been hiring since 2012 and watching the recruitment industry change up close ever since, and turned that screening process into Hirevire's video screening platform. LinkedIn
