QA and Test Engineer Interview Scorecard

A 4-factor weighted scorecard for evaluating QA and test engineers on manual and automation testing experience, automation framework ownership, API testing depth, and team culture fit. Built for product teams and agencies hiring QA engineers across SaaS, mobile, and enterprise software.

When to use this scorecard

Use this AI scorecard when you need to separate QA engineers who have built automation from those who have only used it, and surface API testing depth before a technical screen.

Use this for any QA or test engineering role where the candidate is expected to do both manual and automation testing — not a purely manual role. It is most useful when you need to distinguish engineers who use automation frameworks from those who have built or maintained them, which is the critical bar for mid-to-senior QA hires.

Pair this scorecard with a short take-home — a sample test plan for a feature or a brief API testing exercise. The scorecard evaluates thinking and communication; the exercise validates execution. Use video responses for the manual/automation experience and framework ownership questions where depth of answer reveals real versus surface-level familiarity.

The full scorecard

The scorecard has four weighted factors: Manual & Automation Testing Experience (35%), Automation Framework Ownership (30%), API Testing Knowledge (20%), and Team & Culture Fit (15%).

4 factors · 100% weightage · 1–5 scoring rubric

Evaluates the depth and balance of hands-on experience across both manual and automation testing, and honest self-assessment of strengths.

What to look for

Clearly distinguishes between manual and automation testing and knows when each applies
Gives concrete examples of manual testing (exploratory, regression, edge case, usability)
Gives concrete examples of automation testing with specific tools or frameworks
Honest self-assessment — not just claiming to be strong in both without evidence
Three or more years of experience signals, ideally in a SaaS or agile environment

Score	Rating	Description
1	Poor	Cannot meaningfully describe experience in either area; experience appears insufficient for the role.
2	Needs Improvement	Predominantly manual with very limited automation exposure, or vice versa; struggles to give real examples.
3	Satisfactory	Adequate experience in both areas but leans heavily on one; examples are somewhat vague.
4	Very Good	Good experience in both areas; concrete examples; minor gaps in one area compensated by depth in the other.
5	Excellent	Articulates strong, balanced experience in both; gives specific, credible examples; honest about relative strengths; clearly 3+ years in a relevant environment.

Assesses specific tooling knowledge and whether the candidate has gone beyond using frameworks to actually building or maintaining them.

What to look for

Names specific frameworks: Selenium, Cypress, Playwright (the most common production-grade tools)
Distinguishes between using a framework versus building or maintaining one
Describes their personal contribution to framework architecture or upkeep
CI/CD integration of automated tests (GitHub Actions, Azure DevOps, Jenkins)
Bonus: mentions mobile automation (Appium) or stack-specific tooling (C#/.NET, Python)

Score	Rating	Description
1	Poor	No real framework experience; cannot name relevant tools; automation experience is not credible.
2	Needs Improvement	Limited framework experience; only surface-level tool knowledge; cannot describe a meaningful contribution.
3	Satisfactory	Has used established frameworks but hasn't built one; knows the right tools but in an execution-only capacity.
4	Very Good	Has maintained and extended existing frameworks; solid tool knowledge; CI/CD exposure; credible contributor.
5	Excellent	Has built or significantly maintained a framework from the ground up; names relevant tools; describes CI/CD integration with specifics.

Evaluates practical API testing experience, tooling knowledge, and understanding of what good API test coverage looks like.

What to look for

Confirms real, hands-on API testing experience — not just theoretical
Names specific tools: Postman, RestAssured, Insomnia, or similar
Describes their approach — validates responses, status codes, error handling, data integrity
Awareness of automated API test suites beyond manual Postman collections
Understanding of authentication, headers, payload validation, and test data management

Score	Rating	Description
1	Poor	No meaningful API testing experience; cannot describe tools or a testing approach.
2	Needs Improvement	Limited API testing; vague on tools and approach; mostly aware of APIs rather than experienced testing them.
3	Satisfactory	Has done API testing but mostly ad hoc or manual; knows Postman but hasn't built automated suites.
4	Very Good	Good API testing experience; uses the right tools; solid validation approach; minor gaps in automating API tests.
5	Excellent	Strong hands-on API testing; names Postman or equivalent; describes a structured approach including automation; clear understanding of payload and auth validation.

Assesses whether the candidate genuinely fits the team's working environment and collaboration expectations.

What to look for

Clear and direct answer on working preferences without heavy qualification
Genuine comfort with the team's collaboration style (in-office, hybrid, or remote depending on your setup)
Articulates why they value or enjoy the team's preferred working style
No strong signals of mismatch with the role's day-to-day structure

Score	Rating	Description
1	Poor	Expresses a strong preference incompatible with the role's structure; likely a culture mismatch.
2	Needs Improvement	Hesitant; heavily qualifies their answer; raises incompatible working preferences unprompted.
3	Satisfactory	Confirms fit but answer feels neutral or obligatory; no red flags but no strong alignment signal.
4	Very Good	Clear and positive; mentions what they appreciate about the team's working style.
5	Excellent	Enthusiastically confirms; articulates genuine appreciation for the team's culture; aligns naturally and specifically.

Sample interview questions linked to factors

Use these five questions to probe all four factors. Questions two and three are diagnostic — shallow QA candidates cannot sustain a specific answer about framework contribution or API validation approach.

Question	Factors evaluated
Walk me through your testing experience — both manual and automation. Where are you strongest and where have you had to stretch?	Manual & Automation Testing Experience
Tell me about the most complex automation framework you've worked on. What was your specific contribution — did you build it, extend it, or maintain it?	Automation Framework Ownership
Describe your API testing process. Which tools do you use and what specifically do you validate beyond status codes?	API Testing Knowledge
A developer ships a feature with no test coverage. How do you approach building test coverage from scratch — what do you prioritize and why?	Manual & Automation Testing Experience · Automation Framework Ownership
Tell me about a bug you caught that everyone else missed. How did you find it and what did it take to reproduce it?	Manual & Automation Testing Experience · API Testing Knowledge

Customization notes

Adjust significantly for seniority and specialization. Manual-only roles need a different factor set; senior QAs need a test strategy factor; mobile QAs need a platform-specific factor.

Manual QA only (no automation required)
Remove Automation Framework Ownership and redistribute its 30% across Manual & Automation Testing Experience (raise to 50%) and API Testing Knowledge (raise to 30%). Replace the framework factor with an Exploratory Testing & Bug Reporting factor.
Senior QA Engineer or QA Lead
Add a fifth factor for 'Test Strategy & Coverage Planning' at 20% drawn from Manual & Automation Testing and Culture Fit. Senior QAs need to own test strategy decisions, not just execute them.
Mobile QA (iOS/Android)
Add a Mobile Testing Platforms factor at 15% covering Appium, Detox, Espresso, or XCUITest, drawn from API Testing Knowledge. Mobile testing requires device farm management and OS-specific tooling knowledge.
Startups or early QA hires
Raise Manual & Automation Testing Experience to 45% and reduce Culture Fit to 10%. First QA hires need broad coverage ability and the judgment to prioritize — framework ownership matters less when the codebase has no existing test suite.

Why a weighted rubric matters for qa / test engineers

Why framework ownership and manual-automation balance account for 65% of the score, and what the research says about hiring QA engineers who can scale test coverage without slowing delivery.

Manual and automation experience together form the practical foundation of any QA hire — without that, the role cannot be executed regardless of how well other factors score. Framework ownership is weighted at 30% because it distinguishes engineers who can scale a test suite from those who can only add to one, which is the gap that most QA interviews fail to surface.

Lower variance

Adding a scoring rubric to interviews materially reduces interviewer rating variance and bias compared with unstructured judgment calls.

Highhouse, 2008 (Industrial and Organizational Psychology)

#1 priority

Quality of hire is the top hiring priority for talent leaders, and structured interviews are the method most cited for improving it.

LinkedIn Future of Recruiting Report, 2024

Up to 30%

Bad hires cost employers up to 30% of the employee's first-year earnings, which is why structured screening pays back fast.

U.S. Department of Labor (via SHRM)

Frequently asked questions about hiring qa / test engineers

Common questions when using this scorecard to screen QA engineers, from how to calibrate automation expectations to what framework ownership actually means.

What's the most common mistake teams make screening QA engineers?

Accepting 'I have Selenium experience' as a passing answer on automation. Ask instead: 'Did you write the framework from scratch, extend an existing one, or run tests someone else set up?' The distinction maps directly to the gap between a QA engineer who can multiply coverage and one who can maintain it.

Should I include API testing for roles that are purely UI-focused?

Yes, but reduce the weighting to 10–15%. UI-focused QA still benefits from API knowledge for understanding the data flowing behind the UI — it makes UI test failures easier to diagnose and reduces reliance on developers for root cause analysis.

How do I use this scorecard alongside a take-home exercise?

Use the take-home to validate Technical Proficiency and API Testing specifically. Score the video response for Manual & Automation Testing Experience, Framework Ownership, and Culture Fit first, then use the take-home result to confirm or adjust your API testing and automation scores. Don't rescore everything — use the exercise to verify, not replace, the interview scorecard.

Can this scorecard work for SDET (Software Development Engineer in Test) roles?

Yes, with the Automation Framework Ownership weighting raised to 40% and Manual & Automation Testing dropped to 25%. SDETs are primarily engineers who test — framework architecture and CI/CD integration are the primary skills. Add a code review factor if the role includes peer-reviewing developer code.

Related scorecards

For teams building engineering capacity alongside QA, pair this scorecard with the Software Engineer rubric to maintain consistent scoring across technical hires.

Software Engineer Scorecard AI-Augmented SDR Scorecard

Drop this scorecard into Hirevire

Use this rubric and the linked sample questions to score every video answer automatically. Hirevire's AI does the first pass, so you focus on the candidates worth your time.

See how AI Scorecards work