science investigationambient air monitoringaqs reportingdata validationfrm fem methods
Complete evaluation framework
What to assess and how to score it
Review the evidence signals before interviewing. Then use the anchored descriptions—not instinct alone—to choose the score that best matches each answer.
01
Evaluation factor
Method and rigour
35% weight
Check command of 40 CFR Part 58 QA requirements: zero/span/precision checks, flow audits on BAM-1020 or T-API analyzers, and EPA QA Handbook Volume II procedures.
Evidence to listen for
Follows and can justify an established methodology
Understands contamination, bias, and chain of custody as they apply to the field
Knows the limits of their techniques and says so
Documents procedure so results are reproducible and defensible
Five-point scoring guide
1
Poor
Careless method; unaware of contamination, bias, or procedural integrity.
2
Needs Improvement
Knows procedures but applies them inconsistently; gaps in documentation.
3
Satisfactory
Sound standard practice; less certain outside familiar techniques.
4
Very Good
Rigorous and well documented; understands the limits of each method.
5
Excellent
Names specific instruments, audit tolerances and calibration intervals, and cites QAPP sections governing their own monitoring site operations.
02
Evaluation factor
Real casework
25% weight
Probe hands-on network experience: number of sites and parameters (PM2.5, ozone, NO2, SO2) run, siting evaluations, telemetry outages, and annual data completeness achieved.
Evidence to listen for
Brings specific cases, sites, or projects rather than general description
States their own role and what they personally handled
Can describe an ambiguous or degraded case and how they proceeded
Knows what happened to the work afterwards
Five-point scoring guide
1
Poor
No hands-on casework; experience is entirely academic.
2
Needs Improvement
Limited exposure; cannot describe their own contribution clearly.
3
Satisfactory
Real casework with adequate detail; ownership sometimes vague.
4
Very Good
Specific cases with clear personal scope and outcomes.
5
Excellent
Describes a real monitoring network by site count and pollutant, with completeness percentages and specific outages they resolved.
03
Evaluation factor
Interpretation and judgement
25% weight
Assess how they validate suspect data: distinguishing wildfire smoke or construction impacts from instrument drift, flagging exceptional events, and applying null codes before AQS submission.
Evidence to listen for
Separates what the evidence shows from what they infer
States confidence levels and what would change their conclusion
Comfortable saying the result is inconclusive
Handles pressure to reach a preferred conclusion without bending
Five-point scoring guide
1
Poor
Overstates findings; no separation of evidence from inference.
2
Needs Improvement
Reaches conclusions the evidence does not support; uneasy with uncertainty.
3
Satisfactory
Reasonable judgement; qualifies findings when prompted.
4
Very Good
Clearly separates evidence from inference and states confidence unprompted.
5
Excellent
Walks through a concrete flagged dataset, explains the diagnostic reasoning, and defends why data were invalidated or retained.
04
Evaluation factor
Reporting and testimony
15% weight
Look for written outputs: annual network plans, data certification packages, AQI public notifications, and how they explained a NAAQS exceedance to officials or residents.
Evidence to listen for
Writes findings that a non-specialist can act on
Has presented or defended work to an external audience: court, client, review board, publication
Withstands challenge without overclaiming or retreating
Keeps records that hold up to scrutiny
Five-point scoring guide
1
Poor
Cannot communicate findings; records would not withstand review.
2
Needs Improvement
Reporting is unclear or incomplete; avoids external scrutiny.
3
Satisfactory
Adequate reports; limited experience defending work externally.
4
Very Good
Clear reporting and real experience presenting to an external audience.
5
Excellent
Produced certification memos or network reviews accepted by the regional EPA office and briefed non-technical audiences without overstating findings.
Put this rubric to work
Score every candidate against the same standard
Add these weighted factors to Hirevire and let AI evaluate recorded answers against your rubric.
