operations administrationbatch schedulingconsole monitoringdata center operationsshift handover
Complete evaluation framework
What to assess and how to score it
Review the evidence signals before interviewing. Then use the anchored descriptions—not instinct alone—to choose the score that best matches each answer.
01
Evaluation factor
Execution and reliability
35% weight
Check hands-on running of batch schedules in Control-M, Autosys or z/OS JCL: nightly job volumes, console monitoring, tape and backup cycles, and SLA windows they held.
Evidence to listen for
Describes a workload they owned and how they kept it from slipping
Names the tools and systems they ran day to day
Can talk about volume: tickets, inboxes, orders, uptime
Nothing quietly falls through when they are busy
Five-point scoring guide
1
Poor
Cannot describe their own workload; things slip without them noticing.
2
Needs Improvement
Handles routine volume; drops work under pressure.
3
Satisfactory
Reliable on steady-state work; struggles when volume spikes.
4
Very Good
Consistently reliable at real volume with a system for staying on top.
5
Excellent
Names the scheduler and platform operated, nightly job counts, cutoff times met, and specific restart or rerun procedures followed without escalation.
02
Evaluation factor
Improving the process
25% weight
Probe changes they drove to runbooks, job dependencies or alert thresholds; look for eliminated false alarms, automated manual checks, or shortened overnight batch windows.
Evidence to listen for
Has changed a process rather than only following one
Can name what was slow or error-prone and what they did about it
Documents so the improvement survives them
Knows when a process is worth automating and when it is not
Five-point scoring guide
1
Poor
Follows instructions only; no sense that process can change.
2
Needs Improvement
Notices problems but escalates rather than solving.
3
Satisfactory
Makes small improvements; impact is local and undocumented.
4
Very Good
Has redesigned a real process with measurable effect.
5
Excellent
Cites a concrete fix, such as scripting a repetitive tape mount check or rewriting a runbook, with time or incident reduction quantified.
03
Evaluation factor
Judgement and autonomy
25% weight
Test unsupervised night shift judgement: which ABENDs they restart themselves, when they wake an on-call application owner, and how they decide to hold downstream jobs.
Evidence to listen for
Knows what to decide alone and what to escalate
Handles an exception without freezing or improvising recklessly
Protects confidentiality and access appropriately
Asks a clarifying question rather than guessing on something costly
Five-point scoring guide
1
Poor
Either escalates everything or acts recklessly on their own.
2
Needs Improvement
Needs frequent direction; uneasy with exceptions.
3
Satisfactory
Sound judgement on familiar decisions.
4
Very Good
Clear sense of their own authority; handles exceptions well.
5
Excellent
Draws a clear line between self-resolved failures and genuine escalations, referencing severity criteria, impact on downstream jobs, and documented escalation trees.
04
Evaluation factor
Communication
15% weight
Assess shift handover quality and ticketing habits: log entries, ServiceNow or Remedy updates, and how they brief application teams on overnight failures.
Evidence to listen for
Writes clearly enough that people act without a follow-up
Manages expectations before a deadline slips, not after
Handles a frustrated colleague or customer calmly
Works across time zones or async where the role needs it
Five-point scoring guide
1
Poor
Unclear written communication; goes quiet when things slip.
2
Needs Improvement
Communication needs chasing; raises problems late.
3
Satisfactory
Clear enough day to day; proactive updates are inconsistent.
4
Very Good
Clear, proactive, and calm under pressure.
5
Excellent
Describes structured handover logs and precise incident notes that let the next operator or app owner act without re-investigating.
Put this rubric to work
Score every candidate against the same standard
Add these weighted factors to Hirevire and let AI evaluate recorded answers against your rubric.
