Our methodology
How we audit AI hiring tools.
Our audits combine statistical analysis with psychological assessment expertise. The phases below describe how we conduct an audit. The exact areas examined depend on the tool, how it is used, the relevant requirements, and the agreed scope.
The audit process
How the audit is carried out
Discovery & scoping
We map the tool’s purpose and role in hiring, including its features, data sources, data flows, model objectives, and human decision points. We review internal documentation and speak with relevant product, technical, compliance, or HR stakeholders.
Fairness & bias testing
We conduct quantitative testing for potential adverse impact. Depending on the data, this may include adverse impact ratios using the 4/5ths rule, subgroup comparisons, threshold analysis, and false positive or false negative rates, interpreted alongside group sizes and uncertainty.
Validity & integrity
A fairness result does not, on its own, show that a tool measures something relevant to the job. We assess construct, content, and criterion-related validity, job relevance, psychometric soundness, scoring design, and efforts to minimize bias in the assessment.
Transparency & explainability
We evaluate whether outputs can be understood, whether candidates are informed about AI use and consent, whether past decisions can be reviewed, and whether user documentation is adequate. Where appropriate, we may use technical interpretability methods such as SHAP, LIME, or counterfactual explanations.
Risk mapping with Bowtie analysis
We map potential hazards and consequences alongside preventive controls, detective controls, recovery actions, and remaining gaps. The resulting visual framework supports governance, planning corrective actions, and stakeholder assurance.
Evidence base
What we examine
The evidence depends on the audit scope and what is available. We document any gaps and how they affect the conclusions.
- System and technical documentation. The tool’s purpose, features, data sources, data flows, model objectives, version, and configuration.
- Validation and fairness studies. Existing evidence supporting the tool’s claims, job relevance, and performance across groups.
- Scores and hiring outcomes. Rankings, selections, thresholds, error rates, and demographic information where it exists and can lawfully be used.
- Jobs and hiring processes. The roles, candidate populations, decisions, and points of human judgment involved.
- Candidate and governance materials. Disclosures, consent, user guidance, oversight, data handling, and records of how decisions are reviewed.
- Stakeholder input. Perspectives from relevant product, technical, compliance, HR, or hiring teams.
Reviewing the findings
How we read the results
Findings in context
Results relate to the version, configuration, population, and period examined, and are not applied more broadly than the evidence supports.
Clear limitations
Small groups, incomplete records, and limited access are documented along with their implications.
Methods matched to the question
We distinguish required calculations from supplementary testing and explain why each method was used.
Concerns and recommendations
If we identify fairness, bias, validity, or oversight concerns, we explain the evidence and recommend what to address. We do not design or implement changes to the tool we audit.
What you receive
Your final report
The final report distinguishes supported conclusions from evidence gaps, explains any concerns identified, and sets out practical recommendations through a risk and recommendations roadmap. We then walk through the findings, limitations, and next steps with your team. Where applicable, the agreed deliverables may also include a public summary or verifiable seal.