Bias in the algorithm: How AI hiring tools may be working against diversity

Artificial intelligence has transformed the way many organisations recruit. From resume screening software to automated interview scoring, AI tools promise speed, consistency and objectivity. But there’s a growing body of evidence suggesting that in practice, these tools can do the opposite — reinforcing historical bias rather than eliminating it.

For employers committed to building diverse, inclusive workforces, understanding this risk isn’t optional. It’s essential.

How AI hiring tools work, and where the problem starts

Most AI recruitment tools are trained on historical hiring data. The system learns which candidates were previously hired, performed well, or progressed through the business, and uses those patterns to score or rank new applicants.

The problem is that historical hiring data reflects historical biases. If a company has traditionally hired from a narrow demographic, an AI trained on that data will replicate those patterns at scale.

A well-documented example: Amazon scrapped an internal AI recruiting tool in 2018 after discovering it systematically downgraded resumes from women, having been trained on a decade of predominantly male hires. The story, reported by Reuters, became a cautionary tale for the industry, but the underlying dynamic remains present in many tools in use today. Closer to home, The Australian Human Rights Commission has flagged AI bias in hiring as a growing concern, warning that automated recruitment tools can constitute unlawful discrimination under the Disability Discrimination Act 1992 if they disproportionately screen out candidates with disability or non-linear work histories.

The specific impact on diverse jobseekers

AI screening tools can disadvantage candidates in ways that aren’t immediately visible. This includes jobseekers who:

• Have non-linear employment histories (common among people with disability, carers, or those re-entering the workforce)
• Used different job titles for equivalent roles in different sectors
• Have employment gaps due to health conditions, caregiving, or participation in programs like Workforce Australia
• Come from educational backgrounds that differ from the ‘standard’ pathway
• Use different language or communication styles in written applications

None of these factors necessarily reflect capability. But algorithm-based screening often can’t make that distinction, and candidates are filtered out before a human ever sees their application.

The Australian Human Rights Commission has flagged AI bias in hiring as an area of increasing concern, noting that automated tools can constitute unlawful discrimination under the Disability Discrimination Act 1992 if they disproportionately exclude protected groups.

This isn’t just a legal risk. It’s a talent risk

Businesses using AI screening tools without human oversight may be systematically excluding some of their best potential hires. At a time when skills shortages are acute across many industries, this is a problem employers can’t afford to ignore.

The candidates most likely to be screened out by automated tools are often those with the highest motivation to succeed in a supported role — people who have worked hard to re-enter the workforce, developed resilience across varied circumstances, and who, given the right opportunity, become long-term, loyal employees.

What employers can do

The answer isn’t to abandon technology. It’s to use it more thoughtfully. Some practical steps:

• Audit your screening tools regularly. Ask vendors directly how their algorithm was trained and what bias testing has been done. The Diversity Council Australia provides resources to help organisations assess their recruitment practices.
• Don’t let automation replace human judgement at early stages. Automated shortlisting should inform human decision-making, not replace it.
• Review your job ad language. Identify language that inadvertently narrows your applicant pool.
• Broaden your sourcing channels. Working with specialist employment service providers means you’re accessing candidates through relationships, not just keyword matching.
• Consider structured, skills-based assessment as an alternative to resume screening. What can the candidate do? That’s the question that matters.

The human advantage

There’s something AI can’t replicate: the nuanced understanding that comes from a real conversation with a candidate, or a relationship built between a specialist employment consultant and an employer over time.

Employment service providers working within Workforce Australia and Inclusive Employment Australia do this work every day. They know their candidates, they understand employer needs, and they can make matches that no algorithm would surface.

In a world increasingly driven by automation, that human layer isn’t a weakness in the hiring process. It’s one of its greatest strengths.

At CoAct Connect, our national network of service partners takes a people-first approach to recruitment — connecting employers with motivated, work-ready candidates and providing the support needed to make those placements last. If you’re concerned your current process may be creating unintended barriers, let’s talk.