If you’ve applied for a job at a large company in the last few years, there’s a good chance that an algorithm sorted your resume before a human being even saw it. Companies that make AI hiring algorithms are out to disrupt recruiting and hiring by making it faster and easier to find the “right” candidate.
But what does that mean for applicants who are a good fit but don’t “check all the boxes”? And what does it mean for women and people of color who are historically underrepresented in many industries?
The “right people,” faster and cheaper
The use of AI or “deep learning” (a subset of AI) in the hiring process refers to employing algorithms to find patterns in employment data (whether they be résumés or new “gamified” assessments). These patterns and identifiers theoretically serve as signals that a person will be successful in a position based on historical data.
By applying these measurements to potential employees, those who deploy the technology hope to determine whether a person is a “good fit” for their company. Humans become a series of data points that are plotted against the parameters set by the company.
There are plenty of examples of biased algorithms that devalue the skills of women and minorities. Still, companies around the world are implementing this technology in the hopes of diversifying their workforce. The stated goal of such employers is to have these algorithms pick out candidates based on skills alone rather than more subjective criteria. But as we’ll see, the desire for more data snowballs, and we end up in pseudoscientific territory as a result.
Garbage in, garbage out
Not only are algorithms designed by humans, but they’re also trained on historical data. And because women and minorities have been underrepresented in many professions for years, algorithms are trained on the status quo. Garbage in, garbage out.
Take, for example, the short-lived hiring algorithm created by Amazon. While it was supposed to be kept on the down-low, five people from within the company revealed to Reuters that the company had been building the hiring algorithm since 2014 based on the resumes of previously successful employees. That sounds good until you realize that most of those employees were men. That means the algorithm learned to value the skills and language used in men’s résumés to predict who would be a good hire.
“Everyone wanted this holy grail,” one of the employees said. “They literally wanted it to be an engine where I’m going to give you 100 résumés, it will spit out the top five, and we’ll hire those.”
Unsurprisingly, that didn’t leave them with a diverse pool of candidates.
One of the many problems facing the use of AI in hiring for diversity is that computer science itself isn’t a terribly diverse field. In fact, in 2019, the AI Now Institute reported on the state of computer science, which exhibited what The Guardian called “a disastrous lack of diversity.”
The vast majority of AI workers – the ones training these algorithms – are men, with estimates ranging from 78-80% of the total workforce. AI Now reports only 15% of AI researchers at Facebook, and 10% of AI researchers at Google are women. And with 71% of the applicant pool made up of men, we’re a long way from seeing changes any time soon unless companies make proactive efforts to hire women.
For example, Facebook announced an ambitious plan in June to make its workforce at least 50% “women, people who are Black, Hispanic, Native American, Pacific Islanders, people with two or more ethnicities, people with disabilities, and veterans” within the next five years, and to “double [their] number of women globally and Black and Hispanic employees in the US.” Currently, its US workforce (which is the only one it reports these statistics for) is 44.2% white, 43% Asian, 5.2% Hispanic, and 3.8% Black (the rest are listed as multi-ethnic or “other”).
How companies use AI to “disrupt hiring”
While many people are questioning the legitimacy of AI in the hiring process, dozens of companies are selling this software and promising a diverse candidate pool and staff as a result. These companies claim to be able to make job postings appeal to a wider variety of candidates, objectively vet résumés, build a diverse list of candidates to interview, and rediscover candidates who applied to the company in the past but whose résumés were initially overlooked.
When it comes to letting AI assess job postings, companies like Textio promise to eliminate language that discourages women and minorities from applying to a job. By performing what’s called a “sentiment analysis,” they claim they can make a job ad appealing to all genders and ethnicities. Their algorithm is trained on hundreds of millions of previous job listings.
In terms of narrowing down candidates from a large pool, AI promises to do what humans cannot. Companies claim that their algorithms can identify and strip résumés of extraneous information that doesn’t relate directly to the hiring criteria. Some think this naturally leads to more diversity because the algorithms are looking for specific skill sets required for the job.
Two business psychologists – Tomas Chamorro-Premuzic and Reece Akhtar – have suggested in the Harvard Business Review that using AI in assessing interviews might even be a good thing. Traditional interviews can be unstructured and follow different lines of inquiry based on candidates’ answers. The thinking is that if interviews were conducted digitally and the transcript assessed later by algorithms, AI could use only the relevant information and ignore physical appearance, facial expression, and body language. Regardless of how scientific this sounds, there’s very little evidence to back this digital pseudoscience.
Once again, there’s very little research showing that “talent analytics,” which attempt to decode non-verbal behavior or assess the results of surveys and games, are effective and non-biased. The best we can say is that they may be less biased than humans. But that’s not a reason to market them as “objective” – and the quantification of human beings risks putting us at the mercy of algorithms we know nothing about.
Perhaps one of the most promising uses of AI in hiring for diversity is its use in uncovering bias in past hiring decisions by mining the data of candidates who have previously been passed over for jobs. After all, machines are built to do high volume, repetitive tasks. While there are likely to be problems with the identification criteria, algorithms can still gather data such as phrases or words that are common in rejected résumés and cross-check that language with gender and ethnic language data. Having humans assess that might give us some more insight into how we have unfairly judged female and minority candidates in the past.
Gaming the system
To add to the complexity of using AI in hiring, companies are increasingly using tests and games, thinking it will dovetail nicely with the use of AI as one more metric for their algorithms. Take, for example, a company such as Unilever, which has touted its success in hiring more diverse candidates after having all applicants play neuroscience games.
Canadian researchers have found that these tests can be reliable in predicting job performance when used in hiring selection decisions. However, such tests “invariably lead to lower job selection rates for minority groups because they score lower on average than majority group members.” That’s not because minority candidates lack the knowledge or aptitude for the job, but because these tests aren’t merit-based. They’re designed to test for the status quo, which is usually white and middle class. These tests may predict job performance, but is that because continuing to hire white men is easier than instituting diversity and inclusion initiatives that make workplaces more inviting to women and minorities? Of course, white men will thrive in an environment tailored to them in the first place.
There’s no doubt that the goal of many of these software firms is to help level the playing field. The good intentions are there. So are the beliefs that this works at the corporate level. The Boston-based consulting group BCG Henderson and MIT Sloan Management Review found that 85% of executives surveyed believed AI would give their companies a competitive advantage; 60% said that having an AI strategy is urgent.
But are cognitive ability and intelligence tests reliable predictors of work success, or do they bring us one step closer to an era of digital eugenics?
The problems inherent in digital hiring initiatives
Even under the best of circumstances, there are multiple issues facing companies that use AI in their hiring processes.
The first is a lack of transparency concerning the inner-workings of the algorithm itself. Companies that sell AI hiring services use proprietary algorithms, making their inner-workings opaque, even to the company that buys it. These “black box” algorithms leave us guessing about how judgments are made, meaning that hiring diversity can’t be judged until farther down the line.
That brings us back to transparency. If we can’t build objective algorithms, the least we can do is open them up to public scrutiny. That way, you potentially get a diverse set of eyes on them to help ferret out issues that the development team may have missed. But again, these are the intellectual property of the companies that own them, and making that material open access is akin to financial suicide.
In an effort to address this, in April of 2019, US Senators Cory Booker (D-NJ) and Ron Wyden (D-OR), and Rep. Yvette D. Clarke (D-NY) introduced the Algorithmic Accountability Act, which would require companies “to study and fix flawed computer algorithms that result in inaccurate, unfair, biased or discriminatory decisions impacting Americans.” The bill acknowledges that “algorithms have authors,” according to Rep. Clarke, and that they may act contrary to current anti-discrimination laws. But it also assumes that bias is straightforward and easy to ferret out with the right intentions – and we have yet to find sure-fire ways of ensuring that. The bill has been referred to the House Committee on Energy and Commerce, where it currently sits.
The next issue is one of retention. Even if we were able to ensure a diverse workforce with the use of AI, hiring is only the first step. Taking on new employees of different genders and ethnicities requires inclusion in the workplace so that those employees stay with the company. That’s why diversity and inclusion so often go together. AI algorithms don’t do much to assess the employee life cycle or ensure that women and minority workers will be treated fairly once they are hired.
More often than not, fields dominated by white men have a shallow candidate pool. Potential workers might be hesitant to enter fields where they are not well-represented or won’t have mentors who look like them. Even at the high school and college level, these fields may produce relatively few women and minorities, in which case we’re going to need more than an algorithm to solve the problem.
While it’s true that diversity and inclusion are on the minds of managers, C-suite professionals, boards, and HR personnel, there are still companies that don’t even track this data. Or, if they do, they don’t release it, making it nearly impossible to gauge their commitment to the endeavor.
The problem of privacy
Let’s pretend for a moment that we are able to develop a reliable algorithm that allows companies to hire a diverse workforce. Let’s even assume that retention is not an issue. We are still left with the question of how we got there and whether or not it was ethical or respectful of people’s privacy. There’s a big difference between what we can know about candidates and what we should know about them. How much data is a company entitled to gather about a (potential) worker?
Companies that use the excuse that candidates can opt-out of such data collection are saying that they opt-out of their chances of being considered for the job. And promises to treat data with the utmost sensitivity should be met with skepticism. A company can undoubtedly pledge to keep the data they’ve gathered private, but no one is capable of ensuring it will never be leaked or hacked.
AI algorithms also raise questions about whether this data gathering violates the Americans with Disabilities Act and other employment laws. Take scores on neurological games or even social media deep-dives as examples – will these give employers information they aren’t entitled to, such as family status, political orientation, sexual orientation, or whether a candidate is pregnant or physically or mentally ill? AI may be able to glean some of this from its analytics and behavioral measurements. But how will that data be used (and secured)?
Finally, there’s a good argument to be made that we shouldn’t even try to strip résumés or interview answers of their gender or ethnicity data.
When a field lacks diversity, leveling the playing field may require more than objective analysis; it may require deliberate hiring strategies. Companies may need to actively seek out minority or female candidates, not expect an algorithm to produce them.
Resisting the hype
The idea of handing over the complex problem of ensuring diversity and inclusion in the workplace to a computer is tempting, but success is still years away (if it comes at all).
Ethicists have warned all along that technology that is employed before we assess its social implications will likely lead to negative consequences. Nevertheless, we’ve moved ahead at full speed despite the lack of proof that AI is accurate, unbiased, or ethical at any stage. That means that the only way to get companies to curb their use of these tools now (and therefore backtrack on their investments) may be lawsuits filed by those who have been discriminated against by AI. That’s certainly not a time-saving or cost-effective way to conduct business.
Companies are eager to claim that they’re disrupting recruitment (in a good way) with AI. But we should be ready to accept that algorithms might never be a reliable solution to solving the diversity issues we face in hiring and retention.
After all, technology is only as good as we are.
“The Legal and Ethical Implications of Using AI in Hiring,” by Ben Dattner, Tomas Chamorro-Premuzic, Richard Buchband, and Lucinda Schettler. Harvard Business Review, April 2019.
“The Next Frontier In Hiring is AI-Driven,” by Megan Farokhmanesh. The Verge, January 2019.
“Tackling Bias In Artificial Intelligence (and In Humans),” by Jake Silberg and James Manyika/ McKinsey Global Institute, June 2019.
“Help Wanted – An Exploration of Hiring Algorithms, Equity and Bias.” Upturn, December, 2018.