Wednesday, 2024 October 30

Are AI interviews reliable?

“Please tell us more about yourself.”

The premise of a job interview is often to get to know the other party — that is, the potential employer and employee — better, and to put a face to the CV. However, this was made nearly impossible during the safe-distancing measures put in place during the pandemic. Virtual interviews over Zoom, Teams, and Google Meet became the norm.

Some companies are even taking a step further by introducing AI into the mix. Instead of sending a recruiter or interviewer to do the job, Ai interview systems with robotic voices are used, usually with a set of predetermined questions that have to be answered within a time limit.

Answers are first recorded, then converted from audio to text, which the candidate can then refine further. In such interviews, skills such as thinking on one’s feet and sustaining eye contact — basic elements that make up a conversation — are not evaluated, leaving many candidates disoriented.

The popularity of AI interviews is going up, particularly on social and e-commerce platform Little Red Book (the Chinese equivalent of Instagram and Pinterest together). Terms like “AI interview acceptance rate,” “AI interview techniques,” and “AI interview skills” are being shared in posts, particularly in relation to companies in industries such as fast-moving consumer goods, foreign investments, and banking.

Advantages of AI Interviews

In recent years, the recruitment process has undergone significant changes due to external factors such as the COVID-19 pandemic and the massive supply-demand imbalance in the labor market. Many companies have turned to AI recruiting tools in the first round of interviews to address these challenges.

The number of job-seekers is also a factor. When an HR department receives an overwhelming number of resumes, AI screening and interviews emerge as a simple solution that can overcome the time and space constraints of traditional recruitment processes. The resulting time and labor cost savings of using AI interview systems have made them widely popular among employers, with reported time savings ranging between 65% and 95%, according to popular channel AI Talent on video-sharing platform Bilibili.

One of the main advantages of AI interviews is their objectivity and fairness. “AI interviews can eliminate subjective factors that may interfere with the interview results and reduce bias from factors such as from discrimination of appearance and regional accents, all of which can be set in the algorithm,” said Tian Juanjuan, executive director of the Zhilian Recruitment Evaluation Research Institute.

This advantage of AI interviews may be the reason why the AI interview market is expanding globally. According to data from Facts & Factors, the global AI interview market reached USD 610 million in 2021 and is expected to reach USD 890 million in 2028, at a compound annual growth rate of 6.5%.

The drawbacks

However, it’s worth paying attention to the possible drawbacks of AI interviews.

Some experts in the industry warn that AI systems designed by humans can have both positive and negative effects. While they can improve objective factors, they can also unintentionally reinforce prejudice, particularly during the initial machine-learning stage where the system can learn conscious systemic bias. Recent evidence on several social platforms suggests that many new graduates who have participated in AI interviews suspect that they were rejected because of insufficient education qualifications or gender discrimination.

“In my opinion, information asymmetry is the main reason why interviewees are calling AI interviews into question,” said Liu Wei, director of the Human-Computer Interaction and Cognition Engineering Laboratory of Beijing University of Posts and Telecommunications. “Manufacturers of AI interview technology should prioritize ensuring clear communication between candidates and the AI interview system, rather than solely focusing on serving HR departments.”

Without a human in the equation, job candidates are also unable to gain insight into the company or interview experience by asking their own questions or having a real conversation. This leads to mechanically repeating answers in interview after interview, with no real learning experience, particularly if the questions are the same. As such, the system will favor candidates who prepare template answers and recite them, rather than those who can truly think on their feet.

AI interviews then and now

AI interviews appeared on the scene as early as 2009. At that time, the financial crisis was sweeping the world. Many of the world’s top Fortune 500 companies began large-scale layoffs, and competition in the job market intensified. A large number of positions could not fill their demand, but companies needed to reduce costs and increase efficiency, forcing HR departments to complete recruitment with as little manpower as possible, resulting in the creation of AI interview systems.

The economy soon recovered, and the market demand for AI interviews slowed down for the next decade. The pandemic — together with its accompanying labor crunches and the need for social distancing — revived that demand.

The head of campus recruitment at L’Oréal China said the company introduced an  AI text analysis system in the recruitment of trainees in 2020. “The reason we use the AI text analysis system is that we need to screen a vast number of resumes during the recruitment process, and we need to be able to match candidates efficiently and accurately,” he said. “The system is perfect for meeting the requirements of efficiency and diversified initial screening.”

Another instance of an AI interview tool is Duomian, which can conduct more than 4,000 interviews within 24 hours, the equivalent of the workload of three interviewers working 48 hours without stopping. It can also answer some questions that are frequently asked by candidates, such as those concerning compensation, office location, and company benefits.

“These questions are customized by recruiting companies, which are answered by our virtual interviewers to facilitate candidates’ initial understanding of the company at the AI interview stage, which will also save HR interview time in the future,” said a Duomian representative.

While candidates may not have the benefit of eye contact and a genuine connection with the system, they are still being scrutinized nonetheless. Tools such as Yimian, which belongs to recruitment company Zhaopin Recruitment, can capture attributes categorized into hard skills, soft skills, personality, and motivation.

HireVue, the American company that pioneered the use of AI interviews, also recognizes this aspect. Within a 30-minute interview, its system can obtain 500,000 data points about candidates across 15,000 different dimensions, including body language, language mode, eye movement, speed of response, and the volume of the candidate’s voice.

Are AI interviews reliable?

In recent years, there have been numerous instances of AI interviewers failing to recruit talents due to a lack of human judgment. According to a report from QbitAI, AI interviewers may fail to recruit talent due to a lack of human judgment. For example, a blogger once participated in an AI interview where he used a white wall background in the first interview, and a bookshelf as the background in the second one, while keeping all other factors exactly the same during the interview process. Surprisingly, with the bookshelf background, the AI interviewer’s preference for the blogger increased by 15%. In addition, another woman changed the variable of whether she wore glasses, and the score of the AI interview system showed a difference of 10 points.

“AI algorithms have blind spots and lack the human insight that actual people possess. For example, the candidates mentioned above may not have high academic achievements, but they are good at finding other methods to fool the AI. On the other hand, it would be relatively difficult to fool human interviewers through the same methods,” said Liu.

“The reason that AI interview systems have been called into question is really about a power imbalance. The technology manufacturers and recruitment companies are in a more powerful position because they have so much information about the candidate at their fingertips, while candidates are in a disadvantaged position of operating blindly without any information available to them — they do not know what the actual screening criteria for AI interviews are,” Liu explained.

It’s worth noting that companies that manufacture AI interview systems are continually striving to enhance them to approach the level of experienced human interviewers.

For example, Tongdao Liepin Group, the manufacturer of Duomian, has introduced intelligent follow-up questioning to AI interviews. These will analyze the answers of candidates in real-time, and then choose an appropriate direction to expand on the questions, leading to more customized interview experiences.

In addition, some AI interview systems and recruiting companies are also starting to make the interview a two-way experience. In some instances, the company sends an interview report to the candidate with a summary of their soft skills and compatibility.

Will AI interview systems ever replace human interviewers? At the moment, the answer appears to be no. Instead, a hybrid approach of using AI and follow-up interviews may be the key to balancing efficiency and connection in the future.

This article was adapted based on a feature originally written by Tao Tao and published on Ranciyuan (WeChat ID: chaintruth. 36Kr Global is authorized to translate, adapt, and publish its contents.

KrASIA Connection
KrASIA Connection
KrASIA Connection features translated and adapted high-quality insights published on 36Kr.com, the largest and most influential technology portal in Chinese language with over 150 million readers across the globe.
MORE FROM AUTHOR

Related Read