Hire developers Community Blog Find a dev job Log in
Close menu
Hiring Tips & Insights: Online tests are in decline due to AI. Here’s how you can adapt your process
Less noise, more data. Get the biggest data report on software developer careers in South Africa.

Online tests are in decline due to AI. Here’s how you can adapt your process

05 April 2024, by Marcelle van Niekerk

In this article, we explore AI’s impact on tech assessments and what companies need to do to adapt their processes and ensure they’re still providing a great candidate experience.

OfferZen_How to Build a Standout Job Search Portfolio as an Entry-Level Developer_inner article.png

Coding challenges are declining as AI tools rise in popularity

AI has become an integral part of software development for South African developers, and shows no signs of going away: The majority of local developers now use code-writing assistants and ChatGPT is their most used AI tool.

This has led many companies to worry about developers using these tools to gain an edge in technical assessments. We can see this playing out in the data:

Developers assessed with an online coding challenge

Bar Graph Example

The number of companies using online coding challenges to assess developers has fallen 21.6% over the past 12 months, while whiteboard interviews have also seen their popularity decline.

It’s no secret that AI tools like ChatGPT can solve problems commonly found in online coding challenges. Developers are also aware of the the advantages of AI tools in assessments: We asked the dev community about this and the majority said that using AI tools in assessments can misrepresent their skills:

At the same time, it’s not enough to ban AI from your assessment process, as it’s becoming an integral part of developers’ day-to-day workflow. How you conduct your hiring process is one of the top factors developers use to assess your company culture. On top of that, an irrelevant assessment or asking too many technical questions that are unrelated to the role are significant contributors to a negative experience.

This made us ask: What can companies do to adapt their hiring processes in response to the rise of AI?

Work with AI to improve your technical assessments

One approach companies can take to adapt their hiring processes is working with AI tools to improve their assessments. Karin Bothma, Engineering Manager at OfferZen, suggests running your technical tests through ChatGPT to see what answers it gives you. You can use this information to refine your questions or the problems you’re asking candidates to solve, as anything that relies on straightforward solutions wouldn’t make for a good test.

Instead, focus on questions that test candidates’ creativity, ability to give thoughtful answers, familiarity with specific technologies, and application of technical concepts beyond what ChatGPT can provide.

Test how potential team members leverage AI

A second option is to work with AI rather than trying to circumvent it. Getting great answers from ChatGPT requires good domain knowledge and creativity — these are skills you’d want in a good candidate. So, you can let candidates use ChatGPT or another LLM-based tool and make it part of your assessment.

Ilya Sakharov, CTO of Codility, suggests testing for AI-specific skills as part of your process. This can include things like proofreading and reviewing of AI-generated code, or how they prompt AI tools to achieve a desired outcome.

Alternatively, you can ask how developers are already using AI to determine if they have the fundamentals they need to use these tools to improve their productivity or if they’re masking shortcomings.

For more data and insights to make your hiring process as efficient as possible, get our 2024 Developer hiring & retention report.

Download the full report

Further reading:

200+ developers are waiting for you each week
Get access to a curated list of skilled developers, ready to interview.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Subscribe to our blog

Don’t miss out on cool content. Every week we add new content to our blog, subscribe now.

By subscribing you consent to receive OfferZen’s newsletter and agree to our Privacy Policy and use of cookies.