Testing ChatGPT With Coding Interview Questions
3 min read

Testing ChatGPT With Coding Interview Questions

Here's an in-depth test to see how ChatGPT compares to humans in a coding interview. Spoiler alert; the result is IMPRESSIVE!
Testing ChatGPT with coding interview questions
Photo by Tirza van Dijk on Unsplash

We've interviewed and hired hundreds of software engineers over the past few years.

And it seems like ChatGPT is the last nail on the coffin for coding interviews.

Key Takeaways:

  • ChatGPT solves challenges in seconds that humans would take days.
  • AI will accelerate software development, but it's also a nightmare for hiring managers in the recruitment process.
  • People primarily doing non-creative coding work could easily get replaced by AI.

This Is How ChatGPT Performed in Coding Interview Challenges!

These are 3 real-life examples of code challenges we used to interview them. These were solved by real (human) engineers:

Challenge #1

We started with the simplest question. This is an icebreaker for juniors. They usually take a few minutes.

ChatGPT delivered a correct solution in a few seconds:

ChatGPT solved this question in a few seconds

Challenge #2

Now we went for a classic challenge with a 2d array.

This one is nuanced, and even senior candidates take some time. They need to find the zeros, then solve for one rectangle, and only then solve for n rectangles.

We're also looking for algorithmic complexity here.

question for the second challenge

We used to have this code challenge for 1-hour interviews and would break it into those 3 parts.

Very few candidates would complete the full scope within one hour.

Chat GPT solved the full scope in less than 30 seconds:

ChatGPT solved the second challenge in less than 30 seconds
ChatGPT even explained clearly how the function works
ChatGPT also shared an example of how the function can be used

Challenge #3

Now, we had this one as a take-home challenge (with a bit less context).

It's a classic risk-scoring problem in Fintech:

  • Grab someone's bank account.
  • Fetch a year of transactions (using Plaid API in this case).
  • Find salaries received in the past year.

Candidates would usually take some time looking up the Plaid API, especially the transactions endpoint. Then they'd code the logic to pull the transactions and filter only the desired ones.

ChatGPT impressed us again in just a few seconds:

ChatGPT solved the final challenge in another few seconds
a real software engineer usually submits the function in a few days

Now, the question is, if AI can do this for coding interview challenges, how much can it disrupt a software engineer's daily job?

We're sitting tight to see all developments that come up in the coming months. But we can only assume it will impact a lot how software is built.

Follow us for more knowledge about remote work

We'll be publishing new articles every week, and new social media content every day. If you enjoyed this article, follow us on Twitter or Linkedin, and stay in the loop. Share our content and drop us a comment there. Let's help more people learn about remote work.

Mastering Remote Work, our course for high performing remote workers and teams.