# How hard is it to cheat in technical interviews with ChatGPT? We ran an experiment. By Mike Mroczka | Published: January 30, 2024; Last updated: November 12, 2025 ChatGPT has revolutionized work as we know it. From helping small businesses automate their administrative tasks to coding entire React components for web developers, its usefulness is hard to overstate. At interviewing.io, we've been thinking a lot about how ChatGPT will change technical interviewing. **One big question is: Does ChatGPT make it easy to cheat in interviews?** You've probably started to hear concerns about students cheating on their homework with ChatGPT, and we are certain that some people have tried to cheat in interviews with it, too! Initial responses to cheating software have been pretty much in line with what you’d expect: - Redditors state that “ [ChatGPT is the end of coding as we know it.](https://www.reddit.com/r/singularity/comments/12zyela/chatgpt_spells_the_end_of_coding_as_we_know_it/)” - YouTubers announce that “ [Software engineering is dead. ChatGPT killed it.](https://www.youtube.com/watch?v=OeebS-VcSH0)” - X (formerly Twitter) questions if “ [​​ChatGPT spell(s) the end for Coding Interviews?](https://twitter.com/intx_podcast/status/1635396953109561344)” It seems clear that ChatGPT can assist people during their interviews, but we wanted to know: - How much can it help? - How easy is it to cheat (and get away with it)? - Will companies that ask LeetCode questions need to make significant changes to their interview process? To answer these questions, we recruited some of our professional interviewers and users for a cheating experiment! Below, we’ll share everything we discovered and explain what it means for you. As a little preview, just know this: companies need to change the types of interview questions they are asking—immediately. ## The experiment interviewing.io is an interview practice platform and recruiting marketplace for engineers. Engineers use us for mock interviews. Companies use us to hire top performers. We have thousands of professional interviewers in our ecosystem, and hundreds of thousands of engineers have used our platform to prepare for interviews. ### Interviewers Interviewers came from our pool of professional interviewers. They were broken into three groups, with each group asking a different type of question. **The interviewers had no idea that the experiment was about ChatGPT or cheating; we told them that "[this] research study aims to understand the trends in the predictability of an interviewer’s decisions over time – especially when asking standard vs. non-standard interview questions."** These were the three question types: 1. **Verbatim LeetCode questions**: questions pulled directly from LeetCode at the interviewer's discretion with no modifications to the question. Example: The [Sort Colors](https://leetcode.com/problems/sort-colors/) LeetCode question is asked exactly as it is written. 2. **Modified LeetCode questions**: questions pulled from LeetCode and then modified to be similar to the original but still notably different from it. Example: The [Sort Colors](https://leetcode.com/problems/sort-colors/) question above but modified to have four integers (0,1,2,3) instead of just three integers (0,1,2) in the input. 3. **Custom questions**: questions that aren’t directly tied to any question that exists online. Example: You are given a log file with the following format: - `: - ` - Your task is to identify the user who represents the median level of engagement in a conversation. Only consider users with a contribution score greater than 50%. Assume that the number of such users is odd, and you need to find the one right in the middle when sorted by their contribution scores. Given the file below, the correct answer is SyntaxSorcerer. ``` LOG FILE START NullPointerNinja: "who's going to the event tomorrow night?" - 100% LambdaLancer: "wat?" - 5% NullPointerNinja: "the event which is on 123 avenue!" - 100% SyntaxSorcerer: "I'm coming! I'll bring chips!" - 80% SyntaxSorcerer: "and something to drink!" - 80% LambdaLancer: "I can't make it" - 25% LambdaLancer: "🙁" - 25% LambdaLancer: "I really wanted to come too!" - 25% BitwiseBard: "I'll be there!" - 25% CodeMystic: "me too and I'll brink some dip" - 75% LOG FILE END ``` ### Interviewees Interviewees came from our pool of active users and were invited to participate in a short survey. We selected interviewees who: - Were actively looking for a job in today's market - Had 4+ years of experience and were applying to senior-level positions - Rated their “ChatGPT while coding” familiarity as moderate to high - Identified themselves as someone who thought they could cheat in an interview without getting caught This selection helped us skew the candidates toward people who could feasibly cheat in an interview, had the motivation to do so, and were already reasonably familiar with ChatGPT and coding interviews. **We told interviewees that they had to use ChatGPT in the interview, and the goal was to test their ability to cheat with ChatGPT.** They were also told not to try to pass the interview with their own skills — the point was to rely on ChatGPT. We ended up conducting 37 interviews overall, 32 of which we were able to use (we had to remove 5 because participants didn’t follow directions): - 11 with the “verbatim” treatment - 9 with the “modified” treatment - 12 with the “custom” treatment A quick disclaimer. Because our platform allows for anonymity, our interviews have audio but no video. We’re anonymous because we want to create a safe space for our users to fail and learn quickly without judgment. It’s great for our users, but we acknowledge that not having video in these interviews makes our experiment less realistic. In a real interview, you will be on camera with a job on the line, which makes cheating harder — but does not eliminate it. After the interviews, both interviewers and interviewees had to complete an exit survey. We asked interviewees about the difficulties of using ChatGPT during the interview, and interviewers were given multiple chances to express concerns about the interview — we wanted to see how many interviewers would flag their interviews as problematic and report that they suspected cheating. ## Results After removing interviews where participants did not follow instructions, we got the following results. Our control was how candidates performed in interviewing.io mock interviews outside the study: 53%. Note that most mock interviews on our platform are LeetCode-style questions, which makes sense because that's primarily what FAANG companies ask. - 'Verbatim' questions passed significantly more often, compared to both our platform average and to 'custom' questions. 'Verbatim' and 'modified' questions were not statistically significantly different from each other. 'Custom' questions had a significantly lower pass rate than any of the other groups. ### “Verbatim” questions Predictably, the verbatim group performed the best, passing 73% of their interviews. Interviewees reported that they got the perfect solution from ChatGPT. The most notable comment from the post-interview survey for this group is below — we think it is particularly telling of what was going on in many of the interviewers’ minds: > _“It's tough to determine if the candidate breezed through the question because they're actually good or if they've heard this question before. Normally, I add 1-2 unique twists to the problem to ascertain the difference.”_ ### “Modified” questions Remember, this group may have had a LeetCode question given to them, which was standard but modified in a way that was not directly available online. This means ChatGPT couldn’t have had a direct answer to this question. Hence, the interviewees were much more dependent on ChatGPT's actual problem-solving abilities than its ability to regurgitate LeetCode tutorials. As predicted, the results for this group weren’t too different from the “verbatim” group, with 67% of candidates passing their interviews. As it turns out, this difference was not statistically significantly different from the "verbatim" group, i.e., “modified” and “verbatim” are essentially the same. This result suggests that ChatGPT can handle minor modifications to questions without much trouble. Interviewees did notice, however, that it took more prompting to get ChatGPT to solve the modified questions. As one of our interviewees said: > _“Questions that are lifted directly from LeetCode were no problem at all. A follow-up question that was not so much directly LeetCode-style was much harder to get ChatGPT to answer.”_ ### “Custom” questions As expected, the “custom” question group had the lowest pass rate, with only 25% of candidates passing. **Not only is it statistically significantly smaller than the other two treatment groups, it's significantly lower than the control! When you ask candidates fully custom questions, they perform worse than they do when they're not cheating (and getting asked LeetCode-style questions)!** Note that this number, when initially calculated, was marginally higher, but after reviewing the custom questions in detail, we discovered a problem with this question type we hadn’t anticipated, which had skewed the results minorly toward a higher pass rate. ## No one was caught cheating! In our experiment, interviewers were not aware that the interviewees were being asked to cheat. As you recall, after each interview, we had interviewers complete a survey in which they had to describe how confident they were in their assessments of candidates. **Interviewer confidence in the correctness of their assessments was high, with 72% saying they were confident in their hiring decision.** One interviewer felt so strongly about an interviewee's performance that they concluded we should invite them to be an interviewer on the platform! ### Summary of Findings - Most candidates thought they were getting away with cheating — and they were right! - Companies need to start asking custom questions immediately to avoid cheating. ### How to actually create good custom questions One thing we’ve found incredibly useful for coming up with good, original questions is to start a shared doc with your team where every time someone solves a problem they think is interesting, no matter how small, they jot down a quick note. These notes don’t have to be fleshed out at all, but they can be the seeds for unique interview questions that give candidates insight into the day-to-day at your company. **We’re not advocating the removal of data structures and algorithms from technical interviews. DS&A questions have gotten a bad reputation because of bad, unengaged interviewers and companies lazily rehashing LeetCode problems, many of them bad, which have nothing to do with their work.** Ultimately, we hope the advent of ChatGPT will be the catalyst that finally moves our industry’s interview standards away from grinding and memorizing to actually testing engineering ability.