AI Resume Screening and What 200 Shortlists Get Wrong

For decades, job seekers have been taught to game the system. They stuff their profiles with buzzwords, format them with complex multi-column layouts, and write long, winding descriptions of their daily tasks. They are writing for the legacy Applicant Tracking Systems (ATS) of the past.

But recruitment technology has evolved. At JobTwine, our AI hiring platform sits at the intersection of agentic AI and talent acquisition. Recently, we reviewed 200 AI-scored shortlists generated by our platform. We wanted to see exactly how modern Large Language Models (LLMs) evaluate talent and more importantly, where human resumes are missing the mark under modern AI resume screening.

We found that excellent professionals are being left behind because their resumes are optimized for 2021 bots, not modern AI.

If your talent acquisition team is still relying on old keyword-matching filters for candidate shortlisting, you are missing the diamond-in-the-rough candidates while letting over-optimized applications slip through. 

Here is exactly what resumes get wrong and how the paradigm of automated candidate shortlisting has fundamentally changed.

1. Keyword Stuffing vs. Semantic Context

Legacy ATS platforms looked for exact keyword matches. If a resume had the word Python ten times, it ranked higher. Job seekers figured this out, leading to bloated, unreadable resumes.

Modern AI does not care how many times a word appears; it looks for semantic context.

When our platform evaluates a candidate, it reads the resume like an expert hiring manager. It looks at how a skill was applied. For example, an AI score drops when a resume simply lists "Data Analysis" in a skills matrix, but it spikes when it explains how the candidate used data analysis to reduce churn by 14%.

The AI Reality Check: Resumes get rejected during screening because they list skills as static nouns rather than demonstrating them as measurable business actions.

2. The Rise of ChatGPT and Resume Manipulation

Here is a trend we caught clear as day across the 200 shortlists: resume manipulation via AI-generated resumes.

Candidates are increasingly using tools like ChatGPT to copy-paste job descriptions and perfectly mirror the prompt back to the employer. On paper, it looks like a 100% perfect match.

Legacy systems are blind to this. However, advanced automated candidate shortlisting engines are built to catch this. JobTwine's engine analyzes response patterns and phrasing cadences to flag hyper-optimized, AI-crafted phrasing that does not reflect the candidate’s actual background.

If your screening tool cannot detect text that has been heavily prompted, your shortlist will be filled with expert prompt engineers, not expert technical executors.

3. Formatting Cleverness Breaks AI Parsers

We have all seen them: beautiful, highly stylized resumes designed on Canva with dual columns, progress bars for skill levels, and custom graphics.

While visually appealing to a human eye, these resumes are a nightmare for standard algorithms. When an LLM ingests a multi-column PDF, the text layout often flattens incorrectly, reading horizontally across columns instead of vertically. This turns a logical career history into a scrambled mess of incoherent sentences.

The highest-scoring resumes in our data review were simple: clean, single-column, standard PDF formats.

Why Legacy Candidate Shortlisting is Costing You Great Hires

Seeing these mistakes firsthand made one thing abundantly clear: if your HR team is still using traditional filtering, you are shortlisting the best resume writers, not the best candidates.

Recruiters are swamped, often spending less than six seconds glancing at an application. They miss the hidden gems because the candidate did not use the exact corporate buzzwords. Alternatively, they pass through candidates who have optimized their resumes using AI to bypass old filters. This is exactly why high-volume talent leaders are moving toward smarter, agentic automation.

Look Beyond the Paper with JobTwine

Fixing the resume problem should not be the candidate’s burden alone. As talent leaders, we need tools that are smart enough to read between the lines, look past formatting errors, and uncover the true intent and capability of a professional.

JobTwine’s AI Shortlisting Agent (powered by JayT) goes completely beyond legacy keyword bags. It reads every resume against the true context of the role you are hiring for. It checks for logical career progression, filters out resume manipulation, flags AI-generated resumes, and gives your recruiters an auditable, ranked list with explicit reasoning attached for every decision.

Stop letting outdated resume habits or clever AI prompts dictate the quality of your talent pipeline. Ready to see how intelligent automation can eliminate your screening backlog and save your team 48 hours per role?

Book a Demo for JobTwine’s AI Shortlisting Agent Today.