Evolution of Applicant Tracking Systems from Paper to AI
From paper resumes to AI-driven platforms, see how applicant tracking systems have evolved alongside the workplace, as each phase solved the challenges of its time.
A benefits vendor can win or lose a place on a buyer’s shortlist within a single AI conversation. Its analytics will show nothing either way.
In a Gartner survey published in May 2026, 45% of B2B buyers reported using generative AI during a recent purchase, primarily to gather information about vendors and products.
Almost none of that research reaches a marketing dashboard. For teams asking why use AI search monitoring tools, the answer begins with understanding that blind spot.
Web analytics were designed for a world where interest travels through links. Generative assistants, however, rarely register that interest.
A Pew Research Center study of 900 U.S. adults found that people clicked a traditional search result in 8% of visits when Google showed an AI summary, against 15% when it did not. Links inside the summary performed even worse, at just 1% of visits.
In fact, Conductor’s 2026 AEO/GEO Benchmarks Report, covering 13,770 domains, put AI referrals at 1.08% of total website traffic.
Meanwhile, buyers rely on AI far more than the traffic suggests.
6sense’s 2025 Buyer Experience Report, based on more than 4,000 buyers, found that 94% used large language models to summarize reviews or analyze data during a purchase.
How is that buyers increasingly use AI, yet the traffic reaching vendor websites barely reflects that behavior?
The gap exists because the influence and the visit happen at different times. A buyer who reads a recommendation in an AI chat and acts on it three days later arrives through a branded search, a direct visit, or a broker’s forwarded email. None of those carries an AI referrer.
As a result, teams that judge AI by referral traffic alone conclude it “isn’t driving anything.” That false negative is exactly why you should monitor brand mentions in AI search results directly.
Most monitoring platforms report some version of the same four metrics.
HubSpot’s AEO tool, still labeled beta, makes a useful example of what these metrics look like in practice. The standalone version costs $50 a month and tracks 25 prompts across ChatGPT, Gemini, and Perplexity.
It runs those prompts across the three AI engines and records how often the brand appears in their responses, along with metrics such as citations, competitor visibility, and sentiment.
That routine is the practical answer to the question of why use AI search monitoring tools: no marketing team will put 25 questions to three assistants every day and log the results by hand.
The visibility score is the percentage of tracked answers that mention the brand. For example, if a vendor appears in 12 out of 75 answers, its score is 16%. The metric is best used to answer one question – is the brand showing up more often over several months?
However, it is easy to read too much into this single score.
The score naturally moves up and down from week to week, so a move of a point or two is noise, and only a steady trend means something.
HubSpot makes a similar point on its product page, cautioning that the first few weeks of data are directional and become more reliable as more data accumulates.
Share of voice divides the brand’s mentions by all mentions across a competitor set that the vendor chooses. Put simply, it answers a positioning question: when AI assistants name anyone in the category, how often is it this company?
The weakness is that the denominator is a choice.
A set of five legacy competitors, for example, can make the share of voice look healthy.
However, adding the two venture-backed entrants that brokers actually shortlist can halve it overnight, even though nothing in the market has changed.
Ideally, the competitor list should mirror the names that frequently appear in lost-deal notes and RFP finalist lists. Once set, it should also stay fixed long enough to produce a trend.
Sentiment is measured on a scale from −100 to +100, based on how favorably the assistant characterizes the brand.
Its best use is catching factual and recurring problems, such as a benefits platform being characterized as difficult to implement.
However, sentiment is calculated only from answers that mention the brand, which sharply reduces the sample size. In a small sample, even one unfavorable description can substantially move the aggregate.
Therefore, a score based on four mentions deserves far less confidence than one based on hundreds.
Citation analysis lists the domains and pages an assistant drew on when building its answer.
For a benefits vendor, the list usually mixes the vendor’s own site with review platforms, marketplace category pages, broker blogs, trade press, and discussion forums.
Of the four metrics, citation analysis is the only one that names something a team can change. That is why it gets a fuller explanation below.
For most of its history, U.S. television advertising was priced on a sample.
Nielsen tracked a small group of households and treated their habits as a stand-in for the whole country, until late 2025, when it retired panel-only measurement and began blending that panel with cable box and streaming records.
A prompt set works the same way, because the questions a team chooses to test are its panel.
Tracking twenty-five versions of “What is [Vendor]?“ may produce reassuring visibility while saying almost nothing about whether the product really surfaces.
The real choice is which prompts are worth tracking in the first place. And where do those questions come from?
The best raw material sits in discovery-call notes, RFP questionnaires, and broker emails.
Buyers rarely just type “corporate wellness platform.” Instead, they usually describe a symptom, such as low turnout for last year’s fitness challenge or rising burnout scores in the engagement survey.
RFPs are particularly useful because their questions have grown longer and more open-ended, a pattern Shortlister documented in The Death of the Yes/No Question. HubSpot also offers CRM-based prompt suggestions, which work best as a draft to edit and customize.
Cost pressure shapes the wording, too. With average family premiums reaching $26,993 in 2025, up 6% from a year ago, many employer questions arrive framed around savings, so price and ROI prompts belong in the set.
Most teams start and stop at the category prompt, tracking some version of “best corporate wellness platforms.”
The trouble is that a category question captures only one early moment of the buyer’s journey. In reality, Shortlister’s observed enterprise evaluation cycles typically last 90 to 120 days from RFP send to close.
A stronger prompt set follows the buyer through the five stages of the purchasing journey:
Each stage also points to a different fix. A vendor missing early needs more third-party coverage, while one described incorrectly later needs its sources corrected. A category-only set catches neither problem.
Of course, knowing the stages is easier than fitting them into 25 slots. Because the allowance forces trade-offs, the allocation should follow where deals are won or lost.
The table below shows one reasonable split for a hypothetical vendor in the corporate wellness category.
| Stage | Prompts | Example prompt |
|---|---|---|
| Problem | 5 | "Half our staff say they're burned out, and almost nobody joined last year's wellness challenge. What should HR try instead?" |
| Category | 6 | "What are the best corporate wellness platforms for a 2,000-person hybrid company?" |
| Shortlist and comparison | 6 | "Which wellness apps sync with Google Calendar and fitness trackers and report participation by office?" or "[Vendor] vs. [Competitor] for a workforce split between remote and frontline staff" |
| Validation | 5 | "Is there evidence that [Vendor]'s program lowers health claims?" |
| Commercial | 3 | "How much does a corporate wellness platform cost per employee per month?" |
To get there from a longer list, cut any prompt whose answer would not change a decision. Merge near-duplicates as well, since “top” and “best” versions of one question are likely to return overlapping answers.
It also helps to reserve two or three prompts for broker phrasing, which tends to mention headcount, workforce mix, and the tools employees already use.
Similarly, a wellness vendor that also sells weight management or EAP services should resist splitting 25 slots across categories, because two thin sets measure neither market well.
Finally, freeze the set for at least a quarter.
For vendors, the citation list is the most actionable part of the report, but it is also the least stable.
Profound re-ran roughly 80,000 prompts per platform a month apart in mid-2025 and found that 40% to 60% of cited domains had changed for identical questions. Over six months, drift climbed to between 70% and 90%.
A single appearance, then, means very little.
A more useful rule is to separate recurring sources, which show up in most weekly snapshots and on more than one engine, from one-off sources that appear once and vanish.
Recurring sources are where effort pays off.
In benefits, the likely candidates for recurring sources are broker blogs, law-firm alerts, HR trade publications, and benefits marketplaces such as Shortlister. An earlier Shortlister article explains why third-party pages carry so much weight.
Interestingly, the most valuable finding in a citation report is often an absence.
If a review platform appears in most category answers and the vendor’s profile there is thin or out of date, fixing that profile will likely do more than any new blog post.
Beyond reporting, some tools like HubSpot turn this data into prioritized recommendations: new content to write, outreach to pursue, and social posts.
Suggestions like these optimize the likelihood of being cited. Yet they cannot judge whether a piece of content deserves to exist.
For that reason, a sound AEO strategy puts every suggestion through four questions before acting:
Of the four, the last question also carries legal weight.
The FTC’s final rule on fake reviews, announced in August 2024, bans AI-generated reviews attributed to nonexistent people and compensation conditioned on positive sentiment, with civil penalties for knowing violations.
Suggestions that pass all four questions are worth doing. Rejected ones, on the other hand, should be logged with a reason, so the decision is on record if the same advice comes up again.
Even a well-built report has limits, and the first is that sampled prompts are not real conversations.
In reality, an HR director rarely asks a single question. The exchange runs several turns, carries specific details, and may happen inside an enterprise assistant that no external tool can see.
On top of that, answers vary from run to run, so trend lines over months matter more than any single reading. Most importantly, none of the four metrics, on its own, connects to a sales pipeline.
They show how often and how favorably assistants describe a vendor, but not whether any buyer subsequently requested a demo.
The simplest fix is a required “How did you first hear about us?” question on demo request forms, with “AI assistant” as one of the answers.
When visibility and those form responses rise over the same quarter, the link between AI recommendations and new business becomes much harder to dismiss.
Senior Content Writer at Shortlister
Browse our curated list of vendors to find the best solution for your needs.
Subscribe to our newsletter for the latest trends, expert tips, and workplace insights!
From paper resumes to AI-driven platforms, see how applicant tracking systems have evolved alongside the workplace, as each phase solved the challenges of its time.

Buyers now ask AI assistants which vendors to consider, and the answers never appear in a traffic report. Tracking them takes a well-chosen list of questions and a healthy distrust of small samples.

With the rise of AI in the workplace, its impact has become multi-dimensional. While there is no doubt about its ability to reduce manual labor, this technology also reshapes job expectations, creating new opportunities and challenges.

Buyers now ask AI assistants which vendors belong on the list. The answer gets assembled from pages the vendor did not write, and few vendors have checked which pages those are.
We're working hard to make it easy for you. Sign up for news, trends and insights.