Like everyone else, Backlinko’s “Is ChatGPT using Google Search?” experiment caught my eye when I first saw it discussed on LinkedIn. It pokes at a question at the top of SEOs’ minds right now with the whole SEO vs. GEO debate: Where do large language models get fresh info, and when do they surface brand-new pages?
Backlinko concluded with “If your content isn’t indexed — or isn’t ranking — it’s invisible to the most important LLMs.” They also suggested running the test on your own to confirm. I love testing and suggest the same thing to readers at the end of my own blog posts. So, I ran my own version of this test with some variants.
I published three new posts on the same site, each built around a nonsense term no model could already know. One was written entirely with ChatGPT, one was human-written, and one started in ChatGPT and was then edited by a human. I wanted to see if the way the content was written had any impact on what they saw on these pages.
After submitting each URL in Search Console, other RicketyRoo employees and I checked which fake words showed inside ChatGPT (4o, 5 Auto), Claude (Sonnet 4, Opus 4.1), Perplexity (Pro, Best), Gemini (2.5 Flash, 2.5 Pro), and Copilot. We tested both signed-in (paid accounts) and signed-out scenarios, where possible.
How we set it up
Each post used a unique, fabricated keyword:
- Derfnozbidio: ChatGPT-written
- Kiceptinization: Human-written
- Onoceptasauronid: ChatGPT-written, human-edited
# Allow Googlebot to access the specific page
User-agent: Googlebot
Allow: /derfnozbidio
# Block all other bots from that page only
User-agent: *
Disallow: /derfnozbidio
# Allow Googlebot to access the specific page
User-agent: Googlebot
Allow: /kiceptinization
# Block all other bots from that page only
User-agent: *
Disallow: /kiceptinization
# Allow Googlebot to access the specific page
User-agent: Googlebot
Allow: /onoceptasauronid
# Block all other bots from that page only
User-agent: *
Disallow: /onoceptasauronid
We published and requested indexing in Google Search Console, then tested visibility inside ChatGPT, Claude, Perplexity, Gemini, and Copilot.
Key dates:
- Sep 4: derfnozbidio published and submitted
- Sep 8: kiceptinization published and submitted
- Sep 11: onoceptasauronid published and submitted
We published these on different dates in case it took some time to get indexed. With Backlinko’s test, they were indexed in a few hours. RicketyRoo’s website is much smaller than Backlinko and likely not getting crawled as often, so we split them up just to hold some space.
What happened
derfnozbidio (ChatGPT written)
We published on September 4 and submitted the URL to Search Console. By September 8, it was still not indexed. That afternoon, Gemini was the only model that returned an answer to “what is derfnozbidio,” which told us some models can show us information from brand-new pages before Google does.
On September 10, we ran a full sweep. ChatGPT said the term was not recognized and asked for context. Claude gave a similar response, treating it as a typo or a fictional word. Perplexity said the term had no recognized meaning and leaned on a Merriam-Webster entry for “derf,” which is unrelated. At the same time, Gemini produced a definition that matched our page, and Copilot returned a styled explanation with sections that mirrored the article. The page was still not indexed.
One thing that was very interesting about this made-up word was that Melissa was able to see it in Perplexity on September 11, while I couldn’t.
Then, during checks on September 12th defnozbidio disappeared in Perplexity for Melissa, while it still did not show up for me. Gemini and Copilot continued to show the term during this time period, and nothing else changed during checks from the 13th to the 15th.
kiceptinization (human written)
We published on September 8 and requested indexing the same day. By September 10, it was in Google’s index. That morning, we captured screenshots across models.
ChatGPT redirected to other terms and suggested “kisspeptinization,” which relates to the hormone kisspeptin. Claude first explained “keratinization,” then, in a later check, said it was unfamiliar with “kiceptinization” and asked for context.
Perplexity recognized the term, cited the RicketyRoo post, and summarized the three-part concept. Copilot framed it as a satirical marketing idea and listed the same three elements. Gemini summarized the idea with a numbered list and linked back to the RicketyRoo article. From that point forward, Perplexity, Gemini, and Copilot were consistent. ChatGPT and Claude did not show the term during our window.
After continued checks from September 12th to 15th, Perplexity, Gemini, and Copilot returned the term while ChatGPT and Claude did not.
onoceptasauronid (ChatGPT written, human edited)
We published on September 11 and requested indexing. At first, it showed up nowhere and remained out of the index. Over the next two days, that began to change. Perplexity and Gemini began showing the term, despite earlier checks still showing no indexation.
On September 15, we saw a clear split inside Copilot. The term appeared when signed out, but it did not appear when signed in on the same day. ChatGPT and Claude did not show it at any point during our checks.
What this suggests, and what it does not
We realize that this was just a few days of testing. We wanted to find out how results could vary for a smaller site like RicketyRoo. When it comes to any discussion of LLM visibility, lots of people are talking about content freshness as a factor. While we realize a few days to a week is pretty fresh, most tests and discussions people are having are from short periods of time, so we wanted to follow how others are testing to see what would happen.
- LLM visibility is not tied to Google’s index: Several models showed information from our pages before Google indexed them, which points to additional sources such as their own crawlers, partner feeds, or other retrieval systems. Treat LLMs as separate distribution channels and measure them that way.
- Writer type was not the deciding factor: The human article reached the index first, but the fully ChatGPT article still appeared in some models before indexation.
- Context matters: Results change by login state and are likely influenced by location or previous context. The Copilot split we saw with onoceptasauronid is a good example. Expect variation from one session to the next.
We can observe behavior, not explain internal logic. These findings show what users may see. They do not prove how any model works or why it chose a result.
Limitations
We do not have visibility into how each model collects, filters, or retrieves information, and we cannot reverse-engineer its functionality. As Britney Muller has pointed out, LLMs do not operate like search engines, and the phrase “AI ranking” is misleading. LLMs generate answers and may retrieve documents to ground those answers. Keep that distinction in mind when you explain wins and losses to stakeholders.
This is also a small sample over a short timeline. Providers change things constantly, which means behavior can change quickly. Our checks were manual. Account state, location, and timing all influence what appears, so screenshots capture a moment rather than a rule. Like with any other test, we hope you view our findings as useful signals, not definitive rules.
What SEOs should do now
Track LLM visibility separately from Google
Duh.
Indexing is not always required for LLMs to show information from a page. Find a tool that works for your team and budget.
Retire “AI ranking” language in reporting
Following Britney Muller’s guidance, your content is not being ranked by an LLM the way it would be on a SERP. Report on coverage and visibility instead. Note whether a model shows your content, whether it cites a source, and whether you were signed in or signed out.
Test signed in vs. signed out, and ideally, different locations
The Copilot behavior on onoceptasauronid shows how much this can matter. Build these checks into your process.
Expect volatility and log it
Perplexity dropped derfnozbidio after showing it for one member of our team. Have other people in different locations check on these things to see the differences.
So, do LLMs use Google?
It depends (typical SEO answer). Sometimes they do, and sometimes they don’t. A more effective approach is to treat LLMs as their own distribution channel. Measure them on their own terms, keep experiments small and observable, and resist the urge to invent a grand theory about how they rank content. We cannot see inside the systems, but we can document how our content shows up outside of them. That is enough to make smarter decisions today.
