What sources do you see when you ask ChatGPT to pull a specific data point? Along with that, what results do you get if you ask a different chatbot the same question? And importantly, what happens when you revisit the exact question to those chatbots, just one month later?
Chances are, those answers will not be identical. But why is this the case?
If you’ve ever wondered why you get varying results per AI system, it’s because for the first time in history, we’re seeing search results filtered by algorithms that have the capacity to change wildly over the span of a few weeks.
To put it plainly, just like our social media feeds deliver us content designed to personalize our scrolling experience, AI is adopting the same standard when it comes to generating information. When we interact with AI tools, the machine is deciding what we can see to maximize our user engagement.
But unlike static webpages or news articles, AI systems can alter not just how they deliver information, but which sources they draw from, and how frequently they are swapped out.
According to AI experts, AI’s search volatility is alarmingly greater than traditional search.
In a recent data study about AI, findings showed that across platforms like ChatGPT, Google AI Overviews, Microsoft Copilot, and Perplexity, AI search is never the same twice, where nearly half of the domains shift their citations within a single month. Of those large language models (LLMs), Google AI Overviews have the highest turnover, with 59.3% of its cited domains being new.
This rapid evolution is giving rise to the significance of generative engine optimization (GEO), or the practice of optimizing website content so that it appears in AI-generated search responses. The more AI algorithms shuffle their sources, the harder it becomes for credible sites to maintain visibility.
While high data quality is the backbone of knowledge, when chatbots increasingly alter its content, several risks emerge. Users question whether AI responses are consistent or reliable, while its increased bias can skew accurate information tremendously. A frequent shift in sources also leads to a loss in credibility, where longstanding sites may be replaced by newer, less substantiated ones.
Yet, although search results are out of the user’s control, there are practical strategies for utilizing AI in a smarter way.
As Shane Tepper suggests, a creative director and expert in the GEO landscape, he says that in order to cheat the system, the key lies in crafting prompts that are more intentional.
“It’s about having one perfect passage that answers a specific question,” Tepper states. “Look at your best content. Could each paragraph work in isolation? If not, you’re optimized for the wrong paradigm.”
For users, this means cross-referencing information and making strategic decisions when resorting to AI. In completing important projects, for example, that could look like not relying on a single answer run by AI, confirming the facts before using them, consulting its sources, and finding your own additional data to back up its response.
More than anything, consumers must also have the willingness to revisit searches periodically in order to notice trends. Doing so can help understand when an AI algorithm might be prioritizing bias over accuracy.
As we begin to utilize AI in our everyday lives, this is not simply a warning call. If chatbots are dictating how we access everyday information, we can approach them as viable suggestions, rather than the sole choice for streamlining our workflow. And as long as we’re aware, authentic information can win AI over.
Time and time again, AI has proven its relevance in harnessing the power of knowledge. But if its algorithmic output lacks the integrity to support its users, the future of AI could fall dramatically behind.
In a world where technology is always changing, how will humanity defeat it so that worthy information is not lost in the chaos?
