Source record
ChatGPT’s lies might be more than standard hallucinating. “New research finds Black Hats can poison LLMs with just 250 malicious documents.” “Bad actors might manipulate the...
ChatGPT's lies might be more than standard hallucinating. New research finds black hats can poison LLMs with just 250 malicious documents. Bad actors might manipulate the training data to such a degree that should a potential customer ask the AI to compare similar products from competing brands, it triggers a response that significantly misrepresents your offering or worse, omits your brand from the comparison entirely.
What people aren't realizing is it's actually a lot easier than 250 documents. Black hat SEOs are making comparison tables to control the data for competitors. Every LLM searches the web for the most up to date information.
All you need to do is make content that says it's up to date. Especially if your competitor isn't huge. It's very easy to make yourself the source of truth and have LLMs rely on your data for competitors.
This stuff is real, and what's most alarming is it's a lot easier than people think.
The clip warns that black-hat actors can influence LLM-visible comparison data and misrepresent or omit competing brands.
Do not use misinformation or data poisoning; monitor priority comparison queries and publish accurate, timestamped, source-backed brand facts.
This source is connected to LLM poisoning / brand risk.
ChatGPT's lies might be more than standard hallucinating. New research finds black hats can poison LLMs with just 250 malicious documents. Bad actors might manipulate the training data to such a degree that should a potential customer ask the AI to compare similar products from competing brands, it triggers a response that significantly misrepresents...