Why does AI make up facts about your competitors?
At times, it seems like AI (Bard AI, Chat GPT) is making things up. Ars Techica has a good article about this trait, which it calls confabulation, althought the article doesn’t quite get to the bottom of why AI does this.
If you try AI for various typical competitor analysis questions, maybe half of them will include made-up facts. Here is an example, asking Bard who leads competitive intelligence at Google:
There are three different ways here in which Bard has invented facts:
Misinterpretation: Rossman does not lead any competitive intelligence team at Google, but it is possible that Bard found this competitor analysis webinar by Rossmann and decided it was indicative of her job.
Possible extrapolation: Rossmann has never worked at Microsoft or Salesforce; it is surprising that Bard gets this wrong, because the information is provided in known structured form at LinkedIn, but perhaps Bard has assumed that people who work at Google often have also worked at Salesforce and Microsoft.
Complete invention: Bard claims that Rossmann produced 20% more reports in the last year than the year before that. Assuming this is a relevant metric, and that Rossmann would track this if she even worked in competitive intelligence, it shows a lack of self-awareness by Bard that it’s not the kind of thing an external researcher would know. OK, it’s possible that Rossmann might have shared this metric publicly, in a blog post or presentation. But, ironically, a Google search shows she has not (and why would she - let’s not forget she isn’t the Lead for Competitive Intelligence at all). Bard must know this, but not realize that this is not the category of fact it can extrapolate, because it’s not credible that Bard would have this information.
We don’t know why AI makes things up. A human would, for the most part, know that it’s ludicrous to be typing out random facts into a competitor report - not least because someone is bound to realize. Making things up is the main reason why we can’t use AI in competitor analysis. A human can correct for poor writing, or information gaps, but if the AI is going to confidently, and with precision, assert things that are not true, it becomes unusable.
Need a quote for competitor research?
