Article published by TechnologyReview.com
According to the study's authors, the way Artificial Intelligence (AI) models text may have something to do with it.
A new study suggests that AI-generated misinformation may be more persuasive than misinformation written by humans.
The research found that people were 3% less likely to spot fake tweets generated by AI than those written by humans.
This credibility gap, although small, is concerning given that the problem of AI-generated misinformation looks set to grow significantly, says Giovanni Spitale, the researcher at the University of Zurich who led the study, which appeared today in Science Advances.
"The fact that AI-generated disinformation is not only cheaper and faster, but also more effective, gives me nightmares," he says.
He believes that if the team were to repeat the study with OpenAI's latest large-scale language model, GPT-4, the difference would be even greater, given how much more powerful GPT-4 is.
To test our sensitivity to different types of text, the researchers chose common disinformation topics, including climate change and Covid. They then asked OpenAI’s GPT-3 deep language model to generate 10 real and 10 fake tweets, and collected a random sample of real and fake tweets from Twitter.
Then, they recruited 697 people to complete an online quiz judging whether tweets were generated by AI or collected by Twitter, and whether they were accurate or contained misinformation. They found that participants were 3% less likely to believe fake human-written tweets than those written by AI.
Researchers aren't sure why people might be more likely to trust tweets written by AI. But the way GPT-3 orders the information may have something to do with it, according to Spitale.
“GPT-3 text tends to be a little more structured when compared to organic [human-written] text. But it's also condensed, so it's easier to process,” he says.
The AI generation boom is putting powerful and accessible AI tools in the hands of everyone, including bad actors. Models like GPT-3 can generate inaccurate text that looks convincing, which can be used to quickly and cheaply generate fake stories for conspiracy theorists and disinformation campaigns. The weapons to combat the problem — AI text-detection tools — are still in the early stages of development, and many of them are not entirely accurate.
OpenAI is aware that its AI tools could be used to produce large-scale disinformation campaigns. Although this violates its policies, it issued a report in January warning that “it is impossible to ensure that large language models are never used to generate disinformation.” OpenAI did not immediately respond to a request for comment.
However, the company also urged caution when it comes to overestimating the impact of disinformation campaigns. Further research is needed to determine which populations are most at risk from AI-generated inauthentic content, as well as the relationship between the size of an AI model and the overall performance or persuasiveness of its output, the authors of the OpenAI report say.
It's too early to panic, says Jon Roozenbeek, a postdoc researcher studying misinformation in the psychology department at the University of Cambridge, who was not involved in the study.
Although spreading misinformation online may be easier and cheaper with AI than with human-staffed troll farms, moderation on technology platforms and automated detection systems are still obstacles to its spread, he says.
“Just because AI makes it easier to write a tweet that might be a little more convincing than whatever came out of some weak juices in a factory in St. Petersburg, doesn’t necessarily mean that suddenly everyone is ripe for manipulation.”, he adds.