A new study says AI-generated short stories are being rated as good as, or better than, stories written by humans. The research was published in the journal Judgment and Decision Making.
Researchers asked 1,682 adults to read one of six short stories. Three were written by humans and three were written by ChatGPT.
Each AI story matched the theme of one human-written story. Participants were told whether their story was written by a human or by AI.
That label was not always true. Researchers wanted to see if the label itself changed how people rated the writing.
Participants then rated each story on how absorbing, engaging, and high quality it felt. People who read an AI-written story gave it higher ratings on average than people who read a human-written one.
Labels Changed How People Rated the Stories
Even so, the label mattered. Stories marked as human-written got higher ratings than stories marked as AI-written, no matter who actually wrote them.
Dr Deena Skolnick Weisberg of Villanova University, a senior author of the study, said AI can already produce short stories seen as being as good as human writing. She said people should update their view of AI's abilities.
Weisberg is also a creative writer. She said this does not mean writing should be left to AI.
She said AI-written novels will likely differ from human-written ones. She added there should be room for both, and for stories written jointly by humans and AI.
Weisberg said people write for self-expression, personal challenge, and to make sense of their own lives. She said AI producing human-like stories does not change that.
Most People Could Not Tell AI Apart From Humans
The team ran two more experiments with 905 adults. Each person read two stories on the same theme, one written by a human and one by AI, and had to guess which was which.
In one experiment, about 40% guessed correctly. That is worse than random chance. In the second experiment, 52% guessed correctly, close to a coin flip.
The researchers said there was no clear evidence people could reliably tell human writing from AI writing. Fiction expertise did not help participants guess correctly.
People with more experience using AI tools were better at spotting AI-written stories. Weisberg said AI writing has its own style that people can learn to recognize with practice.
Weisberg also said the preference for stories labeled as human-written may come from a wish for authenticity. She said people actually enjoyed the AI stories more because they were simpler to read.
Luke Kennard, a professor of film and creative writing at the University of Birmingham, was not involved in the study. He said AI is not a neutral tool like spellcheck.
Kennard said AI systems are trained on large amounts of existing human writing and run on data centers with real environmental costs. He said the more important question is whether using AI for writing is worth those costs, and his own view is that it is not.
