Readers Prefer AI-Generated Stories Over Human-Written Fiction, Cambridge Study Finds

A Villanova University study published by Cambridge University Press reveals that readers prefer ChatGPT-generated short stories over human-written fiction. Involving 1,682 adult participants aged 18 to 81, the research shows AI stories were rated 6% higher in quality and 8% higher in engagement. Readers exhibited a pro-human bias, giving the highest scores to AI stories they believed were written by humans. Furthermore, participants failed to reliably identify AI writing, guessing correctly only 40% to 52% of the time.

ChatGPT versus human story ratings

• ▪A study published in the journal Judgment and Decision Making found that readers rated short stories generated by ChatGPT higher in quality and engagement than stories written by human authors
• ▪In the first experiment of the Villanova University study, 1,682 participants rated ChatGPT-generated stories 6% higher in quality and 8% higher in engagement than human-written stories

Reader AI detection failure

• ▪In the third experiment of the Villanova University study, adult readers correctly identified the author of the stories 51.97% of the time, which is statistically no different from chance
• ▪Self-reported expertise in fictional literature did not improve a participant's ability to distinguish between human-written and AI-generated stories in the Villanova University study
• ▪Familiarity with AI systems, measured by self-reported expertise or the Artificial Intelligence Literacy Scale, predicted a higher likelihood of correctly identifying AI-generated writing
• ▪In the second experiment of the Villanova University study, adult readers correctly identified the author of the stories only 39.93% of the time, which is worse than a coin flip

Human authorship attribution bias

• ▪Participants with more positive general attitudes toward AI gave higher ratings to stories they were told were produced by ChatGPT than those with less positive views
• ▪Believing a story had a human author increased its rating by an additional 3% in the first experiment of the Villanova University study, revealing an implicit pro-human bias
• ▪The highest ratings in the Villanova University study were given to stories generated by ChatGPT that participants were falsely told had been written by a human

AI creative writing capability

• ▪Deena Skolnick Weisberg explained that AI writing tends to be clearer, more direct, and easier to process, whereas human-written stories are often more subtle, complex, and less predictable
• ▪Senior author Deena Skolnick Weisberg stated that the study's findings reveal public assumptions about the creative capabilities of artificial intelligence are increasingly out of date
• ▪Assistant professor Cameron Jones of Stony Brook University stated that large language models are optimized against human preferences, leading to writing with high overall appeal

Participant age range demographics

• ▪The second and third experiments of the Villanova University study involved 905 adult participants, with 424 in the second experiment and 481 in the third
• ▪The Villanova University study on AI and human fiction was built around six short stories, with participants ranging in age from 18 to 81