Massive Study Detects AI Fingerprints In Millions of Scientific Papers

A team of U.S. and German researchers analyzed over 15 million biomedical papers and found that AI-generated content has subtly infiltrated academic writing, with telltale stylistic shifts -- such as a rise in flowery verbs and adjectives. 'Their investigation revealed that since the emergence of LLMs there has been a corresponding increase in the frequency of certain stylist word choices within the academic literature,' reports Phys.Org. 'These data suggest that at least 13.5% of the papers published in 2024 were written with some amount of LLM processing.' From the report: The researchers modeled their investigation on prior COVID-19 public-health research, which was able to infer COVID-19's impact on mortality by comparing excess deaths before and after the pandemic. By applying the same before-and-after approach, the new study analyzed patterns of excess word use prior to the emergence of LLMs and after. The researchers found that after the release of LLMs, there was a significant shift away from the excess use of 'content words' to an excess use of 'stylistic and flowery' word choices, such as 'showcasing,' 'pivotal,' and 'grappling.'

By manually assigning parts of speech to each excess word, the authors determined that before 2024, 79.2% of excess word choices were nouns. During 2024 there was a clearly identifiable shift. 66% of excess word choices were verbs and 14% were adjectives. The team also identified notable differences in LLM usage between research fields, countries, and venues.
The findings have been published in the journal Science Advances.

Read more of this story at Slashdot.