Artificial intelligence versus journalists: The quality of automated news and bias by authorship using a Turing test



The integration of Artificial Intelligence (AI) in the media results in the publication of thousands of automated news articles in Spanish every day. This study uses a Turing test to compare the quality of news articles written by professional journalists (from Efe) with those produced by natural language generation (NLG) software (from Narrativa). Based on Sundar’s dimensions (1999) crucial to news perception – credibility, readability and journalistic expertise – , an internationally validated experimental methodology is employed, exploring a novel topic in Spanish: health information. The experiment deliberately varied real and declared authorships – AI and human journalists – to detect potential biases in assessing authorship credibility. A self-administered questionnaire adapted for online surveys was used (N=222), and gender imbalances were minimized to ensure gender equality in the sample (N=128). The study reveals that there are no significant differences between news articles generated by the AI and those written by professional journalists. Both types of news are considered equally credible, though some biases are detected in the evaluation of declared authorship: the AI author is perceived as more believable than the human, while the human journalist is perceived as creating a more lively narrative. The study concludes that it is feasible to produce automated news in Spanish without compromising its quality. In the global media landscape, automated systems employing NLG, machine learning and sophisticated databases successfully advance into new domains such as health information.


automated journalism, automated news, artificial intelligence, Turing test, COVID-19


ARGUEDAS, A.R. and SIMON, F.M. (2023). Automating Democracy: Generative AI, journalism, and the future of democracy. Oxford: Oxford Internet Institute.

BECKETT C. and YASEEN, M. (2023). Generating Change. A global survey of what news organisations are doing with AI. London: LSE. Retrieved from

CALVO-RUBIO, L.-M. and UFARTE-RUIZ, M.-J. (2020). “Percepción de docentes universitarios, estudiantes, responsables de innovación y periodistas sobre el uso de inteligencia artificial en periodismo”. Profesional de la información, 29 (1).

CARLSON, M. (2015). “The Robotic Reporter”. Digital Journalism, 3 (3), 416-431.

CHANG, Y., WANG, X., WANG, J., WU, Y., YANG, L., ZHU, K., CHEN, H., YI, X., WANG, C., WANG, Y., YE, W., ZHANG, Y., CHANG, Y., YU, P.S., YANG, Q. and XIE, X. (2023). “A Survey on Evaluation of Large Language Models”. ArXiv preprint arXiv:2307.03109v7, 1-26.

CLERWALL, C. (2014). “Enter the Robot Journalist. Users’ perceptions of automated content”. Journalism Practice, 8 (5), 519-531.

EUROPEAN COMMISSION (2018). A definition of Artificial Intelligence. Brussels: European Commission. Retrieved from

DANZON-CHAMBAUD, S. (2021). “Tow Report. Covering COVID-19 with automated news”. Columbia Journalism Review. Retrieved from

DANZON-CHAMBAUD, S. (2023). Automated news in practice: Changing the journalistic doxa during COVID-19, at the BBC and across media organizations [Doctoral thesis, School of Communications Dublin City University]. Doras. Retrieved from

DIAKOPOULOS, N. (2019). Automating the News. MA: Harvard University Press.

DOMINGOS, P. (2015). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. St Ives: Allen Lane.

DÖRR, K.N. (2016). “Mapping the Field of Algorithmic Journalism”. Digital Journalism, 4 (6), 700-722.

EFRON, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. California: Standford University.

FANTA, A. (2017). Putting Europe’s Robots on the Map: Automated journalism in news agencies. Oxford: Reuters Institute for the Study of Journalism.

FLANAGIN, A.J. and METZGER, M.J. (2000). “Perceptions of Internet Information Credibility”. Journalism & Mass Communication Quarterly, 77 (3), 515-540.

GRAEFE, A. and BOHLKEN, N. (2020). “Automated journalism: A meta-analysis of readers’ perceptions of human-written in comparison to automated news”. Media and Communication, 8 (3), 50-59.

GRAEFE, A., HAIM, M., HAARMANN, B. and BROSIUS, H.-B. (2018). “Reader’s perception of computer-generated news: credibility, expertise, and readability”. Journalism, 19 (5), 595-610.

GUISADO-CLAVERO M., ASTIER-PEÑA, M.P., GÓMEZ-BRAVO, R. and ARES-BLANCO, S. (2022). “Open data for monitoring COVID-19 in Spain: descriptive study”. Enfermedades Infecciosas y Microbiología Clínica, 42 (2), 80-87.

HAIM, M. and GRAEFE, A. (2017). “Automated News: Better than expected?”. Digital Journalism, 5 (8), 1044-1059.

INSTITUTO CERVANTES (2022). El español: una lengua viva. Informe 2022. Retrieved from

JIA, C. (2020). “Chinese automated journalism: A comparison between expectations and perceived quality”. International Journal of Communication, 14, 2611-2632. Retrieved from

JIA, C. and JOHNSON, T. (2021). “Source Credibility Matters: Does Automated Journalism Inspire Selective Exposure?”. International Journal of Communication, 15, 3760-3781.

JUNG, J., SONG, H., KIM, Y., IM, H. and OH, S. (2017). “Intrusion of software robots into journalism: the public’s and journalists’ perceptions of news written by algorithms and human journalists”. Computers in Human Behavior, 71, 291-298.

LATAR, N. (2014). “The robot journalist in the age of social physics: the end of human journalism?”. In: G. EINAV (Ed.). The New World of Transitioned Media. New York: Springer International Publishing, 65-80.

LATAR, N. and NORDFORS, D. (2009). “Digital Identities and Journalism Content: How Artificial Intelligence and Journalism May Co-Develop and Why Society Should Care”. Innovation Journalism, 6 (7).

LECOMPTE, C. (2015). “Automation in the Newsroom”. Nieman Reports, 69 (3), 32-45. Retrieved from

LEE, S.M. and KIM, T.Y. (1998). “A news on demand service system based on robot agent”. In: International Conference on Parallel and Distributed Systems Proceedings, 528-532.

LERMANN-HENESTROSA, L., GREVING, H., KIMMERLE, J. (2023). “Automated journalism: The effects of AI authorship and evaluative information on the perception of a science journalism article”. Computers in Human Behavior, 138.

LINDEN, C. and TUULONEN, H. (2019). News Automation. The rewards, risks and realities of ‘machine journalism’. WAN-IFRA. Retrieved from

MATSUMOTO, R., NAKAYAMA, H., HARADA, T. and KUNIYOSHI, Y. (2007). “Journalist Robot: Robot System Making News Articles from Real World”. In: 2007 IEEE Intl conf on robotics and automation, 1234-1241.

MEEHAN, J.R. (1977). “Tale-Spin, an Interactive Programme That Writes Stories”. In: Proceedings of the Fifth International Joint Conference on Artificial Intelligence, 91-98. Retrieved from

MEYER, P. (1973). Precision journalism: A reporter’s introduction to social science methods. Bloomington: Indiana University Press.

MEYER, P. (1988). “Defining and measuring credibility of newspapers: Developing an index”. Journalism & Mass Communication Quarterly, 65 (3), 567-574.

MORAN, R.E. and SHAIKH, S.J. (2022). “Robots in the news and newsrooms: Unpacking meta-journalistic discourse on the use of artificial intelligence in journalism”. Digital Journalism, 10 (10), 1756-1774.

MORAVEC, V., MACKOVÁ, V., SIDO, J. and EKŠTEIN, K. (2020). “The Robotic Reporter in The Czech News Agency: Automated Journalism and Augmentation in the Newsroom”. Communication Today, 11 (1), 37-52. Retrieved from

MURCIA VERDÚ, F. J., RAMOS ANTÓN, R. and CALVO RUBIO, L. M. (2022). “Comparative analysis of the sports chronicle quality written by artificial intelligence and journalists”. Revista Latina de Comunicación Social, 80, 91-111.

NAPOLI, P.M. (2014). “Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption”. Communication Theory, 24 (3), 340-360.

PARRA, S. and TORRENS, M. (2018). La inteligencia artificial: el camino hacia la ultrainteligencia. Barcelona: National Geographic.

RUTKIN, A. (2014). “Rise of Robot Reporters: When Software Writes the News”. New Scientist, 221. Retrieved from

SANDOVAL-MARTÍN, T. and LA-ROSA, L. (2018). “Big Data as a differentiating sociocultural element of data journalism: the perception of data journalists and experts”. Communication & Society, 31 (4), 193-209.

SANDOVAL-MARTÍN, T. and LA-ROSA, L. (2023). “Investigación sobre la calidad de las noticias automatizadas en la producción científica internacional: metodologías y resultados”., 55, 114-136.

SCHWAB, K. (2016). La Cuarta Revolución Industrial (The Fourth Industrial Revolution). Barcelona: Debate.

SEPI (2020). Informe Anual 2020. Sociedad Estatal de Participaciones Industriales. Retrieved from

SEPI (2021). Informe Anual 2021. Sociedad Estatal de Participaciones Industriales. Retrieved from

SUNDAR, S. (1999). “Exploring receivers’ criteria for perception of print and online news”. Journalism & Mass Communication Quarterly, 76 (2), 373-386.

SUNDAR, S. (2008). “The MAIN model: A heuristic approach to understanding technology effects on credibility”. In: M.J. METZGER and A.J. FLANAGIN (Eds.). Digital media, youth, and credibility. Cambridge, MA: The MIT Press, 73-100.

TANDOC Jr, E.C., YAO, L.J. and WU, S. (2020). “Man vs. Machine? The Impact of Algorithm Authorship on News Credibility”. Digital Journalism, 8 (4), 548-562.

TEJEDOR-CALVO, S., CERVI, L., PULIDO, C.M. and PÉREZ-TORNERO, J.M. (2021a). “Análisis de la integración de sistemas inteligentes de alertas y automatización de contenidos en cuatro cibermedios”. Estudios sobre el mensaje periodístico, 27 (3), 973-983.

TEJEDOR, S., CERVI, L., TUSA JUMBO, F. and PORTALÉS, M. (2021b). “La información de la pandemia de la Covid-19 en las portadas de los diarios. Estudio comparativo de Italia, Reino Unido, España, Francia, Portugal, Estados Unidos, Rusia y Alemania”. Revista Mexicana de Ciencias Políticas y Sociales, 66 (242), 251-291.

TÚÑEZ-LÓPEZ, M., TOURAL-BRAN, C. and VALDIVIEZO-ABAD, C. (2019). “Automatización, bots y algoritmos en la redacción de noticias. Impacto y calidad del periodismo artificial”. Revista Latina de Comunicación Social, 74, 1411-1433.

TÚÑEZ-LÓPEZ, J. M., FIEIRAS CEIDE, C. and VAZ-ÁLVAREZ, M. (2021). “Impact of Artificial Intelligence on Journalism: transformations in the company, products, contents and professional profile”. Communication & Society, 34 (1), 177-193.

TURING, A. (1950). “Computing Machinery and Intelligence”. Mind, LIX (236), 433-460.

UFARTE-RUIZ, M.J. and MANFREDI SÁNCHEZ, J.L. (2019). “Algoritmos y bots aplicados al periodismo. El caso de Narrativa Inteligencia Artificial: estructura, producción y calidad informative”. Doxa Comunicación, 29, 213-233.

VAN-DER-KAA, H. and KRAHMER, E. (2014). “Journalist versus news consumer: The perceived credibility of machine written news”. In: Proceedings of the Computation + Journalism conference, Columbia University, New York, 24–25, October. Retrieved from

WADDELL, T.F. (2018). “A Robot Wrote This? How perceived machine authorship affects news credibility”. Digital Journalism, 6 (2), 236-255.

WADDELL, T.F. (2019). “Can an Algorithm Reduce the Perceived Bias of News? Testing the Effect of Machine Attribution on News Readers’ Evaluations of Bias, Anthropomorphism, and Credibility”. Journalism & Mass Communication Quarterly, 96 (1), 82-100.

WÖLKER, A. and POWELL, T.E. (2021). “Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism”. Journalism, 22 (2), 86-103.

ZHENG, Y., ZHONG, B. and YANG, F. (2018). “When algorithms meet journalism: The user perception to automated news in a cross-cultural context”. Computers in Human Behavior, 86, 266-275.

WU, Y. (2019). “Is automated journalistic writing less biased? An experimental test of auto-written and human-written news stories”. Journalism Practice, 14 (8), 1008-1028.

Author Biography

Teresa Sandoval-Martín, Universidad Carlos III de Madrid

Teresa Sandoval-Martín, tenured  professor  of  the  Department  of  Communication  at  the  Universidad Carlos III de Madrid, specialized in journalism and new technologies. Lead researcher of the project Identificación de Sesgos de Género en Inteligencia Artificial (Identification of Gender Biases in Artificial Intelligence), funded by the Spanish State Research Agency. She directs the research group Journalism and Social Analysis: Evolution, Effects and Trends. She has directed five doctoral theses and a European postdoctoral fellowship. She has participated in more than a dozen research projects and 70 publications.




Download data is not yet available.