Limits and challenges of large language models for academic and scientific writing: a critical review based on “expert use”
Abstract
Since 2022, the scientific world has acknowledged the impact of the massification of ChatGPT. Its use, both in scientific and academic contexts, has raised questions about theoretical problems such as the authorship of works written at least partially with this technology, and practical problems, such as the lack of adaptation of the texts to academic genres. The present article is an organized presentation of the problems and limits of the use of large language models in academic and scientific contexts. It is based on the analysis of twenty scientific papers that report systematic analysis of data provided by surveys or by systemic literature review. We chose to divide the contexts of use based on the criterion of expert use, which is defined by the convergence of a complete academic literacy and the broad domain of the field about which the chatbot is being inquired. We will address the untrained use and some of its problematic expressions, and we will also analyse the challenges presented by texts produced by chatbots that appear even in the presence of expert use: lack of traceability and biases, hallucinations, parroting, plagiarism, weak argumentative development and inconsistent, incoherent and contradictory expositions.Downloads
References
AlAfnan, Mohammad A., Samira Dishari, Marina Jovic, y Koba Lomidze. 2023. “ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses”. Journal of Artificial Intelligence and Technology 3: 60-68. Fecha de consulta: 15 de febrero de 2025. ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses | Journal of Artificial Intelligence and Technology.
Alkaissi Hussam y Samy I. McFarlane. 2023. “Artificial Hallucinations in ChatGPT: Implications in Scientific Writing”. Cureus. 15: e35179. Fecha de consulta: 15 de febrero de 2025. http://doi.org/10.7759/cureus.35179.
American Psychological Association (APA). 2018. APA Dictionary of Psychology. Washington: Amerian Psychological Association.
Archuby, Gustavo. 2025. “Repositorios institucionales y plataformas de inteligencia artificial ¿acceso abierto para todo?”. Acceso Abierto. Última modificación: 11 de febrero de 2025. Repositorios institucionales y plataformas de inteligencia artificial ¿acceso abierto para todo?.
Barman, Kristian, Nathan Wood, y Pawel Pawlowski. 2024. “Beyond Transparency and Explainability : On the Need for Adequate and Contextualized User Guidelines for LLM Use.” ETHICS AND INFORMATION TECHNOLOGY 26 (3). https://doi.org/10.1007/s10676-024-09778-2.
Barroso, Julio y Julio Cabero. 2011. La investigación educativa en TIC. Madrid: Síntesis.
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”. Trabajo presentado en Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘21). Association for Computing Machinery, Nueva York, Nueva York, Estados Unidos, 1 de marzo de 2021. On the Dangers of Stochastic Parrots | Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.
Bonneau, Jonathan, Laurence Grondin-Robillard, Marc Ménard, André Mondoux. 2022. “Fighting the ‘System’: A Pilot Project on the Opacity of Algorithms in Political Communication”. En New Perspectives in Critical Data Studies. Transforming Communications – Studies in Cross-Media Research, editado por Andreas Hepp, Julianne Jarke y Leif Kramp, 97-120. Cham: Palgrave Macmillan. http://doi.org/10.1007/978-3-030-96180-0_5.
Chan, Alan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bhatt, Adrian Weller, David Krueger, y Tegan Maharaj. 2023. Harms from Increasingly Agentic Algorithmic Systems. En: 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘23), June 12--15, 2023, Chicago, Illinois, Estados Unidos. Fecha de consulta: 28 de abril de 2025. Harms from Increasingly Agentic Algorithmic Systems | Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency.
Chaudhry, Ifat S., Sayed A. M. Sarwary, Ghaleb A., el Refae, Habeb Chabchoub. 2023. “‘Time to revisit existing Student’s performance evaluation approach in higher education sector in a new era of ChatGPT — a case study”. Cogent Education 10 (1): 1-30. Time to Revisit Existing Student’s Performance Evaluation Approach in Higher Education Sector in a New Era of ChatGPT — A Case Study.
Christou, Prokopis A. 2024. “A critical inquiry into the personal and societal perils of Artificial Intelligence”. AI Ethics 2024. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.1007/s43681-024-00556-w.
Conroy, Gemma. 2023. “Scientific sleuths spot dishonest ChatGPT use in papers”. Nature. News. Fecha de consulta: 15 de febrero de 2025. http://doi.org/10.1038/d41586-023-03144-w.
Darwin, Diyenti R., Nur Mukminatien, Nunung Suryati, Ekaning D. Laksmi, and Marzuki. 2023. “Critical Thinking in the AI Era: An Exploration of EFL Students’ Perceptions, Benefits, and Limitations.” Cogent Education 11: 1-18. Full article: Critical thinking in the AI era: An exploration of EFL students’ perceptions, benefits, and limitations.
Dempere Juan, Kenedy Modugu, Hesham Allam y Lakshmana K. Ramasamy. 2023. “The impact of ChatGPT on higher education”. Frontiers in Education 8: 1-13. Fecha de consulta: 15 de febrero de 2025. The impact of ChatGPT on higher education.
Floridi, Luciano. 2023. The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities. Oxford: Oxford University Press.
García Sánchez, Omar V. 2023. “Uso y percepción de ChatGPT en la educación superior”. Revista de Investigación en Tecnologías de la Información 11: 98-107. Fecha de consulta: 15 de febrero de 2025. Uso y percepción de ChatGPT en la educación superior | Revista de Investigación en Tecnologías de la Información.
García-Peñalvo, Francisco J., Faraón Llorens-Largo, y Javier Vidal. (2024). “La nueva realidad de la educación ante los avances de la inteligencia artificial generativa”. RIED-Revista Iberoamericana de Educación a Distancia 27: 9–39. Fecha de consulta: 15 de febrero de 2025. La nueva realidad de la educación ante los avances de la inteligencia artificial generativa | RIED-Revista Iberoamericana de Educación a Distancia.
Grassini, Simone. 2023. “Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings”. Education Science 13: 692-711. Fecha de consulta: 15 de febrero de 2025. Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings.
Imran, Muhammad y Norah Almusharraf. 2023. “Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature”. Contemporary Educational Technology 15 (4): 1-14. Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature.
Kostka, Ilka y Rachel Toncelli. 2023. “Exploring Applications of ChatGPT to English Language Teaching: Opportunities, Challenges, and Recommendations”. The Electronic Journal for English as a Second Language 27: 1-19. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.55593/ej.27107int.
Lillis, Theresa. 2003. “Student Writing as ‘Academic Literacies’: Drawing on Bakhtin to Move from Critique to Design. Language and Education”. Language and Education 17: 192-207. Fecha de consulta: 15 de febrero de 2025.
Lillis, Theresa. 2021. “Academic Literacies: Intereses Locales, Preocupaciones Globales? Academic Literacies: Local Interests, Global Concerns”. En Multilingual Contributions to Writing Research Toward an Equal Academic Exchange, editado por Natalia Ávila Reyes, 35-59. Colorado: University Press of Colorado.
Liu, Sijia, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, Yang Liu. 2024. Rethinking Machine Unlearning for Large Language Models. Nature Machine Intelligence. En prensa. https://doi.org/10.48550/arXiv.2402.08787.
Lo, Chung K. 2023. “What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature”. Education Sciences 13 (410): 1-15. Fecha de consulta: 15 de febrero de 2025. What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature.
Macaranas, Anna, Alissa N. Antle, Bernhard E. Riecke, 2015. “What is Intuitive Interaction? Balancing Users’ Performance and Satisfaction with Natural User Interfaces”. Interacting with Computers 2: 357–370. Fecha de consulta: 15 de febrero de 2025. What is Intuitive Interaction? Balancing Users’ Performance and Satisfaction with Natural User Interfaces | Interacting with Computers | Oxford Academic.
Mai Duong T. T., Can Van Da y Nguyen Van Hanh. 2024. “The use of ChatGPT in teaching and learning: a systematic review through SWOT analysis approach”. Frontiers in Education 9: 1-17. Fecha de consulta: 15 de febrero de 2025. The use of ChatGPT in teaching and learning: a systematic review through SWOT analysis approach.
Martino, Ariana, Michael Iannelli, Coleen Truong. 2023. “Knowledge Injection to Counter Large Language Model (LLM) Hallucination”. En The Semantic Web: ESWC 2023 Satellite Events. Lecture Notes in Computer Science, editado por Catia Pesquita, Hala Skaf-Molli, Vasilis Efthymiou, Sabrina Kirrane, Axel Ngonga, Diego Collarana, Renato Cerqueira, Mehwish Alam, Cassia Trojahn, Sven Hertling, vol 13998, 182-185. Cham: Springer. Knowledge Injection to Counter Large Language Model (LLM) Hallucination | SpringerLink.
Montenegro-Rueda, Marta, José Fernández-Cerero, José María Fernández-Batanero, Eloy López-Meneses. 2023. “Impact of the Implementation of ChatGPT in Education: A Systematic Review”. Computers 12: 1-13. Fecha de consulta: 15 de febrero de 2025. Impact of the Implementation of ChatGPT in Education: A Systematic Review.
Niloy, Ahnaf Chowdhury, Muhammad Ashraful Bari, Jakia Sultana, Rup Chowdhury, Fareha Meem Raisa, Afsana Islam, Saadman Mahmud, et al. 2024. “Why Do Students Use ChatGPT? Answering through a Triangulation Approach”. Computers and Education: Artificial Intelligence 6: 200-208. Why do students use ChatGPT? Answering through a triangulation approach - ScienceDirect.
OpenAI. 2019. Better language models and their implications. Última modificación: 14 de febrero de 2019. Better language models and their implications | OpenAI.
Ouyang, Shuyin, Jie M. Zhang, Mark Harman, and Meng Wang. 2025. „An Empirical Study of the Non-Determinism of ChatGPT in Code Generation”. ACM Transactions on Software Engineering and Methodology 34: 1-28. Fecha de consulta: 15 de febrero de 2025. An Empirical Study of the Non-Determinism of ChatGPT in Code Generation | ACM Transactions on Software Engineering and Methodology.
Pasquinelli, Matteo y Vladan Joler. 2021. The Nooscope manifested: AI as instrument of knowledge extractivism. AI & Society 36, 1263–1280. Fecha de consulta: 28 de abril de 2025. The Nooscope manifested: AI as instrument of knowledge extractivism | AI & SOCIETY.
Qi, Xiang, Zheng Zhu, Bei Wu, X. 2023. “The promise and peril of ChatGPT in geriatric nursing education: What we know and do not know”. Aging and Health Research, 3: 100-136. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.1016/j.ahr.2023.100136.
Ray, Partha P. 2023. “ChatGPT: A Comprehensive Review on Background, Applications, Key Challenges, Bias, Ethics, Limitations and Future Scope”. Internet of Things and Cyber-Physical Systems 3: 121-154. Fecha de consulta: 15 de febrero de 2025. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope - ScienceDirect.
Scotto, Victoria. 2024. El ChatGPT y el fin de la historia: una elaboración teórica sobre sus peligros y promesas leídos desde la filología. Recial, 15 (25), 118-146. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.53971/2718.658x.v15.n25.45626.
Song, Yifan, Guoyin Wang, Li y Lin. 2024. “The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism”. ArXiv. Última modificación: 15 de julio de 2024. arXiv:2407.10457v1 [cs.CL] 15 Jul 2024.
Stokel-Walker, Chris. “AI bot ChatGPT writes smart essays - should professors worry?”. Nature. Última modificación: 9 de diciembre de 2022. AI bot ChatGPT writes smart essays — should professors worry?.
Sullivan, Miriam, Andrew Kelly, y Paul McLaughlan. 2023. “ChatGPT in Higher Education: Considerations for Academic Integrity and Student Learning”. Journal of Applied Learning & Teaching 6: 1-10. Fecha de consulta: 15 de febrero de 2025. ChatGPT in higher education: Considerations for academic integrity and student learning | Journal of Applied Learning and Teaching.
Thorp, Holden H. 2023. “ChatGPT Is Fun, but Not an Author”. Science 379: 6630 (27 de enero de 2023): 313-313. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.1126/science.adg7879.
Tinmaz, Hasan, Mina Fanea-Ivanovici, Hasnan Baber. 2022. “A snapshot of digital literacy”. Library Hi Tech News 40.1: 20-23. https://doi.org/10.1108/LHTN-12-2021-0095.
Transformer, Chat G. P. T. y Alex Zhavoronkov. 2022. “Rapamycin in the context of Pascal’s Wager: Generative pre-trained transformer perspective”. Oncoscience 9 (82): 82-84. Fecha de consulta: 15 de febrero de 2025. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective.
Vargas-Murillo, Alfonso R., Ilda N. M. A. Pari-Bedoya, y Francisco De Jesús Guevara-Soto. 2023. “Challenges and Opportunities of AI-Assisted Learning: A Systematic Literature Review on the Impact of ChatGPT Usage in Higher Education”. International Journal of Learning, Teaching and Educational Research 22: 122-135. Fecha de consulta: 15 de febrero de 2025. Challenges and Opportunities of AI-Assisted Learning: A Systematic Literature Review on the Impact of ChatGPT Usage in Higher Education | Vargas-Murillo.
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention Is All You Need”. Advances in Neural Information Processing Systems. Fecha de consulta: 15 de febrero de 2025. [1706.03762] Attention Is All You Need.
Voosen, Paul. 2017. “How AI detectives are cracking open the black box of deep learning”. Science. News. 6 de julio de 2017. Fecha de consulta: 15 de febrero de 2025. How AI detectives are cracking open the black box of deep learning | Science.
Xames, Doulotuzzaman y Jannatul Shefa. 2023. “ChatGPT for research and publication: Opportunities and challenges”. Journal of Applied Learning & Teaching 6: 1-6. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.37074/jalt.2023.6.1.20.
Zhang, Peng y Gemma Tur. 2024). “A systematic review of ChatGPT use in K-12 education”. European Journal of Education 59: e12599. Fecha de consulta: 15 de febrero de 2025. https://doi.org/10.1111/ejed.12599.
Zhuk, Alesia. 2023. “Artificial Intelligence Impact on the Environment: Hidden Ecological Costs and Ethical-Legal Issues”. Journal of Digital Technologies and Law 4: 932-954. Fecha de consulta: 15 de febrero de 2025. Artificial Intelligence Impact on the Environment: Hidden Ecological Costs and Ethical-Legal Issues | Zhuk.
- Authors keep the copyright and give the journal the right of the first publication, with the work registered with the Creative Commons Attribution-ShareAlike 4.0 International License, which allows third parties to use what is published whenever they mention the authorship of the work and the first publication in this magazine.
- Authors can make other independent and additional contractual agreements for the non-exclusive distribution of the article published in this journal (eg, include it in an institutional repository or publish it in a book) as long as they clearly indicate that the work It was published for the first time in this magazine.
- Authors are allowed and recommended to publish their work on the Internet (for example on institutional or personal pages).
