Abstract

The persuasive capability of large language models (LLMs) in generating mis/disinformation is widely recognized, but the linguistic ambiguity of such content and inconsistent findings on LLM-based detection reveal unresolved risks in information governance. To address the lack of Chinese datasets, this study compiles two datasets of Chinese AI mis/disinformation generated by multi-lingual models involving deepfakes and cheapfakes. Through psycholinguistic and computational linguistic analyses, the quality modulation effects of eight language features (including sentiment, cognition, and personal concerns), along with toxicity scores and syntactic dependency distance differences, were discovered. Furthermore, key factors influencing zero-shot LLMs in comprehending and detecting AI mis/disinformation are examined. The results show that although implicit linguistic distinctions exist, the intrinsic detection capability of LLMs remains limited. Meanwhile, the quality modulation effects of AI mis/disinformation linguistic features may lead to the failure of AI mis/disinformation detectors. These findings highlight the major challenges of applying LLMs in information governance.

Data availability

The raw Toutiao and MCFEND data are available under restricted access because they may be used to train harmful artificial intelligence systems. Access can be obtained by submitting a handwritten ethical statement, as illustrated in Supplementary Table 12, to the corresponding author. The processed Toutiao and MCFEND data are available at https://github.com/GovAIx/QualityModulation.

Code availability

Code will be made available on request. Experiments in psycholinguistics and computational linguistics were conducted using open-source projects stated in the methods section, and the closed-source LLMs were accessed through provider-supplied APIs. The code required to reproduce the experiments can be obtained from https://github.com/GovAIx/QualityModulation.

References

Farquhar, S. et al. Detecting hallucinations in large language models using semantic entropy. Nature 630, 625–630 (2024).

Google Scholar 1.

Lin, Z. et al. Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models. Artif. Intell. Rev. 57, 243 (2024).

Google Scholar 1.

Kumar A., Kumar P., Yadav A., Ahlawat S. & Prasad Y. KGFakeNet: A Knowledge Graph-Enhanced Model for Fake News Detection. In: Proceedings of the Workshop on Generative AI and Knowledge Graphs (GenAIK) 109-122 (International Committee on Computational Linguistics, 2025). https://aclanthology.org/2025.genaik-1.12. 1.

Liu, N., Zhang, T. & Liang, P. Evaluating Verifiability in Generative Search Engines. 7001-7025 (Association for Computational Linguistics, 2023). https://doi.org/10.18653/v1/2023.findings-emnlp.467. 1.

Venkit, P.N., Laban, P., Zhou, Y., Mao, Y. & Wu, C.-S. Search Engines in the AI Era: A Qualitative Understanding to the False Promise of Factual and Verifiable Source-Cited Responses in LLM-based Search. In: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency 1325–1340 (Association for Computing Machinery, 2025). https://doi.org/10.1145/3715275.3732089. 1.

Jeptoo, K.N. & Sun, C. Enhancing Fake News Detection with Large Language Models Through Multi-agent Debates. In: CCF International Conference on Natural Language Processing and Chinese Computing 474-486 (Springer Nature Singapore, 2025). https://doi.org/10.1007/978-981-97-9434-8_37. 1.

Shah, A. et al. Detecting and unmasking AI-generated texts through explainable artificial intelligence using stylistic features. Int. J. Adv. Computer Sci. Appl. 14, (2023). 1.

Hamed, A. A. & Wu, X. Detection of ChatGPT fake science with the xFakeSci learning algorithm. Sci. Rep. 14, 16231 (2024).

Google Scholar 1.

Su, J. et al. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. In: Findings of the Association for Computational Linguistics: EMNLP 12395-12412. https://aclanthology.org/2023.findings-emnlp.827 (2023). 1.

Mitchell, E., Lee, Y., Khazatsky, A., Manning, C.D. & Finn, C. DetectGPT: Zero-shot machine-generated text detection using probability curvature. In: Proceedings of the 40th International Conference on Machine Learning 24950-24962 (PMLR, 2023). https://proceedings.mlr.press/v202/mitchell23a.html. 1.

Laban, P., Fabbri, A., Xiong, C. & Wu, C.-S. Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems. 9885-9903 (Association for Computational Linguistics, 2024).https://doi.org/10.18653/v1/2024.emnlp-main.552. 1.

Khaliq, M. A., Chang, P. Y.-C., Ma, M., Pflugfelder, B. & Miletić, F. RAGAR. Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models. 280-296 (Association for Computational Linguistics, 2024).https://doi.org/10.18653/v1/2024.fever-1.29. 1.

Chen, C. & Zhang, S. RetrieverGuard: Empowering Information Retrieval to Combat LLM-Generated Misinformation. 4399-4411 (Association for Computational Linguistics, 2025).https://doi.org/10.18653/v1/2025.findings-naacl.249. 1.

Wang, H. & Shu, K. Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models. In: Findings of the Association for Computational Linguistics: EMNLP 6288-6304. https://doi.org/10.18653/v1/2023.findings-emnlp.416 (2023). 1.

Li, X. et al. Large language model-driven multi-agent simulation for news diffusion under different network structures. arXiv preprint arXiv:241013909, (2024). 1.

Crothers, E. et al. Adversarial robustness of neural-statistical features in detection of generative transformers. In: 2022 International Joint Conference on Neural Networks (IJCNN) 1-8 (IEEE, 2022). https://doi.org/10.1109/IJCNN55064.2022.9892269. 1.

McGovern, H. et al. Your Large Language Models Are Leaving Fingerprints. In: Proceedings of the 1stWorkshop on GenAI Content Detection 85-95 (International Conference on Computational Linguistics, 2025). https://aclanthology.org/2025.genaidetect-1.6. 1.

Su, J., Cardie, C. & Nakov, P. Adapting Fake News Detection to the Era of Large Language Models. In: Findings of the Association for Computational Linguistics: NAACL 1473–1490 (2024). 1.

Kuntur, S. et al. Under the influence: a survey of large language models in fake news detection. IEEE Trans. Artif. Intell. 6, 458–476 (2025).

Google Scholar 1.

Clark, E. et al. All That’s ‘Human’Is Not Gold: Evaluating Human Evaluation of Generated Text. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing 7282-7296, https://aclanthology.org/2021.acl-long.565 (2021). 1.

Schlichtkrull, M., Guo, Z. & Vlachos, A. Averitec: A dataset for real-world claim verification with evidence from the web. Adv. Neural Inf. Process. Syst. 36, 65128–65167 (2023).

Google Scholar 1.

Tian, J.-J. et al. Web Retrieval Agents for Evidence-Based Misinformation Detection. In: First Conference on Language Modeling 1-28, https://doi.org/10.48550/arXiv.2409.00009 (2024). 1.

Haupt, M. R. et al. Evaluating the influence of role-playing prompts on ChatGPT’s misinformation detection accuracy: quantitative study. JMIR Infodemiol. 4, e60678 (2024).

Google Scholar 1.

Augenstein, I. et al. Factuality challenges in the era of large language models and opportunities for fact-checking. Nat. Mach. Intell. 6, 852–863 (2024).

Google Scholar 1.

Garimella, K. & Chauchard, S. How prevalent is AI misinformation? What our studies in India show so far. Nature 630, 32–34 (2024).

Google Scholar 1.

Feuerriegel, S. et al. Research can help to tackle AI-generated disinformation. Nat. Hum. Behav. 7, 1818–1821 (2023).

Google Scholar 1.

Shoaib, M.R. et al. Deepfakes, Misinformation, and Disinformation in the Era of Frontier AI, Generative AI, and Large AI Models. In: 2023 International Conference on Computer and Applications (ICCA*)* 1-7 (IEEE, 2023). https://doi.org/10.1109/ICCA59364.2023.10401723. 1.

Spina, D. et al. Human-AI cooperation to tackle misinformation and polarization. Commun. ACM 66, 40–45 (2023).

Google Scholar 1.

Mende, M., Ubal, V. O., Cozac, M., Vallen, B. & Berry, C. Fighting infodemics: labels as antidotes to mis- and disinformation?!. J. Public Policy Mark. 43, 31–52 (2024).

Google Scholar 1.

Caramancion, K.M. Harnessing the Power of ChatGPT to Decimate Mis/Disinformation: Using ChatGPT for Fake News Detection. In: 2023 IEEE World AI IoT Congress (AIIoT) 0042-0046 (IEEE, 2023). https://doi.org/10.1109/AIIoT58121.2023.10174450. 1.

Ahmed, K. et al. Social media’s dark secrets: a propagation, lexical and psycholinguistic oriented deep learning approach for fake news proliferation. Expert Syst. Appl. 255, 124650 (2024).

Google Scholar 1.

Liu, Z. et al. Emotion detection for misinformation: a review. Inf. Fusion 107, 102300 (2024).

Google Scholar 1.

Chaudhari, S. et al. RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs. ACM Comput. Surv. 58, Article 53 (2025). 1.

Bai, Y. et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:220405862, (2022). 1.

Kumar, A. & Taylor, J. W. Feature importance in the age of explainable AI: Case study of detecting fake news & misinformation via a multi-modal framework. Eur. J. Operational Res. 317, 401–413 (2024).

Google Scholar 1.

Li, H. et al. Multi-step Jailbreaking Privacy Attacks on ChatGPT. 4138-4153 (Association for Computational Linguistics, 2023). https://doi.org/10.18653/v1/2023.findings-emnlp.272. 1.

Yu, Z. et al. Don’t listen to me: Understanding and exploring jailbreak prompts of large language models. In: 33rd USENIX Security Symposium (USENIX Security 24) 4675-4692 (2024). https://www.usenix.org/conference/usenixsecurity24/presentation/yu-zhiyuan. 1.

Wu, J. et al. Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. In: Proceedings of the 31st International Conference on Computational Linguistics 10275-10292, https://aclanthology.org/2025.coling-main.684 (2025). 1.

Mindner, L., Schlippe, T. & Schaaff, K. Classification of human-and AI-generated texts: Investigating features for ChatGPT. In: International Conference on Artificial Intelligence in Education Technology 152-170 (Springer Nature Singapore, 2023). https://doi.org/10.1007/978-981-99-7947-9_12. 1.

Lin, Z. et al. ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation. In: Findings of the Association for Computational Linguistics: EMNLP 2023 4694-4702 (Association for Computational Linguistics, 2023). https://aclanthology.org/2023.findings-emnlp.311. 1.

Liu, X., Zhu, H. & Lei, L. Dependency distance minimization: a diachronic exploration of the effects of sentence length and dependency types. Humanit. Soc. Sci. Commun. 9, 420 (2022).

Google Scholar 1.

Wang, L.Z. et al. Megafake: a theory-driven dataset of fake news generated by large language models. arXiv preprint arXiv:240811871 (2024). 1.

Chen, C. & Shu, K. Can llm-generated misinformation be detected? In: The Twelfth International Conference on Learning Representations (ICLR 2024) 1-40 (2024). 1.

Zhou, J., Zhang, Y., Luo, Q., Parker, A.G. & De Choudhury, M. Synthetic lies: Understanding AI-generated misinformation and evaluating algorithmic and human solutions. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems 1-20 (Association for Computing Machinery, 2023). https://doi.org/10.1145/3544548.3581318. 1.

Ayoobi, N., Shahriar, S. & Mukherjee, A. Seeing Through AI’s Lens: Enhancing Human Skepticism Towards LLM-Generated Fake News. In: Proceedings of the 35th ACM Conference on Hypertext and Social Media 1-11 (Association for Computing Machinery, 2024). https://doi.org/10.1145/3648188.3675136. 1.

Simchon, A., Edwards, M. & Lewandowsky, S. The persuasive effects of political microtargeting in the age of generative artificial intelligence. PNAS Nexus 3, 35–40 (2024).

Google Scholar 1.

Wilcox, E. G., Futrell, R. & Levy, R. Using computational models to test syntactic learnability. Linguistic Inq. 55, 805–848 (2024).

Google Scholar 1.

Lan, N., Chemla, E. & Katzir, R. Large Language Models and the Argument from the Poverty of the Stimulus. Linguistic Inquiry, 1-28 (2024). 1.

Mischler, G., Li, Y. A., Bickel, S., Mehta, A. D. & Mesgarani, N. Contextual feature extraction hierarchies converge in large language models and the brain. Nat. Mach. Intell. 6, 1467–1477 (2024).

Google Scholar 1.

Mahowald, K. et al. Dissociating language and thought in large language models. Trends Cogn. Sci. 28, 517–540 (2024).

Google Scholar 1.

Sorscher, B., Ganguli, S. & Sompolinsky, H. Neural representational geometry underlies few-shot concept learning. In: Proceedings of the National Academy of Sciences e2200800119. https://doi.org/10.1073/pnas.2200800119 (2022). 1.

Penedo, G. et al. The refinedweb dataset for Falcon LLM: outperforming curated corpora with web data only. Adv. Neural Inf. Process. Syst. 36, 79155–79172 (2023).

Google Scholar 1.

Myntti, A., Henriksson, E., Laippala, V. & Pyysalo, S. Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation. In The 2nd Conference on Language Modeling 1–24 (COLM, 2025). https://doi.org/10.48550/arXiv.2504.01542. 1.

Ecker, U. K. H. et al. The psychological drivers of misinformation belief and its resistance to correction. Nat. Rev. Psychol. 1, 13–29 (2022).

Google Scholar 1.

Vykopal, I. et al Disinformation Capabilities of Large Language Models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics 14830-14847 (Association for Computational Linguistics, 2024). https://aclanthology.org/.acl-long.7932024. 1.

Wu, J., Guo, J. & Hooi, B. Fake News in Sheep’s Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 3367-3378 (Association for Computing Machinery). https://doi.org/10.1145/3637528.3671977 2024. 1.

Wan, H. et al. DELL: generating reactions and explanations for LLM-based misinformation detection. In: Findings of the Association for Computational Linguistics: ACL 2024 2637-2667 (Association for Computational Linguistics, 2024). https://aclanthology.org/2024.findings-acl.155. 1.

Chen, D., Chen, X., Lu, P., Wang, X. & Lan, X. CNFRD: a few-shot rumor detection framework via capsule network for COVID-19. Int. J. Intell. Syst. 2023, 2467539 (2023).

Google Scholar 1.

Yu, X., Sheng, Z., Lu, W., Luo, X. & Zhou, J. RaCMC: residual-aware compensation network with multi-granularity constraints for fake news detection. In: Proceedings of the AAAI Conference on Artificial Intelligence 986-994 (AAAI, 2025). https://doi.org/10.1609/aaai.v39i1.32084. 1.

Hu, A. & Ou, M. From passive to active: How does algorithm awareness affect users’ news seeking behavior on digital platforms. Telemat. Inform. 100, 102291 (2025).

Google Scholar 1.

Li, Y., He, H., Bai, J. & Wen, D. MCFEND: A Multi-source Benchmark Dataset for Chinese Fake News Detection. In: Proceedings of the ACM Web Conference 2024 4018-4027 (Association for Computing Machinery, 2024). https://doi.org/10.1145/3589334.3645385. 1.

Sosnowski, W., Modzelewski, A., Skorupska, K., Otterbacher, J. & Wierzbicki, A. EU DisinfoTest: a Benchmark for Evaluating Language Models’ Ability to Detect Disinformation Narratives. In: Findings of the Association for Computational Linguistics: EMNLP 2024 14702-14723 (Association for Computational Linguistics, 2024). https://aclanthology.org/2024.findings-emnlp.862. 1.

Xiong, L. et al. DeepSeek: paradigm shifts and technical evolution in large AI models. IEEE/CAA J. Autom. Sin. 12, 841–858 (2025).

Google Scholar 1.

Liang, W., Wu, H., Gao, J., Shang, Y. & He, Z. SafeLLMs: A Benchmark for Secure Bilingual Evaluation of Large Language Models. In: CCF International Conference on Natural Language Processing and Chinese Computing 437-448 (Springer Nature Singapore, 2025). https://doi.org/10.1007/978-981-97-9434-8_34. 1.

Du, X. et al. Supergpqa: Scaling LLM evaluation across 285 graduate disciplines. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track 1–74 (NeurIPS, 2025). https://neurips.cc/virtual/2025/poster/121825. 1.

Peters, U. & Chin-Yee, B. Generalization bias in large language model summarization of scientific research. R. Soc. Open Sci. 12, 241776 (2025).

Google Scholar 1.

Xu, Z. et al. Magpie: Alignment data synthesis from scratch by prompting aligned LLMs with nothing. In: The Thirteenth International Conference on Learning Representations: ICLR 1-32 (2025). https://doi.org/10.48550/arXiv.2406.08464. 1.

He, H. & Choi, J.D. The stem cell hypothesis: dilemma behind multi-task learning with transformer encoders. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing 5555-5577 (Association for Computational Linguistics, 2021). https://aclanthology.org/2021.emnlp-main.451.

Download references

Acknowledgements

X.Z. acknowledges the funding support from the Social Science Foundation of Shaanxi Province (No. 2024R055), and the Shaanxi Province Key Industrial Innovation Chain (Group) Project in Industrial Domain (No. 2022ZDLGY06-04). M.W. acknowledges the funding support from the Natural Science Basic Research Program of Shaanxi Province (No. 2025JC-YBMS-1100).

Author information

Authors and Affiliations

School of Management, Xi’an University of Architecture and Technology, Xi’an, Shaanxi, China

Yulong Ma, Xinsheng Zhang, Jinge Ren & Minghu Wang 1.

School of Economics and Management, Xi’an University of Posts and Telecommunications, Xi’an, Shaanxi, China

Runzhou Wang 1.

College of Humanities, Zhejiang Normal University, Jinhua, Zhejiang, China

Yang Chen

Authors

1. Yulong Ma
2. Xinsheng Zhang
3. Jinge Ren
4. Runzhou Wang
5. Minghu Wang
6. Yang Chen

Contributions

Y.M. is the first author, X.Z. is the second and corresponding author of this article. X.Z. and M.W. acquired funding for the studies. Y.M. and J.R. conceptualized the study. Y.M. drafted the article. Y.C. provided feedback for the experiment. X.Z., J.R., R.Z. and M.W. all contributed to the review and revision process, improving the writing, data interpretation, and literature analysis.

Corresponding author

Correspondence to Xinsheng Zhang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Communications thanks Antonio Moreno-Ortiz, Chun-Ming Lai and the other anonymous reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Cite this article

Ma, Y., Zhang, X., Ren, J. et al. Linguistic features of AI mis/disinformation and the detection limits of LLMs. Nat Commun (2025). https://doi.org/10.1038/s41467-025-67145-1

Download citation

Received: 19 July 2025

Accepted: 24 November 2025

Published: 11 December 2025

DOI: https://doi.org/10.1038/s41467-025-67145-1