Agentic Detection of Online Conspiracies

What it is about

The authors ask whether a Hebrew tweet endorses a conspiracy theory, or only mocks, quotes or reports it. They argue that this is a question about the speaker's intent, so the surrounding context matters. Their data is a Hebrew tweet archive of about 270 million tweets from 2018 to 2023. From it they build an adversarial set of 33,121 candidates, of which 504 were labelled by two experts (331 non-conspiracy, 173 conspiracy). An LLM agent can call four tools: tweet metadata, author profile, the user's recent posts, and the retweet ego network. Text-only classification reaches F1 0.535. The agent with all tools reaches accuracy 0.790 and F1 0.730, and it beats the same context pasted into the prompt (F1 0.67). False positives fall from 142 to 76 and false negatives from 58 to 30. The agent reads about 1.5 times more input characters than the prompt-only version. A two-agent debate is weaker, at F1 0.558.

Ideas

• I1. A speaker-commitment flag in prag: when a sentence quotes a claim and then ridicules it ("the storm is Pfizer's plot, there is no hope for you"), mark the claim as reported or mocked, not as asserted. The paper's own error classes give the test cases: strong criticism of a government versus a conspiracy claim, and a quote that is mocked.
• I2. A tool-economy gate in the world reader: consult outside context only when the sentence's reading is unresolved. The paper finds that 10.9% of cases use no tool, and in those the agent agrees with text-only classification 94.5% of the time. This is a measurable rule for when the world needs more evidence.
• I3. Use the paper's error categories as a checklist of hard pragmatic cases for prag, written in our own examples. The labelled set is non-commercial, so it stays a measurement, never part of the build.

For Mova

I1 goes into the pragmatics rules of prag (the speaker-commitment part of the frame). I2 goes into world, as a rule for when to read further context. I3 is a test list for prag, written in our own words and examples. Nothing from the dataset is carried into a release.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2609.30250

@misc{biton2026agentic,
  title = {{Agentic Detection of Online Conspiracies}},
  author = {Biton, Lior and Tsur, Oren},
  year = {2026},
  eprint = {2609.30250},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.30250}
}

References

50 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allen, J.; Watts, D. J.; and Rand, D. G. 2024. Quantifying the impact of misinformation and vaccine-skeptical content on Facebook. Science, 384(6699): eadk3451. find
2. Austin, J. L. 1975. How to do things with words, volume 5. Harvard university press. find
3. Bai, L.; Huang, Z.; Wang, X.; Sun, J.; Mihalcea, R.; Brynjolfsson, E.; Pentland, A.; and Pei, J. 2026. How do ai agents spend your money? analyzing and predicting token consumption in agentic coding tasks. arXiv preprint arXiv:2604.22750. ↗
4. Bergemann, D.; Bonatti, A.; and Smolin, A. 2025. The Economics of Large Language Models: Token Allocation, Fine-Tuning, and Optimal Pricing. In Proceedings of the 26th ACM Conference on Economics and Computation. find
5. Bian, T.; Xiao, X.; Xu, T.; Zhao, P.; Huang, W.; Rong, Y.; and Huang, J. 2020. Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 549–556. find
6. Caspit, B. 2015. [Survey on Conspiracy Beliefs Concerning the Assassination of Yitzhak Rabin]. Maariv. Hebrew-language news article. find
7. Castillo, C.; Mendoza, M.; and Poblete, B. 2011. Information Credibility on Twitter. In Proceedings of the 20th International Conference on World Wide Web, 675–684. New York, NY, USA: Association for Computing Machinery. find
8. Chase, H. 2022. LangChain. Software. find
9. Corso, F.; Pierri, F.; and De Francisci Morales, G. 2025. Conspiracy Theories and Where to Find Them on TikTok. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8346–8362. Vienna, Austria: Association for Computational Linguistics. find
10. Del Vicario, M.; Bessi, A.; Zollo, F.; Petroni, F.; Scala, A.; Caldarelli, G.; Stanley, H. E.; and Quattrociocchi, W. 2016. The Spreading of Misinformation Online. Proceedings of the National Academy of Sciences, 113(3): 554–559. find
11. Diab, A.; Nefriana, R.; and Lin, Y.-R. 2024. Classifying Conspiratorial Narratives at Scale: False Alarms and Erroneous Connections. Proceedings of the International AAAI Conference on Web and Social Media, 18(1): 340–353. find
12. DiResta, R. 2024. Invisible Rulers: The People Who Turn Lies into Reality. New York: PublicAffairs. find
13. Douglas, K. M.; and Sutton, R. M. 2023. What are conspiracy theories? A definitional approach to their correlates, consequences, and communication. Annual review of psychology, 74(1): 271–298. find
14. Douglas, K. M.; Sutton, R. M.; and Cichocka, A. 2017. The Psychology of Conspiracy Theories. Current Directions in Psychological Science, 26(6): 538–542. find
15. Douglas, K. M.; Uscinski, J. E.; Sutton, R. M.; Cichocka, A.; Nefes, T.; Ang, C. S.; and Deravi, F. 2019. Understanding Conspiracy Theories. Political Psychology, 40(S1): 3–35. find
16. Enders, A. M.; Klofstad, C. A.; and Uscinski, J. E. 2024. The Relationship Between Conspiracy Theory Beliefs and Political Violence. Harvard Kennedy School Misinformation Review, 5(6). find
17. Fort, M.; Tian, Z.; Gabel, E.; Georgiades, N.; Sauer, N.; Dakota, D.; and Kübler, S. 2023. Bigfoot in Big Tech: Detecting Out of Domain Conspiracy Theories. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, 353–363. Varna, Bulgaria: INCOMA Ltd., Shoumen, Bulgaria. find
18. George, A. R.; Ahrens, M.; Pierrehumbert, J. B.; and McMahon, M. 2024. Conspiracy Detection Beyond Text: Exploring the Feasibility of Adding Psycho-Linguistic Features to Enhance Conspiracy Detection Models. In Disinformation in Open Online Media, volume 15175 of Lecture Notes in Computer Science, 32–45. Cham: Springer. find
19. Grinberg, N.; Joseph, K.; Friedland, L.; Swire-Thompson, B.; and Lazer, D. 2019. Fake News on Twitter During the 2016 U.S. Presidential Election. Science, 363(6425): 374–378. find
20. Han, C.; Zheng, W.; and Tang, X. 2025. Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 15114– 15129. Suzhou, China: Association for Computational Linguistics. find
21. Hilberts, S.; Govers, M.; Petelos, E.; and Evers, S. 2025. The impact of misinformation on social media in the context of natural disasters: narrative review. JMIR infodemiology, 5: e70413. find
22. Jackson, S. J.; Bailey, M.; and Welles, B. F. 2020. # HashtagActivism: Networks of race and gender justice. Mit Press. find
23. LangChain, Inc. 2024. LangGraph. Software. find
24. Langguth, J.; Schroeder, D. T.; Filkuková, P.; Brenner, S.; Phillips, J.; and Pogorelov, K. 2023. COCO: An Annotated Twitter Dataset of COVID-19 Conspiracy Theories. Journal of Computational Social Science, 6(2): 443–484. find
25. Lei, Y.; and Huang, R. 2023. Identifying Conspiracy Theories News based on Event Relation Graph. In Findings of the Association for Computational Linguistics: EMNLP 2023, 9811–9822. Singapore: Association for Computational Linguistics. find
26. Li, H.; Wang, A.; Li, K.; Wang, Z.; Zhang, L.; Qiu, D.; Liu, Q.; and Su, J. 2025. A Multi-Agent Framework with Automated Decision Rule Optimization for Cross-Domain Misinformation Detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 5709–5725. Suzhou, China: Association for Computational Linguistics. find
27. Li, Q.; Zhang, Q.; and Si, L. 2019. Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 1173–1179. Florence, Italy: Association for Computational Linguistics. find
28. Li, X.; Zhang, Y.; and Malthouse, E. C. 2024. Large Language Model Agent for Fake News Detection. arXiv preprint arXiv:2405.01593. ↗
29. Liang, T.; He, Z.; Jiao, W.; Wang, X.; Wang, Y.; Wang, R.; Yang, Y.; Shi, S.; and Tu, Z. 2024. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 17889–17904. Miami, Florida, USA: Association for Computational Linguistics. find
30. Liaw, S. Y.; Huang, F.; Benevenuto, F.; Kwak, H.; and An, J. 2023. YouNICon: YouTube’s CommuNIty of Conspiracy Videos. Proceedings of the International AAAI Conference on Web and Social Media, 17(1): 1102–1111. find
31. Liu, Y.; Liu, Y.; Zhang, X.; Chen, X.; and Yan, R. 2025. The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake News. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, 504–514. New York, NY, USA: Association for Computing Machinery. find
32. Liu, Z.; Liu, B.; Thompson, P.; Yang, K.; and Ananiadou, S. 2024. ConspEmoLLM: Conspiracy Theory Detection Using an Emotion-Based Large Language Model. arXiv preprint arXiv:2403.06765. ↗
33. Ma, J.; Gao, W.; Mitra, P.; Kwon, S.; Jansen, B. J.; Wong, K.-F.; and Cha, M. 2016. Detecting Rumors from Microblogs with Recurrent Neural Networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 3818–3824. AAAI Press. find
34. Mahl, D.; Schäfer, M. S.; and Zeng, J. 2023. Conspiracy theories in online environments: An interdisciplinary literature review and agenda for future research. New media & society, 25(7): 1781– 1801. find
35. Muhammed, S. T.; and Mathew, S. K. 2022. The disaster of misinformation: a review of research in social media. International Journal of Data Science and Analytics, 13(4): 271–285. find
36. Phillips, S. C.; Ng, L. H. X.; and Carley, K. M. 2022. Hoaxes and Hidden Agendas: A Twitter Conspiracy Theory Dataset: Data Paper. In Companion Proceedings of the Web Conference 2022, 876– 880. New York, NY, USA: Association for Computing Machinery. find
37. Pustet, M.; Steffen, E.; and Mihaljević, H. 2024. Detection of Conspiracy Theories Beyond Keyword Bias in German-Language Telegram Using Large Language Models. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), 13–27. Mexico City, Mexico: Association for Computational Linguistics. find
38. Qazvinian, V.; Rosengren, E.; Radev, D. R.; and Mei, Q. 2011. Rumor Has It: Identifying Misinformation in Microblogs. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 1589–1599. Edinburgh, Scotland, UK: Association for Computational Linguistics. find
39. Ruchansky, N.; Seo, S.; and Liu, Y. 2017. CSI: A Hybrid Deep Model for Fake News Detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 797– 806. New York, NY, USA: Association for Computing Machinery. find
40. Schick, T.; Dwivedi-Yu, J.; Dessı̀, R.; Raileanu, R.; Lomeli, M.; Hambro, E.; Zettlemoyer, L.; Cancedda, N.; and Scialom, T. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems, volume 36, 68539–68551. Curran Associates, Inc. find
41. Shahsavari, S.; Holur, P.; Wang, T.; Tangherlini, T. R.; and Roychowdhury, V. 2020. Conspiracy in the Time of Corona: Automatic Detection of Emerging COVID-19 Conspiracy Theories in Social Media and the News. Journal of Computational Social Science, 3(2): 279–317. find
42. Shi, F.; Chen, X.; Misra, K.; Scales, N.; Dohan, D.; Chi, E. H.; Schärli, N.; and Zhou, D. 2023. Large language models can be easily distracted by irrelevant context. In Proceedings of the 40th International Conference on Machine Learning, 31210–31227. PMLR. find
43. Spanakis, P.; Lymperaiou, M.; Filandrianos, G.; Voulodimos, A.; and Stamou, G. 2026. AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), 1175–1204. San Diego, California, USA: Association for Computational Linguistics. find
44. Starbird, K. 2017. Examining the Alternative Media Ecosystem Through the Production of Alternative Narratives of Mass Shooting Events on Twitter. In Proceedings of the International AAAI Conference on Web and Social Media, volume 11, 230–239. find
45. Starbird, K.; Arif, A.; and Wilson, T. 2019. Disinformation as Collaborative Work: Surfacing the Participatory Nature of Strategic Information Operations. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW): 1–26. find
46. Steffen, E. 2025. More than Memes: A Multimodal Topic Modeling Approach to Conspiracy Theories on Telegram. Proceedings of the International AAAI Conference on Web and Social Media, 19(1): 1831–1844. find
47. Vosoughi, S.; Roy, D.; and Aral, S. 2018. The Spread of True and False News Online. Science, 359(6380): 1146–1151. find
48. World Economic Forum. 2026. find
49. The Global Risks Report 2026. Technical report, World Economic Forum, Geneva, Switzerland. find
50. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; and Cao, Y. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations. find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/langstream-2026-biton-agentic-detection-online-conspiracies/