Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models
Oct 1, 2024·,,,,,,,,,,·
0 min read
James Chapman
Xinyi Leng
Jason Liang
Jack Mauro
Xu Wang
Andrea L. Bertozzi
Junyuan Lin
Bohan Chen
Chenchen Ye
Temple Daniel
P. Jeffrey Brantingham
Abstract
Narrative data spans all disciplines and provides a coherent model of the world to the reader or viewer. Large Language Models (LLMs) have advanced natural language analysis but still struggle with complex, conflicting narrative arcs. This work analyzes true-crime podcast data (Serial) using knowledge graphs (KGs) with both classical NLP and LLM approaches, and directly compares KG-augmented LLMs (KGLLMs) with classical methods for KG construction, topic modeling, and sentiment analysis. The KGLLM enables natural-language querying of the knowledge base, factual Q&A, and robustness testing under adversarial prompting. Results indicate KGLLMs outperform standard LLMs on multiple metrics, show greater robustness to adversarial prompts, and better summarize text into topics.
Type
Publication
Proceedings of ACM Conference (GTA3 Workshop at the 33rd ACM International Conference on Information and Knowledge Management, 2024)