AI, Large Language Models, and the Limits of Narrative Analytics

AI has revolutionised the way that we work and live in todays society, but in the world of narrative data analytics the reliance on LLM's comes with a health warning.

LLMs are designed to generate plausible language, not to preserve meaning.

This distinction matters. In narrative analysis, the challenge is not producing insight-like text but holding meaning stable enough to measure, compare, and act on over time.

Accuracy and Reliability

Generative AI can produce outputs that appear authoritative while containing subtle errors or distortions. If models are trained on biased or incomplete data, those biases are reproduced at scale. Crucially, probabilistic language models cannot reliably convert narrative into stable, repeatable metrics. Without expert governance, this leads to inconsistent results and false confidence.

Context, Correlation, and Causation

LLMs struggle to understand human intention, behaviour, and lived experience. They detect patterns and correlations but cannot reliably distinguish cause from coincidence or integrate cross-domain understanding without human constraint. This limits their usefulness for qualitative and behavioural insight.

 

Stability Over Time

Because LLMs are probabilistic and continually updated, the same narrative can produce different interpretations across runs, models, or versions. This makes them unsuitable as the primary engine for longitudinal analysis, benchmarking, or safety-critical decision making.

Transparency and Auditability

LLMs operate largely as black boxes. Their outputs cannot be traced back in a transparent, auditable way to specific words or phrases in the source data. This undermines trust, reliability, and regulatory confidence.

Ethics, Cost, and Oversight

AI systems carry risks around privacy, bias, and compliance, particularly under regulations such as GDPR. They also demand significant computational resources and ongoing supervision. Human oversight is not optional. It is essential.

How Akumen Reliably Analyses Stories and Narrative Data

Akumen starts where generative AI cannot. We uncover what people are really trying to say, then align that understanding to the purpose of the organisation or research. This immediately directs attention to the relevant why information — the reasons, drivers, risks, and lived experiences behind the words.

Meaning is never guessed or generated. Every meaningful phrase is mapped to a theme defined by the data itself, using transparent, human-governed ontologies that prevent drift and hallucination (not fabricating answers).

Once meaning is structured, measurement becomes possible. Akumen transforms raw narrative into precise, stable, and repeatable meaning-driven metrics. Our ontologies evolve carefully, but once stabilised they do not fundamentally change, ensuring long-term consistency. Trends, benchmarks, and meta-metrics remain fully traceable to the original language.

This is what it means to make sense and to make a difference. Akumen converts unstructured meaning into structured insight, and structured insight into transparent, reproducible metrics that organisations can trust and act on with confidence.

To talk with us more on this subject email Eross@akumen.co.uk. We would also love to hear your opinions and perspectives!