Narratives as data

A narrative is what an organisation or a person says about itself, repeatedly, across channels: in interviews, on its website, in press releases, in job advertisements, in filings. Read one statement at a time, it is text. Read across thousands of statements and several years, it becomes a dataset with structure, a time dimension and an audience dimension. This note describes how statements can be turned into comparable records, what can then be measured, and where the method fails.

The unit: a claim, not a document

Documents are the wrong unit of analysis. An hour-long interview contains dozens of distinct statements, on different subjects, with different degrees of certainty. The useful unit is the claim: a single assertion, attributed to a speaker, at a point in time, in a given source.

A claim can be represented as a structured record:

• speaker: the resolved person or organisation making the statement;
• subject: the entity the statement is about;
• predicate and object: what is asserted, for example “headcount = 120”, “is independent”, “will launch in Germany”;
• type: numerical, positioning, forward-looking or biographical;
• modality: asserted, hedged, conditional, denied;
• time: when it was said, and the period it refers to, which are often different;
• channel and audience: interview, filing, recruitment page, investor presentation, social post;
• provenance: document, page or timestamp, and the exact span of text or speech;
• confidence: how certain the extraction is, for each field.

Every downstream comparison depends on this representation. A claim without a speaker cannot be attributed; a claim without a reference period cannot be compared with a filing; a claim without provenance cannot be checked.

From speech and text to claims

The extraction pipeline has three stages.

Attribution comes first. For audio and video, speaker diarisation segments a recording by voice; the segments are then linked to identities using introductions, captions and context. For text, the speaker is usually the author or the person quoted, which requires quotation detection. Attribution errors are the most damaging errors in the pipeline, because they assign a statement to the wrong person.

Segmentation and extraction follow. Long passages are split into self-contained statements, and each statement is mapped to the claim schema. Language models can do this reliably when constrained: a fixed schema, typed fields, an obligation to quote the supporting span, and a refusal option when the passage contains no claim. Unconstrained summarisation is not extraction; it produces paraphrases that cannot be traced.

Normalisation closes the loop. Coreference resolution links “we”, “the company” and its legal name to the same canonical entity. Numbers are parsed with their units and scope (“about 120 people”, “over a hundred engineers”). Relative time expressions such as “last year” or “since we started” are resolved against the date of the statement.

Kinds of claims

Claims differ in how they can be checked.

• Numerical claims (revenue, headcount, customers, funding) are compared with filed values, provided period, scope and definition match.
• Positioning claims (“independent”, “profitable”, “European”) are compared with structural facts: ownership, accounts, registered addresses.
• Forward-looking claims (“we will launch”, “we expect to break even”) are neither true nor false when made; they are compared later with what happened.
• Biographical claims (past roles, previous companies) are compared with registries, archives and earlier versions of the same biography.

They also differ in who makes them. First-party claims are what an entity says about itself. Third-party claims are what others say about it: the press, analysts, employees in reviews, counterparties in court. The two are extracted with the same schema but kept distinct, because the most informative comparisons are often between them: what an organisation says, what others say about it, and what it has filed.

Grouping claims into narratives

A narrative is a cluster of claims that express the same idea in different words over time. Grouping them is a semantic problem.

The standard approach embeds each claim as a vector and clusters the vectors, so that “we are fully independent”, “no outside shareholder controls us” and “we answer to no investor” fall together. Topic-modelling methods built on embeddings, such as BERTopic, combine clustering with a readable description of each cluster. The output is a set of narratives per speaker, each with its member claims, its first and last appearance, and its frequency over time.

Embeddings have a known blind spot: polarity. “We are independent” and “we are not independent” sit close together in vector space, because they share almost every word and topic. Clustering alone will place a claim and its negation in the same narrative, and a contradiction will be counted as repetition. Each cluster therefore needs a second pass that separates stance, using the modality field or an inference model, before frequencies are computed.

Two further parameters matter. Granularity: too coarse, and distinct narratives merge; too fine, and one narrative splits into paraphrases. Stability: as new statements arrive, clusters should be updated without reshuffling everything already assigned.

Change over time

With claims timestamped and grouped, the time dimension becomes measurable.

Narrative lifecycle: when a narrative first appears, how often it is repeated, when it fades, and whether it disappears abruptly, particularly when that coincides with a filing, a departure or a rewritten web page.

Drift within a narrative: the same idea expressed with changing strength or content. “We will be profitable next year”, repeated for several years, is a stable narrative whose reference period keeps moving. Tracking the referenced period, not just the wording, makes this visible.

Numerical trajectories: the same quantity quoted over time forms a series that can be compared with the filed series for the same periods.

Differences between audiences

The audience dimension compares what is said to whom. The same organisation describes itself differently to investors, customers, recruits and regulators, and some difference is normal: emphasis changes with the reader.

The meaningful differences are those that cannot both be true at the same time. Growth described as accelerating to investors while job advertisements are withdrawn and filings show a falling headcount. A product described as available to customers while recruitment pages describe it as still in development. Measuring this requires aligning claims by subject and period across channels before comparing them.

Change or contradiction

Two claims that differ are not necessarily in contradiction. “We have 80 employees” in 2021 and “we have 120 employees” in 2023 describe a change. A contradiction requires the same subject, the same predicate and the same reference period: two different headcounts for the same year, or an “independent” claim made while a filing shows a controlling shareholder on that date. Confusing change with contradiction is the main source of false alarms, which is why the reference period is a required field and not an optional one.

Detection then combines two techniques. For text, natural language inference models classify a pair of statements as entailment, contradiction or neutral; they are effective on clearly phrased pairs and weaker on hedged ones, so they are best used to rank candidates rather than to decide. For numbers and structure, rules do better: a quoted figure is compared with the filed figure for the same scope and period, within a tolerance.

Comparing every claim with every other is quadratic. Candidates are generated by blocking on subject, predicate type and reference period, so that only claims about the same thing at the same time are compared.

Evaluation

Each stage needs its own measure.

• Attribution: diarisation error rate on audio, and attribution accuracy on a hand-checked sample of quotes.
• Extraction: precision and recall of claims against passages annotated by hand, with a separate check that modality and reference period are preserved.
• Grouping: cluster coherence as judged by annotators, and the rate at which opposite stances are wrongly merged.
• Contradictions: precision among the highest-ranked candidates, since that is what an analyst reviews first.

Without these measures, a narrative analysis is an impression with better formatting.

Failure modes

• Attribution errors: a statement assigned to the wrong speaker, often in panel discussions or quoted speech.
• Lost modality: “we hope to” or “if all goes well” stripped during extraction, turning an intention into an assertion.
• Scope mismatch: a group figure compared with a subsidiary’s, or a calendar year with a fiscal year.
• Translation effects: nuance lost or added when statements in different languages are compared through translation.
• Extraction drift: a model paraphrasing instead of extracting. The defence is to require the supporting span for every claim and to reject claims whose span does not contain what is asserted.
• Coverage gaps: archives capture websites at irregular intervals, and recordings disappear. A narrative that seems to stop may simply have stopped being captured. Absence from the record is not absence of the statement.

What this makes possible

Treated as data, a narrative stops being an impression. It becomes a set of records that can be counted, dated, grouped and compared with everything else in the public record: what was said, to whom, when, how often, what others said, and whether it held.