Information provenance and conflicting sources

The research system delivers a neat report with cited figures. Someone checks a citation and the source does not say that. Another figure is attributed to a report that never published it. At some step, the data got separated from its origin.

Where the source gets lost

Every time a subagent summarises in prose, it compresses the findings and the attribution falls off: “the market grew 12%” survives, “according to report X, page 4, with 2023 data” does not. Later, the synthesis cites whole websites or the wrong source.

Provenance is preserved with claim-source mappings: subagents do not return prose, they return structured records with

• the claim;
• the URL or document name;
• the exact excerpt that supports it;
• the publication or data collection date.

The synthesis has to preserve and merge those links, not rewrite them. A claim that arrives without a source does not enter the report as fact: it is left out or marked as unverified until its origin is found.

When two sources disagree

Two credible sources give different figures for the same thing. What you do not do is pick one at random or average them: both invent or hide information. You make a conflict annotation: both figures, each with its source and date, and the conflict flagged.

Who resolves it? The analysis subagent finishes its work with both values included and annotated, and lets the coordinator decide how to reconcile them before synthesis. If the subagent silently picks one, the coordinator, which may have had the context to see that the sources used different definitions, never finds out.

One nuance: if they cover the same period with the same definition and one is official and the other a blog, the official one prevails and the other is shown as unverified.

Differences over time

One survey says 34% and another 51%. The synthesis calls it a contradiction, but one is from 2021 and the other from 2024: it is a trend. Requiring the publication date and data collection date in the subagents’ output prevents a temporal difference from being mistaken for a conflict.

The synthesis also cannot change the time frame or the certainty of what the source says. If an analyst wrote in 2023 that they “expect” something, the report cannot present it as a forecast for next year.

How the report is built

• Settled separated from disputed. Explicit sections for widely supported findings and for contested findings, keeping how the original source characterises them and their methodological context (“preliminary”, “pilot with 40 patients”). Otherwise a small pilot shows up with the same certainty as a consolidated result.
• Each content type in its format. In a multi-source synthesis, financial data goes in tables, news in prose and technical findings in structured lists. Turning everything into a single bulleted list makes the report harder to use.

The typical traps

• Subagents that return prose summaries without sources.
• Picking or averaging a figure between conflicting sources.
• Calling data from different years a contradiction.
• Rewriting what the source says with more certainty or a different time horizon.
• A single format for the whole report.

On the exam

If the report miscites, the answer is for subagents to return claim, source, excerpt and date. If there are different figures, show both with their source and let the coordinator decide. If the figures are from different years, require dates. If the problem is presentation, separate settled from disputed or use each content type’s format.