Categories
Experiments

Assessing an Assessment

Using LLMs and formalisation to closely examine a finished intelligence analytical product

Version

Originally published: 13 September 2026.
Revised: 11 October 2026.

v1.1 — An interactive account of the audit and formal representation. The full audit manuscript remains unchanged.

What can a reader evaluate in a finished intelligence assessment when its supporting reporting and production records are unavailable? The CIA’s January 1981 memorandum Kashmir: A Simmering Trouble Spot provided the test case. A project-developed audit procedure examined its judgments, attribution, uncertainty and reasoning. The audit showed where the memorandum explains its judgments and where its wording leaves questions open. Some passages explain why a judgment was made or state when it would hold. Elsewhere, it is unclear how the surrounding statements lead to the judgment. For selected major judgments, the released copy does not show the intelligence reports they drew on. Formalisation made selected relationships available for inspection and checking.

Choose a stage to explore the work.


Discover more from Tarqeq Research

Subscribe to get the latest posts sent to your email.