Pipeline weld inspection looks, on the surface, like exactly the kind of task modern vision models should be good at: look at an image, find the defect. We tested that assumption directly, and the result is a big part of why we built Fathom Seam the way we did.
The test
We ran five frontier vision models against full weld radiograph films, the actual images inspectors use to certify a weld under API 1104. The task was straightforward to describe: find the defects. The models scored between 0% and 4.1% recall.
That is not a rounding error or a hard-mode benchmark. It means, on average, these models missed almost every defect present on a full film. A general-purpose vision model, however capable it is at describing a photo or reading a chart, is not built for the specific, high-resolution, low-signal pattern recognition that weld defect detection requires. A full film is enormous relative to the size of the defects that matter, and the failure modes, cracks, slag, porosity, are subtle enough that a model trained on general imagery has no real basis for finding them reliably.
What actually works: measure, then decide
Fathom Seam does not ask a single model to look at a whole film and guess. The architecture separates the job into three steps: classification finds candidate regions, physics-based measurement characterizes what is actually there, and a deterministic API 1104 ruleset makes the accept or reject decision. No single component is asked to do more than it is reliable at.
The result is 91.2% per-pore recall, at the region level 98.7%, with candidate reduction of 82.4% to 95.5%, meaning the system narrows a full film down to a small set of regions worth an inspector's attention rather than asking a human to scan the whole thing cold. A band-first triage approach speeds up review by roughly 11.95x over an unassisted pass.
Fail-closed by design
Seam runs fully offline, with no per-weld cost tied to a hosted model, and it fails closed: anything the system is not confident about gets routed to a human auditor rather than silently passed or silently rejected. In an industry where a missed defect in a pipeline weld is a safety and liability event, "confident and wrong" is a far worse failure mode than "uncertain and slow." Fail-closed is a design decision, not a limitation.
The case that proves why this matters
One film had already been accepted by a human inspector. Seam flagged it. On review, the film showed 5 missed slag voids across 68 inches of weld, defects a trained human had passed on a first review. That is not a claim about human inspectors being unreliable in general. It is the exact scenario a second, physics-grounded pass exists to catch: a low-frequency, high-consequence miss that a purely visual, first-pass review can let through.
What this means for an operator or engineering partner
The 0% to 4.1% frontier model result is the reason general-purpose AI is not a shortcut to reliable weld inspection. The 91.2% per-pore recall and the fail-closed architecture are the reason a purpose-built, physics-grounded system is a different category of tool entirely, one built to catch what a single pass, human or model, can miss.
Want this cut for your territory? Book a demo.