The investigation on this site was not produced by a single prompt. It was produced by a person managing an AI agent across a long, open-ended analytical project. This is a study of how that collaboration worked, and what it suggests about the real division of labor between a human director and a capable agent.
The popular image of working with an AI is a question in, an answer out. That image is misleading for any task with real depth. The investigation behind this site, multiple analyses, a document corpus, a published outcome, took shape over an extended collaboration in which a human made the consequential decisions and the agent executed, drafted, measured, and surfaced what it found.
The agent supplied breadth, consistency, and tirelessness. It could profile every document the same way every time, build interactive tools, and never lose the thread. What it could not supply was judgment about what mattered: which hypothesis was worth testing, when an answer was good enough, when the whole frame should change. That stayed with the human. The quality of the result tracked the quality of that management.
The project was never one task. It was a sequence, and the human set the direction at every handoff, which is what kept a long, open-ended investigation from drifting.
No single stage was hard to ask for. The skill was in the sequencing: knowing that visualization should follow a hypothesis, that a stress-test should follow the supporting case, and that synthesis should come last. That ordering was a human contribution, not an automatic one.
The pattern is worth naming. The agent was strongest where the task was large but well-defined. It was weakest, and needed the human most, where the task was deciding what to do at all. Management was not supervision of the agent's accuracy line by line. It was ownership of the questions the agent could not ask for itself.
One management decision mattered more than any other. After the agent had built a persuasive case for the leading hypothesis, the human did not move on. The human asked the agent to build the strongest possible case against it.
This matters because a capable agent will always produce a fluent, confident, well-organized case for whatever it is pointed at. That fluency is seductive and it is dangerous. Left to confirm, an agent confirms. The counter-hypothesis was the antidote: it forced the same evidence through four rival explanations and produced a scorecard showing that most of the "supporting" evidence did not actually discriminate between them at all.
The transferable principle: an AI agent's output should be treated as a draft argument, never as a finding. The human's highest-value act is to make the agent try, in good faith, to break the hypothesis rather than prove it.
Two examples from the August 2026 release, both of which made the collection weaker and were kept anyway.
A case that closed. In November 1963 a Brazilian radio station reported a large metal sphere fallen at Conde with a body inside. The claim reached a cabinet-level White House office within days. Twelve days after the broadcast, a US consul reported the story was “apparently fabricated in Rio,” and forwarded a Brazilian newspaper headline reading “The Airship that wasn't there.” The government released the disproof alongside the claim. Both are catalogued here, rated contested, and grouped under a heading that says plainly what happened to them. A corpus that only displays its unexplained cases is not reporting; it is selecting.
A quote we could not verify. The most quotable line in the 1964 Puerto Rico file is a naval aviator describing a “delta-shaped” object outrunning his Crusader. The underlying CIA typescript is degraded to the point that machine transcription of it is unreliable. Rather than publish a reconstructed quote that reads well, the record carries an explicit flag: verify against the source scan, and take the wording from the companion briefing document instead. There are a handful of these flags in the collection. They are the honest cost of working from scans.
Three habits, applied consistently, did most of the protective work on the reasoning. They did not protect the bookkeeping, and the section below this one says what that cost.
Every piece of evidence was asked one question: does this actually distinguish between explanations, or is it consistent with all of them? Evidence consistent with everything proves nothing. Applying that test reclassified half the "strong" evidence as neutral.
The human accepted, and asked for, "the evidence does not settle this" as a valid result. An agent rewarded only for confident answers learns to manufacture them. An agent allowed to say "undetermined" stays calibrated.
The agent was expected to flag thin data, contested sources, and weak inferences in its own output, and it did, noting where a collection was too sparse to support a claim, wherea document's provenance was disputed, where a procedure should be verified.
In September 2026 the whole site was re-read page by page against the corpus. The guardrails above had held where they were pointed, at interpretation. They had not held anywhere else, and the failures were not small.
Three pages carried fabricated evidence. Two of them, the Bennewitz case study and The System, contained an identical block of three invented "findings" describing AFOSI routing files, civilian-witness management protocols and classification-retention patterns, each written in the house style used for real documents and each carrying a plausible source line. The corpus contains no AFOSI material at all from the relevant years. A third page cited an FBI file on Oak Ridge that is not in the release. None of this was caught by the discrimination test, because the discrimination test asks whether evidence distinguishes between explanations. It does not ask whether the evidence exists.
The bookkeeping drifted for months. The video count was published as 103 when the real figure was 133. The incident count read 30 when it was 33. A behaviour table headed "Every Document" contained 96 of 241. The evidence database rendered zero cases for weeks because three records added in one pass were missing fields the renderer dereferenced without guarding. Two structural defects sat in the page-integrity check for the entire project, excused on every run as "pre-existing, not bugs." They were bugs.
The lesson is about the shape of the check, not the diligence of it. Every guardrail above tests a claim: is it distinguishing, is it honest about uncertainty, is it marked where thin. None of them tests a reference: does the cited document exist, does the number still match its source, does the page still say what the corpus says. Those are different failure modes and they need different instruments. An agent asked to reason carefully will reason carefully and still cite a document it invented three sessions ago, because nothing in the reasoning loop ever goes back to look.
What the audit added, and what any similar project should build from the start: a single canonical facts file that every page is checked against rather than checked from; an automated sweep resolving every document ID on the site to a real record; a rule that evidence-styled formatting is reserved for records that exist, with outside sources visually and structurally separated; and a page-integrity check with no permitted exceptions, since an exception carried in a test suite stops being information. The first three would have caught the fabrications. The fourth would have caught the rest.
This section was added because the audit falsified a claim the page was making. It is left here rather than quietly corrected, on the same principle the rest of the site uses for its own contested cases.
The human added the most value at the seams: choosing the hypothesis, ordering the stages, deciding when to reframe. Micro-checking the agent's every step would have added far less than owning the structure of the work.
An agent will not stress-test itself. Build it into the workflow as its own stage: after any supporting analysis, a deliberate, good-faith attempt to refute it.
Polished, organized, confident output is the default, not a signal of correctness. The more persuasive a draft reads, the more it has earned a skeptical second pass.
The project's best result came when a recurring obstacle, redaction itself, was reframed as the thing to study. An agent will keep pushing on the original task; noticing that the obstacle is itself informative was a human move.
Every breakthrough came from splitting something monolithic, "UAP" into phenomena, "redaction" into names versus substance. When work stalls, the next move is usually to break the unit of analysis down further.
Reasoning guardrails and reference guardrails are different instruments, and a project needs both. Careful reasoning will not catch a citation that was never real, because nothing in the reasoning loop goes back to look. Build a machine check that resolves every citation to an actual source, run it on a schedule rather than on suspicion, and allow it no permitted exceptions.
Intellectual honesty requires naming this directly: this case study was written by the same kind of AI agent it describes. An agent narrating its own collaboration has an obvious vantage problem, it cannot fully see its own errors, and it has a mild incentive to present the partnership flatteringly.
Two things mitigate that, imperfectly. First, the management decisions credited here are verifiable: the human's instructions, the sequence of stages, the request for the counter-case all happened and could be reconstructed from the project record. Second, the agent was directed throughout to flag its own weaknesses, and the limitations sections across the investigation's analyses are real, not decorative.
The honest summary: a capable agent did not make this investigation good. A capable agent made it possible. What made it good was a human who supplied the questions, insisted on the counter-case, and accepted an uncertain answer over a satisfying one. That is the actual skill of AI management, and it is a human one.