I don’t want an AI system to finish a job with a confident sentence and nothing behind it.
If I ask for a report, I want the report. If I ask for an email draft, I want the correct draft saved in the right place. If a tool fails, I want to know where it failed. A polished explanation can be useful, but it cannot take the place of the thing I asked the system to do.
That question—what actually happened?—is why I’m building SRR, the SupRrrAi ToolKit. It is also why my development videos include failures, repairs, and another run of the same test. The interesting part is often under the hood.
The gap between an answer and an action
A language model can describe a completed task without having completed it. That does not require some dramatic intention to deceive. It can be an incorrect assumption, a misunderstood tool response, or a summary that gets ahead of the work.
The practical result is the same: the operator receives confidence where they needed confirmation.
There are several places for that gap to appear. A model may propose an action without calling the tool. A tool may run but return an error. A file may exist but contain the wrong information. An application window may look correct while its saved state is incomplete.
Treating all those situations as “done” hides the part we need to investigate. I want the system to keep them distinguishable: planned, attempted, observed, and checked against the requested result.
What I started testing
In my August 12 development update, I discussed the Linux execution work behind SRR and the start of a planned 50-prompt testing series. The point was to challenge what the harness could demonstrate, rather than accept what the model said about itself.
That work brought together tools, execution, and the authority that decides whether an action may proceed. The model contributes its plan. The surrounding system has to enforce the relevant boundaries and inspect what comes back.
I used strong language in that video about keeping models honest. The engineering meaning is more specific: a completion claim should be checked against observations that the model cannot replace with another sentence. That is a development objective, not a promise that software can never be wrong.
The records matter after the demonstration
In the August 15 freeze-files walkthrough, I showed the documentation surrounding the Prompt 01 work: handoffs, readmes, snapshots, and the records kept with the project.
Those records matter because a demonstration is easy to lose in a long conversation. Later, I need to distinguish what was requested, what was run, which attempt the result belongs to, and what still needs attention. A screenshot without that context can raise as many questions as it answers.
The video also showed the development repository as private at that point. That is part of the project’s history, not a public release announcement. Showing work in progress should make its status clearer, not imply that every component is ready for everyone to install.
What useful evidence looks like
Evidence depends on the task. For a file operation, it may include the saved file’s contents and location. For an application workflow, it may include the actual saved item and its identity. For a terminal operation, it may include the command result and the state that operation was supposed to change.
A content digest can help identify a particular artifact. It cannot decide whether that artifact is the correct one for the user’s request. The wrong report has a digest too. We still need the connection between the instruction, the selected input, the action, and the result.
The same applies to a successful exit code. It tells us something about the program’s execution. Whether it proves the requested outcome depends on what the program actually checked.
I want those connections to be visible enough that another review can challenge them.
Strict checks should not make ordinary work unusable
There is a balance here. I am not trying to make somebody complete a certification exercise every time they want a folder created or a browser opened.
An ordinary job needs an understandable request, appropriate authorization, execution, and a useful result. A deliberately demanding acceptance test can ask for additional evidence, isolation, cleanup checks, and independent review. Those extra requirements belong to the test being requested.
Similarly, a model should not have to guess the harness’s internal field names to express a reasonable plan. Translating that intent into a safe internal action is infrastructure work. Making the output format more rigid does not, by itself, make the underlying observation stronger.
The aim is usable software with clear boundaries—not a growing ceremony around every action.
What these tests do and do not establish
A successful test can establish that a particular build performed particular actions under the conditions that were measured. That is worthwhile. It gives us something more concrete than an enthusiastic summary.
It does not establish universal reliability. The host, controller, tools, storage, or evidence collector can contain defects. Another environment may behave differently. A later code change may introduce a regression.
That is why I treat a passing result as a reason to preserve the record and keep testing. It is not permission to announce that the whole system is solved.
Failures are useful for the same reason. When a test exposes a missing handoff or an incorrect check, that gives the next repair a specific target. Hiding the failure would hide the lesson.
What comes next
My development loop is straightforward: build it, test it, understand the failure, repair it, run it again, and verify the result. Sometimes that takes longer than I expected. Sometimes the problem is in the surrounding machinery rather than the model I originally suspected.
This first month of articles follows that process. Next comes the difference between an agent and the harness around it. Then I’ll use an email workflow and a terminal workflow to show why the surrounding details matter.
SRR is being developed as a Linux-first, local-first AI agent harness. The point of sharing the work is to make the questions and the limits visible while I build.
Watch the related development update below, follow The Sp_ceman Channel, or visit SupRrrAi to follow the project.