CYBER-HOBO
concepts
in practice
how it is built

The problem

A capable language model is not, by itself, an effective professional worker. The shortfall sits in the model's working properties, and each one is measurable.

Five of them shaped this system:

Fluent text, unverified facts.The model writes references as fluently as prose, whether or not they are right. When I audited the first regulatory notes we had produced together, every note in the batch carried citation errors. Depending on the note, 37 to 84 percent of checked claims verified against the source text.
Written rules do not hold.Prose instructions are followed while fresh and drift afterwards. Measured on this system's own writing conventions: about three of four new documents fell off the written specification, until an automated check began enforcing it.
No memory.A session ends and the model retains nothing. Without an external record, the next session re-derives what the previous one already knew.
Volume outruns review.The model produces text many times faster than one person can check it. Unchecked output carries error rates like the ones above straight into deliverables.
Silent choices under ambiguity.Given an instruction with two readings, the model picks one and continues without saying so. The record of this practice shows that damage begins at these points.

The first number above comes from an audit anyone can repeat on their own AI-produced material:

  1. Collect the produced documents and extract every checkable claim: article numbers, scopes, figures, quoted definitions, named duties.
  2. Open the source each claim points at and compare. Four outcomes: verified, wrong, imprecise, untraceable.
  3. Score each document by the share of its claims that verified directly.
  4. Read the failures by class. Where they cluster shows where checking has to live. In this practice they clustered in structured references, and grew where the source material was thin.

The method needs the source texts at hand, and it needs every claim checked rather than only the suspicious ones. Sampling by suspicion misses the class that matters most: the claims that read most fluently.

Asking the model to do better does not move these properties. Structure moves them: checks that run automatically, records that persist between sessions, and human review directed at the claims that carry risk. The change is measurable:

How the output was producedMeasured result
Notes written with the model and taken as produced, with no checking step between the model and the noteEvery note in the audited batch carried citation errors. Per note, 37 to 84 percent of checked claims verified against the source.
Deliverables produced under the system: each factual claim graded and checked against its source before deliveryBetween 0 and 1.7 percent of claims failed the pre-delivery check and were corrected. No delivered document has a wrong claim on record.

The mechanisms that produced the difference are the subject of this site.