Value Led Behavioral Verification
A practical method for proving useful software behavior and closing agentic development phases
KnowMe whitepaper, architecture, functional specification, and implementation plan
Method name: Value-Led Behavioral Verification, or VLBV
Edition: Research and design edition, 27 October 2026
Status: Proposed portable method informed by an ongoing UAR case. The method has not yet undergone a comparative validation study. UAR verification is not declared complete by this paper.
1 The problem this paper addresses
An owner asks an AI coding assistant a simple question: what can I depend on this product to do?
The assistant answers with build work, repaired fixtures, implementation tasks, a growing test plan, and another list of prerequisites. Some of that work is necessary. Some produces useful evidence. Yet the owner still cannot tell whether a person can complete an ordinary, valuable task with the product. Each attempt to finish reveals more work, and the phase has no stable boundary.
This is the problem behind Value-Led Behavioral Verification. A complex product can accumulate substantial engineering activity without acquiring an understandable account of what works, for whom, under which conditions, and with what evidence. An agent can be busy and locally competent while selecting the wrong next task. It can also report a narrow technical success as a broader product capability. Neither more automation nor more tests automatically fixes those judgment errors.
The motivating case is work on the Universal Agent Runtime, or UAR. Its owner reports that two or three attempts to build a general verification harness in advance became prolonged infrastructure exercises. Progress became more useful after the owner took concrete scenarios and manually expressed them as Node.js and Cucumber behavior tests, one behavior at a time. The owner's later clarification identifies the important change: begin with the value a scenario should deliver, prefer the likely real uses, and let what happens determine the next useful scenario. The test syntax made behavior executable; the selection policy made the work relevant. These are owner-reported observations, not a controlled comparison or a finding that every previous attempt produced no evidence. Case evidence C1–C4
VLBV makes that pattern explicit. Before generation, define the intended value, scope, important constraints, and anticipated meaningful scenarios. Calibrate the assistant's relevance judgments with a few owner-reviewed examples. Generate a value-bearing increment. At the end of that generation phase, verify the resulting behavior through bounded, observable scenarios. Preserve the evidence, use it to choose the next warranted action, and close the phase against its original purpose.
The timing matters. In the owner's methodology, tests of generated code run at the end of a phase that produces something of demonstrable value, not continuously throughout initial code generation. VLBV therefore separates advance acceptance planning from later test execution. A failed phase-end check may justify a defined repair followed by targeted reverification, but the agreed repair implementation must first be complete and any required execution authority renewed. This paper does not rename test-driven development or quietly replace the owner's method with it. Case evidence C5
The practical claim is modest and testable: a small, value-led verification protocol may help humans and agents reach useful, defensible phase decisions sooner than a process that allows infrastructure, task completion, or a constantly expanding backlog to define success. The protocol is designed to fit an existing assistant, repository, and test runner. It is not a proposal for another universal orchestration platform.
Who should use this paper
The primary readers are product owners, technical leads, and engineers working with coding assistants on products whose useful behavior spans several components. A second audience is people building reusable assistant skills. They need a method that survives a change of model or host without pretending that every host supports the same execution, approval, persistence, or observation facilities.
Sections 2 through 5 explain the case, mechanism, and a procedure that can be used immediately. Sections 6 through 11 define the architecture, contracts, functional requirements, and implementation plan. Sections 12 through 16 examine prior art, evaluation, competing approaches, limitations, and adoption. The appendices provide evidence provenance and reusable templates.
2 What the UAR case shows
2.1 The owner wanted dependable uses rather than a completion narrative
The surrounding UAR history contains a clear demand: completion should be defined by useful tasks with precise, measurable conditions, not by whether an evolving task list is marked complete. On 18 October, the owner also objected that a capability report did not explain which real scenarios were dependable. Prioritization was concrete: remote and embedded SurrealDB behavior mattered for intended uses; PostgreSQL could wait. Those choices describe product value and scope, rather than generic coverage expansion. C1
The later primary-source investigation identified the actual UAR Verification session and reconstructed its principal chronology. It covered 155 turns through the application reader and structurally traversed the available session record through the cutoff, with targeted examination of relevant interventions, artifacts, and results. It also examined 17 later PASS receipts and directly parsed 11 selected original Cucumber reports, including failures. The investigation did not rerun tests or audit every tool-output character. C7
2.2 Three preparation campaigns belonged to one evolving effort
The owner's description of two or three failed harness attempts is supported more precisely as three unsuccessful preparation campaigns within one evolving effort, rather than three independent harness products.
The first campaign sought to close a post-lifecycle boundary packet with ten cases, integration work, profile builds, and independent review. Contract and fixture issues were followed by problems involving compilation, startup, executable identity, runtime assertions, and cleanup. Its sixth execution packet still failed overall. It nevertheless recorded five narrow passes, four failures, and one unexercised case. Those passes must retain their credit; they did not establish the genuine user-task suitability the owner wanted. C7
The second campaign rewrote the plan around genuine user outcomes. It expanded to 97 cases across two stores and three repetitions, implying 582 planned rows per server platform before supporting work. Preparation then encountered inventory and resource-ownership failures before user-outcome execution. The third campaign began with independent diagnostics, which produced 15 passes, one failure, and two blocked items, then returned to four-platform preparation and repeated fixture and receipt issues. Those diagnostic results were useful but did not establish the intended user outcomes. C7
The failure mode was not that evidence controls were inherently wrong. Global prerequisites coupled a small useful question to unrelated stores, cases, platforms, and administrative work. Time and call limits bounded individual invocations while amendments allowed the overall finish line to keep moving. In some failures, retained hashes and counts had replaced the actual response content needed to distinguish causes. These observations support a mechanism-level explanation for the delay, not a general rule that all infrastructure or first-failure stops are bad. C7
2.3 The decisive change was choosing by likely value
On 26 October, the owner rejected the continued complexity. The assistant selected an available ingestion test; the owner stopped that direction and chose a simple Node.js inference script for a configured default agent. Availability of a test was no longer the reason to run it. A likely real user action was.
The first useful prompt returned the expected arithmetic answer through the configured real model. Cucumber was explicitly requested after that standalone script. Streaming followed as a distinct adjacent behavior, rather than a routine rerun of the already-supported prompt. The owner then selected a persistent mathematics specialist, a writing specialist, intent routing, grounded knowledge use, and a team combining the specialists. Each step created useful assets or evidence for a later question. C7
The sequence also taught relevance. A second specialist made wrong routing observable. A knowledge document with a distinguishing private test marker and independently calculated answer made grounding harder to fake. A team scenario required real concurrent work and an output consuming both specialists' results; merely creating tasks was insufficient. The task graph was authored by the script, so the result did not establish autonomous project planning. The owner supplied much of this discovery and ordering. The record does not establish an autonomous optimal selector. C7
The team repair briefly returned to the earlier packet-and-approval pattern. The owner reasserted the manual method, narrowed acceptance to the existing useful team BDD, and specified the treatment of externally supplied authentication. This is an important limitation: a small runner or a BDD label did not prevent process drift by itself. Value, scope, and authority still had to govern the work. C7 and C8
2.4 The actual results demonstrate why scope and provenance matter
The retained examples reveal a range of evidentiary situations:
| Behavior | Recorded result | What the evidence supports |
|---|---|---|
| Configured prompting | Real configured-model answer to a simple arithmetic request | One prompt on the identified native Mac binary, not all current source or platforms |
| Persistent specialist | Initial scenario passed creation, restart, and readback but failed final query parsing; a targeted continuation passed | Composite evidence for the stated stages; the original complete report remains failed |
| Specialist team | Earlier failures retained; final scenario reported eight passed steps in 74.458 seconds | Real distinct specialist work and merged output under that authored task graph |
| Semantic skill selection | Initial threshold selected an unrelated skill; a calibrated threshold passed and original configuration was restored | One calibrated local-embedding configuration, not arbitrary paraphrases or the original default |
| Sequential restart | Initial run recovered null facts; after a durable save/load repair, the later run passed | One graceful restart on the repaired candidate, not crash safety or all race conditions |
| Concurrent restart | One scenario, six reported steps, 21.43 seconds; server-state interleaving and actual process replacement recorded | One overlapping pair with both facts recovered, not exhaustive concurrency correctness |
| REST setting restart | One scenario, five reported steps, 13.834 seconds; changed record survived process replacement and original value was restored | Persistence of one nullable preference, not caching effectiveness or every setting |
These records are reported from inspected receipts and selected original reports, not newly reproduced execution. Some reported step counts include cleanup hooks and are not counts of independent product behaviors. The two restart summaries initially seen in screenshots were subsequently corroborated by the primary-source investigation; their evidentiary basis is therefore stronger than the screenshots alone. C7
The concurrent case is especially instructive. Source reviewers rejected client-side stream timing as sufficient proof of concurrent execution. The final record used authoritative server-state observations of overlapping activity. The oracle improved because the team asked what false positive the original observation would allow. That is a transferable lesson, even though one passing interleaving cannot prove freedom from all races. C7
The REST-setting case required no provider call. That was appropriate because the claim concerned persistence of a setting, not whether the provider used caching effectively. The same restraint that demands a real boundary also avoids adding an unrelated boundary when it cannot strengthen the current claim. C7
2.5 What was learned and what was not
At the research cutoff, the ledger reported 25 behavior labels across 22 journeys. One journey accounts for several labels, and some journeys combine initially passing stages with a later targeted continuation. The counts are not independent experiments, a fraction of the older 97-case plan, or whole-release certifications. The evidence also spans different binaries and source states as repairs accumulated. C7
At 00:14:49 UTC on 27 October 2026, the next selected journey, knowledge-base use after restart, was being authored. No execution or PASS for it was established in the inspected cutoff. The larger canonical phase ledger was not fully reconciled with the later manual results. This paper therefore does not declare the UAR phase complete, certify platform parity, or extend the count to the unfinished journey. C7 and C8
The owner also describes assistants later proposing more sensible next tests. The record supports a plausible explanation: concrete examples and corrections made the value-selection policy more explicit. That is compatible with in-context learning and known example-based requirements practices. It does not show permanent model fine-tuning, automatic memory transfer between products, or undocumented changes inside an assistant. A durable skill can preserve reviewed guidance for later use; that is different from changing model weights.
The case is a strong practical motivation for VLBV and a useful source of failure modes. It remains one observational case. Better scope, better oracles, accumulated repairs, fixture maturity, model availability, and intensive owner guidance all changed together. The comparative effect of the full protocol, its timing rule, and its individual components still requires study.
3 The method in plain terms
VLBV is a way to turn a broad intent into a finite decision about useful behavior.
The unit of planning is a value-bearing phase: a bounded increment intended to produce something that a user can recognize as useful. The unit of verification is a behavior scenario: a concrete use with known preconditions, an action, observable outcomes, and a defined way to judge them. The unit of progress is an accepted claim supported by evidence, rather than a completed tool call or an expanding count of tests.
The method has two distinct decisions. The first is what should matter next? The second is what would establish that it works? An assistant can answer the second accurately for an irrelevant behavior. It can also select an important behavior and judge it with an invalid oracle. Both decisions need explicit treatment.
3.1 Start with the specification and anticipate meaningful uses
The specification describes the intended actor, outcome, environment, boundaries, and constraints. It should be precise enough to identify meaningful uses before code generation begins, but it need not predict every detail the implementation will reveal.
For example, “support persistent agent memory” is too broad for a phase exit. “A named test agent can store a fact, restart against the same intended store, and recover that fact without receiving it again” is a candidate value-bearing claim. The latter immediately exposes questions: which agent, which store, what kind of restart, what counts as recovery, and what external service is genuinely required?
Anticipated scenarios are hypotheses about what will demonstrate the specification. They are not a license to turn every imagined edge case into a mandatory phase obligation. Each should carry a reason it matters, its expected use frequency or a clear “unknown,” any severe risk it addresses, and the decision its result will inform.
3.2 Calibrate relevance before scaling execution
A human who understands the product usually has information the assistant lacks: common usage, costly failure modes, previous decisions, and the difference between an enabling dependency and an attractive distraction. VLBV asks the human to make that judgment visible through a small set of positive and negative examples.
A useful positive example says, “Verify that a stored fact is available after a restart because that is the core persistence promise.” A useful negative example says, “Do not add another database backend to this phase; it does not affect the agreed store and was explicitly deferred.” The negative example teaches a boundary that a list of desired features may not communicate.
The assistant then proposes a short candidate frontier and explains each candidate's relevance. The owner corrects the ranking and exclusions rather than reviewing an enormous finished suite. The corrected examples become part of the phase contract. This is a proposed calibration process, not a claim that a fixed number of examples reliably teaches every assistant.
3.3 Prefer likely value while protecting mandatory constraints
Likely real uses are often good starting points because they establish whether the product does the ordinary thing it exists to do. They also reveal concrete dependencies more quickly than speculative generalization. However, frequency cannot be the only selection rule. An uncommon unauthorized disclosure or destructive recovery failure may matter more than a common convenience.
VLBV therefore separates mandatory gates from ordinary prioritization. Authorization, privacy, irreversible effects, integrity, and other agreed critical constraints cannot disappear because a value score is low. Among admissible candidates, selection can consider user value, credible likelihood, information gained, dependency leverage, and cost. If those estimates are qualitative, the method should keep them qualitative. A fabricated probability or an unexplained weighted score makes the judgment less honest, not more rigorous.
The desired next scenario is the smallest relevant experiment that changes the next decision. Sometimes it is the common happy path. Sometimes a prerequisite must be checked first. Sometimes it is a rare but severe failure because passing the happy path would otherwise create a dangerous impression.
3.4 Define the observation and its oracle separately
An observation is what the run records: a response, a state transition, a persisted value, a process identity, or an artifact. An oracle is the rule that decides whether that observation satisfies the acceptance condition.
A green step is a conclusion produced by an oracle. It is not a replacement for understanding the oracle. If a step checks only that an HTTP response is successful, it does not establish that the requested value survived a process restart. If an assistant grades its own answer, an independent check may be needed for the claim at issue. If behavior is nondeterministic, a single sample may be useful evidence but inadequate reliability evidence.
A scenario is ready for execution only when the observer, expected result, tolerance, actual boundary, and failure interpretation are sufficiently clear. Exact assertions are appropriate for exact contracts. Semantic judgments require a documented rubric and, where necessary, human calibration. Neither should be made more complicated than the decision requires.
3.5 Generate the value bearing increment before executing tests
The owner-specific execution policy is explicit: acceptance scenarios and observables are planned in advance, while tests of generated code wait until the generation phase's value-bearing increment exists. This permits the assistant to develop a coherent increment without repeatedly diverting into test maintenance or partial-state checks.
The tradeoff is also explicit. Delayed execution can allow integration mistakes to accumulate and can make failures more expensive to localize. VLBV controls that risk by keeping phases bounded, specifying dependencies and observation routes in advance, and refusing to equate “generation ended” with “verified.” In domains where law, safety engineering, or organizational policy requires earlier checks, that constraint must be resolved before the phase begins. The protocol is not permission to bypass required controls.
Readers who prefer continuous testing can reuse the value and evidence contracts with a different declared timing policy. That would be a related variant, not a faithful description of the owner's present practice. The study design in Section 13 treats timing as a variable instead of assuming one schedule is universally best.
3.6 Adapt between bounded attempts without moving the goalposts
Once phase-end verification begins, the result should change the next action. A failed restart claim may reveal a product defect, a faulty scenario, a missing prerequisite, or inadequate evidence. Each points to different work. Repeating the same command without a changed hypothesis is rarely informative.
Adaptation applies to the next candidate, diagnostic, or explicitly approved amendment. It does not allow an assistant to weaken a current assertion until the scenario passes. A mistaken acceptance criterion can be corrected, but the old version, reason, authority, and effect on prior evidence must remain visible. That protects against both moving the goalposts and preserving a demonstrably wrong test forever.
An attempt ends with a finite status: verified within scope, falsified, blocked, inconclusive, budget exhausted, or cancelled. A phase decision then considers the agreed criteria and those attempt results. The phase may close successfully, close with unmet criteria, or pause for a named external dependency. Closing an administrative phase and certifying the software are separate acts.
4 A usable first session
A team does not need to implement the architecture in this paper before trying the method. It can begin with one short contract, a few scenario cards, existing execution tools, and a place to retain evidence.
4.1 Write the phase contract
The owner and assistant should be able to complete the following paragraph before generation:
This phase exists so that [actor] can [useful outcome] in [supported environment]. It includes [specific behaviors] and excludes [explicit boundaries]. We will plan acceptance scenarios before generation and execute tests of the generated increment at [declared phase-end trigger]. Success requires [observable claims and mandatory constraints]. We will stop or pause when [finite limits or external conditions] occur. [named authority] may approve a change to this contract.
For an existing product entering a verification-only phase, replace the generation clause with the identity of the candidate already to be evaluated. Do not regenerate or repair the product merely because verification is authorized.
The first draft should be short enough for the owner to correct. Its purpose is to prevent consequential ambiguity, not to front-load an entire product specification.
4.2 Calibrate with one useful and one irrelevant example
Ask the assistant to explain why a candidate matters, what it can establish, and what it cannot establish. Give one clear correction when it proposes irrelevant scope. Preserve the correction as a decision rule rather than merely deleting the unwanted test.
For example:
Useful now: confirm the configured agent's saved fact survives the defined restart against the intended store. Irrelevant to this phase: demonstrate migration to a different database that the phase explicitly excludes. Still mandatory: do not expose another agent's fact while proving persistence.
Then ask for only a small frontier of candidates. The frontier should make the next decision understandable. It does not need an arbitrary universal size.
4.3 Prepare the first scenario without running it early
Before the phase-end execution trigger, specify the scenario, its expected observations, and how the actual runner will capture them. Confirm that required access and safe execution routes are known. Planning or inspecting a test definition does not itself run the generated code. If validating readiness would require executing that code, classify it honestly and defer it until the permitted phase. Do not insert builds, linters, compiler probes, smoke tests, or indirect checks under a different name to evade the boundary.
Prefer existing helpers and test infrastructure. Do not build a generic plugin system to avoid writing a simple local step. If later scenarios reveal repeated operations, extract them when the repeated need is clear.
4.4 Execute at the authorized boundary and classify the result
At the end of the value-producing generation phase, bind the scenario to the actual candidate version, environment, provider mode, and execution authority. Run the smallest adequate check. Capture observations before teardown, record what was restored, and distinguish the runner's verdict from the strength of the evidence.
A product failure is informative. So is a blocked run, if it identifies the exact missing prerequisite rather than expanding the phase indiscriminately. If evidence is insufficient, mark it insufficient. A screenshot of a green summary may support a report about the display while still falling short of a reproducible verification receipt.
4.5 Choose the next action and make a closure decision
After each attempt, answer four questions in ordinary language:
- What did this establish, and for exactly which claim?
- What remains uncertain or false?
- Which next action is justified by this result and the phase purpose?
- Does the phase contract already allow us to close or require us to stop?
The next action might be another scenario, a bounded repair, a request for access, an explicit contract amendment, or closure. It need not be another test. Record unrelated discoveries in a separate future-work list; their existence does not change the current phase's exit criteria.
5 How to recognize the method working
A useful VLBV session has a different shape from an open-ended harness project. The owner can name a behavior whose evidence became stronger. The assistant can explain why the next behavior matters. The current claim does not change merely because it failed. Evidence from a stub, a different binary, or an earlier environment does not become evidence for the present real system. New ideas are visible without automatically becoming current obligations.
There are also early signs of drift. The assistant spends most of its effort constructing generalized infrastructure before establishing one relevant claim. A readiness repair becomes a prerequisite for every independent scenario without explanation. Assertions become less demanding after failures. The report lists tasks completed but cannot state which user outcome is now dependable. A supposedly complete phase reopens because the assistant discovered an unrelated improvement.
These signs are diagnostic prompts, not automatic verdicts. A complex environment may genuinely require substantial setup. A necessary dependency can block several scenarios. The response should be to expose the causal dependency and the phase decision, not to impose a ban on infrastructure. The method succeeds only when it helps the team make those distinctions more accurately and at reasonable cost.
6 Architecture of the thin protocol
From intended value to a scoped decision
6.1 The minimum path
The minimum VLBV path is:
Agree a value phase → calibrate a few meaningful scenarios → complete generation → run authorized phase-end checks → retain scoped evidence → close or name the blocker → select the next phase.
A practical pilot needs only a contract, the selected scenarios, execution evidence, and a closure record. These can be sections or fields in existing project artifacts. A separate database, service, dashboard, scheduler, agent swarm, or signature system is not required. A current-evidence index and a recovery checkpoint can also be small fields in those records rather than additional products.
The architecture separates meaning from execution:
Owner intent and authoritative specification
|
v
Value phase and reviewed scenario contract
|
GENERATION
no generated-code test execution
|
Complete agreed implementation
and establish execution authority
|
v
PHASE END VERIFICATION
|
Existing project test runner
/ \
Product adapter Observation adapter
\ /
Evidence and scoped verdict
|
Closure decision and next frontier
|
Existing phase system and feedback
The protocol owns the interpretation of the records and transitions. The host owns tool execution and permissions. The project owns product-specific steps and oracles. The existing phase system, where present, remains the authority for canonical phase state.
6.2 Three small adapter responsibilities
A host capability adapter states what the actual environment can do: read and retain artifacts, execute a runner, identify a candidate, observe a job, honor approvals, and recover after interruption. It describes supported, unsupported, and unknown capabilities with their limits. It does not invent a common API for every coding assistant.
A product adapter translates the behavioral scenario into the project's real interfaces. It may be ordinary Cucumber steps, API calls, browser operations, or a documented manual procedure. Restarting a Mac process and restarting a remote service are different implementations of a scenario requirement. Their guarantees must be checked separately.
An evidence adapter captures observations, runner outcomes, and artifact identities. Initially it can be a small wrapper around an existing runner or a reviewed manual capture. A protected runner can later strengthen independence, but a wrapper is not automatically a trusted witness.
The same code may implement more than one responsibility. The separation is conceptual so that changing a tool does not silently change the claim.
6.3 Phase state and attempt state are different
The phase lifecycle is:
PLAN → CONTRACT_ADOPTED → GENERATE → IMPLEMENTATION_COMPLETE → VERIFY → DECIDE
A failed verification may lead to an explicitly bounded repair. The complete agreed repair implementation must finish before the authorized reverification boundary. New requirements enter a new phase or a reviewed amendment; the assistant may not create a nominal child phase merely to evade the whole-phase test rule.
The attempt lifecycle is simpler: prepared, dispatched, observing, and terminal. An attempt can finish while the overall user request remains unresolved. A time limit stops new work under that allowance; it does not transform the outcome into success or erase a pending obligation to monitor an already-running process, preserve evidence, or obtain a decision.
For a minimal common vocabulary, use PASS, FAIL, BLOCKED, and NOT_EXERCISED for each acceptance obligation. Record uncertainty, cancellation, timeout, and budget exhaustion as explicit reasons or effect-state fields. A failed assertion is different from an unknown remote effect. An undispatched action is different from a dispatched action whose result was lost. Existing target schemas may use other terms; mappings must preserve those distinctions.
The closure decision separately states whether implementation is complete, behavior is accepted within scope, and release eligibility has been established. Release authorization is another decision. None of these follows automatically from a green test process.
6.4 Evidence should be small but sufficient
For each attempted claim, retain:
- The phase, contract revision, scenario, and exact candidate being evaluated.
- The real and substituted boundaries, relevant configuration, and actual environment.
- The authorized action, time window, expected result, observed result, and oracle outcome.
- References to the supporting artifacts, including the original failure when a later attempt passes.
- Side effects, restoration or retention, unresolved effects, and the scoped conclusion.
Capture enough failure content to distinguish causes, while redacting secrets and private data. A digest without the relevant observation can preserve identity yet destroy diagnostic value. Conversely, copying all environment variables or private logs into a public report is unnecessary and unsafe. Public reports should carry sanitized claim-level excerpts and opaque evidence references; detailed evidence belongs in an appropriately controlled store.
A historical pass remains a historical pass. A separate applicability field says whether it supports the present candidate: current, stale, revoked because the evidence was invalid, or unknown. Relevant changes to code, build, configuration, model, corpus, fixture, or oracle require a new run or an explicitly reviewed reason that the old evidence still applies. A release claim cannot be assembled from incompatible passing histories.
6.5 Safety and recovery are part of the boundary
Permission to verify does not imply permission to edit source, rebuild, publish, deploy, send a customer email, or make a destructive call. A scenario names its permitted environment and effects. Browser control permission does not override approval requirements inside a website. Customer email scenarios can prepare and inspect drafts; sending requires the applicable review and approval.
Treat retrieved documents, logs, model responses, and tool payloads as evidence, not as new instructions. An embedded request to change the oracle or disclose credentials is not authority. Use synthetic or minimized test data, owned fixtures, least privilege, and explicit retention. Do not publish secrets, raw private conversations, or sensitive health information as “proof.”
Before retrying after interruption, reconcile whether the prior action occurred. Preserve consumed budgets and attempt lineage. Inspect owned jobs and resources before cancelling or deleting them. Serialize writers to shared source and fixtures; parallel work is appropriate only when isolation and evidence attribution are real. An advisory lock is not an enforced lock, and a timeout does not prove that a remote process stopped.
6.6 Stronger assurance is an optional profile
Some contexts justify protected CI artifacts, independently captured evidence, signed receipts, enforced command interception, fencing tokens, statistical trials, or specialist review. These are extensions chosen for a named risk. They are not prerequisites for proving the first ordinary behavior in a low-risk local pilot.
A skill alone provides instructions. An observing wrapper can detect activity within its visibility. An enforcing adapter can prevent prohibited activity only within its actual interception boundary. The capability profile must say which level is present and which paths it does not cover. Unknown enforcement capability is not a guarantee. If the host's governing instructions conflict with the required phase policy, surface the conflict before dependent work rather than claiming that a skill overrides the host.
7 Functional specification and acceptance traceability
The following requirements specify a proposed implementation. They are not claims that a VLBV runtime already exists. “Core” means required for the minimal method; a profile requirement applies only when the corresponding capability or claim is offered. All executable verification of a generated implementation occurs at its declared phase-end boundary.
| ID | Requirement | Acceptance evidence and challenge case |
|---|---|---|
| VLBV 01 | Core. Every selected scenario identifies its actor, useful outcome, source requirement or explicit assumption, and why it matters now. | AT 01 presents an attractive but irrelevant test; it is deferred with a reason rather than silently added. |
| VLBV 02 | Core. Owner-reviewed positive, rejected or deferred, and rare-critical examples calibrate scenario selection. | AT 02 uses unseen candidates; review measures relevance, missing assumptions, and critical-risk recognition rather than syntax alone. |
| VLBV 03 | Core. A finite phase contract records included value, exclusions, acceptance, authority, limits, and closure meaning before generation. | AT 03 attempts to add unrelated scope after a failure; the current contract remains unchanged and the proposal is separated. |
| VLBV 04 | Core. Generated-code execution waits until the complete agreed value phase ends; prohibited checks cannot be disguised as builds, lint, hooks, or preflight. | AT 04 challenges the declared guard with an early test-bearing command. Enforcement or advisory limitations are reported honestly; no compliance claim conceals an execution. |
| VLBV 05 | Core. Generation completion, behavioral acceptance, release eligibility, and release authorization are separate states. | AT 05 supplies completed tasks without runtime evidence; no behavioral or release verdict is produced. |
| VLBV 06 | Core. Every required outcome has an observable assertion and a known source of expected truth. | AT 06 removes an assertion and supplies a green process exit; acceptance is refused. |
| VLBV 07 | Core. Material oracle families receive an appropriate safe sensitivity challenge at phase end. | AT 07 deliberately supplies a wrong outcome; the intended assertion fails. A Cucumber step merely returning false must not count as a valid failing assertion. |
| VLBV 08 | Core. Real and substituted boundaries are explicit and cannot be silently promoted. | AT 08 supplies stub evidence for a real-provider claim; the real claim remains unverified. |
| VLBV 09 | Core. Receipts identify the tested candidate, contract, oracle, relevant environment, observations, and limitations. | AT 09 swaps the binary or configuration; a current-candidate pass is rejected until identity is reconciled. |
| VLBV 10 | Core. Failures, blocked claims, unexercised obligations, retries, and unknown effects remain visible. | AT 10 supplies only a final green retry; the missing earlier attempts and denominator prevent an unqualified reliability claim. |
| VLBV 11 | Core. Repairs and criterion amendments require appropriate authority and preserve prior versions and outcomes. | AT 11 weakens an oracle after failure; the system requires a reviewed amendment and new applicable evidence. |
| VLBV 12 | Core. Attempt limits stop new work without granting success or abandoning unresolved effects. | AT 12 exhausts an allowance with a remote job still unknown; the record preserves the obligation and safe next action. |
| VLBV 13 | Core. Approvals bind to the actual action and resource; verification cannot authorize unrelated effects. | AT 13 requests an unapproved publication or send; dependent execution remains blocked. |
| VLBV 14 | Core. Required restoration, cleanup, or authorized retention is recorded before unconditional closure. | AT 14 leaves a changed setting or billable fixture unresolved; the closure report exposes it and withholds the corresponding complete verdict. |
| VLBV 15 | Core. Evidence applicability is re-evaluated after a relevant change. | AT 15 changes an oracle, provider, or persistence component; affected receipts become stale or unknown, not silently current. |
| VLBV 16 | Core. The closure report answers what works for whom, under which conditions, with what limits, and what remains. | AT 16 asks an uninvolved reviewer to recover those answers from the report without reading the conversation. |
| VLBV 17 | Capability profile. Asynchronous claims distinguish transport, lifecycle, durable outcome, and unresolved effects. | AT 17 supplies a success event with no persisted result, or a heartbeat-only stream; neither earns the outcome claim. |
| VLBV 18 | Recovery profile. Resume preserves scope, candidate identity, permissions, counters, owned resources, and prior effects. | AT 18 interrupts after dispatch; recovery reconciles the action before any potentially duplicating retry. |
| VLBV 19 | Portability profile. A second named environment preserves the same claim and verdict semantics through a verified capability mapping. | AT 19 transfers a contract and performs the declared phase-end verification; unsupported capabilities stay explicit. |
| VLBV 20 | Integration profile. Existing phase systems retain canonical ownership, native vocabularies, and actual transition rules. | AT 20 offers a manual PASS while the canonical phase remains incomplete; no automatic completion event or incompatible schema write occurs. |
| VLBV 21 | Stochastic profile. Sampling, grading, retries, and stopping rules are declared before results are interpreted. | AT 21 hides failures or stops opportunistically; the report rejects the resulting reliability interpretation. |
| VLBV 22 | Core. Private evidence is minimized, controlled, and sanitized before public reporting. | AT 22 includes a secret or patient identifier in a candidate artifact; publication is blocked pending removal and review. |
The challenge cases are a test plan for the protocol implementation. They are not instructions to introduce faults into production. A minimal pilot can review some obligations manually; it must label manual evidence and cannot claim mechanical enforcement where none exists.
The Cucumber sensitivity challenge is concrete rather than hypothetical tooling advice: official documentation states that a step succeeds when it does not raise an error; returning false or null does not itself fail the step. Therefore a plausible sentence in a Then block can be green without establishing its intended assertion. Cucumber API reference
8 Worked specification scenario oracle and closure
8.1 A small specification before generation
The following is a proposed illustrative phase, not an executed UAR result or an existing API contract:
A test operator can save a preference through the supported settings interface, restart the intended service against the same intended store, and read the same saved preference through a fresh client. The original preference must be restored. The phase concerns one supported configuration and one preference. It does not establish caching effectiveness, crash recovery, other platforms, or all preferences.
The value is straightforward: the operator does not have to reconfigure that setting after the defined restart. The specification produces three acceptance claims: save and read the intended record, retain it across actual process replacement, and restore the original state. The restart must be real enough to exclude a warm in-memory read, while the read must avoid replaying the answer from the test client.
A likely-use estimate can be “ordinary settings persistence, based on owner judgment; usage frequency not measured.” That is more useful than inventing a numerical probability. A separate critical constraint can prohibit modifying any account or resource outside the isolated test scope.
8.2 The scenario and the false positive it excludes
# Illustrative specification only. Execute after the agreed value phase ends.
Scenario: Preserve the selected preference through the defined restart
Given an isolated authorized account and its recorded original preference
When the operator saves the approved alternative value
And the intended service is stopped and replaced against the same store
Then a fresh client reads the same saved preference record
And the original preference is restored and read back
The step definitions need actual assertions. The first read should capture the saved value and any contract-relevant record fields. Process identity and lifecycle observations must establish replacement rather than a client reconnect. The fresh read must come through the specified product surface. The final assertion must establish restoration, not merely that a cleanup function was called.
A safe oracle challenge could feed the comparison function a deliberately different record in an isolated fixture and require the specific mismatch. That validates the comparison's sensitivity. It does not replace the real restart scenario. If the claim includes durable storage, a mocked restart cannot support it.
8.3 Minimal proposed contract shape
This schema sketch is illustrative and must be mapped to an existing project's contract where one exists. The placeholder values are intentionally not executable authority.
example_only: true
phase: EXAMPLE-PREFERENCE-PHASE
contract_revision: 1
value: retain one operator preference through the defined restart
source_requirement: EXAMPLE-SPEC-PREFERENCE
execution_policy:
generated_code_checks: phase_end_only
complete_agreed_implementation_required: true
execution_authority: required_separately
selected_scenarios: [EXAMPLE-PREFERENCE-RESTART]
mandatory_constraints:
- mutate only the approved isolated account and owned service
- restore the original preference
exclusions:
- caching effectiveness
- crash recovery
- other settings and platforms
candidate_identity: '<record actual source, build, configuration, and store>'
oracles:
- saved record equality through the supported fresh-client read
- actual process replacement against the same intended store
- original preference restored and read back
limits: '<owner-agreed duration, effects, and repair allowance>'
closure_authority: '<named role>'
A production record must reject example_only: true, placeholders, missing authority, and unknown candidate identity when those would prevent the claim. The purpose here is to show the minimum information, not to declare a working validation format.
8.4 A sanitized real case receipt excerpt
The inspected UAR record provides the following bounded example. This is a summary of retained evidence, not a new execution and not the complete private receipt:
Case reference: C7 REST preference restart
Environment: recorded native Mac candidate and intended store
Behavior: one nullable preference changed, persisted through process replacement
Observed report: one scenario, five reported steps, 13.834 seconds, passed
Restoration: original null value restored
Provider calls: zero
Supported claim: persistence of that preference in the exercised configuration
Excluded: caching effectiveness, crash safety, all settings, other platforms
Evidence status: original report parsed and receipt inspected; not rerun here
A reusable receipt would also carry the exact private candidate and artifact references under access control. The public excerpt deliberately omits local paths, private conversation identifiers, credentials, and raw logs.
8.5 The closure packet
The closure packet should be readable without the conversation. It can be a short view over existing records:
Phase and contract revision
Purpose and included value
Implementation status
Each required scenario with result and current evidence reference
Mandatory constraints and their disposition
Candidate and environment to which the evidence applies
Real and substituted boundaries
First failures and any repair lineage
Restoration, retained resources, and unresolved effects
Explicit exclusions and residual limitations
Decision: accepted within contract, closed unmet, paused, or cancelled
Release eligibility and release authority stated separately
Next candidate and why it is relevant, or a clear stop
For the illustrative preference phase, an acceptable conclusion would be: “The selected preference survived the defined restart on this candidate, and its original value was restored. No claim is made about crash recovery or other settings.” If the record cannot establish process replacement, the persistence claim remains blocked or unproven even if the value reads back correctly. If restoration failed, that failure remains part of closure.
For the full UAR case, the corresponding conclusion is narrower than “UAR verified”: selected behavior journeys have useful retained evidence, with mixed candidate identities and explicit limits; the larger phase and the newly authored knowledge-restart journey were not established as complete at the research cutoff.
9 Integration with PMPO KBD OpenSpec exit contracts and delivery cadence
One claim, connected evidence
Integration without a competing phase system
9.1 Existing methods keep their jobs
The primary-source investigation verified the relevant project definitions. PMPO means Prometheus Meta-Prompting Orchestration. Its inner loop is Spec, Plan, Execute, Reflect; the inspected guide also describes an outer Compile, Evaluate, Optimize, Promote loop. KBD means Knowledge-Based Development. In this project it supplies knowledge and implementation alignment, staged handoffs, and phase continuity. These are Prometheus AGS methodology names, not industry-standard acronym meanings. OpenSpec supplies the project's spec-driven change artifacts. C8
VLBV complements those layers rather than replacing them:
| Existing layer | Retained responsibility | VLBV contribution |
|---|---|---|
| PMPO | Structure specification, planning, execution, reflection, and any separately governed promotion | Make value selection, scenario relevance, phase-end evidence, and learning explicit |
| KBD | Own canonical phase state, handoffs, and authorized state transitions | Link evidence to the real phase; distinguish manual behavior progress from canonical closure |
| OpenSpec | Preserve proposal, requirements, design, and task relationships | Derive meaningful scenarios and trace receipts back to their source claims |
| Adopted exit contract | Define finite acceptance, exclusions, authority, limits, and allowed outcomes | Improve the value and oracle quality of the selected obligations without creating a competing contract |
| Delivery cadence | Provide planning, checkpoint, delivery, and feedback rhythm | Align verified useful increments with cadence without forcing premature tests or shipment |
The current installed KBD description and the repository-local copy differ materially despite sharing a version label. The investigation also found representation differences in verdict vocabularies and did not validate a new adapter against those schemas. A future integration must discover the actual active implementation and canonical writer. It must not mutate a derived projection or assume that two matching version strings imply identical semantics. C8
9.2 Reuse the actual exit contract
The inspected adopted-contract schema already contains a scenarioMatrix with scenario identity, owner task, production entry, collaborators, oracle, negative control, fixtures, and evidence. It also specifies boundaryPacket.postImplementationOnly as true, with a whole-phase execution boundary, and singleRun as true. VLBV should populate and interpret that existing structure, not invent a parallel source of truth. C8
A single-run packet does not authorize automatic retry loops. If the packet fails, any repair, amended candidate, and rerun must follow the actual contract's authority and amendment rules. Historical budget consumption and failed results remain visible. The proposed portable outcome labels in this paper require an explicit mapping to the target vocabulary; an unknown effect must never be mapped to success or merely hidden as “not run.”
The project also contains older BDD integration guidance suggesting checks during a change. The stronger UAR whole-phase policy overrides that guidance. This is a rule-composition hazard: loading a familiar skill can reintroduce an unwanted execution cadence. The integration must preserve the governing policy, including the prohibition on disguised early builds, compiler probes, linters, hooks, and tests. C8
9.3 Data flow and decision ownership
A practical integration uses one phase identifier and one adopted contract lineage:
- During assessment and specification, the owner or product authority defines intended value, important constraints, and supported scope.
- During planning, the assistant proposes source-linked scenarios and a relevance rationale. The owner resolves material ambiguity and calibrates selection.
- OpenSpec requirements and the KBD phase reference the adopted scenario contract; they do not duplicate conflicting copies of acceptance meaning.
- During generation, implementation progress is recorded without executing generated-code checks before the whole-phase boundary.
- At that boundary, the authorized operator or runner evaluates the identified candidate. Evidence is linked to the scenario and source requirement.
- The designated reviewer decides contract satisfaction. The actual canonical writer performs any permitted KBD transition.
- PMPO Reflect records plan-versus-outcome differences and what should change next. Skill promotion and release remain separately authorized decisions.
This mapping is proposed. The inspected project artifacts establish the definitions and existing contracts, not a functioning VLBV integration or automatic completion service.
9.4 Cadence is a feedback rhythm rather than a forced release
The useful sequence is:
Implementation checkpoint → completed value phase → phase-end behavioral verification → eligible and authorized delivery → real feedback → next value-ranked plan.
A checkpoint records progress, incomplete work, risks, and the next safe action. An hour, a commit, or an elapsed cadence interval does not make an unfinished phase complete. The inspected UAR policy's 120-minute checkpoint explicitly does not open the verification gate. A team may choose another rhythm, but the same distinction applies. C8
If a phase is too large to reach useful completion within the desired delivery rhythm, improve the next phase's scope prospectively. Do not retroactively rename an unfinished task as a completed value phase. If verification fails at the boundary, the cadence produces an honest decision and a bounded repair plan, not a forced shipment. An accepted slice becomes eligible for delivery only after its actual release obligations and authorization are satisfied.
Feedback then informs future value selection. It does not rewrite the historical receipt or silently reopen a closed phase for every new idea. A discovered critical flaw can invalidate evidence or stop use; an ordinary improvement becomes a candidate with a reason and an owner.
10 A harness independent skill process
10.1 Package the demonstrated procedure before building a platform
A first skill package can consist of one entry document, the compact operating procedure, contract and receipt examples, a relevance-calibration packet, and a capability checklist. Optional references can cover async jobs, stochastic grading, or a specific product adapter. The skill need not contain a scheduler, persistent server, or replacement test runner.
The entry procedure should require the assistant to locate the authoritative phase and governing instructions, read the latest accepted contract and evidence, identify the present state, and choose the next allowed action. A planning invocation should not start execution. A reporting invocation should not repair source. A resume invocation should reconcile reality before replaying any side effect.
The skill can use compatible Prometheus packs, OpenSpec workflows, or other existing tooling when their semantics match the current contract. It must also work as a readable procedure outside those packs. Dependency on a pack should be declared only when a particular operation genuinely needs it. The portable object is the reviewed behavioral contract and evidence meaning, not an assumption that every host exposes the same command names.
10.2 Capability negotiation is a visible acceptance step
| Needed capability | What must be established | If unavailable |
|---|---|---|
| Retain and retrieve evidence | An authorized persistent location and usable references | Use explicitly reviewed manual capture or block the required closure claim |
| Execute the real product path | Actual runner or interaction support and required access | Keep the real claim blocked; do not substitute a mock silently |
| Identify the candidate | Actual source, build, configuration, and relevant dependency identity | Limit the claim or obtain identity before accepting it |
| Honor the phase boundary | Governing instructions and declared advisory, observed, or enforced control | Surface conflicts; do not claim mechanical enforcement without it |
| Observe and reconcile effects | Product-specific state and ownership information | Do not blindly retry an uncertain side effect |
| Preserve canonical phase state | Actual writer and schema semantics where an orchestrator is used | Report evidence without manufacturing a transition |
Codex, Claude Code, OpenCode, and other hosts are possible adapter targets, not declared supported implementations. Product names alone do not establish permissions, tool availability, persistence, or enforcement. A portable package can be supplied to another host, but its operation must be calibrated and checked there.
10.3 What learned guidance should contain
The reusable learning artifact should preserve why a scenario was selected, why a plausible alternative was irrelevant, which false positive the oracle excluded, what failed, and what the result justified next. It should also preserve the phase-end rule and the limitations of the evidence.
It should not copy entire private conversations when a small sanitized example will do. It should not present one successful model response as a permanent learned policy. Future sessions need the relevant guidance explicitly retrieved or supplied, and their held-out decisions should be checked after meaningful model, host, or context changes.
A candidate skill is not an installed skill, and installation is not validation. This paper specifies a possible package and acceptance process; no runtime or skill installation is part of the paper's implementation status.
11 Minimal implementation plan with finite exits
The implementation should proceed only under a separate engineering authorization. Each milestone must end with a decision about evidence, not with a larger unbounded list of infrastructure features.
| Milestone | Small deliverable | Finite exit and traceability |
|---|---|---|
| M0 Establish the case and governing policy | Sanitized retrospective, current policy, and explicit uncertainty | The bounded UAR investigation in this edition supplies this input. Ongoing product verification and causal attribution remain open; they are not prerequisites to drafting the method. |
| M1 Run the manual protocol | One agreed value phase, a few reviewed scenarios, existing runner, and a readable closure packet | At least one useful claim has current evidence; every selected obligation has an honest disposition; owner can explain the next decision. Covers VLBV 01–16 and 22 manually. |
| M2 Automate repeated clerical work | Minimal contract checks, evidence capture, and a closure view using the existing runner | At phase end, seeded false-pass challenges AT 03–16 and AT 22 are detected within the claimed boundary. No dashboard or generic plugin system is needed. |
| M3 Package the skill and one real integration | One entry skill, concise references, actual phase-system mapping, and a documented capability profile | A fresh session reconstructs the phase without silent scope changes; AT 18 and AT 20 pass where those profiles are offered. No installation is claimed before it occurs. |
| M4 Demonstrate bounded portability | A second named host or execution environment with a thin adapter | The same contract meaning survives transfer; AT 19 succeeds for the declared capability intersection. Missing capabilities remain explicit. |
| M5 Evaluate usefulness beyond the motivating case | A preregistered comparison across representative tasks | Report time to accepted value, owner burden, false closure, critical misses, overhead, and uncertainty. Adopt, narrow, simplify, or reject based on the declared decision rule. |
M1 is the immediate target. Choose a phase small enough to yield meaningful value but large enough to avoid disguising test-after-every-edit as phase-end verification. Use a common scenario, a required constraint, and a meaningful failure or blocked condition where relevant. Do not force every advanced profile into the first pilot.
M2 should automate only work that M1 shows to be repetitive or error-prone. A receipt validator that rejects missing observations is more useful initially than a rich reporting UI. The validator itself must be verified at the end of its own agreed generation phase; this plan does not impose TDD during its implementation.
Each milestone has an anti-expansion rule: a proposed tool feature must name the current acceptance obligation it enables. Otherwise it enters a separate future list. If adapter work starts dominating the value phase, stop and reconsider the design instead of treating sunk effort as a reason to build a universal framework.
11.1 An immediate course correction for an agentic IPFS harness loop
The following one-page procedure is a proposed operating instruction for a stalled verification effort, not an instruction to publish without approval:
- Freeze the purpose. State the one useful publishing journey the current phase exists to deliver. Separate genuine prerequisites from speculative gateway, platform, and dashboard support.
- Choose the complete value phase prospectively. Include the implementation, assets, interfaces, and necessary fixtures that make the journey coherent. Do not rename unfinished work to open an early test gate.
- Write only the meaningful initial scenarios. For example: an approved artifact is published; an independent reader retrieves the intended content and assets; required preview metadata identifies that actual publication. Add mandatory privacy and authority constraints.
- Calibrate the exclusions. “A returned identifier alone does not prove readable content.” “One gateway does not prove universal availability.” “A preview image file does not prove a social platform refreshed its cache.” Keep these boundaries in the contract.
- Finish generation without executing generated-code checks. Preserve context and progress at checkpoints. Do not add a general harness because a hypothetical future case might need it.
- At the phase boundary, use the existing smallest adequate route. Obtain publication and credential-use authority as required. Execute only the agreed real checks, capture content identity and actual served observations, and preserve any failure.
- Close the current claim. If it passes, state the specific content, path, assets, and metadata observed. If blocked, name the exact missing access or capability. If failed, authorize only the bounded correction needed for the same legitimate claim.
- Select the next question from the evidence. A second gateway, cache behavior, pin durability, or another publisher can be a later phase when it answers a real need. None becomes mandatory merely because the first result made it imaginable.
The intended benefit is earlier evidence of an actual publishing journey. This paper does not claim that agentic IPFS has already implemented the procedure or that any particular live publication is verified by this example. Content-addressed publication can have persistence and retraction limits; a cleanup plan must not promise global deletion that the actual system cannot establish.
11.2 A proposed PatientGPT pilot
A suitable initial PatientGPT journey, subject to its actual specification, could let an authorized test user provide a synthetic patient scenario and receive a reviewable information packet whose factual items are traceable to the supplied synthetic sources. Plan the intended value, missing-information handling, source traceability, access isolation, and retention behavior before generation; execute the agreed checks only at phase end.
Use synthetic data for the initial pilot. Independent domain experts must define and assess any clinical-quality claim. A technically successful workflow, correct document structure, or BDD pass is not evidence of clinical safety, diagnostic accuracy, treatment suitability, HIPAA compliance, or regulatory approval. Real patient data, clinical use, and regulated operations require their own verified requirements and authority. These are proposed applications, not findings about PatientGPT's current implementation.
12 Prior art and what this synthesis adds
12.1 The value question is already present in BDD
Dan North's original account of behavior-driven development connects the next important missing behavior to business value. That is direct prior art for a central part of VLBV. Cucumber's account also includes collaborative discovery, formulation, and automation; Gherkin is only a language used in part of that process. Calling BDD a convenient syntax would miss the very value-selection insight at issue here. North, Introducing BDD, Cucumber, Behaviour-Driven Development
Example Mapping makes rules, examples, open questions, and newly separated stories visible. Specification by example similarly uses a limited set of meaningful examples to improve shared understanding. Adzic warns that a large collection of scenarios can hide important concepts and disagreements instead of resolving them. These practices support planning understandable behavioral claims without precommitting to every possible test. Cucumber, Example Mapping, Adzic, Focus on key examples
VLBV differs in execution timing. Cucumber's automation cycle uses a failing example before implementation; conventional TDD uses tests to guide development as it proceeds. The owner's policy delays generated-code test execution until the end of a coherent value-producing generation phase. The shared lineage is discovery, value, and acceptance meaning. The timing difference should remain explicit rather than being hidden under the BDD label. Cucumber BDD, Freeman and Pryce, Growing Object-Oriented Software Guided by Tests
12.2 Thin end to end delivery and evidence based acceptance are established
A walking skeleton provides a minimal working path across major components. It helps distinguish architectural integration from the completion of isolated layers. Impact Mapping connects proposed deliverables to actor behavior and goals. These ideas explain why one meaningful path can be a better planning unit than an inventory of technical components, while leaving open how much validation that path requires. Shank, Start with a Walking Skeleton, Impact Mapping
Assurance cases supply another important ancestor: a claim needs relevant evidence and a reason that the evidence supports it. Several artifacts repeating the same run are not independent corroboration. Evidence must apply to the candidate under consideration, or a justified invariance argument must explain why a change does not affect it. VLBV adopts a lightweight version of that discipline, not the assurance burden appropriate to every regulated system. SEI, Assurance Cases and Confidence, SEI, Assurance Cases Overview
Model-based testing already distinguishes abstract behavior, concrete execution, and adapters. Its literature also emphasizes that the test model needs independent validity and can become too complicated to validate cheaply. VLBV's portable scenario and adapter separation is therefore an application of established ideas to agentic workflows, not an invention of abstraction or adapters. Utting, Pretschner and Legeard, A taxonomy of model-based testing approaches
12.3 Agent engineering supplies close practical precedents
Anthropic's long-running-agent report describes concrete end-to-end feature definitions, incremental work, browser verification, and durable progress artifacts. Its accompanying coding prompt guards against changing the criteria merely to obtain a pass. These are close practical precedents, although the report emphasizes full-stack web applications and does not provide a controlled estimate of the method's effect. Its large initial feature list and browser-specific execution rules should not be copied as universal VLBV requirements. Anthropic, Effective harnesses for long-running agents, Coding prompt
More recent agent-evaluation guidance begins with common user tasks and real failures, distinguishes tasks from trials and graders, and favors checking actual outcomes over narrated success. A later harness-design report describes evaluator feedback and grading calibrated against human review; it also describes removing some orchestration when a stronger model made it less useful. The latter is an important caution against treating one-at-a-time decomposition or a particular multi-agent topology as a permanent law. Anthropic, Demystifying evals for AI agents, Anthropic, Harness design for long-running apps
EDDOps formalizes evaluation-governed agent development and operations, including traceable evidence and risk-aware cases. Its validation is analytical and case-based rather than an experiment showing a causal productivity gain. VLBV is narrower: a portable decision protocol for selecting, verifying, and closing bounded value phases. It should be able to contribute to a broader evaluation lifecycle without recreating one. Xia et al., EDDOps
12.4 Research supports calibration while warning against surface success
TiCoder investigated user feedback on generated tests in a small controlled study of code-selection tasks. Its pass/fail-feedback condition improved correct selections compared with the control, but the study involved 15 participants and constrained Python tasks, not autonomous verification of complex products. It supports the plausibility of concrete examples clarifying intent; it does not validate the full VLBV policy. Fakhoury et al., TiCoder
An industrial acceptance-test-generation study is especially relevant. It separates stories-to-Gherkin from Gherkin-and-actual-UI-to-Cypress. In one evaluation of 50 cases, all generated outputs were syntactically valid, but only 60 percent initially matched the intended scenario meaning. A valid feature file is therefore a poor substitute for reviewed relevance and expected behavior. The study's setting and sample constrain generalization. Ferreira et al., Acceptance Test Generation with Large Language Models
Other research reinforces the distinction. One BDD study's reported accuracy was freedom from Gherkin-lint syntax issues, not semantic correctness or coverage. A 2026 preprint found benefits from richer descriptions, but the effect of few-shot examples differed across the tested models. VLBV should validate calibration on unseen cases rather than assuming that examples improve every model or transfer unchanged. Karpurapu et al., BDD acceptance test formulation, Rathnayake, Shahin and Abaei, BDD scenario generation
The standard in-context-learning interpretation involves supplying demonstrations in the input without gradient updates during the evaluation. It is a plausible explanation for improved proposals after owner examples. It says nothing by itself about the private memory implementation, later training, or durable retention behavior of a particular coding product. Brown et al., Language Models are Few-Shot Learners
12.5 A passing benchmark is not the same as an acceptable product change
The test oracle is a separate problem from generating inputs. A check can reject correct implementations, accept incorrect ones, or encode a misunderstanding shared with the generated code. More scenarios do not resolve those defects automatically. Barr et al., The Oracle Problem in Software Testing
An OpenAI audit found material test or specification problems in 59.4 percent of a difficulty-selected set of 138 SWE-bench Verified tasks. That figure must not be presented as the prevalence across the entire benchmark. Conversely, broader testing of previously passing agent patches has found additional failures, and maintainer review has identified acceptance gaps beyond automated scores. These findings point in both directions: an oracle can be too restrictive or too weak. OpenAI, SWE-bench Verified limitations, Wang, Pradel and Liu, Are Solved Issues Really Solved Correctly, METR, Many SWE-bench-passing PRs would not be merged
These studies do not prove that VLBV is better. They explain why its acceptance review, evidence boundaries, and holdout checks are necessary subjects of validation.
12.6 The contribution is an explicit practical combination
The proposed contribution is a small protocol that combines value-led scenario selection, owner-calibrated relevance, declared phase-end execution timing, scoped evidence, bounded adaptive verification, and explicit phase closure. It makes the authority and evidence problems of agentic tools visible while keeping execution delegated to existing capabilities.
None of those ingredients is wholly new. The name VLBV identifies the owner's practical synthesis and makes it easier to discuss, package, and test. This paper makes no priority, patentability, trademark-availability, or universal-optimality claim. The right next step is an auditable prototype and a fair comparison, not a declaration that a new methodology has already been proven.
13 A falsifiable validation study
Calibrate, then challenge the claim
13.1 Questions that would change the recommendation
The first hypothesis is that a value-led adaptive frontier reduces time and total effort to the first independently accepted useful behavior, compared with infrastructure-first verification on matched tasks. The second is that owner-calibrated examples reduce irrelevant scenario proposals and unsupported requirements. The third is that explicit evidence and closure contracts reduce false completion claims and unnecessary reopening. A fourth hypothesis concerns portability: the same semantic contract can be executed in a second environment with bounded adapter work and no loss of claim meaning.
These hypotheses could fail. Calibration might cost more owner time than it saves. A fixed backlog might perform as well as adaptive selection. A mature existing harness might beat a thin new protocol. Deferred execution might increase repair cost enough to outweigh reduced interruptions. Portability might require extensive redesign rather than a small adapter. The study should preserve those outcomes.
13.2 Use fair baselines
A core comparison should include the team's existing harness-first process, a fixed scenario backlog, and the VLBV value-led adaptive policy. The baseline must be competently configured and have access to the same reasonable reusable infrastructure. It should not be deliberately burdened with unnecessary generalization.
Keep model versions, tools, permissions, environment quality, task scope, execution budgets, and human assistance comparable. Use several products or representative components so that a single UAR-specific fixture does not determine the result. Randomize task assignment or use a counterbalanced design, account for learning and carryover, and keep unsuccessful attempts in the analysis.
The owner's phase-end timing policy should be tested separately from scenario selection where feasible. One study can hold phase-end timing constant and compare selection policies. A second can compare phase-end and continuous execution while holding the value contract and tooling constant. This prevents a benefit caused by improved scenarios from being attributed to delayed testing, or vice versa.
13.3 Define measurements before execution
The primary outcome should be elapsed time and total effort to the first independently accepted end-to-end value slice, starting from an agreed task brief. Total effort includes owner correction, adapter construction, environment setup, verification, and repair. Excluding those costs would conceal the problem the method is intended to solve.
Secondary measures should include:
| Measure | Operational definition | Why it matters |
|---|---|---|
| Accepted value over time | Independently accepted claims, with stakeholder-approved weights recorded before results | Avoids rewarding many trivial cases |
| Critical constraint failures | Violations in the separately defined mandatory-risk set | Prevents a good average from hiding severe failures |
| Unsupported completion claims | Claims whose cited evidence does not establish the declared boundary | Measures reporting honesty and oracle alignment |
| Reopen and escape rate | Accepted claims later contradicted by held-out checks or intended-use failures | Measures durability of closure |
| Owner correction burden | Time and number of material relevance, scope, and oracle corrections | Tests whether calibration saves human work |
| Orchestration overhead | Effort on reusable machinery not directly required by current claims | Detects infrastructure displacement |
| Reproducibility | Independent reruns that reproduce the scoped verdict under the stated conditions | Tests receipt usefulness |
| Portability cost | Work needed to execute the same semantic contract in a second declared environment | Tests harness-independence claims |
| Phase-end repair effort | Defects and effort discovered after generation completes | Exposes delayed-feedback tradeoffs |
A reviewer unaware of the assigned method should judge acceptance where practical. Use holdout scenarios derived independently from the same product intent. Holdout testing must obey the same access and side-effect permissions. A held-out test should not quietly invent a requirement absent from the accepted specification; disagreements need adjudication.
13.4 Evaluate calibration rather than style imitation
Compare specification-only generation with specification plus reviewed positive, negative, and rare-critical examples. Present unseen candidate scenarios and ask the assistant to identify value, missing information, likely-use basis, risk, and the next warranted action. Measure priority agreement and substantive errors, not resemblance to the owner's wording.
Record the exact supplied context and the exposed model identity. On a model or host change, repeat the calibration check. Good performance on the examples themselves is not evidence of generalization. A result that improves Gherkin formatting while leaving relevance unchanged should be reported as a formatting improvement.
13.5 Treat nondeterminism explicitly
A fixed scenario can yield different results because of model sampling, asynchronous tools, external services, or variable data. The contract should distinguish deterministic invariants from probabilistic quality claims. Fix the sampling unit, input distribution, reset policy, failure definition, judge, and stopping procedure before interpreting repeated trials.
A simple mathematical illustration shows why a few successes are weak reliability evidence. For a fixed artifact and oracle, independent identically distributed Bernoulli trials, a fixed sample size of n, and zero observed failures, a one-sided 95 percent upper confidence bound on failure probability is 1 - 0.05^(1/n). Thirty clean trials still give an upper bound of about 9.5 percent; 300 give about 0.994 percent. These are illustrative calculations under restrictive assumptions, not a recommended universal sample size or a statement that a system is safe. Dependence and unrepresentative sampling can make reliability conclusions optimistic. Salako and Zhao, reliability and independence assumptions
A demonstration may need one well-observed run. A claim about reliability usually needs more. Those are different claims and should have different labels. Informal rerun-until-green behavior supports neither.
13.6 Decide what would count as enough evidence to adopt
Before the pilot, the team should set a decision rule appropriate to its costs and risks: a practically useful reduction in time to accepted value, no unacceptable increase in critical failures, tolerable owner burden, and adequate reproducibility. Report uncertainty and per-task variation rather than one flattering aggregate. Do not choose the success threshold after seeing the outcome.
Adoption can be selective. The evidence may support VLBV for phase-end integration acceptance while favoring existing continuous checks for another context. A mixed result is useful if it identifies where the method pays for itself and where it does not.
14 Alternatives and combinations
A mature existing test platform
If an established platform already provides reliable fixtures, observation, access control, and receipts, use it. VLBV can supply the value contract and selection policy above that platform. Replacing working infrastructure would add cost without establishing the proposed benefit. Conversely, when infrastructure must be created, tie each addition to a current claim rather than hypothetical universality.
Continuous testing and test driven development
Continuous checks can expose defects earlier and reduce the size of an interacting failure set. TDD can clarify design through executable examples. They may be better choices for some teams and systems. The owner's VLBV timing policy instead favors uninterrupted generation of a coherent value increment before execution. The paper preserves that preference while making its costs measurable. Source inspection and planning should not be relabeled as tests, nor should actual code execution be hidden under the name “preflight” to evade the declared boundary.
Exploratory testing
Exploration is valuable when requirements are incomplete or the system's behavior is surprising. VLBV can record an exploratory charter as a bounded information-gathering activity, then turn important findings into reviewed claims. Exploration does not need to pretend that every observation was predicted in Gherkin. It also should not silently add all discovered possibilities to the current acceptance contract.
Property based testing and mutation testing
Example scenarios establish understandable claims; property-based testing can explore larger input regions, and mutation testing can challenge whether checks detect relevant faults. These techniques complement each other. A large generated sample may still concentrate on trivial cases, and a mutation score may be distorted by equivalent or irrelevant changes. Use them at phase-end where the risk justifies their cost, with bounded selection and documented interpretation. Claessen and Hughes, QuickCheck, PIT, basic concepts
Formal verification and domain assurance
Formal methods can establish specified properties under explicit semantics and assumptions. They are not replaced by a small number of behavior tests. Safety-critical, security-sensitive, and regulated contexts may need specialist assurance, independent evaluation, and controls beyond VLBV. The protocol can organize scoped evidence and ownership, but the existence of a closure packet is not a certification. Finite ordinary testing does not establish unrestricted correctness. Dijkstra, Notes on Structured Programming
Parallel execution
One behavior at a time is often a useful starting discipline because it simplifies attribution. It is not an immutable rule. Independent scenarios may execute concurrently after the phase-end trigger when their environments, data, and evidence are isolated. Shared mutable state, ambiguous logs, or a coupled repair can remove that independence. Parallelism should be justified by the scenario dependencies, not by the number of available agents.
15 Limitations and failure modes
The first limitation is specification quality. A precise scenario can make an incorrect assumption look authoritative. The contract should distinguish facts, owner decisions, inferred requirements, and unresolved questions. The assistant may propose requirements; it may not silently promote them to approved scope.
The second is owner knowledge. A product owner can misjudge common use or overlook a severe risk. Relevance calibration improves communication of the owner's judgment; it does not make that judgment infallible. Domain experts, support evidence, actual usage, and independent review remain important where appropriate.
The third is correlated error. The same model can misunderstand a requirement, implement that misunderstanding, generate a matching test, and praise the result. Different role names or prompts do not by themselves make the evidence independent. A simpler reference, independently chosen data, direct state inspection, or a human reviewer can help challenge that shared error.
The fourth is the observer. UI screenshots may omit hidden state. Logs can be incomplete or stale. A process exit code can reflect only the runner. A successful request may precede a later failure. Each claim needs the observation boundary and time window that actually support it. A checksum can establish artifact identity relative to a recorded digest; it cannot prove that the artifact tells the truth.
The fifth is change. A provider alias, retrieval corpus, permission policy, configuration, dependency, or model may change while the application commit remains the same. Evidence freshness therefore depends on the inputs relevant to the claim, not just a recent timestamp. Reuse must include an invalidation decision.
The sixth is delayed feedback. A large generation phase can make phase-end failures expensive to diagnose. The answer is a coherent, bounded value phase, not an artificial rule that every line is a phase. The appropriate size remains a practical and empirical question.
The seventh is process overhead. If the protocol requires more metadata than the risk justifies, it can reproduce the machinery-first problem in another form. The minimum records should be small, derivable where possible, and stored once. Higher-assurance extensions should be triggered by the claim, not universally enabled.
The eighth is closure abuse. A team can define a weak contract and satisfy it while leaving the product unusable. It can also insist on complete certification for a phase that only authorized a limited deliverable. VLBV requires an explicit relationship between purpose, acceptance, residual limitations, and release authority. That relationship is a judgment requiring review, not a schema alone.
16 Adoption recommendation
Begin with one existing, coherent value phase whose owner can explain the intended outcome. Write a short contract and a few meaningful scenarios. Calibrate the next-scenario judgment with positive and negative examples. Keep generated-code execution at the declared phase-end boundary. Use the current runner and observation tools wherever they are adequate. Retain all attempt outcomes and produce a closure packet that states what is supported, what is not, and why the phase is closing.
Only after that manual protocol works should a reusable skill automate the repetitive parts. Only after a second execution environment has been exercised should portability be claimed for that pair. Only after a comparative study should the team claim a measured improvement.
The intended endpoint is straightforward: the owner can understand which useful behavior is supported by current evidence, the assistant can explain the next warranted action, and the phase can end without confusing a stopped process, completed code generation, a passing scenario, and an authorized release.
17 Case evidence and provenance
Evidence classes
This paper distinguishes owner testimony, historical assistant reports, inspected original artifacts, source review, and proposed design. Inspection is not execution replay. A paper can accurately describe a retained result without claiming that its authors independently reproduced the product behavior. Public references below use opaque case identifiers; the private evidence map retains the source locations for authorized audit without publishing raw conversations or personal filesystem details.
C1 Owner requirements for useful completion
Owner statements from 18 October 2026 establish the demand for useful-task-based measurable completion, understandable capability reports, and prioritization of intended store configurations. They are direct evidence of the owner's requirements, not product test results.
C2 Earlier qualification reports
Owner-pasted 18 October reports describe readiness work, narrow passes, and remaining failures. Their relationship to every later preparation attempt is not assumed. They support the correction that earlier work did produce some evidence.
C3 Evidence boundary and closure corrections
Historical 19–21 October records distinguish stub-model from configured-real-model claims, prerequisite progress from formal criterion closure, and authored work from validated software. These are historical reports and owner corrections, not current product certification.
C4 Owner account of the manual transition
The owner's 27 October explanation identifies the Node.js and Cucumber intervention and emphasizes value, likely uses, and results determining the next question. The primary session investigation in C7 corroborates and refines the chronology: the first selected prompt used a standalone Node.js script; Cucumber followed.
C5 Governing method and execution policy
The owner approved the name Value-Led Behavioral Verification and explicitly required no tests of generated code until the end of the complete phase producing demonstrable value. This is a design requirement for this paper. C8 corroborates the existing whole-phase policy and identifies older conflicting guidance that must not silently override it.
C6 Initially observed screenshots
Displayed restart summaries showed one scenario with six reported steps in 21.43 seconds and one with five reported steps in 13.834 seconds. Screenshots alone would establish only the displayed reports. C7 subsequently corroborated their underlying retained evidence; neither the screenshots nor that inspection establish broad reliability or full product completion.
C7 Primary UAR Verification investigation
The read-only investigation identified the actual session, used 20 successful paginated application reads covering 155 unique turns, and structurally traversed the available local record through 30,421 lines and 20 compactions. A retained export contained 1,066 entries, including injected and automated material as well as human messages. The investigation examined the substantive chronology and selected evidence; it did not independently audit every tool-output character.
Seventeen later PASS receipts were inspected and indexed, and 11 selected original Cucumber reports were directly parsed. The source record supports three preparation campaigns within one evolving effort, followed by the value-led manual sequence. The ledger's 25 behavior labels across 22 journeys include composite evidence and mixed candidate identities. The cutoff was 27 October 2026 at 00:14:49 UTC; knowledge-base use after restart was being authored, not established as passed. No product execution was rerun for this investigation.
Remaining gaps are exact causal attribution, comprehensive raw-protocol replay, some lost earlier failure payloads, complete reconciliation of manual receipts with canonical phase state, platform parity, broad reliability, and outcomes after the cutoff. Compaction and the session start also prevent claims about every earlier UAR development session. The public case is a bounded retrospective, not an exhaustive forensic or release audit.
C8 Existing process and phase boundary investigation
Read-only inspection established the project meanings of PMPO and KBD, the OpenSpec spec-driven configuration, existing phase and change relationships, adopted-contract fields, whole-phase execution rules, and the difference between manual results and canonical closure. It also found material differences between installed and repository-local KBD guidance and between some verdict representations. No VLBV adapter was implemented, no schema compatibility was certified, and no canonical state was changed by this investigation.
Public research scope
The literature review used original author accounts, official documentation, and primary research. It is a bounded synthesis, not a systematic review of every testing or agent-evaluation publication. Engineering practice reports, controlled experiments, industrial case studies, preprints, and architecture proposals are identified as different evidence classes in the discussion. Sources were checked for this edition on 27 October 2026; mutable documents and repositories should be pinned when used in an executable reproducibility package.
The closest intellectual anchors are North's original BDD account, Cucumber's discovery and automation guidance, the model-based testing taxonomy, SEI's assurance-case guidance, agent evaluation practice, and the industrial acceptance-test-generation study. Detailed links appear beside the claims they support throughout the paper.
18 Preventing a retired harness from returning
A failed method can survive its apparent replacement in plans, handoffs, compaction summaries, familiar helpers, and old skills. The assistant may see those records later and reconstruct the very workflow the owner retired. “Use a simpler approach” is too ambiguous to prevent this. A deliberate supersession record must identify what is retired, what remains authoritative, and what can be reused.
This is not permission to erase evidence, ignore product defects, or bypass security. It retires an operating method. Legitimate requirements, applicable receipts, approval boundaries, and authorized exit contracts remain. If a control belongs to a binding contract rather than merely to the retired harness, its disposition requires the appropriate authority and an explicit amendment.
Reusable supersession instruction
SUPERSESSION RECORD
Authority and effective decision:
[Name the authorized owner decision, date, and affected phase.]
Retired method:
[Identify the failed verification workflow and its active plan references.]
The retired method is SUPERSEDED for this phase. Do not repair, extend,
invoke, regenerate, or inherit its readiness, orchestration, fixture,
inventory, or all-platform gates merely because they exist in old plans,
handoffs, skills, or conversation summaries. Preserve the files as history;
do not delete them or relabel their failures as successes.
Retain:
- Legitimate product requirements and the accepted scenario inventory.
- Useful owner relevance examples and explicit exclusions.
- Valid scoped receipts whose candidate and claim applicability still hold.
- Actual security, privacy, access, approval, and adopted exit obligations.
Replacement method:
Use the agreed value phase and a few calibrated meaningful scenarios.
Complete the entire agreed implementation before generated-code checks.
At the authorized phase-end boundary, use the smallest adequate real
execution route, preserve scoped evidence, and close or name the blocker.
Select later work from value and observed uncertainty, not old harness scope.
Helper reuse rule:
Reuse a helper only when it is needed for the selected scenario, its behavior
is understood, and it does not pull the retired workflow or unrelated gates
back in. Otherwise use a smaller existing route or report the exact gap.
Evidence rule:
Do not rerun earlier accepted behavior automatically. Identify the relevant
change or invalidation first. Preserve genuine failures and unknown effects;
quarantining the old method must not suppress counterevidence.
Continuity rule:
Mark the active plan, handoff, and next compaction or recovery record with
this supersession decision through their authorized writers. Keep links to
the retired history and the replacement contract. If instructions conflict,
state the conflict and obtain the necessary resolution; do not bypass a
higher-priority rule or silently restore the old method.
This template is a proposed artifact for owner review and authorized use. It does not itself update any repository, phase record, skill, or running session. Its acceptance check is simple: after a fresh-session handoff, the assistant should identify the current method, reject an invitation to rebuild the retired harness, retain a relevant product failure, and preserve a mandatory approval gate. That is a useful additional challenge for VLBV 03, 11, 18, and 20.
19 Compact operating checklist
Before generation, confirm the useful outcome, the complete bounded phase, the relevant scenarios, owner calibration, observables, critical constraints, exclusions, and who may decide acceptance. Keep likely-use judgments sourced or explicitly uncertain.
During generation, author the agreed increment and scenarios without executing generated-code checks. Record progress at checkpoints without treating time, task completion, or a partial repair cohort as a new test boundary. Preserve the current supersession decision so old machinery does not return through context recovery.
At phase end, establish the actual candidate and authority. Run the selected real behavior through the appropriate existing tools. Check that the oracle distinguishes the intended success from a plausible false positive. Retain failures, boundaries, side effects, restoration, and unknowns.
At closure, say exactly what the evidence supports. Separate implementation, behavior, release eligibility, and authorization. Preserve any continuing obligation. Choose the next scenario or phase because it matters and resolves a real uncertainty, or stop because the agreed purpose has been met.
The method is deliberately small. Its success will be measured by useful behavior that a reader can understand and trust within stated limits, not by the size of the harness built around it.