Compare Saved Decisions Again

Use Native Lispex and the npm CLI, Node.js, and Bash. Set NATIVE to the Native executable's absolute path. Set NPM_CLI to the installed npm package's bin/lispex.js path. This example first runs Native to create reports. The npm vouch replay command then compares those saved reports.

Prepare two cases

Work in a new directory. Create the report directories.

mkdir -p corpus/expected candidate

The command prints nothing and exits with code 0. Save these four complete files as UTF-8 with LF line endings and a final line feed.

refund-old.lspx

(if (<= input 14) '(decision allow) '(decision deny late))

refund-new.lspx

(if (<= input 13) '(decision allow) '(decision deny late))

fault.lspx

(/ input 0)

input.datum

14

The old rule accepts day 14. The new rule accepts days up to 13. Both runs of fault.lspx attempt division by zero.

Save the case list as corpus/manifest.json. A corpus is the set of cases you choose before comparing results.

{
  "decision_gallery": "csk.profile-decision-gallery/v0",
  "cases": [
    {
      "stem": "refund",
      "expected_transcript": [
        "(decision allow)"
      ]
    },
    {
      "stem": "fault",
      "expected_transcript": []
    }
  ]
}

Each stem names a JSON file under corpus/expected. The expected_transcript field lists the expected recorded values.

Record both rules

Run each source with the same input.datum.

"$NATIVE" diff-receipt --input input.datum refund-old.lspx > corpus/expected/refund.json
"$NATIVE" diff-receipt --input input.datum refund-new.lspx > candidate/refund.json
"$NATIVE" diff-receipt --input input.datum fault.lspx > corpus/expected/fault.json
"$NATIVE" diff-receipt --input input.datum fault.lspx > candidate/fault.json

The first two commands exit with code 0. The last two exit with code 1 because division by zero prevents comparison. Each command writes a complete csk.differential-receipt/v0 report to the named file; stderr is empty.

Here, agree compares two execution methods within one run. Native evaluates the source and its derived Meaning representation. The next command compares the old run with the new run.

Compare the saved reports

node "$NPM_CLI" vouch replay corpus --against candidate > replay-report.json

The command exits with code 1, creates replay-report.json, and writes this message to stderr.

replay found behavioral differences
- refund: reference-transcript, meaning-env-transcript

The report has format csk.replay-report/v0. Its complete summary is:

{
  "status": "changed",
  "exit_code": 1,
  "total": 2,
  "byte_changed": 1,
  "decision_changed": 1,
  "not_projectable": 2,
  "faults": 1
}

One refund decision changed. One case contains an execution fault. not_projectable: 2 counts the old and new sides of that fault case.

Read each case

For refund, decision.old.datum is (decision allow). decision.new.datum is (decision deny late). For fault, both decision sides have status: "not_projected" and reason not-attempted-runtime-or-meaning-fault.

changed: false on the fault case means its compared report fields stayed the same. not-comparable means execution supplied no usable comparison. decision_changed: null means the tool could not compare decision values. Read those fields together before accepting a result.

Choose the comparison input

corpus/expected holds the earlier reports. --against candidate selects the candidate report directory. Preserve both sets, the source and input bytes, and the executable identifiers.

Version mode reads the existing records. Run Native separately when you need results from another rule or executable. The report can say unchanged, changed, or failed-precondition. An unchanged report can still contain faults, so also read faults, not_projectable, and the case details. Exit 0 means no differences under the selected comparison. Exit 1 reports a difference or failed precondition. Usage and file-reading errors use exit 2.

The npm command checks stored evidence. Vouch authentication and a fresh Native gate are separate steps. The calling application decides whether requests or grants are fresh and may be used again. It also authorizes and performs external actions.

Lispex Vouch · Vouch reports · Application handoff