You can repeat a rule under the same execution conditions and compare its result, output, warnings, and diagnostics. Keep the source, input, evaluator, language profile, and resource limits fixed. Getting the same result under the same conditions is called determinism, and the set of limits is called a resource profile.
Read a resource stop
In the embed example, the decision evaluator
runs (if (< input 15) "allow" "deny") with integer input 14.
The rule returns "allow". Its inspection JSON contains these fields.
{
"category": "semantic-complete",
"code": "complete"
}Set eval_work to 0 in a separate evaluation limits file and run the prepared
rule again. The new record contains these fields.
{
"category": "semantic-exhausted",
"code": "eval_work"
}The second run stops at its work limit. It has a resource outcome instead of
the rule's answer. Both embed evaluate commands exit with code 0 because
they successfully write an outcome file. Read category and code to tell
the outcomes apart. The guide provides every file and command.
Keep the execution conditions fixed
Determinism means the same result under the same stated execution conditions. Use the following information when repeating or comparing runs.
| Keep fixed | What to record |
|---|---|
| Source and input | Exact submitted bytes |
| Evaluator | Exact executable or WASM artifact and its identifier |
| Language profile | Supported forms and procedures |
| Resource profile | Selected limits and resource usage counter version |
| Output channels | Values, stdout, warnings, diagnostics, and their order |
A resource usage counter measures work as the selected evaluator defines it. Wall-clock duration measures elapsed time. Changing the runtime implementation can change its work counts, so compare exact usage within the same implementation.
For a comparison report, also keep the test inputs and expected results selected before execution. The report identifies those cases, executors, profiles, and channels. Read agreement, mismatch, or not-comparable within that named set.
Choose limits for the execution method
A resource profile groups the limits used by one execution method. Depending on the method, these cover input and output bytes, evaluation steps, control depth, and allocation. Execution finishes, reports a runtime error, or stops when it would exceed a limit.
The reference interpreter and Lispex VM run trusted source for
writing rules, local automation, and comparisons. The embed evaluators run
supplied rules with explicit work and allocation limits. Each preparation or
evaluation starts a fresh instance. Preparation and evaluation have separate
limits.
Keep engine failures and operational interruptions separate from a rule's value, runtime error, or resource outcome. The evaluator records them as engine status.