Required outcomes
Verify useful responses, structured fields, citations, actions, and tool completion.
Version positive, negative, boundary, and tool-use cases. Automate replay against every candidate and explain which behavior moved, why it matters, and whether it blocks release.
$ prooflane-inspector prompt regression prompt-suite.json \ --include-content \ --report prompt-report.html ✗ 1/3 · Return-window policy · FAIL · 850 ms ✓ 2/3 · Credential-handling policy · PASS · 910 ms ✓ 3/3 · Billing-owner verification · PASS · 980 ms 2 pass · 0 drift · 1 fail · 0 error Score 67 · required 100 Decision BLOCK HTML report: prompt-report.html
Verify useful responses, structured fields, citations, actions, and tool completion.
Reject disallowed actions, secret disclosure, unsupported claims, and unnecessary tools.
Test clarification, abstention, missing context, invalid input, and uncertain evidence.
Assert selected tool, call count, argument shape, order, and final task outcome.
Prefer deterministic assertions, then apply bounded rubrics only where semantics require them.
Compare case evidence and trace behavior instead of reducing every change to one score.
Author cases visually, generate adversarial variants with AI, and promote stable behavior into a release gate.
Convert incidents, requirements, and edge cases into versioned tests.
Define deterministic, schema, semantic, tool, and safety expectations.
Run the same cases across model, prompt, tool, or application changes.
Accept intentional goldens or block regressions with trace-level evidence.
prooflane-inspector prompt regression prompt-suite.json \ --include-content \ --out prompt-report.json \ --report prompt-report.html
Detailed CLI reports show the system instruction, input prompt, expected answer, observed model answer, and assertion explanation. Failed cases open expanded; credential-shaped values are redacted.
Prompt Regression CLI guideTurn expected behavior into a release gate your team can inspect.
Start using Prooflane free for 30 days. No credit card required.
Prooflane is beta software. Automated and model-based graders may be incomplete or wrong and do not guarantee future model behavior. Terminal output is illustrative and not a customer result.