Required outcomes
Verify useful responses, structured fields, citations, actions, and tool completion.
MCP prompt regression automation
Change the model or prompt without changing the promise.
Version positive, negative, boundary, and tool-use cases. Automate replay against every candidate and explain which behavior moved, why it matters, and whether it blocks release.
Golden casesStructured assertionsTool behavior
assistant-v12 → v13
$ prooflane gate --prompts release cases 96 passed 91 failed 3 inconclusive 2 + better citation format − tool argument lost account scope − refusal on allowed refund preview BLOCK · 2 required behaviors regressed
Behavior coverage
Test what the agent must do—and must not do.
Forbidden behavior
Reject disallowed actions, secret disclosure, unsupported claims, and unnecessary tools.
Ambiguous edges
Test clarification, abstention, missing context, invalid input, and uncertain evidence.
Agent traces
Assert selected tool, call count, argument shape, order, and final task outcome.
Hybrid graders
Prefer deterministic assertions, then apply bounded rubrics only where semantics require them.
Explain the change
Compare case evidence and trace behavior instead of reducing every change to one score.
Prompt Studio walkthrough
Turn an idea or incident into a permanent test.
Author cases visually, generate adversarial variants with AI, and promote stable behavior into a release gate.
REAL PRODUCT · Prompt Studio
Prompt StudioAuthor reusable cases, assertions, and regression suites visually.
REAL PRODUCT · Prompt generation
Generate prompts with AICreate positive, negative, boundary, and adversarial cases directly in the product.
Workflow
A memory for expected agent behavior.
Capture
Convert incidents, requirements, and edge cases into versioned tests.
Assert
Define deterministic, schema, semantic, tool, and safety expectations.
Replay
Run the same cases across model, prompt, tool, or application changes.
Review
Accept intentional goldens or block regressions with trace-level evidence.
prooflane gate --prompts release --baseline .prooflane/prompts.json prooflane runs show latest --failures-only
Part of the spine
Behavior tests become stronger with adjacent evidence.
Keep the promise
Make every hard-earned fix permanent.
Turn expected behavior into a release gate your team can inspect.
30-day full-access trial. No credit card required.