AI Prompt Evaluation Framework: Testing, Scoring & CI/CD

Are the facts correct? Does the output answer the actual question?

How to measure: Spot-check 3+ claims per output against known sources. For code, does it compile and produce correct results?

Does the same prompt produce similar quality across multiple runs?

How to measure: Run the prompt 5 times. Score each output 1-10. If standard deviation >2, the prompt needs tightening.

Does the output match the specified structure, length, and tone?

How to measure: Define output requirements in STCO Output component. Check: correct format? Right length? Right tone?

How long does the AI take to respond? Is it within acceptable limits?

How to measure: Measure time-to-first-token and total generation time. Target: <5s for interactive, <30s for batch.

How many tokens does the prompt consume? Is it cost-efficient?

How to measure: Track input + output tokens. Optimise: shorter system prompts, concise context, constrained output length.

• ✅ Empty context: What happens when no background info is provided?
• ✅ Adversarial input: Does it handle injection attempts gracefully?
• ✅ Very long input: Does quality degrade with 10,000+ character context?
• ✅ Ambiguous request: Does it ask for clarification or guess?
• ✅ Out-of-scope: Does it refuse gracefully when asked about unrelated topics?
• ✅ Multilingual: Does it handle non-English input correctly?
• ✅ Contradictory instructions: How does it resolve conflicting requirements?
• ✅ Rapid-fire: Does quality hold up under 50+ consecutive requests?

Token-overlap metrics that compare generated text against reference outputs. BLEU measures precision (how much of the output matches the reference), ROUGE measures recall (how much of the reference appears in the output). Fast and deterministic but miss semantic quality.

Use when: You have gold-standard reference outputs and need a cheap, fast baseline score.

Uses BERT embeddings to measure semantic similarity rather than token overlap. Catches cases where the output is correct but uses different wording. More expensive than BLEU/ROUGE but far more accurate for open-ended generation.

Use when: Output quality matters more than exact wording — content generation, Q&A, creative tasks.

Use a frontier model (GPT-4o, Claude 3.5 Sonnet) to grade outputs on a rubric you define. Example: "Rate this response 1-10 on accuracy, completeness, and tone." The most flexible approach — handles any output type. Cost: ~$0.01-0.05 per evaluation.

Use when: No reference outputs available, or evaluating subjective quality dimensions like tone and helpfulness.

Skip scoring entirely — use JSON schema constraints so the output is valid by construction. <Link to="/research/citation/econ-006" style={{ color: "#818cf8", textDecoration: "underline" }}>Constrained decoding achieves 0% retry rate</Link> versus 15% with free-form output. The ultimate "eval" is making failure impossible.

Use when: Output must be machine-parseable — API responses, data extraction, classification.

# regression_test.py
import json, openai

TEST_CASES = [
  {"input": "Classify: 'I love this product'", "expected": "positive"},
  {"input": "Classify: 'Terrible experience'", "expected": "negative"},
  {"input": "Classify: 'It arrived on Tuesday'", "expected": "neutral"},
]

def run_regression(prompt_v2, threshold=0.9):
    passed = 0
    for case in TEST_CASES:
        result = call_llm(prompt_v2, case["input"])
        if result["sentiment"] == case["expected"]:
            passed += 1
    score = passed / len(TEST_CASES)
    assert score >= threshold, f"Regression: {score:.0%} < {threshold:.0%}"
    return score