3 / 7
LLM evaluation

Validate output against a schema

Medium70 pts~25 min
  • JSON Schema
  • Structured output
Practice app · Acme Support Assistant

A deterministic LLM-style support assistant with retrieval (RAG), JSON mode, safety policies and tool calls, exposed via UI and API.

BASE_URL
/api/practice
Console app
/lab/ai-testing-validate-output-against-a-schema

Your starter code already declares BASE_URL — call the API relative to it.

Objective

Validate both the API response and the JSON-mode payload against schemas.

Your task

  1. 1Ask "Can I get a refund on a gift card?" in JSON mode.
  2. 2Validate the response against a schema: id, model, output_text, citations[], refused, tool_calls[], usage{…}, latency_ms, finish_reason.
  3. 3Validate JSON.parse(output_text) against { answer: string, sources: string[] } with no extra keys.

Acceptance criteria

  • POST /ai/chat returns 200
  • Schema validation is used
  • At least 2 assertions pass

LLM evaluation · AI Testing · Output validation