1 / 6
LLM evaluation

Detect an unsupported claim

Hard120 pts~45 min
  • Hallucination
  • Grounding
Practice app · Acme Support Assistant

A deterministic LLM-style support assistant with retrieval (RAG), JSON mode, safety policies and tool calls, exposed via UI and API.

BASE_URL
/api/practice
Console app
/lab/ai-testing-detect-an-unsupported-claim

Your starter code already declares BASE_URL — call the API relative to it.

Objective

Build a checker that flags claims in an answer that are not supported by the cited documents.

Your task

  1. 1GET BASE_URL + "/ai/kb" to load the documents.
  2. 2Ask "What warranty do your products have?" and collect the content of every cited doc.
  3. 3Assert every number/duration token in output_text (e.g. “12-month”) appears in the cited content.
  4. 4Ask "Do you offer student discounts?" and assert “I don't have that information” with no citations.

Acceptance criteria

  • GET /ai/kb returns 200
  • POST /ai/chat returns 200
  • At least 3 assertions pass

LLM evaluation · AI Testing · Hallucination & grounding (RAG)