Structured output with tool_use and JSON Schema

A system extracts invoice data by asking “return a JSON with these fields”. 3% of the responses cannot be parsed: an extra comma, a comment before the JSON. Asking for the format in text leaves the syntax up to the model following an instruction.

The output as a tool

The most reliable way to get output that complies with a schema is tool use: you define an extraction tool, for example extract_invoice, whose input_schema is a JSON schema with the fields you want. The model responds with a tool_use block, and its arguments (input) are your structured data. Nothing needs to run: you read the input and you are done. This eliminates syntax errors.

Which tool_choice to use

Value	What the model does	When
auto	May call a tool or answer in text	Normal conversation; does not guarantee the extraction
any	Must call a tool, but chooses which	Several schemas (invoice, receipt, credit note) and you do not know which document type arrives
{"type": "tool", "name": "extract_metadata"}	Must call that tool	A specific extraction that has to run no matter what, for example before the enrichment steps

With auto, the model may answer “Here is the data:” in text and your pipeline breaks. If you need the call every time, auto is the wrong answer. (Some recent models do not accept forcing the tool; there the documentation suggests auto with strict tool use or structured outputs.)

What the schema does not guarantee

The schema guarantees structure and types, not that the values are true. Semantic errors still happen:

• line items that do not add up to the total;
• the issue date in the due date field;
• a number with the right type but misread.

Those errors are caught with separate content validation, in code (covered in the next lesson).

Designing the schema so it does not invent

• Nullable fields for what may be missing. If due_date is required and the contract has no due date, the model invents one to comply. With "type": ["string", "null"], it returns null.
• Enum with “other” + detail for extensible categories: category: ["hardware", "software", "services", "other"] plus a free-text category_detail filled in when it picks “other”. That way it does not force an unusual case into the wrong category.
• “unclear” for ambiguity: if the document shows two contradictory statuses, a value that says so beats a random pick.
• Required with an empty array allowed for lists: it separates “this invoice has no line items” from “the field did not come back”.
• A field for extraction problems (unreadable document, missing page): it reports instead of guessing.
• Nest only what is needed: every extra level is harder to fill correctly.

Normalising formats

The schema says total is a number; the document says “1.250,00 $”. Add normalisation rules in the prompt alongside the strict schema (decimal point, dates as YYYY-MM-DD, currencies as ISO codes) and an enum of ISO codes for the currency instead of a free string that comes back as “USD”, “usd” or “US$”.

The typical traps

• Asking for JSON in the prompt, or “validating the JSON” with another model, instead of using a tool.
• tool_choice: auto when the extraction must always happen.
• Believing a strict schema prevents totals that do not add up.
• Making a field required when the data is sometimes absent.

On the exam

In the structured data extraction scenario, if parsing fails, the answer is a tool with a schema and a forced call. If the model invents values, the field becomes nullable. If some categories do not fit, “other” with detail. If there are several unknown document types, any; if one step has to run first, the tool forced by name.