Structured outputs and function calling
Structured output makes the model reply in a fixed format that code can read, usually JSON that matches a schema. Function calling lets the model ask your code to run one of your functions, with the arguments it chose.
Why it matters
Free-form text is hard for programs to use reliably: a missing field or an extra sentence breaks parsing. And models can't look things up or take actions by themselves. Structured outputs make answers reliable to parse, and tool calling connects the model to live data and real actions — the basis of every agent.
The idea
Structured output means making the model reply in a fixed format, usually JSON that matches a schema, so your code can read it reliably. Function calling (tool use) goes one step further. You describe functions to the model. Instead of answering in prose, it replies with a function name and the arguments to call it with. Your code runs the function and sends the result back.
A form with labelled boxes instead of a blank page. The model fills in the boxes, and your program reads them.
A person filling in a form knows the boxes are strict. A model can still put a wrong value in the right box, like a real date that is the wrong date, so you check the content as well as the shape.
How it works
- Define the shape you want with a JSON Schema: field names, types, which are required, allowed values.
- Send the schema with the request (many APIs can enforce it during generation), then validate every reply in code anyway.
- For tools, describe each function: its name, what it does, and a schema for its arguments.
- When a tool would help, the model replies with a tool call, such as {tool: get_weather, city: Leeds}, instead of prose.
- Your code checks the arguments, runs the function, and sends the result back as a tool-result message.
- The model then writes the final answer, or calls another tool. This loop is how agents work.
- If validation fails, send the error back and ask once for a corrected reply.
Key points
- The model chooses the call; your code runs it. Never execute arbitrary model output.
- Valid JSON can still contain wrong values: check content, not just shape.
- Good tool descriptions matter as much as good prompts.
- Make optional fields explicit, and allow null for missing data instead of invented values.
- Keep tools small and specific, with clear names.
Common mistakes
- Trusting a reply because it parsed. Valid JSON can still hold wrong or invented values, so check them.
- Letting a tool take dangerous actions without checks or confirmation.
- Vague tool descriptions, so the model picks the wrong tool or invents arguments.
- Forcing a required field when the data may genuinely be missing, so the model makes something up.
Not true: Marking a field as required makes the model fill it with correct information.
In fact: When the information is missing from the source, a required field forces the model to invent something. Let such fields be optional or null.
Not true: A reply that parses as valid JSON can be trusted.
In fact: Valid JSON only means the shape is right. The values inside can still be wrong or invented, so check them.
Part of Unit 5: AI engineering in the free ML Foundations course.