Tool calling

Tool calling is the bridge between model reasoning and real software capabilities. Instead of asking a model to invent a result, the application exposes a typed capability that the model can request.

Lifecycle

1. The application sends the model a small set of available tools.
2. The model returns a structured tool request.
3. The application validates the tool name and arguments.
4. Authorization checks whether the signed-in user may perform the operation.
5. The application executes the tool.
6. The result is returned as new context.

The model does not execute your function. Your application does.

Tool design

Good tools are narrow, composable and explicit. Prefer get_order(orderId) over run_database_query(sql). Descriptions should explain purpose, inputs, constraints and outputs. Schemas should reject malformed data before execution.

Read vs write

Read operations still require authorization, but write operations need stronger controls because they create side effects. Use approval gates, idempotency keys, current-state checks and limits for high-impact actions.

Tool results are untrusted

External APIs, webpages and retrieved documents can contain text that attempts to manipulate the next model step. Treat tool output as data, not authority. Keep policy instructions separate from untrusted content.

Errors

Return structured errors such as NOT_FOUND, FORBIDDEN, RATE_LIMITED and RETRYABLE. Avoid exposing stack traces, credentials or internal infrastructure details to the model.

Multiple tools

When several tools can solve a task, reduce ambiguity through clear names and descriptions. Parallelize independent read operations only when doing so does not break ordering, consistency or rate limits.

Example: order assistant

Expose get_order, search_products and cancel_order. The model may request cancel_order, but the backend verifies ownership, cancellation eligibility and idempotency before changing anything. If approval is required, the tool returns an approval-required state instead of executing.

Safe executor

async function executeTool(call: ToolCall, ctx: Context) {
  const tool = registry.get(call.name);
  if (!tool) return { error: "TOOL_NOT_ALLOWED" };
  const args = tool.schema.parse(call.arguments);
  if (!(await tool.authorize(ctx, args))) return { error: "FORBIDDEN" };
  return tool.execute(ctx, args);
}

Never treat model-generated tool arguments as trusted input.

Practice

Turn three APIs into agent tools. For each define the exact name, input schema, authorization rule, side-effect level, retry behavior, maximum frequency and safe error response.

Challenge

Build a tool registry that supports read-only and write tools. Require an idempotency key for every write and reject every tool that is not explicitly registered.