Agent architecture
Agent architecture
Learn the components that turn a model call into a controllable agent runtime.
flowchart LR U[User] --> O[Orchestrator] O --> M[Model] O --> ST[State] M --> P[Policy] P --> T[Tools] T --> S[Business services] O --> K[Knowledge] O --> L[Observability]
Lesson overview
Agent architecture
A production agent is a collection of cooperating layers, not one giant prompt.
Model layer
The model interprets context and proposes the next action. Select models based on task quality, tool-use reliability, context capacity, latency and cost—not only benchmark scores.
Orchestrator
The orchestrator owns the run loop. It loads state, builds model input, accepts a decision, validates it, executes tools and decides whether another iteration is allowed. Business rules and security checks should remain deterministic.
State
State records what happened during a run: goal, messages, tool calls, results, approvals, errors and intermediate decisions. Separate ephemeral run state from durable user data.
Tools
Tools are typed capabilities. Prefer narrow business actions such as get_order or create_ticket over generic SQL, shell or unrestricted HTTP access. Narrow tools reduce ambiguity and blast radius.
Policy
Policy answers whether an action is allowed. It can depend on identity, role, resource ownership, risk, environment and approval. Policy executes before side effects.
Knowledge and memory
Retrieval supplies external knowledge. Memory stores useful state across interactions. They solve different problems and should not be collapsed into one unbounded context.
Observability
Every run should produce a trace that can explain model calls, tool calls, retrieval, policy decisions, latency, cost, errors and final outcome.
Separation of concerns
The strongest architecture has a clear authority boundary: the model decides what it wants to do; the application decides what it is allowed to do; business services perform the operation; observability records what happened.
Learning path
Theory → Example → Code → Practice → Quiz → Challenge → Completion
Step 1
Theory
Agent architecture
A production agent is a collection of cooperating layers, not one giant prompt.
Model layer
The model interprets context and proposes the next action. Select models based on task quality, tool-use reliability, context capacity, latency and cost—not only benchmark scores.
Orchestrator
The orchestrator owns the run loop. It loads state, builds model input, accepts a decision, validates it, executes tools and decides whether another iteration is allowed. Business rules and security checks should remain deterministic.
State
State records what happened during a run: goal, messages, tool calls, results, approvals, errors and intermediate decisions. Separate ephemeral run state from durable user data.
Tools
Tools are typed capabilities. Prefer narrow business actions such as get_order or create_ticket over generic SQL, shell or unrestricted HTTP access. Narrow tools reduce ambiguity and blast radius.
Policy
Policy answers whether an action is allowed. It can depend on identity, role, resource ownership, risk, environment and approval. Policy executes before side effects.
Knowledge and memory
Retrieval supplies external knowledge. Memory stores useful state across interactions. They solve different problems and should not be collapsed into one unbounded context.
Observability
Every run should produce a trace that can explain model calls, tool calls, retrieval, policy decisions, latency, cost, errors and final outcome.
Separation of concerns
The strongest architecture has a clear authority boundary: the model decides what it wants to do; the application decides what it is allowed to do; business services perform the operation; observability records what happened.
Step 2
Example
Example: CRM agent
The model chooses whether to search CRM, inspect a deal or draft an email. The orchestrator enforces an eight-step budget. CRM tools expose only approved operations. A policy layer verifies tenant and record access. Sending an external email requires approval. Traces record tool names, latency and outcome without storing credentials.
Step 3
Code
Typed tool boundary
type Tool<TArgs, TResult> = {
name: string;
schema: Schema<TArgs>;
authorize: (ctx: Context, args: TArgs) => Promise<boolean>;
execute: (ctx: Context, args: TArgs) => Promise<TResult>;
};A tool owns validation, authorization and execution rather than trusting model output.
Step 4
Practice
Practice
Draw seven boxes: model, orchestrator, state, tools, policy, knowledge and observability. For each box write what it owns and what it must never own. Mark every boundary where an incorrect decision could create a real side effect.
Step 5
Quiz
1. Which layer should own the agent loop?
2. What is a tool?
Step 6
Challenge
Challenge
Design an agent that can read a database, search documentation and create support tickets. Add tenant authorization, a five-step budget, tracing and a human approval gate for ticket creation.
Complete every stage
Work through every step in order, then the lesson will be marked complete.
Each chapter and subtopic has its own public URL under /ai-agent.