AI agent architecture: model, tools, state and control
AI agent architecture is the arrangement of the model, instructions, tools, state and controller that turns a request into a sequence of actions. It defines how observations enter the next decision, where permissions are enforced, how failures are recorded and which conditions end or interrupt a run.
Use this guide when an agent diagram looks clear but you cannot explain who actually runs a tool. We will trace a catalogue lookup from the user's request to a checked answer and identify the decisions that belong in application code.
Key ideas
- Keep decision proposals separate from tool execution.
- Record observations with their source and status.
- Validate inputs at the boundary that actually performs the action.
- Make completion, failure and interruption distinct outcomes.
Five components with different jobs
Instructions define the desired task and its boundaries. The model proposes a next action. Tools perform specific operations. State records what has happened. The controller connects these pieces and decides whether to continue. This separation helps answer a practical debugging question: did the failure come from an ambiguous goal, a wrong proposal, an unavailable tool or a check that accepted an unsupported answer? Changing the prompt cannot repair every one of those failures.
Trace one catalogue request
A learner asks for an introductory AI lesson. The application creates a state containing the goal and an empty observation list. A planner proposes a catalogue lookup with the key 'ai'. The controller verifies the tool name and argument shape, then runs the lookup. It stores both the returned record and whether the call succeeded. The next proposal can use that observation to finish. The final check confirms that the answer refers to a returned record rather than a generated course title.
- 1Goal
- 2Proposed action
- 3Validation
- 4Tool call
- 5Observation
- 6Next decision
State is more than conversation history
A message log helps reconstruct what was said, but operational state also needs the remaining budget, completed actions and pending decisions. Record a lookup's identifier instead of copying an entire database into each model request. Keep the state small enough to inspect and explicit enough to resume. If a run stops while awaiting review, its status should remain pending; it should not quietly become successful when the process restarts.
Put checks where they can enforce a rule
A prompt saying 'use only catalogue lookups' expresses a policy. An execution function that rejects every other tool enforces it. Apply the same distinction to argument types, access scope and repeated actions. A returned error must remain an error in the state. For actions that change an external system, define how you recognise an already completed operation before retrying. Otherwise recovery can accidentally repeat a change that succeeded before the connection failed.
Design a small observable loop
Choose explicit transitions: ready, tool requested, observation recorded, awaiting review, complete and stopped. Add only the states your exercise uses, then explain the transition out of each. Log tool metadata and outcomes without collecting unnecessary personal data. In a larger system, an orchestration runtime can help with persistence or interruptions. First make sure you understand the basic loop; a framework can organise it, but it cannot decide what an acceptable result means for your task.
In everyday language
Think of a small workshop. The model suggests the next job, the tools do specific work, the notebook records results and the supervisor checks permissions and progress. The architecture is the arrangement that lets each part do its own job.
Try it yourself
Draw a request passing through five boxes: goal, proposal, validation, lookup and observation. Add the point where a missing record stops the answer from inventing a lesson. Mark which boxes can change data.
Expected result
A diagram that locates execution and checks outside the model, records the lookup's status and includes an explicit completed or stopped outcome.
Check your answer: Where should a forbidden tool call be rejected?
At the execution boundary, before the tool runs. Instructions alone do not enforce that boundary.
Questions
Is the model the controller?
The model may choose a proposed next action, but application code still receives that proposal, checks it and invokes the actual tool. Keeping those responsibilities separate makes permissions and limits inspectable. An implementation can package both parts together, so examine its execution path rather than relying on a diagram label.
Do I need an orchestration framework first?
You can study a small loop with plain code. A framework becomes useful when you need features such as persistence, resumable runs or explicit graph transitions. Compare those needs with the framework's documented behaviour and test a failure case before adopting it for a larger project.
Trace one real request through the design. If you cannot point to its execution boundary, observation and final check, make those parts explicit before adding features.
Sources and further reading
- Anthropic — Building effective agents ↗Sources checked:
- LangChain — LangGraph overview ↗Sources checked:
- Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models ↗Sources checked: