
Most business workflows do not need an AI agent. They need a rule, a scheduled job, or a language model used as one step inside code you control. An agent, software that decides its own next step and calls tools to get there, earns its place only when the steps cannot be written down in advance, the input is unstructured, and a human approves anything that cannot be undone. Both Anthropic and OpenAI, the two vendors with the most to gain from selling agents, publish the same advice: start with the simplest thing that works and add autonomy only when a deterministic solution fails.
Key takeaways
- Anthropic's definition separates "workflows", where "LLMs and tools are orchestrated through predefined code paths", from "agents", where "LLMs dynamically direct their own processes and tool usage". The same model can power either; the difference is who controls the sequence.
- OpenAI's guide names three signals that a workflow may need an agent: complex decision-making, difficult-to-maintain rules, and heavy reliance on unstructured data. Its own caveat: "Otherwise, a deterministic solution may suffice."
- Agents "often trade latency and cost for better task performance" (Anthropic). Every step is a model call; every model call is nondeterministic and billed.
- OWASP lists "excessive agency" as a top-ten LLM risk, with three causes: excessive functionality, excessive permissions, excessive autonomy. The controls are known and cheap compared with the incidents.
- The right question is not "can an agent do this?" but "what is the lowest level on the ladder that handles 95% of cases, and who handles the rest?"
Five levels of automation
The vocabulary is muddled because vendors call everything an agent. Here is the ladder as we use it when scoping, lowest cost and risk first.
| Level | What decides the next step | Typical tool | Good for | Fails when |
|---|---|---|---|---|
| 1. Rules and triggers | Fixed logic written by a developer | Platform automations, webhooks, Shopify Flow-style rules | Known inputs, known outcomes ("if order value > X, tag for review") | Inputs vary in ways the rule did not anticipate |
| 2. Scheduled jobs | A timer and a script | cron, Lambda on a schedule, platform jobs | Batch work on structured data: reports, syncs, clean-ups | Data is messy or the job needs judgement |
| 3. RPA | A recorded sequence of UI actions | RPA bots that "work on the presentation layer of existing applications" (IBM) | Legacy systems with no API | The screen changes; exceptions pile up |
| 4. LLM inside a workflow | Code controls the sequence; a model handles one or more steps (classify, extract, draft, summarise) | Anthropic's prompt chaining, routing, parallelisation, orchestrator-workers, evaluator-optimizer patterns | Unstructured input with a predictable process | The number of steps cannot be fixed in advance |
| 5. Agent | The model plans the steps and picks tools; a human approves high-impact actions | Agent frameworks, MCP servers, approval queues | "Open-ended problems where it's difficult or impossible to predict the required number of steps" (Anthropic) | No evaluation set, no approvals, no kill switch |
Level 4 is where most of the 2026 value sits for a mid-size business: the model reads the email, the invoice or the ticket, turns it into structured fields, and ordinary code does the rest. It is cheaper to run, easier to test and simpler to explain to an auditor than an agent, and it covers the "unstructured data" case that OpenAI's guide cites.
The definitions, from the people who build them
- Anthropic (December 2024): "Workflows are systems where LLMs and tools are orchestrated through predefined code paths." Agents are systems where "LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." Its recommendation: "finding the simplest solution possible, and only increasing complexity when needed."
- OpenAI (2025): "Agents are systems that independently accomplish tasks on your behalf." It adds that "applications that integrate LLMs but don't use them to control workflow execution, think simple chatbots, single-turn LLMs, or sentiment classifiers, are not agents."
- AWS: an AI agent is "a software program that can interact with its environment, collect data, and use that data to perform self-directed tasks."
- Google Search Central (2026): "AI agents are autonomous systems that can perform tasks on behalf of people."
The common thread is control of the sequence. If your code decides what happens next, you have automation with an AI step. If the model decides, you have an agent and you inherit its failure modes.
A seven-question decision procedure
Work through these in order for one workflow. Stop at the first level that passes.
- Can the rule be written on one page? If a developer can list the inputs and the outcomes, build level 1 or 2. Most "AI" requests end here once someone writes the rule down.
- Is the only obstacle a system without an API? Consider level 3, but price in maintenance: IBM notes RPA bots "can follow only the processes defined by an end user", so every screen change is a ticket.
- Is the input unstructured (email, PDF, chat, voice) but the process fixed? Build level 4: a model extracts or classifies, code acts. Write 50 test inputs with expected outputs first.
- Does the number of steps genuinely vary case by case? Only now is level 5 on the table. OpenAI's three signals (nuanced judgement, brittle rule sets, unstructured data) should all be present, not one.
- What is the cost of a wrong action, and can it be reversed? If an action is "sensitive, irreversible, or [has] high stakes" (OpenAI's examples: cancelling orders, authorising large refunds, making payments), it needs a human approval step regardless of level.
- Who owns the evaluation set and reviews failures weekly? No owner, no agent. This is the question that kills most proposals, and should.
- What is the exit? If the vendor, model or framework goes away, can the workflow fall back to level 4 or to a person within a day?
Three worked examples
These are composite scenarios, not client cases. We have built AI workflow agents along these lines; the examples show the reasoning, not a specific project.
Order exceptions at an online store. Orders fail address validation, payment review or stock checks several times a day. Level 1 handles the known cases (tag, hold, notify). Level 4 handles the messages: a model reads the customer's reply, extracts the corrected address or intent, and code updates the order. An agent is not needed; the steps are fixed. What changed with AI is that the unstructured email no longer needs a person to read it first.
Invoice and document intake. Supplier PDFs arrive in a shared inbox in a dozen layouts. Level 3 (RPA) breaks every time a layout changes. Level 4 works: extraction by a model into a fixed schema, validation by code against purchase orders, exceptions routed to a person. The evaluation set is 50 real invoices with the correct fields; accuracy on that set is the acceptance test.
Lead qualification and CRM hygiene. Enquiries arrive from forms, email and chat with missing company names and vague needs. This is the first case where level 5 can be justified: the agent may need to look up the company, check the CRM for duplicates, draft a reply, and ask a clarifying question, with the number of steps depending on what it finds. It still runs with read access to the CRM and a human approving any outbound message until the evaluation set says otherwise. See our guide to MCP for business owners for how an agent reaches the CRM safely, and the RAG buyer's guide for the read-only assistant that should ship first.
Cost and risk by level
Run cost. Levels 1 to 3 cost compute and maintenance. Level 4 adds one model call per document; level 5 adds a model call per step, plus retries, plus tool calls. Multiply expected volume by steps per case and the provider's current per-token price from its pricing page; never estimate from an article. Anthropic's summary is the honest one: agentic systems "often trade latency and cost for better task performance".
Nondeterminism. A rule gives the same answer every time; a model does not. Level 4 contains that variance inside one step that code validates. Level 5 lets it compound across steps, which is why evaluation sets and approvals are not optional.
Security. OWASP's "Excessive Agency" entry describes damage done "in response to flawed LLM outputs", with three root causes: excessive functionality, excessive permissions and excessive autonomy. The same page lists the controls: limit tools "to only the minimum necessary", limit permissions "to the minimum necessary", "utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken", "implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not", and rate-limit actions. Prompt injection through a document the agent reads is the attack to assume; see the OWASP section of our RAG buyer's guide.
Integration. Agents reach systems through APIs, event queues or MCP servers. The integration surface is the same one that ESB and API projects have always had; our refreshed guide from ESB to APIs, events and MCP covers it.
Guardrails before any agent goes live
- Least privilege: separate credentials per tool, read-only by default, write scopes added one at a time.
- Approval queue: every irreversible or external action waits for a named person; OpenAI's guide recommends human oversight for high-risk actions "until confidence in the agent's reliability grows".
- Audit log: every prompt, tool call, result and approval stored, searchable, retained per your privacy policy (see privacy for AI features).
- Spend and rate limits: a daily token budget and a cap on actions per hour, enforced outside the model.
- Kill switch: one flag that stops the agent and routes work to the fallback process.
- Evaluation set: 50 to 100 real cases with expected outcomes, run before every prompt, model or tool change.
- Sandbox first: Anthropic recommends "extensive testing in sandboxed environments, along with the appropriate guardrails"; a staging CRM and a test mailbox cost little.
- Single agent before many: OpenAI's recommendation is to "maximize a single agent's capabilities first"; multi-agent systems add overhead few mid-size workflows justify.
FAQ
What is the difference between an AI agent and automation?
Automation follows a sequence written by a developer; an AI agent decides its own sequence and chooses tools as it goes. Anthropic's terms are "workflows" (predefined code paths) versus "agents" (the model directs its own process). A model used for one step inside a fixed process is automation with an AI step, not an agent.
When does a workflow actually need an agent?
When the number of steps cannot be predicted in advance and the work involves judgement over unstructured input. OpenAI's guide lists complex decision-making, difficult-to-maintain rules and heavy reliance on unstructured data, and advises that otherwise "a deterministic solution may suffice". If a rule fits on one page, build the rule.
Are AI agents expensive to run?
They cost more per case than rules or scheduled jobs because every step is a model call, and Anthropic notes they trade latency and cost for performance. Estimate volume times steps per case times the provider's current token price, add retries and tool calls, and compare with the cost of the people-hours saved.
Is RPA still worth using?
For a legacy system with no API, sometimes. RPA bots work on the user interface and only follow the recorded process, so they break when screens change. In 2026 a model-based extraction step (level 4) usually replaces the brittle part, with RPA kept only where no API exists.
What guardrails does an agent need?
Least-privilege credentials per tool, human approval for irreversible or external actions, a complete audit log, spend and rate limits enforced outside the model, a kill switch, an evaluation set run before every change, and a sandbox before production. OWASP's Excessive Agency guidance and both vendors' guides describe the same controls.
Impinge Solutions builds AI agents and workflow automation with a human in the loop, starting with a two-week pilot on one workflow. Pick a workflow and we will tell you which level it needs.
Sources
- Anthropic, "Building effective agents" (19 Dec 2024): https://www.anthropic.com/engineering/building-effective-agents
- OpenAI, "A practical guide to building agents" (2025, PDF): https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
- AWS, "What are AI agents?": https://aws.amazon.com/what-is/ai-agents/
- Google Search Central, "Optimizing your website for generative AI features on Google Search" (AI agents definition; updated 10 Jul 2026): https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- IBM, "What is robotic process automation (RPA)?": https://www.ibm.com/think/topics/rpa
- OWASP, "LLM06:2025 Excessive Agency": https://genai.owasp.org/llmrisk/llm062025-excessive-agency/
- Model Context Protocol, "What is MCP?": https://modelcontextprotocol.io/
- Anthropic and OpenAI pricing pages (for current per-token prices): https://www.anthropic.com/pricing and https://openai.com/api/pricing/



