How to build reliable AI agent workflows with open-source Kortix
Map AI agent workflow failure modes to concrete controls, with a reference implementation on open-source Kortix.
Map AI agent workflow failure modes to concrete controls, with a reference implementation on open-source Kortix.
Agent workflows fail in ways a generic retry does not fix. A tool reports success with an error payload inside. A schema drifts, a loop never exits, and the fix each time is engineering around the model: name the failure mode, attach a control, instrument the signal that proves it holds. Kortix is the open-source platform that supplies those primitives, from a sandbox per session to a review gate on every change.
Why agent workflows fail in production
Agent workflow failures cluster into a small set of recurring modes. Each one needs its own control and its own signal, and a generic "add retries" fix covers one row while the rest stay exposed.
| Failure mode | What it looks like | Control | Signal |
|---|---|---|---|
| Silent tool failure | A tool returns success with an error payload, an empty result or a stale value | Validate the result content, not the transport status | Tool-error and empty-result rate |
| Schema drift | An upstream API renames or retypes a field while the agent parses the old shape | Validate typed input and output at every step boundary | Schema-validation failures per step |
| Non-determinism | The same input produces different plans or outputs across runs | Pin the model and version, and run a fixed eval set | Eval pass-rate variance |
| Context rot | The model misses a fact as the context fills with distractors | Retrieve only relevant context, compress it, restate the task | Task-success rate against context size |
| Compounding errors | One wrong step becomes the input to the next | Programmatic gates between steps and a bounded verifier pass | Gate-failure and rework-loop rate |
| Runaway loops | The agent iterates because no exit condition ever evaluates true | Cap iterations and set explicit stopping conditions and budgets | Timeouts and cost per task |
| Unguarded side effects | The agent sends, deletes or charges without review | Human approval before irreversible actions, and least-privilege credentials | Unapproved actions, which should be zero |
| Duplicate side effects | A retried call creates a second resource | Idempotency keys, timeouts, capped backoff with jitter, a circuit breaker | Duplicate-action and circuit-open counts |
Context rot shows why a control has to match its mode. Chroma's evaluation found model performance degrading as input length grows, and topically related distractors push it down further (Chroma). One structural choice comes before any of these controls: how much of the flow the model is allowed to decide.
Choose the simplest structure that fits
Anthropic separates a workflow from an agent (Anthropic). A workflow orchestrates models and tools through predefined code paths, while an agent lets the model direct its own process and its own tool use. Both are agentic systems. They carry different failure profiles: the workflow is predictable and rigid, the agent flexible and exposed to compounding errors.
Google Cloud reaches the same conclusion from the design side. Start with a single agent, so you can refine the core logic, the prompt and the tool definitions before adding more (Google Cloud). Performance drops as that one agent takes on more tools and harder tasks, and you see it as latency, wrong tool selection or unfinished work. Anthropic's advice is the same: find the simplest solution possible, and add complexity only when it earns its place.
Bound the loop and make retries safe
The loop is where an agent's flexibility turns expensive. Google Cloud describes a loop pattern that repeats subagents until a termination condition is met. It warns that the "primary trade-off is the risk of an infinite loop" when that condition is not correctly defined (Google Cloud). So set an exit condition the workflow can actually evaluate, plus a cap on iterations and a budget for time and tokens. Anthropic lists stopping conditions such as a maximum number of iterations as part of the design (Anthropic).
Retries help with transient faults. They work only when the operation is idempotent. AWS's guidance: the effect of a retried call should happen once, even when the call is made many times (AWS). Timing matters too. Capped exponential backoff with jitter spreads retries out. Clients stop retrying in synchronized bursts (AWS). A timeout bounds each attempt. A circuit breaker stops calling a service once failures cross a threshold, which gives it room to recover (Azure). For a long run, durable execution takes the workflow to completion and, after a crash, resumes at the point where it stopped (Temporal).
Validate every boundary and gate every side effect
Every boundary between steps is a place where a wrong shape gets in. Define the expected input and output of each step. Validate both against a schema before the next step consumes the result. JSON Schema is a common way to express that shape, and a mismatch fails fast instead of flowing downstream (JSON Schema).
Anthropic warns that agents carry "the potential for compounding errors", so a wrong step should not quietly become the input to the next (Anthropic). Put programmatic checks between steps. Add a bounded verifier pass with explicit pass criteria. Google Cloud's review-and-critique pattern gives a critic predefined criteria to evaluate output against, and Anthropic's evaluator-optimizer runs a generator and a critic in a loop (Google Cloud, Anthropic).
Some actions should never run unattended. Google Cloud asks whether a task involves high-stakes decisions, safety-critical operations or subjective approvals that need human judgment (Google Cloud). Anthropic notes that agents can pause for human feedback at checkpoints or blockers (Anthropic). Gate irreversible actions behind approval. Give the agent only the credentials it needs.
Track the signals that prove a control holds
Observability has to cover the deterministic parts of a workflow and the probabilistic ones. The OpenTelemetry GenAI semantic conventions define spans, metrics and events for GenAI clients and MCP, which gives traces a shared vocabulary instead of a one-off log format (OpenTelemetry).
Read those signals per step. Emit one span for each step. It carries the inputs, outputs, latency and errors. Then track a short list per tool: the tool-error and empty-result rate, schema-validation failures, task-success and rework-loop rates, cost per task. Hold a fixed eval set of representative tasks. Run it before any change to a prompt, a tool or a model. Anthropic recommends comprehensive evaluation and sandboxed testing with guardrails (Anthropic).
Reliable workflows on open-source Kortix
Kortix is the open-source CompanyOS: agents, shared skills, company memory and connectors live in one git repo you own. A session runs an agent in an isolated sandbox. Each session works on its own branch, and the agent opens a change request a human reviews and merges (Kortix docs). Per-agent grants scope connectors, secrets and skills. Connector credentials are brokered server-side, so raw keys never enter the machine, and a full audit trail records what ran (Kortix on GitHub). Kortix is open source. Self-host it, bring any model provider with your own keys, and start free on one project.
# kortix.yaml: governance only; the agent's behavior lives in agents/workflow-runner.md
kortix_version: 2
default_agent: workflow-runner
agents:
workflow-runner:
connectors: [github]
secrets: []
skills: [reliability-checks]
triggers:
- slug: nightly-run
type: cron
cron: "0 0 3 * * *"
agent: workflow-runner
prompt: |
Validate every tool result against its schema, stop after five
iterations, and open a change request instead of writing to main.
The platform supplies the controls that are expensive to build twice: isolation per session, a review gate before work lands, scoped credentials and an audit trail. Run it on Kortix Cloud, in your VPC, or on your own on-prem network (Kortix on GitHub). Self-host is free. You still define the schemas, the exit conditions, the budgets and the eval set for each workflow.
If tool access is the part you are hardening, how to give AI agents tool access safely covers scoped connectors and approval policies. Open-source AI agent platforms compares the options on licence, deployment and model choice. Getting started with Kortix walks through the first session.
Start with one workflow. Name its failure modes, attach a control to each, and measure the signal that proves the control holds. Get started with open-source Kortix, or read the docs to run the reference shape yourself.