Diagnose six agent failure patterns from a real run

An agent's final answer can look convincing while the task it was given remains unfinished. For a Python OpenAI Agents SDK run you own, the useful question is where the observed work first departed from the requirement agreed before the run. This guide shows how to inspect the permitted trace, result, session and application records, check six possible failure patterns, and leave an exact unknown where evidence is missing. It is based on the SDK documentation checked on 25 October 2026. BIG CHANGE did not run an agent or inspect a private trace for this article.

The big change

• What changed: The Agents SDK records model, tool, handoff and guardrail activity in traces by default. Its run result and session records can add context that a final answer omits. Together, these give an agent maintainer a way to investigate a real run step by step.
• Why it matters: A maintainer can locate the first supported departure from an approved task and test a small correction. That matters most when a reassuring answer masks a rejected tool call or an absent external result.
• What to watch: A trace records activity, while task completion depends on the actual answer or effect at the destination. Redacted inputs, missing records and unverified effects still require an explicit unknown.

Before you inspect a run

You need an existing Python OpenAI Agents SDK task agent and a real run you own or have permission to inspect. Get the trace ID and run time; the exact request and acceptance criterion recorded before the run; the agent instructions, configuration revision, reachable tools and handoffs; the final result or expected external receipt; and, for a multi-turn task, the permitted session or application history. Record the installed SDK version. The current manuals describe behavior, but do not establish what an older or modified installation did.

Check access to the correct Traces dashboard project and to the application's own records. The tracing manual says tracing is enabled by default for Runner.run, with generation, function-tool, guardrail and handoff spans. It can be disabled, and it is unavailable for organizations using Zero Data Retention. Trace export can also lag. Generation and function spans can contain sensitive prompts, tool arguments and outputs. Inspect only permitted records, keep identifiers and quotations in a private diagnosis file, and redact anything shared. Do not enable sensitive tracing just to complete this exercise without an approved privacy decision.

Reading existing records needs no new model call. A controlled repeat can consume model and tool allowance and can trigger a real tool effect. Before any repeat, agree on a spending limit, an actual permitted input and a read-only or isolated tool target. If that approval is unavailable, you can still record the supported diagnosis and mark repair verification pending.

Build the evidence record

1. Open the owned trace by ID and time in the correct project. If it is absent, check the project, tracing settings, export delay and retention policy. Record the missing trace as unobservable where it prevents inspection; absence is not evidence of a clean run. Save the requirement source, trace and configuration IDs, session ID if relevant, inspection time and permission scope privately.
2. Follow the observed input, model generations, tool requests and results, handoffs, guardrails and final output. The results manual distinguishes final_output, input, new_items and raw_responses; new_items carries tool, handoff and approval associations that a plain final answer may hide. Compare the run result with the application's delivery or saved-artifact record. For work spanning several runs, inspect the permitted session or application continuation record too. Sessions can merge saved history with new input and can filter that merge, so saved history alone does not prove what a later model saw.
3. Copy the blank grid below into a private record and fill all six rows from the actual run. A second real run may be needed to investigate a category the first does not expose. Use supported, not observed, unobservable or unknown cause for each finding. Not observed means only that this inspected run supplied no evidence of that pattern. For every empty field, write which record is missing or redacted. Record span or application IDs privately, never in a public issue by default.

These six names are investigation labels, not an exhaustive taxonomy or a claim about how often failures occur. A run can support more than one. Keep observation separate from cause: events appearing in trace order do not show that the first caused the second.

Check all six patterns

1. Context degradation. Find a requirement or source fact in the approved earlier input, then check whether it reached the later model input. Compare the generation input with the session record and the application's history limits, merge callback or handoff filter. A missing item may reflect a documented filter, failed retrieval or an omission before the run; a visible item the model ignored points elsewhere. If the later input is redacted, mark that comparison unobservable. For a proven omission, an authorized repair restores the exact approved material through the application's context path. The repeat must show both its presence in the later input and an output meeting the same requirement.
2. Specification drift. Put the pre-run task contract beside the exact instruction and configuration revision used by the agent. Compare each plan, tool request, handoff and final artifact with that contract, and mark the first requirement changed or dropped without approval. Check for a later authorized user instruction before calling a change drift. If the contract's revision is missing, record unknown cause. A bounded correction can restore the specific constraint or add a checkpoint before the divergent step. Verify that the requirement survives that step and the final artifact without dropping other requirements.
3. Sycophantic confirmation. Identify the exact agreement or reassurance, then compare it with an independently checked answer, source, tool result or acceptance criterion available at that point. Politeness and agreement with a true claim are not failures. An untested user claim cannot establish the opposite fact either. A repair check should require the approved evidence before confirmation and use an independently known answer to verify the new response. A model's own rating of its answer is insufficient. If you use a guardrail, inspect its actual boundary and result: SDK input, output and function-tool guardrails apply at different points.
4. Tool-call failure. Match the requested tool, arguments and approval state to its returned status and the actual downstream effect. Distinguish rejection, transport or execution error, invalid returned data, and a missing external effect. A successful function span does not certify the returned data; an error span may have been handled correctly. Redacted tool results leave the failure type unobservable. Correct only the demonstrated boundary, then check the exact call on an approved read-only or isolated target and independently verify the expected effect or rejection. For an uncertain prior write, read the destination before considering another attempt.
5. Cascading failure. Locate the first observed wrong datum or failed step. To connect it to a later error, find the later generation input, tool argument or handoff that actually consumed that datum. Several errors appearing in sequence are not proof of a cascade. A missing middle input makes the claimed link unknown. Put a validation or stop checkpoint at the earliest proven bad output, then check in a permitted controlled run whether downstream calls stop or receive corrected data. Inspect the final artifact as well as the checkpoint.
6. Silent failure. Start with the pre-run completion criterion: a saved file, delivered message, acknowledged update, checked answer or another task-specific receipt. Compare the authoritative destination with the trace's final message and application logs. A fluent final response and absence of an exception do not prove the effect happened. A missing receipt may be delayed processing or a logging gap, so read the destination before labeling it failed. A bounded repair adds a task-specific postcondition and an explicit failure or unknown path. Verify the actual receipt or independent answer in a permitted run, including what the agent reports when that evidence is absent.

Decide what can be repaired

Identify the first proven divergence from the pre-run requirement and list any later failures whose dependency is still unproven. Choose the smallest reversible correction supported by the evidence, record its configuration revision, and compare one authorized isolated repeat with the same acceptance criterion. Check the real tool effect or independent answer and any other required output that the correction could disturb. A green trace status or confident final sentence alone does not establish completion.

When the requirement, model input, intermediate result or destination is unavailable, record the precise missing item and the next permitted way to obtain it. Leave the cause or repair result unknown until the evidence exists. This procedure does not authorize production writes, customer messages, extra model spending or a change in tracing privacy settings.

For a different task after diagnosis, our held-out agent feedback-loop guide covers comparing improvements across variants. Our FunctionTool boundary guide focuses on one guarded tool call. Neither replaces the whole-run evidence check here.