Choosing an AI agent product starts with the work it must do. Write down the real input, required output, permitted actions and conditions under which a person must intervene. Then inspect the product on three axes: where it runs and sends data, who controls each step, and the contract by which your application calls it and receives a result. A polished demo cannot answer those questions for your workflow.
This guide gives you a blank decision worksheet and a versioned documentation example using the OpenAI Agents SDK for Python 0.24.0, released on 15 October 2026. The example maps documented capabilities. BIG CHANGE did not install the SDK, run a model, test an account or verify a deployment. Your own requirements and authorized trials determine the decision.
The big change
- What changed: Agent products offer several ways to run code, route work and connect tools. Their feature lists do not specify where a particular application runs or which permissions it grants. A three-axis map turns those choices into questions an owner can verify.
- Why it matters: A documented handoff or approval feature has little value to a team whose deployed workflow does not use it. Connecting product evidence to the team's mandatory requirements exposes that gap before real data or consequential tool access is added.
- What to watch: The decision changes only when a missing requirement is supplied, an unknown is resolved with evidence, or an authorized normal and failure trial verifies the configured workflow. A newly released version needs its own check.
Before you compare products
Choose one named product, release and access tier for each copy of the worksheet. Gather its current primary manual, API or SDK reference, release notes, pricing and privacy terms. Record when you checked each source. A feature available in a different tier, variant or version does not fill the row for the product under review.
The decision owner supplies the actual task, input and output, users, data categories, allowed side effects, deployment and retention rules, deadline and spending limit. Mark each criterion mandatory, preferred or not required, with the real threshold where one exists. If these inputs are missing, the decision is requirements pending. The worksheet can still map product facts, but it cannot endorse a fit.
Use documented for a claim read in the product's versioned manual, observed or measured only for a real authorized check, and inferred for a conclusion drawn from documented behavior. Give each row a result of supported, unsupported, unknown or not applicable. Record why not applicable is legitimate; it cannot remove a mandatory need.
1. Locate the runner and every data path
Find out whether the runner executes on a reader-operated device or server, on a vendor host, or in a mixed arrangement. Draw the path from input to model, tool, storage and final output. Name the operator at each stop. Check platform and language requirements, network dependency, available regions and service access against the requirements you wrote down. When the manual describes a software component but not your hosting location or residency, mark the deployment row unknown until your architecture and the provider's terms establish it.
Also record who maintains the runtime and who can see logs or traces. “The code runs in our process” does not settle whether prompts, tool arguments or telemetry travel to a provider.
2. Map routing, effects and oversight
Trace who selects the next step: the model, fixed application code or a person. Identify the owner of any manager agent or handoff, the tools reachable after each transition and the effects those tools can have. Look for a real approval point before an effect, a turn limit, the failure stop, a way to resume or recover, and a trace that the operator is permitted to inspect.
An SDK may offer all these controls while an individual application enables only some of them. Record both the documented capability and the actual configuration. Treat an absent application setting or inaccessible run record as unknown for that application. A model API key authenticates model requests; it does not authorize users of your application or grant every downstream tool action. Those boundaries need their own permissions and checks.
3. Write down the complete interface contract
Specify the input type and schema, size or format limits, and what happens to missing or unsupported input. Do the same for final output, partial output, interruption and error. Name the trigger: an SDK call, HTTP endpoint, UI action, event or schedule. State who can invoke it and how that caller is authenticated. Record the package or API version, dependency compatibility, migration notes and the owner of any adapter your team must build.
Finish with access tier, model and tool charges, usage controls, privacy settings, retention, rights to inputs and outputs, licensing and support. A license for client code does not set the price or rights for model calls. Check each vendor term that matters to your data and publication plan; leave the row unknown when the applicable contract is unavailable.
Copy the decision worksheet
Save this blank table for one product and release. In each requirement cell write mandatory, preferred or not required plus the actual threshold. In each evidence cell write the exact URL or authorized record ID, section, version, check date and stage (documented, observed, measured or inferred). The result cell takes supported, unsupported, unknown or reasoned not applicable. The last cell names the owner and next check. Keep private credentials and personal data out of the sheet.
Product/release/access tier: __ Decision owner and date: __
Real task, input, output and allowed effects: __ Users and data category: __
Permitted data locations and retention: __ Trial environment and allowed effects: __
Deadline and spend ceiling: __ Primary documentation and checked dates: __
Axis and criterion | Requirement and threshold | Evidence, version, date and stage | Result | Owner, check or recovery |
|---|---|---|---|---|
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
| __ | __ | __ | __ |
Across axes: access, charges and usage controls | __ | __ | __ | __ |
Across axes: privacy, retention, rights and support | __ | __ | __ | __ |
Decision: __ Mandatory failed/unknown rows: __ Preferred gaps: __
Authorized normal trial and result: __ Denied or missing-input trial and result: __
Actual output, tool effects, failure behavior and usage record: __ Tested scope/date: __
A versioned documentation example
The tagged quickstart installs openai-agents in a Python project, sets OPENAI_API_KEY for the default OpenAI model path and starts an agent with Runner.run. Package metadata identifies version 0.24.0, Python 3.10 or later, an openai dependency in the 3.x range and an MIT license. The standard Agent plus Runner path therefore points to a runner in the application's Python process, with model requests sent through the configured provider; the Agents manual names the Responses API as the OpenAI model default. The process location is an inference from the published code path, not a verified hosting or residency arrangement. The operator must document the chosen host, network route and region.
For routing, the orchestration manual describes model-selected and code-controlled flow, including a manager agent calling specialists as tools and handoffs that change the active agent. The runner manual describes the loop and max_turns. Function tools can wrap Python functions; the human approval manual describes an approval interruption and resume. These are SDK options. Which route, tool permissions, approval rule, turn limit and recovery procedure a particular application uses remains to be inspected.
The runner accepts a string, Responses-format input items or saved run state on resume, according to the runner manual. The results manual documents a final output, run items, last agent and interruptions; a streaming result is incomplete until processing finishes. The quickstart's trigger is a Python call. It does not establish a public endpoint, timer, application input validator or the caller permissions for one. The reader must supply those parts of the contract and test its own incomplete and error handling.
For cost and privacy, the usage manual records requests and tokens per run, while API pricing depends on the chosen model and usage. The tracing manual says tracing is on by default and sensitive model/tool inputs and outputs can be included by default; it documents controls to disable or limit collection. Review the configured tracing path, retention and applicable terms before entering real data. No account entitlement, fixed task cost or operational result follows from these manuals.
Decision for this example: requirements pending. The reader's task, thresholds and deployed configuration are blank. This documentation map establishes capabilities and questions to check. It makes no product-fit claim.
Apply the decision and verify the result
- Fill every mandatory threshold. If the task or a mandatory threshold is missing, write
requirements pendingand name the missing input. Do not assign a score. - Check the evidence row by row. Any mandatory
unsupportedresult meansno documented fitfor the stated scope. Cite the row. Any mandatoryunknownmeanshold for evidence; name the exact manual, account setting, contract or bounded trial needed. Resolve an unsupported requirement before treating any other rows as an endorsement. - Separate documentation from operation. If all mandatory rows are documented as
supported, recorddocumented fit for the stated scope; operational validation pending. Preferred gaps and limits stay visible. The existence of a product option does not prove that the configured application uses it. - Run authorized checks before adoption. With a real permitted input and an agreed spend and effects boundary, run a representative normal case and a denied or missing-input case. Inspect actual inputs, outputs, tool effects, errors, approval behavior and usage. Record
validated fit for tested scopeonly if the required normal and failure behavior passes. State exactly what environment and version were tested. This article ran no such trial.
If a document has moved, the chosen version differs or access is missing, preserve the old citation, mark affected rows unknown, and refresh the exact official clause or arrange an authorized check. A failed trial can make a row unsupported for that configuration; record the observed result before revising the decision. Never turn an empty source response into evidence that a feature is absent, and never send secrets or personal data merely to complete the worksheet.
For a narrower check of one OpenAI SDK function tool's permissions, see BIG CHANGE's tool-boundary guide. The three-axis decision here covers the surrounding product and application contract.
