Can AI Agents Test Your App Like a Human? OpenAI’s New Computer Use Explained
An AI agent can read a test file and tell you what it *should* verify. A browser gives it a different kind of evidence: it can open a rendered page, interact with controls, and observe what the interface actually does. That makes computer use an attractive addition to a developer workflow. It does not make an agent a human tester, nor does it turn one successful browser run into proof that an app is correct.
On September 29, 2026, OpenAI added computer use to the Agents API . An agent can now work in an OpenAI-hosted browser, while the developer's application handles website access decisions and, when needed, sign-in. This is a follow-up to the Agents API launch we covered earlier, not a new launch of the API itself.
The useful question is not “Can it click a button?” It is: Can you give an agent a bounded user journey, collect observable evidence, and decide what the result means?
What is new—and what is not
The Agents API computer-use guide describes a browser running in an OpenAI-hosted environment. Your application starts a session, follows events, responds to access requests, gives the agent a task, inspects activity, and deletes the session when finished. The agent uses what it observes in the browser to choose its next action. To enable that path, the documented configuration adds a computer_use tool and an OpenAI-hosted environment with a desktop enabled.
That is distinct from the existing computer-use guide for the Responses API . In that integration, your application provides and operates the browser or desktop environment and returns screenshots or other observations to the model. Both can support UI work, but they give the developer different responsibilities for the runtime, session state, and action controls. Do not copy a Responses API example into an Agents API session and assume the browser is hosted in the same way.
“Like a human” describes the *surface* of the test: the agent encounters the UI rather than only its source code or an API response. It does not imply human judgment, perfect visual perception, or an exhaustive test. The model's final summary should be treated as a report to verify against the recorded browser activity and the app's final state.
A useful first test: a form with visible states
Imagine a staging app with a newsletter form. A safe first task is to check a non-destructive flow using test data: leave the email field empty, submit, observe validation, enter an invalid address, then enter a known test address and check that the expected confirmation appears. The scenario is illustrative; we have not run it against a live app or measured the new feature's success rate.
Give the agent an expected result for each step. For example:
| Action | Evidence to capture | What remains uncertain |
|---|---|---|
| Submit an empty field | Validation message, unchanged page, focus behavior | Whether every browser and assistive technology announces it correctly |
| Enter an invalid address | Visible rejection without a success state | Whether the backend also rejects malformed requests |
| Submit a designated test address | Confirmation and the observed final page state | Whether a message was delivered or a database record was written |
This is where browser use adds value. It can find a disconnect between code-level expectations and the rendered journey: a hidden error message, a button that appears clickable but never advances, or a success banner that shows before the request completes. But a visible confirmation alone does not prove delivery, persistence, or accessibility. Check those with appropriate assertions, logs, and human review.
For a first trial, use a staging environment and a test account. Avoid checkout, deletion, invitations, production data, and any flow where an unintended click has material consequences. The browser can interact with the same controls a user sees; that is precisely why its permissions matter.
The hosted-browser workflow in plain English
The official guide's lifecycle can be reduced to six steps:
- Start a session with the computer-use tool and hosted desktop browser configured. Keep the session ID so a disconnect does not force a blind restart.
- Send one bounded task with an allowed origin, test data, expected states, and explicit stop conditions.
- Handle access requests. The browser asks for approval before visiting each new website origin, including public sites. Network access being enabled is not the same as approving an origin.
- Handle sign-in only when required. The application, not a free-form chat prompt, should present the authentication request and collect the user's response. For a public staging page, there may be no sign-in step.
- Follow activity and verify the outcome. Browser operations appear as activity items; a completed operation is not the same as a completed task. Inspect the final UI state, relevant screenshots, and any independent assertions.
- Review and clean up. Retrieve the activity you need and delete the session when the work is complete.
Those steps follow OpenAI's hosted computer-use documentation . Screenshots can be included in activity output, but they are excluded by default; even when enabled, some operations may have no screenshot. A screenshot can also contain account or page data, so keep it out of public logs and restrict who can view it. Do not design a test report that silently interprets a missing screenshot as a pass.
A task brief worth reusing
This is an editorial example, not a claim that the exact prompt has been benchmarked. It assumes your application has already set up the session, approvals, and a safe staging browser.
Test the newsletter form at [approved staging URL] using only the supplied
test data. Do not visit other sites, sign in, purchase anything, delete data,
or submit a real person's information. Stop and ask if the page requires an
unexpected origin or authentication step.
Check three states in order: empty submission, invalid email, and a designated
test email. For each state, describe the action taken, the visible result,
and whether that result matches the expected behavior below:
[insert expected states].
Do not infer backend delivery from a success message. If a step fails or the
page is ambiguous, stop and report the last observed state. At the end,
separate observations from inferences and list checks that still need code,
API, accessibility, or human review.The prompt reduces ambiguity, but it is not a security boundary. The application must enforce what the browser can access and which actions are possible. If the agent encounters an unexpected redirect, the access request should be reviewed rather than auto-approved just because the initial task sounded harmless.
The approval boundary is narrower than it sounds
OpenAI's guide says the hosted browser requests approval for each new origin. It also explicitly warns that origin approval does not enforce confirmation before individual actions. If a site permits a purchase, destructive change, or other consequential action after origin access, a prompt asking the agent to confirm first is not a guaranteed control. For that guarantee, OpenAI advises restricting the hosted browser to resources that cannot perform such actions or using a browser runtime you control. Source: Agents API computer-use guide
The page itself is another boundary. Website text, including instructions injected into a page, is untrusted task data. It cannot authorize the agent to leave the test scope or override the user's request. Apply an origin allowlist or tight network policy, use disposable credentials, set time and cost limits, support cancellation, and verify the final state. OpenAI's general computer-use safety guidance makes the same broader point for UI agents: keep consequential actions under human control and do not rely solely on the model's answer.
What should remain a conventional test?
Browser exploration and deterministic tests answer different questions. An agent is useful when a flow is hard to specify in selectors beforehand, when a recent UI change needs exploratory checking, or when you want an intelligible account of what a user-facing path appears to do. A scripted test is better for exact assertions that must run the same way on every commit. API and integration tests are better for backend effects the page cannot prove.
Use a layered acceptance decision:
- Browser evidence: Did the agent reach the intended screen and observe the expected visible state?
- Automated assertions: Do repeatable tests check validation, requests, persistence, and regressions?
- Accessibility review: Are labels, focus, keyboard operation, and announcements actually usable—not merely visually present?
- Human judgment: Does the flow make sense, and are the agent's unexpected actions or omissions acceptable?
If a browser run fails, save the last observed state and reproduce it with a narrower test. If it passes, keep the result proportional: “this particular path worked in this particular environment” is more honest than “the app is tested.”
Should you adopt it now?
Try the hosted browser if your team already builds on the Agents API and has a safe staging surface with clear expected states. Begin with a short, read-only or test-data flow; inspect the activity before considering broader autonomy. The Agents API is in public beta, and its pricing overview says model usage, OpenAI tools, and hosted sandboxes follow their respective rates. Measure actual session usage rather than promising a fixed cost per test.
The new capability is most interesting as a bridge between code changes and user-visible behavior. It can help an agent notice what a unit test never sees. It cannot replace the test suite, permission design, or a person deciding whether the experience is good enough to ship. For a broader measurement framework, see our guide to evaluating coding agents.
