Source: OpenAI Computer Use Guide
OpenAI API guide to computer use in the Responses API: the computer tool loop, code-execution harnesses over Playwright, browser toolsets and safety practices.
Contents
- What: official API documentation (Computer use guide)
- Author/publisher: OpenAI
- Date: living document; read 2026-11-14
- Original: Computer use
Summary
Three integration paths: the built-in computer tool (structured mouse/keyboard actions), a code-execution tool where the model writes Playwright or PyAutoGUI code, and browser function tools. SDK helpers run the loop.
Key claims
- Models named (as of 2026-11):
gpt-6-astraandgpt-6.1-sol; for GPT-6 Astra the guide recommends the code-execution path. - Loop: request with
tools: [{type: "computer"}]; response holds acomputer_callwith an orderedactionsarray (click, double_click, drag, move, scroll, keypress, type, wait, screenshot); you execute and reply withcomputer_call_outputcontaining a screenshot; continue withprevious_response_id. status: "completed"means the model finished generating the call, not that you executed it.- Pending safety checks stop the sample loop; add a human approval callback to handle them.
- Practices: isolated browser/VM with fresh profile and restricted network; treat page content as untrusted; keep credentials out of page text and screenshots; require approval for purchases or data submission; set step, time and cost limits; verify actual outcomes instead of trusting the final answer; always release sessions.
- Alternatives: custom function tools or MCP for known UI operations; hosted browsers (Browserbase, Browser Use, E2B and others); web search when no interaction is needed.