Source: Anthropic, Introducing Computer Use (Claude 3.5 Sonnet)
Anthropic's October 2024 announcement of computer use in public beta, with early OSWorld results and stated limitations.
Contents
- What: vendor announcement
- Author/publisher: Anthropic
- Date: 2024-10-22
- Original: Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
Summary
First frontier-model computer use in public beta: Claude views screenshots and moves the cursor, clicks and types through an API tool.
Key claims
- Claude 3.5 Sonnet scored 14.9% on OSWorld screenshot-only (next best 7.8%), 22.0% with more steps.
- Described as experimental and at times cumbersome and error-prone; scrolling, dragging and zooming were hard.
- Developers advised to start with low-risk tasks.
- New classifiers detect computer use and potential harms (spam, misinformation, fraud).