New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Anthropic Computer Use Tool for AI Chat #44355

Open

petemill opened this issue Mar 1, 2025 · 0 comments

Assignees

Labels

browser-ai priority/P2 release-notes/include

Member

petemill commented Mar 1, 2025

Anthropic’s computer use tool uses screenshots to get coordinates which are sent to click functions. It can also type text, press keys and drag the mouse.
https://docs.anthropic.com/en/docs/build-with-claude/computer-use
Most of these functions can be sent to the renderer via the DevTools agent protocol from the browser process using a content::DevToolsAgentHostClient.
https://chromedevtools.github.io/devtools-protocol/

petemill self-assigned this

mattmcalister added browser-ai priority/P2 release-notes/include labels

mattmcalister added this to Browser AI

mattmcalister moved this to In Progress in Browser AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment