mcpField
G
gemini-computer-use

gemini-computer-use

github.com/am-willAgent workflows1.0k1.2k installsUpdated 15w ago
Repository

About gemini-computer-use

gemini-computer-use is a browser automation skill for agents, using Gemini and Playwright to carry out goal-driven computer tasks. It supports confirmation for actions flagged by safety checks and configurable Chromium browsers.

Key features

  1. 01Screenshot-driven Gemini action loop
  2. 02Playwright browser interaction
  3. 03User confirmation for flagged actions
  4. 04Configurable Chromium browser selection
  5. 05Turn limits and excluded actions

Use cases

  1. 01Finding information on a website
  2. 02Automating browser tasks from a prompt

Install

Copy the command and run it in a terminal.

$npx skills add https://github.com/am-will/codex-skills --skill gemini-computer-use

After installing, type/gemini-computer-usein Claude Code to confirm it loaded.

Bundled files

References and scripts installed along with SKILL.md.

env.example
references/
google-computer-use.md
scripts/
computer_use_agent.py

Details

From the project README

Gemini Computer Use

Quick start

  1. Source the env file and set your API key:

    cp env.example env.sh
$EDITOR env.sh
source env.sh

  2. Create a virtual environment and install dependencies:

    python -m venv .venv
source .venv/bin/activate
pip install google-genai playwright
playwright install chromium

  3. Run the agent script with a prompt:

    python scripts/computer_use_agent.py \
  --prompt "Find the latest blog post title on example.com" \
  --start-url "https://example.com" \
  --turn-limit 6

Browser selection

  • Default: Playwright's bundled Chromium (no env vars required).
  • Choose a channel (Chrome/Edge) with COMPUTER_USE_BROWSER_CHANNEL.
  • Use a custom Chromium-based executable (e.g., Brave) with COMPUTER_USE_BROWSER_EXECUTABLE.

If both are set, COMPUTER_USE_BROWSER_EXECUTABLE takes precedence.

Core workflow (agent loop)

  1. Capture a screenshot and send the user goal + screenshot to the model.
  2. Parse function_call actions in the response.
  3. Execute each action in Playwright.
  4. If a safety_decision is require_confirmation, prompt the user before executing.
  5. Send function_response objects containing the latest URL + screenshot.
  6. Repeat until the model returns only text (no actions) or you hit the turn limit.

Operational guidance

  • Run in a sandboxed browser profile or container.
  • Use --exclude to block risky actions you do not want the model to take.
  • Keep the viewport at 1440x900 unless you have a reason to change it.

Resources

  • Script: scripts/computer_use_agent.py
  • Reference notes: references/google-computer-use.md
  • Env template: env.example

Comments

Markdown supported