capture-screen

Capture Screen

Programmatic screenshot capture on macOS: find windows, control views, capture images.

Select observation or foreground control

For an existing view, enumerate and capture the exact window without activating it. Skip Step 2 when no view change is needed. A window image proves visible content in that capture, not keyboard focus, input delivery or a successful business action.

Before opening/raising windows, scrolling, selecting, changing zoom or sending keys, the executing agent checks user activity and other agents' ownership. Use a concrete task handoff or an existing explicit idle-desktop authorization; under the latter, coordinate exclusive access and proceed without asking again. Idle input alone is not ownership. Busy/unknown occupancy or unexpected input stops foreground actions; continue independent reads and do not restore old focus after human takeover.

Inspect the chosen tool's activation semantics. An app-targeted call or a shell job running in the background does not prove it preserves focus. A failed capture does not authorize raising the window, permission changes or switching tools around a host refusal/GUI lock. Only request a handoff when existing authorization is absent.

Quick Start

Observe an unidentified dialog or the full desktop

Use this route when the user requests a whole-screen view, or when permitted window observation cannot identify the reported obstruction. First establish that the host permits this capture and its scope. An app-specific control refusal does not prove screen capture is unavailable; it also does not authorize using a different tool to read content the host protects. Stop if the refusal covers the proposed capture or a GUI lock prevents it.

Capture without activation, scrolling or input:

capture_dir="$(mktemp -d)"
screencapture -x "$capture_dir/desktop.png"

Inspect the resulting image with the host's permitted image viewer. On a multi-display desktop, inspect each produced display image before claiming to have checked the whole desktop. File creation alone does not prove the requested dialog was captured. Once identified, prefer the exact window for further observations and keep the user's reported interaction failure in scope.

Follow the macOS permission owner's macos-permissions entry for classification and recovery.

Capture a known window

# Find Excel window ID
swift scripts/get_window_id.swift Excel

# Capture that window (replace 12345 with actual WID)
screencapture -x -l 12345 output.png

Step 1: Get Window ID (Swift)

Use the bundled Swift/CoreGraphics route when the host permits window enumeration. Prefer the host's native window inventory when it supplies the required IDs.

Quick inline execution

swift -e '
import CoreGraphics
let keyword = "Excel"
let list = CGWindowListCopyWindowInfo(.optionOnScreenOnly, kCGNullWindowID) as? [[String: Any]] ?? []
for w in list {
    let owner = w[kCGWindowOwnerName as String] as? String ?? ""
    let name = w[kCGWindowName as String] as? String ?? ""
    let wid = w[kCGWindowNumber as String] as? Int ?? 0
    if owner.localizedCaseInsensitiveContains(keyword) || name.localizedCaseInsensitiveContains(keyword) {
        print("WID=\(wid) | App=\(owner) | Title=\(name)")
    }
}
'

Using the bundled script

swift scripts/get_window_id.swift Excel
swift scripts/get_window_id.swift Chrome
swift scripts/get_window_id.swift          # List all windows

Output format: WID=12345 | App=Microsoft Excel | Title=workbook.xlsx

Parse the WID number for use with screencapture -l.

Step 2: Control Window (AppleScript, optional foreground work)

Use these commands only inside the authorized foreground interval above. Bind the exact app/window/document before using any active window or active workbook example; never infer the target from whichever window happens to be active.

Microsoft Excel (full AppleScript support)

# Activate (bring to front)
osascript -e 'tell application "Microsoft Excel" to activate'

# Set zoom level (percentage)
osascript -e 'tell application "Microsoft Excel"
    set zoom of active window to 120
end tell'

# Scroll to specific row
osascript -e 'tell application "Microsoft Excel"
    set scroll row of active window to 45
end tell'

# Scroll to specific column
osascript -e 'tell application "Microsoft Excel"
    set scroll column of active window to 3
end tell'

# Select a cell range
osascript -e 'tell application "Microsoft Excel"
    select range "A1" of active sheet
end tell'

# Select a specific sheet
osascript -e 'tell application "Microsoft Excel"
    activate object sheet "DCF" of active workbook
end tell'

# Open a file
osascript -e 'tell application "Microsoft Excel"
    open POSIX file "/path/to/file.xlsx"
end tell'

Any application (basic control)

# Activate any app
osascript -e 'tell application "Google Chrome" to activate'

# Bring specific window to front (by index)
osascript -e 'tell application "System Events"
    tell process "Google Chrome"
        perform action "AXRaise" of window 1
    end tell
end tell'

Timing and Timeout

Always add sleep 1 after AppleScript commands before capturing, to allow UI rendering to complete.

IMPORTANT: osascript hangs indefinitely if the target application is not running or not responding. Always wrap with timeout:

timeout 5 osascript -e 'tell application "Microsoft Excel" to activate'

Step 3: Capture (screencapture)

# Capture specific window by ID
screencapture -l <WID> output.png

# Silent capture (no camera shutter sound)
screencapture -x -l <WID> output.png

# Capture as JPEG
screencapture -l <WID> -t jpg output.jpg

# Capture with delay (seconds)
screencapture -l <WID> -T 2 output.png

# Capture a screen region (interactive)
screencapture -R x,y,width,height output.png

Retina displays

On Retina Macs, screencapture outputs 2x resolution by default (e.g., a 2032x1238 window produces a 4064x2476 PNG). This is normal. To get 1x resolution, resize after capture:

sips --resampleWidth 2032 output.png --out output_1x.png

Verify capture

# Check file was created and has content
ls -la output.png
file output.png    # Should show "PNG image data, ..."

Multi-Shot Workflow

Foreground example: capture multiple sections of an Excel workbook. Establish the authorization and exclusive ownership above first. If a user takes over, stop the remaining sequence rather than replaying it or restoring your previous focus.

# 1. Open file and activate Excel
osascript -e 'tell application "Microsoft Excel"
    open POSIX file "/path/to/model.xlsx"
    activate
end tell'
sleep 2

# 2. Set up view
osascript -e 'tell application "Microsoft Excel"
    set zoom of active window to 130
    activate object sheet "Summary" of active workbook
end tell'
sleep 1

# 3. Get window ID
#    IMPORTANT: Always re-fetch before capturing. CGWindowID is invalidated
#    when an app restarts or a window is closed and reopened.
WID=$(swift -e '
import CoreGraphics
let list = CGWindowListCopyWindowInfo(.optionOnScreenOnly, kCGNullWindowID) as? [[String: Any]] ?? []
for w in list {
    let owner = w[kCGWindowOwnerName as String] as? String ?? ""
    let wid = w[kCGWindowNumber as String] as? Int ?? 0
    if owner == "Microsoft Excel" { print(wid); break }
}
')
echo "Window ID: $WID"

# 4. Capture Section A (top of sheet)
osascript -e 'tell application "Microsoft Excel"
    set scroll row of active window to 1
end tell'
sleep 1
screencapture -x -l $WID section_a.png

# 5. Capture Section B (further down)
osascript -e 'tell application "Microsoft Excel"
    set scroll row of active window to 45
end tell'
sleep 1
screencapture -x -l $WID section_b.png

# 6. Switch sheet and capture
osascript -e 'tell application "Microsoft Excel"
    activate object sheet "DCF" of active workbook
    set scroll row of active window to 1
end tell'
sleep 1
screencapture -x -l $WID dcf_overview.png

Failed Approaches (DO NOT USE)

These methods were tested and confirmed to fail on macOS:

Method	Error	Why It Fails
System Events → id of window	Error -1728	System Events cannot access window IDs in the format screencapture needs
Python import Quartz (PyObjC)	ModuleNotFoundError	PyObjC not installed in system Python; don't attempt to install it — use Swift instead
osascript window id	Wrong format	Returns AppleScript window index, not CGWindowID needed by screencapture -l

Permission Troubleshooting

If get_window_id.swift cannot enumerate windows or a required field is unavailable, read the actual result before attributing the failure to Screen Recording. Request the helper's diagnostic guidance without enumerating windows:

swift scripts/get_window_id.swift --permission-hint screen
swift scripts/get_window_id.swift --permission-hint microphone

Interpret that output with the helper diagnostic procedure. The hint prints guidance and execution context; it does not check grant state or identify which process should receive a grant. Keep general TCC attribution, permission changes and request recovery with macos-permissions.

Supported Applications

Application	Window ID	AppleScript Control	Notes
Microsoft Excel	Swift	Full (zoom, scroll, select, activate sheet)	Best supported
Google Chrome	Swift	Basic (activate, window management)	No scroll/zoom via AppleScript
Other macOS apps	Swift or permitted host inventory	App-dependent	Capture requires host permission and a capturable surface

AppleScript control depth varies by application. Excel has the richest AppleScript dictionary. For apps with limited AppleScript, keyboard simulation via System Events is a foreground alternative only after the entry gate passes, not an automatic recovery from a failed background action.