跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
浏览器与自动化

native_window_control

When the user needs to operate an already-open native desktop app window (not a browser): list windows, get control tree, click buttons/tabs, or type text into controls. Windows only; requires Python and pywinauto (pip install pywinauto). | 操作已打开的原生桌面窗口(非浏览器):列举窗口、控件树、点击或输入;仅 Windows,需 pywinauto。

hyqibot · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add hyqibot/token-free-openclaw --skill native_window_control

技能说明


name: native_window_control description: "When the user needs to operate an already-open native desktop app window (not a browser): list windows, get control tree, click buttons/tabs, or type text into controls. Windows only; requires Python and pywinauto (pip install pywinauto). | 操作已打开的原生桌面窗口(非浏览器):列举窗口、控件树、点击或输入;仅 Windows,需 pywinauto。" metadata: builtin_skill_version: "1.0" copaw: emoji: "🪟" requires: platform: win32

Native Window Control (Windows UI Automation)

For already-open native desktop app windows: list windows, snapshot controls, click buttons/tabs, type into edit controls. Unlike browser tooling, this operates native desktop windows (exe, desktop apps), not web pages.

When to use

  • User says: "Click the Start button in XX app", "Click the second tab in XX window", "Type 123 into the input in XX window"
  • User wants the agent to operate an already-open desktop app UI (exe window with tabs/buttons)
  • Combine with screenshot tool: capture the window first, then use this skill to click or type

Prerequisites

  • OS: Windows only.
  • Dependencies: pip install pywinauto. Run the script from the skill's scripts/ directory with Python.

How to run (script in this skill)

This skill provides a Python script scripts/native_window.py. Run it from the skill directory (or with the script path pointing at this skill's scripts folder). Working directory should be the skill root or the folder containing native_window.py.

Usage:

python scripts/native_window.py <action> [window_title] [ref] [text]
  • list_windows — List visible windows (title, PID). No extra args.
    Example: python scripts/native_window.py list_windows

  • snapshot — Get control tree for a window (substring match on title).
    Example: python scripts/native_window.py snapshot "Notepad"
    Returns JSON with controls array; each has ref, control_type, name. Use ref for click/type_text.

  • click — Click a control by window title substring and ref.
    Example: python scripts/native_window.py click "Notepad" "2"

  • type_text — Type into a control.
    Example: python scripts/native_window.py type_text "Notepad" "1" "hello"

Recommended flow

  1. list_windows → identify the target window title.
  2. snapshot with that title substring → get controls and refs.
  3. click or type_text with the same title and the chosen ref (and text for type_text).

Notes

  • Window title matching is substring, first match. Use a more specific substring or list_windows if multiple windows match.
  • Some legacy apps do not support UI Automation; those windows cannot be snapshotted or controlled.
  • You can take a screenshot first so the user can confirm the target window is open and in front.