ui_find_element

Screen, windows & UI · macOS · Windows

Effect: read (strongest supported action). Status: gated.

Gated means the tool requires an installed app, a connected account, permission, or runtime availability. Check those requirements on the listed platform before using it.

macOS

Purpose

GUI automation — control a native app's interface. Finds an element (button, field, menu…) in an app's accessibility tree by role and/or label. Scope with app_bundle_id or window_id. Returns an opaque element_ref (usable by ui_click / ui_get_element this session) plus role, label, bounds, focused, and `enabled` only when the app publishes AXEnabled (otherwise `enabled_unknown: true`, which does NOT mean disabled). found=false when the app is reachable but no element matches; app_not_found is an explicit error. Requires Accessibility permission.

Required inputs

No required inputs are declared in this platform's schema. Optional selectors and runtime requirements may still apply.

Permissions and confirmation

No explicit confirmation parameter is exposed in this snapshot. This does not grant permission to act: obtain user authorization before any real action.

Use only the apps, accounts and resources authorized by the user on this platform. Consult this tool's platform-specific description for its connection and permission requirements; this catalog does not assert additional OAuth scopes or OS entitlements.

Full input schema — macOS

{
  "properties": {
    "app_bundle_id": {
      "description": "Scope the search to this app.",
      "type": "string"
    },
    "index": {
      "description": "Which match to return if several (default 0).",
      "type": "integer"
    },
    "label": {
      "description": "AX title/description to match.",
      "type": "string"
    },
    "match": {
      "description": "Default contains.",
      "enum": [
        "exact",
        "contains"
      ],
      "type": "string"
    },
    "role": {
      "description": "AX role, e.g. AXButton, AXMenuItem, AXTextField.",
      "type": "string"
    },
    "window_id": {
      "description": "Alternatively scope by a window_id from list_windows.",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}

Documentation example

Do not execute this example. These concrete inputs refer to a fictional demonstration dataset. Resolve real handles and obtain user authorization before any real call.

{
  "label": "Save"
}

Windows

Purpose

GUI automation — control a native Windows app's interface. Finds an element (button, field, menu…) in an app's UI Automation tree by role and/or label. Scope with `app` (process name like "notepad", or a window-title substring). Returns an opaque element_ref (usable by ui_click / ui_type / ui_get_element this session) plus role, label, bounds, enabled, focused. `role` is a UIA control type (Button, Edit, CheckBox, MenuItem, Hyperlink, Text…) — run ui_read_tree to see the exact names. found=false when the app is reachable but nothing matches; app_not_found is an explicit error.

Required inputs

No required inputs are declared in this platform's schema. Optional selectors and runtime requirements may still apply.

Permissions and confirmation

No explicit confirmation parameter is exposed in this snapshot. This does not grant permission to act: obtain user authorization before any real action.

Use only the apps, accounts and resources authorized by the user on this platform. Consult this tool's platform-specific description for its connection and permission requirements; this catalog does not assert additional OAuth scopes or OS entitlements.

Full input schema — Windows

{
  "properties": {
    "app": {
      "description": "Scope to this app: process name (e.g. notepad) or a window-title substring.",
      "type": "string"
    },
    "index": {
      "description": "Which match to return if several (default 0).",
      "type": "integer"
    },
    "label": {
      "description": "Name/HelpText/Value to match.",
      "type": "string"
    },
    "match": {
      "description": "Default contains.",
      "enum": [
        "exact",
        "contains"
      ],
      "type": "string"
    },
    "role": {
      "description": "UIA control type, e.g. Button, Edit, MenuItem, CheckBox.",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}

Documentation example

Do not execute this example. These concrete inputs refer to a fictional demonstration dataset. Resolve real handles and obtain user authorization before any real call.

{
  "label": "Save"
}