UI Automation and Microsoft Active Accessibility

Note

This documentation is intended for .NET Framework developers who want to use the managed UI Automation classes defined in the System.Windows.Automation namespace. For the latest information about UI Automation, see Windows Automation API: UI Automation.

Microsoft Active Accessibility was the earlier solution for making applications accessible. Microsoft UI Automation is the new accessibility model for Microsoft Windows and is intended to address the needs of assistive technology products and automated testing tools. UI Automation offers many improvements over Active Accessibility.

This topic includes the main features of UI Automation and explains how these features differ from Active Accessibility.

Programming Languages

Active Accessibility is based on the Component Object Model (COM) with support for dual interfaces, and is therefore programmable in C/C++, Microsoft Visual Basic 6.0, and scripting languages. UI Automation (including the client-side provider library for standard controls) is written in managed code, and UI Automation client applications are most easily programmed using C# or Visual Basic .NET. UI Automation providers, which are interface implementations, can be written in managed code or in C/C++.

Support in Windows Presentation Foundation

Windows Presentation Foundation (WPF) is the new model for creating user interfaces. WPF elements do not contain native support for Active Accessibility; however, they do support UI Automation, which includes bridging support for Active Accessibility clients. Only clients written specifically for UI Automation can take full advantage of the accessibility features of WPF, such as the rich support for text.

Servers and Clients

In Active Accessibility, servers and clients communicate directly, largely through the server's implementation of IAccessible.

In UI Automation, a core service lies between the server (called a provider) and the client. The core service makes calls to the interfaces implemented by providers and provides additional services such as generating unique runtime identifiers for elements. Client applications use library functions to call the UI Automation service.

UI Automation providers can provide information to Active Accessibility clients, and Active Accessibility servers can provide information to UI Automation client applications. However, because Active Accessibility does not expose as much information as UI Automation, the two models are not fully compatible.

UI Elements

Active Accessibility presents UI elements either as an IAccessible interface or as a child identifier. It is difficult to compare two IAccessible pointers to determine if they refer to the same element.

In UI Automation, every element is represented as an AutomationElement object. Comparison is done by using the equality operator or the Equals method, both of which compare the unique runtime identifiers of the elements.

Tree Views and Navigation

The user interface (UI) elements on the screen can be seen as a tree structure with the desktop as the root, application windows as immediate children, and elements within applications as further descendants.

In Active Accessibility, many automation elements that are irrelevant to end users are exposed in the tree. Client applications have to look at all the elements to determine which are meaningful.

UI Automation client applications see the UI through a filtered view. The view contains only elements of interest: those that give information to the user or enable interaction. Predefined views of only control elements and only content elements are available; in addition, applications can define custom views. UI Automation simplifies the task of describing the UI to the user and helping the user interact with the application.

Navigation between elements, in Active Accessibility, is either spatial (for example, moving to the element that lies to the left on the screen), logical (for example, moving to the next menu item, or the next item in the tab order within a dialog box), or hierarchical (for example, moving the first child in a container, or from the child to its parent). Hierarchical navigation is complicated by the fact that child elements are not always objects that implement IAccessible.

In UI Automation, all UI elements are AutomationElement objects that support the same basic functionality. (From the standpoint of the provider, they are objects that implement an interface inherited from IRawElementProviderSimple.) Navigation is mainly hierarchical: from parents to children, and from one sibling to the next. (Navigation between siblings has a logical element, as it may follow the tab order.) You can navigate from any starting-point, using any filtered view of the tree, by using the TreeWalker class. You can also navigate to particular children or descendants by using FindFirst and FindAll; for example, it is very easy to retrieve all elements within a dialog box that support a specified control pattern.

Navigation in UI Automation is more consistent than in Active Accessibility. Some elements such as drop-down lists and pop-up windows appear twice in the Active Accessibility tree, and navigation from them may have unexpected results. It is actually impossible to properly implement Active Accessibility for a rebar control. UI Automation enables reparenting and repositioning, so that an element can be placed anywhere in the tree despite the hierarchy imposed by ownership of windows.

Roles and Control Types

Active Accessibility uses the accRole property (IAccessible::get_actRole) to retrieve a description of the element's role in the UI, such as ROLE_SYSTEM_SLIDER or ROLE_SYSTEM_MENUITEM. The role of an element is the main clue to its available functionality. Interaction with a control is achieved by using fixed methods such as IAccessible::accSelect and IAccessible::accDoDefaultAction. The interaction between the client application and the UI is limited to what can be done through IAccessible.

In contrast, UI Automation largely decouples the control type of the element (described by the ControlType property) from its expected functionality. Functionality is determined by the control patterns that are supported by the provider through its implementation of specialized interfaces. Control patterns can be combined to describe the full set of functionality supported by a particular UI element. Some providers are required to support a particular control pattern; for example, the provider for a check box must support the Toggle control pattern. Other providers are required to support one or more of a set of control patterns; for example, a button must support either Toggle or Invoke. Still others support no control patterns at all; for example, a pane that cannot be moved, resized, or docked does not have any control patterns.

UI Automation supports custom controls, which are identified by the Custom property and can be described by the LocalizedControlTypeProperty property.

The following table shows the mapping of Active Accessibility roles to UI Automation control types.

Active Accessibility role UI Automation control type
ROLE_SYSTEM_PUSHBUTTON Button
ROLE_SYSTEM_CLIENT Calendar
ROLE_SYSTEM_CHECKBUTTON Check box
ROLE_SYSTEM_COMBOBOX Combo box
ROLE_SYSTEM_CLIENT Custom
ROLE_SYSTEM_LIST Data grid
ROLE_SYSTEM_LISTITEM Data item
ROLE_SYSTEM_DOCUMENT Document
ROLE_SYSTEM_TEXT Edit
ROLE_SYSTEM_GROUPING Group
ROLE_SYSTEM_LIST Header
ROLE_SYSTEM_COLUMNHEADER Header item
ROLE_SYSTEM_LINK Hyperlink
ROLE_SYSTEM_GRAPHIC Image
ROLE_SYSTEM_LIST List
ROLE_SYSTEM_LISTITEM List item
ROLE_SYSTEM_MENUPOPUP Menu
ROLE_SYSTEM_MENUBAR Menu bar
ROLE_SYSTEM_MENUITEM Menu item
ROLE_SYSTEM_PANE Pane
ROLE_SYSTEM_PROGRESSBAR Progress bar
ROLE_SYSTEM_RADIOBUTTON Radio button
ROLE_SYSTEM_SCROLLBAR Scroll bar
ROLE_SYSTEM_SEPARATOR Separator
ROLE_SYSTEM_SLIDER Slider
ROLE_SYSTEM_SPINBUTTON Spinner
ROLE_SYSTEM_SPLITBUTTON Split button
ROLE_SYSTEM_STATUSBAR Status bar
ROLE_SYSTEM_PAGETABLIST Tab
ROLE_SYSTEM_PAGETAB Tab item
ROLE_SYSTEM_TABLE Table
ROLE_SYSTEM_STATICTEXT Text
ROLE_SYSTEM_INDICATOR Thumb
ROLE_SYSTEM_TITLEBAR Title bar
ROLE_SYSTEM_TOOLBAR Tool bar
ROLE_SYSTEM_TOOLTIP ToolTip
ROLE_SYSTEM_OUTLINE Tree
ROLE_SYSTEM_OUTLINEITEM Tree item
ROLE_SYSTEM_WINDOW Window

For more information about the different control types, see UI Automation Control Types.

States and Properties

In Active Accessibility, elements support a common set of properties, and some properties (such as accState) must describe very different things, depending on the element's role. Servers must implement all methods of IAccessible that return a property, even those that are not relevant to the element.

UI Automation defines many more properties, some of which correspond to states in Active Accessibility. Some are common to all elements, but others are specific to control types and control patterns. Properties are distinguished by unique identifiers, and most properties can be retrieved by using a single method, GetCurrentPropertyValue or GetCachedPropertyValue. Many properties are also easily retrievable from the Current and Cached property accessors.

A UI Automation provider does not have to implement irrelevant properties, but can simply return a null value for any properties it does not support. Also, the UI Automation core service can obtain some properties from the default window provider, and these are amalgamated with properties explicitly implemented by the provider.

As well as supporting many more properties, UI Automation supplies better performance by allowing multiple properties to be retrieved with a single cross-process call.

The following table shows the correspondence between properties in the two models.

Active Accessibility property accessor UI Automation property ID Remarks
get_accKeyboardShortcut AccessKeyProperty or AcceleratorKeyProperty AccessKeyProperty takes precedence if both are present.
get_accName NameProperty
get_accRole ControlTypeProperty See the previous table for mapping of roles to control types.
get_accValue ValuePattern.ValueProperty

RangeValuePattern.ValueProperty
Valid only for control types that support ValuePattern or RangeValuePattern. RangeValue values are normalized to 0-100, to be consistent with MSAA behavior. Value items use a string.
get_accHelp HelpTextProperty
accLocation BoundingRectangleProperty
get_accDescription Not supported in UI Automation accDescription did not have a clear specification within MSAA, which resulted in providers placing different pieces of information in this property.
get_accHelpTopic Not supported in UI Automation

The following table shows which UI Automation properties correspond to Active Accessibility state constants.

Active Accessibility state UI Automation property Triggers State Change?
STATE_SYSTEM_CHECKED For check box, ToggleStateProperty

For radio button, IsSelectedProperty
Y
STATE_SYSTEM_COLLAPSED ExpandCollapseState = Collapsed Y
STATE_SYSTEM_EXPANDED ExpandCollapseState = Expanded or PartiallyExpanded Y
STATE_SYSTEM_FOCUSABLE IsKeyboardFocusableProperty N
STATE_SYSTEM_FOCUSED HasKeyboardFocusProperty N
STATE_SYSTEM_HASPOPUP ExpandCollapsePattern for menu items N
STATE_SYSTEM_INVISIBLE IsOffscreenProperty = True and GetClickablePoint causes NoClickablePointException N
STATE_SYSTEM_LINKED ControlTypeProperty =

Hyperlink
N
STATE_SYSTEM_MIXED ToggleState = Indeterminate N
STATE_SYSTEM_MOVEABLE CanMoveProperty N
STATE_SYSTEM_MUTLISELECTABLE CanSelectMultipleProperty N
STATE_SYSTEM_OFFSCREEN IsOffscreenProperty = True N
STATE_SYSTEM_PROTECTED IsPasswordProperty N
STATE_SYSTEM_READONLY RangeValuePattern.IsReadOnlyProperty and ValuePattern.IsReadOnlyProperty N
STATE_SYSTEM_SELECTABLE SelectionItemPattern is supported N
STATE_SYSTEM_SELECTED IsSelectedProperty N
STATE_SYSTEM_SIZEABLE CanResize N
STATE_SYSTEM_UNAVAILABLE IsEnabledProperty Y

The following states either were not implemented by most Active Accessibility control servers or have no equivalent in UI Automation.

Active Accessibility state Remarks
STATE_SYSTEM_BUSY Not available in UI Automation
STATE_SYSTEM_DEFAULT Not available in UI Automation
STATE_SYSTEM_ANIMATED Not available in UI Automation
STATE_SYSTEM_EXTSELECTABLE Not widely implemented by Active Accessibility servers
STATE_SYSTEM_MARQUEED Not widely implemented by Active Accessibility servers
STATE_SYSTEM_SELFVOICING Not widely implemented by Active Accessibility servers
STATE_SYSTEM_TRAVERSED Not available in UI Automation
STATE_SYSTEM_ALERT_HIGH Not widely implemented by Active Accessibility servers
STATE_SYSTEM_ALERT_MEDIUM Not widely implemented by Active Accessibility servers
STATE_SYSTEM_ALERT_LOW Not widely implemented by Active Accessibility servers
STATE_SYSTEM_FLOATING Not widely implemented by Active Accessibility servers
STATE_SYSTEM_HOTTRACKED Not available in UI Automation
STATE_SYSTEM_PRESSED Not available in UI Automation

For a complete list of UI Automation property identifiers, see UI Automation Properties Overview.

Events

The event mechanism in UI Automation, unlike that in Active Accessibility, does not rely on Windows event routing (which is closely tied in with window handles) and does not require the client application to set up hooks. Subscriptions to events can be fine-tuned not just to particular events but to particular parts of the tree. Providers can also fine-tune their raising of events by keeping track of what events are being listened for.

It is also easier for clients to retrieve the elements that raise events, as these are passed directly to the event callback. Properties of the element are automatically prefetched if a cache request was active when the client subscribed to the event.

The following table shows the correspondence of Active Accessibility WinEvents and UI Automation events.

WinEvent UI Automation event identifier
EVENT_OBJECT_ACCELERATORCHANGE AcceleratorKeyProperty property change
EVENT_OBJECT_CONTENTSCROLLED VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change on the associated scroll bars
EVENT_OBJECT_CREATE StructureChangedEvent
EVENT_OBJECT_DEFACTIONCHANGE No equivalent
EVENT_OBJECT_DESCRIPTIONCHANGE No exact equivalent; perhaps HelpTextProperty or LocalizedControlTypeProperty property change
EVENT_OBJECT_DESTROY StructureChangedEvent
EVENT_OBJECT_FOCUS AutomationFocusChangedEvent
EVENT_OBJECT_HELPCHANGE HelpTextProperty change
EVENT_OBJECT_HIDE StructureChangedEvent
EVENT_OBJECT_LOCATIONCHANGE BoundingRectangleProperty property change
EVENT_OBJECT_NAMECHANGE NameProperty property change
EVENT_OBJECT_PARENTCHANGE StructureChangedEvent
EVENT_OBJECT_REORDER Not consistently used in Active Accessibility. No directly corresponding event is defined in UI Automation.
EVENT_OBJECT_SELECTION ElementSelectedEvent
EVENT_OBJECT_SELECTIONADD ElementAddedToSelectionEvent
EVENT_OBJECT_SELECTIONREMOVE ElementRemovedFromSelectionEvent
EVENT_OBJECT_SELECTIONWITHIN No equivalent
EVENT_OBJECT_SHOW StructureChangedEvent
EVENT_OBJECT_STATECHANGE Various property-changed events
EVENT_OBJECT_VALUECHANGE RangeValuePattern.ValueProperty and ValuePattern.ValueProperty changed
EVENT_SYSTEM_ALERT No equivalent
EVENT_SYSTEM_CAPTUREEND No equivalent
EVENT_SYSTEM_CAPTURESTART No equivalent
EVENT_SYSTEM_CONTEXTHELPEND No equivalent
EVENT_SYSTEM_CONTEXTHELPSTART No equivalent
EVENT_SYSTEM_DIALOGEND WindowClosedEvent
EVENT_SYSTEM_DIALOGSTART WindowOpenedEvent
EVENT_SYSTEM_DRAGDROPEND No equivalent
EVENT_SYSTEM_DRAGDROPSTART No equivalent
EVENT_SYSTEM_FOREGROUND AutomationFocusChangedEvent
EVENT_SYSTEM_MENUEND MenuClosedEvent
EVENT_SYSTEM_MENUPOPUPEND MenuClosedEvent
EVENT_SYSTEM_MENUPOPUPSTART MenuOpenedEvent
EVENT_SYSTEM_MENUSTART MenuOpenedEvent
EVENT_SYSTEM_MINIMIZEEND WindowVisualStateProperty property change
EVENT_SYSTEM_MINIMIZESTART WindowVisualStateProperty property change
EVENT_SYSTEM_MOVESIZEEND BoundingRectangleProperty property change
EVENT_SYSTEM_MOVESIZESTART BoundingRectangleProperty property change
EVENT_SYSTEM_SCROLLINGEND VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change
EVENT_SYSTEM_SCROLLINGSTART VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change
EVENT_SYSTEM_SOUND No equivalent
EVENT_SYSTEM_SWITCHEND No equivalent, but an AutomationFocusChangedEvent event signals that a new application has received the focus
EVENT_SYSTEM_SWITCHSTART No equivalent
No equivalent CurrentViewProperty property change
No equivalent HorizontallyScrollableProperty property change
No equivalent VerticallyScrollableProperty property change
No equivalent HorizontalScrollPercentProperty property change
No equivalent VerticalScrollPercentProperty property change
No equivalent HorizontalViewSizeProperty property change
No equivalent VerticalViewSizeProperty property change
No equivalent ToggleStateProperty property change
No equivalent WindowVisualStateProperty property change
No equivalent AsyncContentLoadedEvent event
No equivalent ToolTipOpenedEvent

Security

Some IAccessible customization scenarios require wrapping a base IAccessible and calling through to it. This has security implications, since a partially trusted component should not be an intermediary on a code path.

The UI Automation model removes the need for providers to call through to other provider code. The UI Automation core service does all the necessary aggregation.

See also