Microsoft Active Accessibility ​

Microsoft Active Accessibility (MSAA) is an older interface used for accessibility. It can interact with screen readers such as Windows Narrator or NVDA, and has a more limited set of available functionality compared to newer interfaces. The base of MSAA is the IAccessible interface, which all a11y-providing classes extend. MSAA methods are prefixed with acc_. In the Flutter engine, AXPlatformNodeWin extends IAccessible. Any IAccessible objects used are COM interfaces, and should be initialized and reference counted accordingly. When a program such a screen reader queries an application for an a11y object, it sends a WM_GETOBJECT window message, to which the window will respond with the result of passing the IAccessible of its root to the function LresultFromObject.

UI Automation ​

UI Automation is one framework for Windows that is used for accessibility, for example, by screen readers such as Windows Narrator or NVDA. The interface class IRawElementProviderFragment must be implemented by any object representing a UIA fragment, i.e. an accessibility item that exists on the screen, such as a text label, a pane, a button, etc.. For the Flutter engine, the class AXPlatformNodeWin extends IRawElementProviderFragment in order to serve this purpose. Abstract methods in IRawElementProviderFragment provide functionality including retrieving the bounds of the fragment on screen, testing if an on-screen point is within the fragment, and retrieving properties of the fragment such as name and role. Each of the methods of IRawElementProviderFragment implemented by AXPlatformNodeWin are listed in ax_platform_node_win.h. Additionally, IRawElementProviderFragment contains a method that returns the fragment root of the fragment. The fragment root represents the root of a fragment tree, and extends the class IRawElementProviderFragmentRoot. Generally, and in the case of Flutter, these correspond 1-1 to a window. We extend this class in AXFragmentRootPlatformNodeWin in ax_fragment_root_win.cc. This class contains methods to retrieve the fragment at a given point, and the current focus of the tree. The Window class’s OnGetObject handles WM_GETOBJECT window messages, which a screen reader will dispatch to access the UIA fragment tree. When the lparam parameter of the message is equal to the constant UiaRootObjectId, the request is asking for the UIA root, and a reference is created out of the window’s fragment root with UiaReturnRawElementProvider, which is then returned to the message sender. Patterns are optional interfaces which UIA elements can provide for additional functionality. For example, providing the ITextProvider allows a fragment to be treated as formatted text, and IToggleProvider indicates that a fragment can be toggled (e.g. a checkbox) and provides its current toggle state (i.e. on, off, mixed). The UIA method GetPatternProvider is used to return an instance of a requested interface type, or none if it is not provided. Provided patterns may or may not be the same as the fragment itself. For example, in AXPlatformNodeWin, when IToggleProvider is requested, a casted pointer to the node is returned, as AXPlatformNodeWin extends IToggleProvider. By contrast, when an ITextProvider is requested, a separate object is returned, rather than AXPlatformNodeWin being a subclass of ITextProvider. In this case, this is necessary, as other inherited methods of AXPlatformNodeWin share a name but different signature to those in ITextProvider, so it cannot extend both.

Whether or not the engine makes use of UI Automation is controlled in the Window class, in Window::OnGetObject. When a UIA message is received, a UIA response is only issued if the engine was built with the FLUTTER_ENGINE_USE_UIA macro defined.