Advanced Pointer Overview

This is an overview of the Window Server extensions that support the use of advanced pointers. These extensions enable mobile devices to respond to events from multiple pointers, including their proximity and pressure. These extensions are available only in ScreenPlay.

Variant: ScreenPlay. Target audience: Application developers.

Architecture

In response to multiple pointer movements, an input driver generates a pointer event for each touch. The Window Server changes and delivers the pointer events to the client application, as shown in the following diagram.

Figure: Pointer events

Pointer-related events move from the driver to the application. The TRawEvent, TWsEvent and TPointerEvent classes encapsulate the event information.

Key concepts/terms

The following terms are introduced here and are used throughout the advanced pointer documentation:

Pointer: A device that points at the screen, for example, a mouse, pen, finger, or touchpad.
Multiple pointer support: Handling more than one pointer at the same time.
Proximity: Proximity in this context means the distance of the pointer from the mobile device. Sensors detect the value of the proximity of the pointer from the mobile device.
Pressure: Pressure in this context is a value that indicates the force applied to the mobile device by the pointer. Sensors detect the value of the pressure of the pointer on the mobile device.

Logical model of pointing devices

A logical model is used to handle all of the different pointing devices correctly. In this logical model, the pointer is performing an action when any of the following scenarios occur:

An EButton1Down event is sent to the client.
EDrag events are sent to the client (optional).
An EButton1Up event is sent to the client.

Pointing devices can be in one of the following states:

State	Description
Out of range	Not detected by the device's sensors.
Up	Detected by sensors, coordinates available, not performing an action.
Down	Detected by sensors, coordinates available, performing an action (as defined above).

The mapping between these states and physical actions performed by the user vary for different pointing device types (mouse, finger) and different phone models. Here are two example interpretations of physical pointing device states:

Mouse. An example interpretation by a driver of the physical state of a mouse is as follows: It is always in the up state except when the left button is pressed, when it is in the down state. It is never out of range.
Fingers with proximity support. The following is an example interpretation of finger physical states when the device has proximity support (that is, it can measure the finger's proximity from the screen):
- Out of range when proximity > 10cm
- Up when 1cm < proximity <= 10cm
- Down when proximity <= 1cm
In this way the finger can perform actions (for example, press buttons, drag items) without touching the screen.

Multiple pointers

The system handles multiple pointers at the same time by identifying each one with a number and sending pointer events separately for each of them. A field (iPointerNumber) in the pointer event classes (TRawEvent and the TAdvancedPointerEvent class which extends TPointerEvent) identifies the pointer for which the event occurred. The order and frequency of pointer events depends on the driver of the pointing device.

On each phone there is a pool of available pointer numbers, ranging from 0 to HALData::EPointerNumberOfPointers - 1. Every time the driver detects a new pointer coming into the range of the device, the driver assigns a number from the pool. After the pointer goes out of the range of the device, the number no longer identifies that particular pointer and the driver sends TRawEvent::EPointer3DOutOfRange with this pointer number. The number is then released back to the pool and may be reused to identify another pointer coming into the device's range.

Pressure and proximity

The pressure of the pointer on the screen is represented as a positive integer. The proximity of the pointer to the screen is represented as a negative integer. It is up to the UI Platform to define units of pressure and proximity, which may be non-linear. In the TRawEvent class, pressure and proximity are expressed in terms of the Z coordinate of the pointer. The definition of the Z coordinate is based on the assumption that the pointer is either hovering over the screen (proximity < 0, pressure = 0) or touching the screen (proximity = 0, pressure >= 0):

If proximity < 0 and pressure = 0 then the Z coordinate = proximity.
If proximity = 0 and pressure >= 0 then the Z coordinate = pressure.

Figure: Relationships between the pointer proximity, pressure and z coordinate

API summary

API	Description
HALData	Includes z coordinate, pressure precision and number of pointers. Also contains threshold values for proximity and pressure.
`TRawEvent`	Includes the pointer number and a structure for proximity and pressure data.
`RWindowBase`	Enables advanced pointer events for the window.
`TWsEvent`	Supports `TAdvancedPointerEvent` (derived from `TPointerEvent)`.
`TPointerEvent`	Includes event types for pressing and closeness.
`TAdvancedPointerEvent`	Inherits from `TPointerEvent`. Provides capabilities for pressure, proximity and multiple pointers.
`RWsSession`	Provides get and set methods to manipulate threshold values for proximity and pressure.

Typical uses

Enabling many pointers for a window

Enables multiple pointers to act on one window.
Enabling concurrent control of many objects

Controls many objects on one or more windows with many pointers.
Getting z coordinates from TPointerEvent

Reads the pressure and proximity values for a pointer.
Pinch zooming

Enables the user to zoom in and out of data by using two fingers to pinch an area of the screen. Pinch zooming is normally used in web-page, map, picture, chart and control applications.
Responding to high pressure events

As the pointer state changes, the pressure value can be checked. When it passes over a threshold value, the Window Server responds appropriately to the client.