High performance touch interactions – TurboTouch
TurboTouch : High-performance touch interactions
Transfer functions and latency in tactile interaction
Touch surfaces have become the preferred way of interacting with interactive systems, such as desktops, smartphones or tablets. Interaction with these surfaces can be direct or indirect, de-pending on whether it is co-located with the interactive content or not. In both cases, the transfer functions define the relationship between moving a finger and changing the state of the virtual objects displayed. The design of these functions is usually reduced to ad-hoc choices. A first objective of the project was to characterize the functions used in existing systems in order to know their performance, in order to improve them and to propose design alternatives. The time between a user's physical action on a touch surface and the update of visual information also affects the use of touch surfaces. The objective has been to develop methods to easily measure these times, which are lacking. Another objective was to develop algorithms to partially or fully compensate for this latency.
Our approach is based on taking into account the motor and perceptual abilities of users as well as the characteristics of the task they are performing. It is also a question of knowing how the current systems work, in order to take into account their characteristics to improve transfer functions and latency compensation. To do this, we started by characterizing the transfer functions for pointing and scrolling tasks. We then developed new devices for measuring and characterizing latency. This work made it possible to work on the development of new latency compensation algorithms based on trajectory extrapolation. We have also developed models for modeling pointing tasks and developing new transfer functions. The various proposals were validated by carrying out controlled experiments and simulations to demonstrate the relevance of the pro-posed solutions.
We have obtained major results on the measurement and compensation of latency. We have developed the first technique for measuring end-to-end latency and characterizing its origin when a user is interacting with a system. We have also developed a new latency compensation algorithm that offers better performance compared to the algorithms of the state-of-the-art. We have also developed new transfer functions for scrolling on smartphones that are more efficient than those in the literature.
We also tested new transfer functions used to characterize the visual-motor performance of such transfer functions. A study on the calibration latency with a new transfer function was also per-formed.
The project opens up perspectives on improving latency compensation techniques and transfer functions. For example, new tools need to be implemented to estimate the prediction error ac-cording to the context, in order to be able to adapt the amount of compensation in real time. The algorithms developed could also be adapted to other contexts, such as virtual reality. In a more distant future, the objective is to be able to measure and compensate in real time for the latency of a system. For transfer functions, the work of modelling pointing tasks still needs to be further developed in order to take into account the use of non-linear transfer functions and thus have more advanced tools to optimize them.
Our work has been published in the best conferences in the fields of Human Machine Interac-tion and Automatics: 3 articles at CHI (CORE A*), 4 at UIST (CORE A) and 2 at CDC (CORE A). Two articles were also published in good journals in Automatics. The work on latency measurement and compensation has also been published in 5 international patents and one national patent. The work on the characteristics of visual-motor performance in relation to different transfer functions is being finalized before submission to international journals in the field of behavioral neuro-sciences.
Touch-based interactions with computing systems are greatly affected by two interrelated factors: the transfer functions applied on finger movements, and latency. Little is actually known on these functions, and latency only recently received attention in this context. This project aims at transforming the design of touch transfer functions from black art to science to support high-performance interactions. We will precisely characterize the functions used and the latency observed in current touch systems. We will develop a testbed environment to support multidisciplinary research on touch transfer functions. We will use this testbed to design latency reduction and compensation techniques, and new transfer functions. Three partners with expertise in Human-Computer Interaction, Control Theory and Human Movement Science will collaborate on this project: the MINT and NON-A teams from Inria Lille, and the "Perceptual-motor behavior group" from the Institute of Movement Sciences.
Monsieur Géry Casiez (Institut National de Recherche en Informatique et Automatique)
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
Inria Institut National de Recherche en Informatique et Automatique
ISM Institut des Sciences du Mouvement
Help of the ANR 455,663 euros
Beginning and duration of the scientific project: September 2014 - 48 Months