Around-Device Interaction

Gesture interaction in the space around small devices is one way of overcoming limitations of touchscreen input. Gestures performed near a device allow users to interact more casually, for example: quickly dismissing incoming phone calls when busy; or checking for new text messages without having to pick the phone up. This style of interaction also allows input when touch is not suitable, for example: when hands are wet and messy from preparing food but you need to quickly check a recipe; or when wearing gloves outside in the cold. Finally, around-device interaction can also avoid problems caused by finger occlusion, where the interacting fingers cover parts of the display when providing touchscreen input. Wearable devices, like smart-watches, would especially benefit from less occlusion as screen sizes are much smaller than on phones.

Gesture Feedback

Good feedback is important to help users gesture effectively, although giving good feedback from small devices can be difficult. Visual feedback is not always ideal because it takes away already limited screen space. On-screen cues may also be difficult to notice from a short distance away. Instead, we look at how other output types can be used to give useful feedback. Audio and tactile feedback in particular are promising output modalities for gesture feedback.

Audio and tactile feedback are not without their own difficulties, however, and our research seeks to address these. One of the challenges of trying to deliver tactile feedback in a gesture interface is that users are not always in contact with the device they are interacting with; instead, they may be gesturing from several feet away or from across the room. Non-contact forms of tactile feedback (such as ultrasound haptics [1]) and distal methods of tactile feedback (such as wearable devices) should also be considered. We looked at these types of tactile feedback in our ICMI 2014 paper [2].

Gesture Design

We’ve also looked at gesture design for around-device interfaces [3]. In our paper we give recommendations for designing usable and acceptable mid-air gestures for mobile phones. We also present a set of evaluated gestures. In that work we also looked at the social acceptability of gestures, with similar findings to our earlier work on gesture acceptability [4].

References

[1] G. Wilson, T. Carter, S. Subramanian, and S. Brewster: Perception of Ultrasonic Haptic Feedback on the Hand: Localisation and Apparent Motion. In Proceedings of acm chi 2014, pp. 1133-1142 (2014).
[Bibtex]
@article{Wilson2014,
author = {Wilson, Graham and Carter, Tom and Subramanian, Sriram and Brewster, Stephen},
journal = {Proceedings of ACM CHI 2014},
keywords = {Ultrasound,haptic feedback,localisation,perception},
pages = {1133--1142},
title = {{Perception of Ultrasonic Haptic Feedback on the Hand: Localisation and Apparent Motion}},
url = {http://dl.acm.org/citation.cfm?id=2557033},
year = {2014}
}
[2] E. Freeman, S. Brewster, and V. Lantz: Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions . In Proceedings of icmi ’14 (2014).
[Bibtex]
@inproceedings{Freeman2014,
author = {Freeman, Euan and Brewster, Stephen and Lantz, Vuokko},
booktitle = {Proceedings of ICMI '14},
publisher = {ACM Press},
title = {{Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions}},
url = {http://research.euanfreeman.co.uk/papers/ICMI\_2014.pdf},
year = {2014}
}
[3] [doi] E. Freeman, S. Brewster, and V. Lantz: Towards Usable and Acceptable Above-Device Interactions . In Proceedings of mobile hci ’14 posters (2014).
[Bibtex]
@inproceedings{Freeman2014a,
author = {Freeman, Euan and Brewster, Stephen and Lantz, Vuokko},
booktitle = {Proceedings of Mobile HCI '14 Posters},
doi = {10.1145/2628363.2634215},
publisher = {ACM Press},
title = {{Towards Usable and Acceptable Above-Device Interactions}},
url = {http://research.euanfreeman.co.uk/papers/MobileHCI\_2014\_Poster.pdf},
year = {2014}
}
[4] J. Rico and S. Brewster: Usable gestures for mobile interfaces: evaluating social acceptability . In Proceedings of the 28th international conference on \ldots (2010).
[Bibtex]
@inproceedings{Rico2010,
author = {Rico, Julie and Brewster, Stephen},
booktitle = {Proceedings of the 28th international conference on \ldots},
file = {:C$\backslash$:/Users/user/AppData/Local/Mendeley Ltd./Mendeley Desktop/Downloaded/Rico, Brewster - 2010 - Usable gestures for mobile interfaces evaluating social acceptability.pdf:pdf},
isbn = {9781605589299},
title = {{Usable gestures for mobile interfaces: evaluating social acceptability}},
url = {http://dl.acm.org/citation.cfm?id=1753458},
year = {2010}
}