feedback for gestures 2017

minimal feedback for gesture-controlled devices

cover_gesture-feedback

why it was made

There are almost no gesture-controlled devices where users can set certain values like explicit numbers precisely.
Gesture controls seem to be imprecise because they are lacking feedback for their users.
We wanted to reduce that imprecision and thereby enhance the safety of gesture-controlled devices.

what it is based on

First of all, we studied different types of feedback and how precise users can set certain values when having feedback support.
Secondly, we tested people's associations with various light blinking patterns.

Before starting the user tests, we thought of "universal" gestures for setting values and confirming them (or triggering something).
We chose an up/down movement of the flat hand as value-setting gesture and closing the hand for confirming.

setvalue_loop
confirmgesture_loop

Testing Feedback Types
We had extensive testing sessions with ten different people where we let them set values by just using the up/down gesture. These test persons were told to set a specific value (using scales 0-3, 0-9, 0-24, 0-50) while receiving auditive, tactile and visual feedback after another. We accomplished this using an Arduino UNO, a Leap Motion, a speaker, a small vibration motor and various LEDs.

versuchsaufbau_visuell

After conducting more than 20 different tests, we came to the conclusion that visual feedback enables users to set the precisest values using gestures.

Blinking Pattern Testing
To see what people associate with various LED blinking patterns, we have asked over 70 people to tell us their thoughts on patterns shown to them.
We used the results of this survey to combine them with the RGB-LED colour values from the previous study.

synthesis

Colours represent the values that are set or will be set.
The brightness of the light indicates the status of the device.

To simulate a use case we built a gesture-controlled ventilator with an integrated RGB-LED that is responsible for the feedback.

ventilator-leap-still

Fan is on and set to a medium intensity
Yellow light is shown with low brightness. Because of its low brightness, it does not interfere with the environment during standby. However, the light is on and shows the device is ready to be used.

User gets into interaction zone
The brightness sets to 100% as the user has to see the colour values properly. Colour changes to the value the user's hand is at immediately.

Hand is being moved and reaches maxima
During the hand's movement, the reached values are shown by the different colour tones. As soon as the hand reaches the lowest or highest point of the scale the LED blinks two times.

User doesn't confirm a value, fan intensity stays the same as before

Fan is off
User starts interaction and confirms value with hand closing gesture
The LEDs brightness changes to 100% at the start of the interaction. As soon as the user has chosen a value and confirms it, the LED shortly becomes less bright to indicate confirmation.

Before leaving the interaction zone, the user looks at different values
Just like before values are shown but since the user does not confirm them they won't be set. So when the user finishes the interaction the value changes back to the last value set.