Freehand gesture-based interfaces in interactive systems are becoming more common, but what if your preferred way to gesture a command -- say, changing the TV to channel 10 -- significantly differed from that of a user from another culture? Would the system recognize your command?

Researchers from the Penn State College of Information Sciences and Technology and their collaborators explored this question and found that some gesture choices are significantly influenced by the cultural backgrounds of participants.

"Certain cultures may prefer particular gestures and we may see a difference, but there is common ground between cultures choosing some gestures for the same kind of purposes and actions," said Xiaolong "Luke" Zhang, associate professor of information sciences and technology and principal investigator of the study. "So we wanted to find out what can be shared among the different cultures, and what the differences are among different cultures to design better products."

In their study, the researchers asked American and Chinese participants to perform their preferred gestures for different commands in three separate settings: answering a phone call in the car, rotating an object in a virtual reality environment, and muting the television.

The team found that while many preferred commands were similar among both cultural groups, there were some gesture choices that differed significantly between the groups. For example, most American participants used a thumbs up gesture to confirm a task in the virtual reality environment, while Chinese participants preferred to make an OK sign with their fingers. To reject a phone call in the car, most American participants made a horizontal movement across their neck with a flat hand, similar to a "cut" motion, while Chinese participants waved a hand back and forth to reject the call. Additionally, in Chinese culture, one hand can represent digits above five, while in American culture an individual can only represent numbers one to five using one hand.

"This project is one of the first kind of research to study the existence of cultural influence and the use of preferences of hand gestures," said Zhang. "We provide empirical evidence to show indeed that we should be aware of the existence of this matter."

On the other hand, Zhang said, from the perspective of design, the study shows that certain gestures can be common across multiple cultures, while other gestures can be very different.

"Designers have to be careful when delivering products to different markets," he said. "(This work could inform companies) to enable users customize the gesture commands, rather than have them pick something that is unnatural to learn from the perspective of the culture."

Story Source:

Materials provided by Penn State. Original written by Jessica Hallman. Note: Content may be edited for style and length.

Journal Reference:

1. Huiyue Wu, Jinxuan Gai, Yu Wang, Jiayi Liu, Jiali Qiu, Jianmin Wang, Xiaolong(Luke) Zhang. Influence of cultural factors on freehand gesture design. International Journal of Human-Computer Studies, 2020; 143: 102502 DOI: 10.1016/j.ijhcs.2020.102502