May the Force be with you: Facebook says its neural wristbands will one day let you control virtual objects at a distance

Facebook: Social media giant Facebook has given a glimpse into its new augmented reality interface plans, which is based on CTRL-Labs technology. Facebook has touted this AI-powered AR interface as one that would not make users choose between interacting with the surrounding world or their devices, integrating users’ surroundings with their devices. The platform has shared a video on its plans, showing off the wristbands, which use electromyography (EMG) and translate subtle neural signals into actions, including swiping, typing and playing games. Haptic feedback is also provided by the bands, making it a system more responsive than the basic hand tracking options.

The social media platform had acquired the CTRL-Labs startup back in 2019.

Simply speaking, the bands can track basic gestures that are supposedly simple as well as easy to execute, and these gestures are called “clicks” by Facebook. And these gestures are tracked with the help of nerve signals running along the arms.

Two critical aspects of this technology include a technology that would allow users to effortlessly communicate with the system, as well as AI that would build a contextual understanding of the user and make inferences about the information the user might need or the actions the user might undertake based on this understanding of the user and their surroundings.

In theory, the bands are capable of doing much more, like tracking the brain’s nerve signals to the fingers while a user is typing, allowing users to type on a virtual keyboard rather than a physical one. The bands are also capable of adapting to the manner in which a user types, learning the ways a person’s fingers move when they make common typographical errors, so that the band can automatically correct those errors, and capture what the user likely wanted to type.

The bands have the potential to change the manner in which most people interact with computers. However, the concept is not very ahead of the work that the CTRL Labs had said it was doing years ago. In actuality, EMG has the ultimate potential to perform these same typing-style gestures if the user only thinks about moving their hands, without actually doing it. Eventually, the technology has the potential to let the user control virtual objects at a distance, “sort of like having a superpower like the Force.”

For now, though, the Mark Zuckerberg-headed company is looking at streamlining user interactions by using AI and augmented reality glasses. Notably, Facebook had announced in 2020 that it was working on the AR glasses.

The controllers are likely to offer an interface that the user could wear at all times, rather than having to pick them up, even in the simpler iterations. This is because the company was looking at a solution that would provide safety, privacy as well as security, while also letting people meaningfully control and personalise their experience with the AR. Intuitive, easy to use, always available and non-obtrusive were some of the requirements needed to be fulfilled by the solution so as to give users maximum comfort and benefit, the company said, which is why the wrist made the most sense, considering that it was the traditional place where watches were worn, along with the fact that it right next to the users hands, which is what people use the most while interacting with the world.

What makes this even more interesting is the haptic feedback. To simplify, haptic feedback is the sensation that we feel when we touch another object. The way humans function is based a lot on haptics or the sense of touch, and what usually distinguishes virtual world of say video games is primarily the lack of these haptics. However, Facebook’s Reality Labs is hoping to bring in the haptic feedback into these wristbands, at least to some extent. The company said that while it would be unable to bring all the sensations of the real world into the virtual one, it is “starting to produce a lot of them”.

According to Facebook, if one were to play a virtual game of bow and arrow, its wristwatch would be able to approximate the sensation of the string of the bow being pulled back. The company hopes that this would allow users to have more confidence in their actions and how they are doing things.

Will this mean that Facebook would be able to read your mind?

In a word – no. Even though the technology has been called a brain-computer interface by CTRL Labs, it is not quite that, at least not at the level of Elon Musk’s Neuralink, which would work by inserting a chip in the brain of the user. Facebook has said that the system would only be able to detect the thoughts that the brain decides to act upon. Basically, the system would only decode the signals at the wrist that the brain would send as commands, so that these signals can be converted into digital instructions for the device.

The band would still be collecting a lot of information, including tracking the way the user’s body moves. Though that is done by all wearable technology, it still needs a lot of trust on the part of users to accept such a technology, especially coming from Facebook, which has been surrounded by privacy-related issues for years now.