You sit down, turn on the game console, pick up a wireless controller, and wait for the system to warm up. You’re ready to game. Well, maybe that’s how it will be for now, but in the near future gaming could potentially be vastly different. You will sit down, put on your headset, and instantaneously use your thoughts to control the game. Brain computer-interface (BCI) has many medical applications, but what about its application in the entertainment industry, specifically in gaming? Last year, Neurable, a startup launched by CEO Ramses Alcaide, came out with the game Awakening, which was premiered as a demo at the Siggraph conference, a tech conference on interactive technology and computer animation. Awakening was the first marketable virtual reality (VR) video game controlled solely by your mind.
The actual technology used in this game is fascinating, combining brain-computer interface and virtual reality. The headband is non-invasive and consists of seven strategically placed EEG electrodes; the headband itself is also attached to VR goggles from HTC, a consumer electronics company. In other words, the EEG-based system analyzes your brain waves and connects the results to a software that interprets the signal and produces a specific outcome in the game. Alcaide equates the technology with providing your brain with its own computer mouse, so that you can scroll among objects in the virtual reality room and select the object that you want to “pick up.”
As always, there is the problem of background noise in EEG recordings. Background noise is everything picked up as a signal that does not actually come from the brain. To distinguish from such background noise, the technology uses machine learning algorithms to pick up on participant behavior and thus better understand player intentions, while also learning to ignore the background noise. This machine learning is utilized through a short training session before the player actually starts playing the game. The training session helps the headset recognize and learn what signals a person produces when they focus on a particular VR object. The session consists of a light bouncing around the VR room and a small, colored, virtual reality ball placed in front of you; you are asked to focus on the ball so that when the light hits the ball, your brain automatically produces an event-
related potential (a specific brain signal that directly arises as a response to a specific sensory event, in this case a flashing light on the object of interest). The system reads this event-related potential and begins to recognize such characteristic electrical spikes in your brain activity, which correlate to when you concentrate on an item. Consequently, when you play the game, the same light from the training session bounces around the VR room, and when it hits the item you think about, the system identifies the same characteristic increase in your brain activity and thus selects that item.
While the technology seems extremely technical and complicated, Alcaide claimed that the storyline was actually harder to devise than the headset, since his team had zero experience in the sector of game design. The science-fiction storyline is based on dystopia. You play a telekinetic child being held for experimentation in a government lab. You’re sole mission: to escape from the lab. How? By picking up objects in the room with your mind and throwing them at the surrounding walls to derive clues and break out of the room. Nevertheless, while the storyline of Awakening is simple, the prospect of the game is exciting for the future of mind-controlled, VR video games.
You know how industries work, when one company comes out with a product, other companies try to improve that model and make it better than ever. There are more companies today working on such BCI video games and trying to improve the precedent that Neurable set. Alcaide himself stated that he hopes to improve the Neurable headset by decreasing its dependency on EEG electrodes over time so that it eventually can consist solely of a more efficient, less bulky EEG sensor rather than multiple dry electrodes. Additionally, with the advent of VR arcades, such mind-controlled VR headsets may be able to roll out to a wider consumer population. VR arcades allow people to try out headsets before actually purchasing them, which is beneficial because oftentimes these headsets are pricey and people do not want to spend so much money on something they are not sure will even work.
We may be far away from a video game that combines both precise mind-controlled VR and an interesting storyline with amazing graphics. However, we are already making progress in that direction, and that could be very exciting for the world of gaming. Check out the video below on a Neurable’s headset at the Siggraph Conference in LA last year!
Works Cited:
https://www.technologyreview.com/s/608574/mind-controlled-vr-game-really-works/#comments
https://www.nytimes.com/2017/08/27/technology/thought-control-virtual-reality.html
https://spectrum.ieee.org/semiconductors/devices/the-first-mindcontrolled-vr-game-will-hit-arcades-in-2018
https://medium.com/inborn-experience/behold-the-next-generation-vr-technology-part-6-brain-interface-89b1d31a0a96