Virtual Reality System Lets You Explore Your Brain in Real-Time

What if it were possible to explore your brain in virtual reality, watching your thoughts flashing before your eyes? A team of neuroscientists and software developers have created a way to do precisely that.

A new system combines brain scanning, brain recording and virtual reality to allow a user to journey through a person's brain in real-time. This "glass brain" was demonstrated here at the South by Southwest (SXSW) Interactive festival today (March 10).

The system was developed bycomputational neuroscientists Tim Mullen and Christian Kothe of the University of California, San Diego, in collaboration with the lab of Adam Gazzaley, a neuroscientist at the University of California, San Francisco, using a virtual reality headset made by the gaming company Oculus Rift.

"We've never been able to step inside the structures [of the brain] and see it in this way," Gazzaley said. "It's biofeedback on the next level."

Independently, Gazzaley has been collaborating with Philip Rosedale, creator of the game Second Life, on using virtual reality to improve neuroscience, and vice versa.

The brain in the demo actually belonged to Rosedale's wife Yvette, who was wearing a cap studded with electroencephalogram (EEG) electrodes that measure differences in electric potential in order to record brain activity. Gazzaley's team had previously scanned Yvette's brain using magnetic resonance imaging (MRI) to reveal its underlying structure and tangle of neural fibers. [3D Images: Exploring the Human Brain]


During the demo, Rosedale wore a virtual reality headset through which he could explore his wife's brain in 3D, as flashes of light displayed her brain activity from the EEG. A projection screen showed a similar view to the audience. When the brain came on the screen, the buzz of excitement in the room was audible.

The glass brain didn't actually show what Yvette was thinking — the EEG signals merely painted a picture of her brain activity more broadly. But the researchers ultimately hope to get closer to decoding brain signals and displaying them using the virtual reality system.

Being able to literally watch your own brain activity is not only fun, it has therapeutic potential. For example, people with traumatic brain injury or other neurological problems could visualize how their condition is affecting their brain activity, and learn to correct it.

So far, the team has focused on visualizing one brain at a time. But Philip Rosedale foresees a day when two people could interact virtually in a way that telegraphs their inner state.

"People often speak of interactions in virtual reality as being impoverished compared with those in the real world", Rosedale said. "But, what if you could communicate in virtual reality in a way that makes real life impoverished?"


Glass brain flythrough

This is an anatomically-realistic 3D brain visualization depicting real-time source-localized activity (power and "effective" connectivity) from EEG (electroencephalographic) signals. Each color represents source power and connectivity in a different frequency band (theta, alpha, beta, gamma) and the golden lines are white matter anatomical fiber tracts. Estimated information transfer between brain regions is visualized as pulses of light flowing along the fiber tracts connecting the regions.


Tanya Lewis
Neuroscapelab