This article is republished from The Conversation under a Creative Commons license. Read the original article, which was published August 1, 2022.
Ever wondered why kissing feels better than holding hands? The tongue is a pretty incredible piece of kit, though notoriously difficult to study, due to its position inside the mouth. Obviously, it gives us access to the wonderful world of taste, but more than that, it has greater sensitivity to touch than the fingertip. Without it, we aren’t able to speak, sing, breathe efficiently or swallow delicious beverages.
So why don’t we use it even more? My new study investigates how to make the most of this strange organ – potentially as an interface to help people with visual impairments navigate and even exercise. I realise this may sound mindboggling, but please bear with me.
My research is part of a field known as “sensory substitution”, a branch of interdisciplinary science that combines psychology, neuroscience, computer science and engineering to develop “sensory substitution devices” (known as SSDs). SSDs convert sensory information from one sense to another. For example, if the device is designed for a person with a visual impairment, this typically means converting visual information from a video feed into sound or touch.
Drawing pictures on the tongue
BrainPort, first developed in 1998, is one such technology. It converts a camera’s video feed into moving patterns of electrical stimulation on the surface of the tongue. The “tongue display” (a small device shaped like a lollipop) consists of 400 tiny electrodes, with each electrode corresponding to a pixel from a camera’s video feed.
It creates a low-resolution tactile display on the tongue matching the output from the camera. The technology can be used to help stroke victims maintain their sense of balance. And in 2015, the US Food and Drug Administration approved its use as an aid for the visually impaired.
Imagine holding your hand up to a camera and feeling a tiny hand simultaneously appear on the tip of your tongue. It sort of feels a bit like someone is drawing images on your tongue in popping candy.
While the BrainPort has been around for years, it hasn’t seen much real-world uptake, despite being ten times cheaper than a retinal implant. I use the BrainPort to test how human attention works on the surface of the tongue, to see if differences in perception might be the cause of this.
In psychology research, there is a famous method to test attention, called the Posner Cueing paradigm, named after the American psychologist Mike Posner who developed it in the 1980s to measure visual attention.
When I say attention, I don’t mean “attention span”. Attention refers to the set of processes that bring things from the environment into our conscious awareness. Posner found that our attention can be cued by visual stimuli.
If we briefly see something moving out of the corner of our eye, attention focuses on that area. We probably evolved this way to quickly react to dangerous snakes lurking around corners and in the edges of our visual field.
This process also occurs between senses. If you’ve ever sat in a pub garden in summer and heard the dreaded drone of an incoming wasp to one ear, your attention is very quickly drawn to that side of your body.
The sound of the wasp captures your auditory attention to the general location of the potentially incoming wasp so that the brain can quickly allocate visual attention to identify the exact location of the wasp, and tactile attention to quickly swat or duck away from the wasp.
This is what we call “cross-modal” attention (vision is one mode of sensation, audio another): things that appear in one sense can influence other senses.
Paying attention to the tongue
My colleagues and I developed a variation of the Posner Cueing paradigm to see if the brain can allocate tactile attention on the surface of the tongue in the same way as the hands or other modes of attention. We know loads about visual attention, and tactile attention on the hands and other body parts, but have no idea if this knowledge translates to the tongue.
This is important because BrainPort is designed, built and sold to help people “see” through their tongue. But we need to understand if “seeing” with the tongue is the same as seeing with the eyes.
The answer to these questions, like almost everything in life, is that it’s complicated. The tongue does respond to cued information in roughly the same way as the hands or vision, but despite the incredible sensitivity of the tongue, attentional processes are a bit limited compared with the other senses. It is very easy to over-stimulate the tongue – causing sensory overload that can make it hard to feel what’s going on.
We also found that attentional processes on the tongue can be influenced by sound. For example, if a BrainPort user hears a sound to the left they can more easily identify information on the left side of their tongue. This could help to guide attention and reduce sensory overload with the BrainPort if paired with an auditory interface.
In terms of real-world use of the BrainPort, this translates to managing the complexity of visual information that gets substituted and, if possible, use another sense to help share some of the sensory load. Using the BrainPort in isolation could be too overstimulating to provide reliable information and could potentially be improved by using other assistive technology alongside, such as the vOICe.
We’re using these findings to develop a device to help rock climbers with visual impairments to navigate while climbing. To prevent information overload, we’re using machine learning to identify climbing holds and filter out less relevant information. We’re also exploring the possibility of using sound to cue where the next hold might be, and then use the feedback on the tongue to precisely locate the hold.
With a few tweaks, this technology may eventually become a more reliable instrument to help blind or deaf or blind people navigate. It may even help paraplegic people, unable to use their hands, navigate or communicate more efficiently.
Written by Mike Richardson, Research Associate in Psychology, University of Bath.