How brain waves might help us see

Dr. Adeeti Aggarwal was the lead author on this study. Her ultimate career goal is to become an academic ophthalmologist whose clinical insights motivate her research in visual processing, and whose research also translates back to patient care. She is fascinated by how cortical networks transform visual sensory information into perception and how defects in sensory processing may alter or abolish perception such as in hallucinations or blindness. This interest has driven her research in graduate school and she hopes to continue studying how visual processing pathways participate in perceptual experience as her career progresses. 

or technically,

Visual evoked feedforward-feedback traveling waves organize neural activity across the cortical hierarchy in mice

[See Original Abstract on Pubmed]

Authors of the study: Adeeti Aggarwal, Connor Brennan, Jennifer Luo, Helen Chung, Diego Contreras, Max B. Kelz, Alex Proekt

Modern cameras do an amazing job of turning the photons of light in the world into pixels on our phone or laptop screen that faithfully capture that moment in time. The fact that we all walk around with the technology to do this sitting in our pockets is the result of decades of innovation and technological advancement. But even with everything that your smartphone’s camera can capture, we have an even more elegant piece of machinery doing all that and more sitting between our ears all day: our brains.

How is our ability to see different than a camera? To start, there’s the obvious difference in materials. Cameras are made of hard, man-made materials, whereas your brain is filled with comparatively squishier biological material. But even more importantly, a camera and your brain are trying to accomplish two different things. The goal of a camera is to recreate the world exactly as it is. The goal of your visual system is to use what you see to interact with the world. Unlike cameras, you need to do things like pay attention to one thing over another, predict what’s coming next, or change your behavior according to what you see.

We can think of the brain as needing to accomplish two things: 1) build up a representation of what is in the world, and 2) integrate that into our current understanding of the world and intended actions to accomplish something. One popular idea, or hypothesis, is that the brain accomplishes the first goal of building up a representation of the world by sending neural signals through several brain regions moving from the back of your head toward the front, termed feedforward communication. The second goal is then accomplished by integrating those signals with neural activity in other brain regions and then passing a signal backwards through the same regions from front to back, which is called feedback. These “traveling waves” of brain activity could coordinate brain activity across different parts of the brain and integrate the two goals of the visual system.

Figure 1

Illustration of the hypothesized direction of the flow of brain activity for feedforward waves (yellow) and feedback waves (blue). Figure made with biorender.com.

Testing this hypothesis has been difficult, because it requires the ability to look at brain activity across large portions of the brain as it changes very quickly and the tools to do this were only recently developed. Until recently, several scientists had used what tools were available to study feedforward and feedback activity, but they could only look for small snapshots of evidence of feedforward and feedback waves. However, last year a team of researchers at the University of Pennsylvania led by Dr. Adeeti Aggarwal, a former PhD student in the Neuroscience Graduate Group, used new technology to visualize these waves of activity across the mouse brain for the first time.

To do this, Dr. Aggarwal and her team recorded brain activity across several areas of the mouse brain while they flashed a green light in front of the mouse’s eye. By using a special kind of analysis that allowed them to get a cleaner look at the data, they were able to see the two kinds of brain waves that the hypothesis predicted. The first feedforward wave fluctuated quickly and moved from the back to the front of the brain, while the second feedback wave fluctuated more slowly and moved from the front to the back of the brain. Importantly, the team found that both waves of activity spread equally far across the brain, despite the feedforward wave fluctuating faster than the feedback wave. Through this and other observations the team concluded that the two waves of brain activity interact and integrate to form a cohesive wave of brain activity that could be combining the information about what the mouse is seeing with other brain signals.

This was exciting evidence that the kinds of feedforward and feedback waves that neuroscientists thought could coordinate visual information are actually present in the brain, but how might they help a mouse to see?  Your brain cells, called neurons, communicate with each other by sending a kind of signal called an action potential, or spike. Whether and how a neuron produces spikes is what ultimately influences what you see and how you behave. To demonstrate that these waves of brain activity could shape these important brain signals, Dr. Aggarwal and her team looked at whether the waves of brain activity had an impact on whether and how neurons produced spikes.  They found that neurons were more likely to produce spikes at the peaks of the slow oscillation than at the lower points. This links the waves of brain activity that they observed directly to spikes, which suggests that these waves are capable of coordinating brain information about what the mouse is seeing with other kinds of signals.

Dr. Aggarwal and her team’s paper provides exciting new evidence for how different parts of the brain can be coordinated through waves of activity, and future work will continue to determine how these waves can be linked to behavior and whether they can be seen in human brains as well. Understanding how the brain coordinates activity across brain regions to turn sight into action could be helpful in many ways. For one, this information could help to engineer better visual prosthetics for people who are blind. If these waves are necessary to coordinate brain activity across parts of the brain, it may be necessary for visual prosthetics to produce signals that work in the same way. Beyond direct human applications, incorporating similar principles into the design of robotic systems that need to coordinate information about the world with a set of goals or actions could produce robots that can better interact with the world to accomplish their goals. As with all scientific advancements, Dr. Aggarwal’s study is one exciting piece in many bigger puzzles.

About the brief writer: Catrina Hacker

Catrina Hacker is a PhD candidate working in Dr. Nicole Rust’s Lab. She is broadly interested in the neural correlates of cognitive processes and is currently studying how we remember what we see. She also co-directs PennNeuroKnow.

Interested in learning more about Adeeti’s work? Check out the full paper here!

Shedding light on migraines: Signals from the eye make people with migraine more sensitive to light

or technically,

Selective amplification of ipRGC signals accounts for interictal photophobia in migraine

[See Original Abstract on Pubmed]

Authors of the study: Harrison McAdams, Eric A Kaiser, Aleksandra Igdalova, Edda B Haggerty, Brett Cucchiara, David H Brainard, Geoffrey K Aguirre

Have you ever stepped outside and had to squint or shield your eyes from the sun? Bright light can be uncomfortable for anyone, but it can be especially painful to those who experience migraines. While headaches are often associated with migraines, another common symptom is photophobia, or sensitivity to light. Harrison McAdams, a neuroscience student in Dr. Geoffrey Aguirre‘s lab at Penn, wanted to find out what part of the eye or brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. might be responsible for the symptom of photophobia in migraines.

In order to discover how photophobia might arise, we need to understand how our eyes allow us to see our surroundings. The first step in vision is when light hits the retina, a sheet of cells that covers the back of the eye. The retina is made of three layers of neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles with specific functions. Some of these neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles can detect light: these are the rods and cones, and they become excited by photons, the particles that make up light. Rods work in dim light, while cones work in bright light and can sense red, green, or blue light. This is how we’re able to see in color! Rods and cones talk to other neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles in the retina like RGCs (retinal ganglion cells), which then send signals out of the eye and to many different areas of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals..

Figure 1: Anatomy of the eye and retina (adapted from webvision.med.utah.edu)

Most RGCs just listen to the rods and cones that talk to them; however, there are special RGCs that are “intrinsically photosensitive” (ipRGCs), which means they can detect light just like rods and cones do. These ipRGCs actually don’t help you see images. Instead, they are responsible for your sense of a day/night cycle and for helping your pupils adjust to the light or dark. There is also evidence that ipRGCs connect to the thalamus, an area of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. which can sense pain. Harrison thought this connection could explain why light can be especially painful to people with migraine.

60 people participated in this study: 40 experienced photophobia from migraines and 20 did not have migraines for comparison. Pulses of different colored light were flashed in one eye to either activate the cones, the ipRGCs, or both. They rated how painful each flash of light was on a scale from 0 to 10, 10 being the most painful. People with photophobia tended to rate the “cone” light as more painful compared to those without, and their rating increased as the light got brighter. Since we know that they’re sensitive to bright light and that cones work in bright light, this makes sense. Interestingly, they also gave higher pain ratings to light which only activated the ipRGCs.

From this test, Harrison knew that ipRGC activity could cause pain. After recording pupil size while the light was flashing, he found no difference in people with or without migraine. This means that not all ipRGC functions are affected: It’s a specific strengthening of the signal from the retina to the pain-sensing area of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals..

Why would our eyes be wired like this? Just like touching a hot stove can damage your hand, staring at the sun or shining a flashlight in your eyes can damage your retinas. And once those neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles are gone, they’re gone for good--so take care of them! Although this signal can be helpful, warning us that what we’re doing is harmful, it seems like when the signal becomes too strong, people can develop migraines. Identifying the root of the problem is the first step in developing treatments to help people who live with this condition.

About the brief writer: Sierra Foshe

Sierra is a PhD student in Josh Dunaief's lab. She is interested in the mechanisms of retinal inflammation and degeneration.

Want to learn more about the role of ipRGCs in photophobia? You can find Harrison’s full paper here!

How you find what you’re looking for.

or technically,

Signals in inferotemporal and perirhinal cortex suggest an untangling of visual target information.

[See Original Abstract on Pubmed]

Authors of the study: Marino Pagan, Luke S Urban, Margot P Wohl, Nicole C Rust

Finding Your Keys
You are late and the Uber is already outside. Where are your wallet and keys? You scan the nearest table. A dirty coffee cup, excessive CVS coupons, and at last, you see your wallet and keys, poking out from under a shirt. While this process might seem effortless, quickly finding what you are looking for in a crowded scene -- a process called “visual search” -- is an ability that even sophisticated computer programs have trouble with ¹. Marino Pagan in Nicole Rust’s lab at the University of Pennsylvania spent his PhD studying exactly how our brainsThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. can quickly and flexibly find what we’re looking for out of everything we are looking at.

However, this question has been difficult for scientists to answer. Before Marino performed his experiments, it was unknown which part of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. was responsible for visual search. In other words, scientists hadn’t yet been able to identify neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles that specifically respond when what you are looking for matches what you are looking at. Additionally, it was unknown how any brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. area would combine the information about what you are looking for and what you are looking at. How do scientists go about answering where and how the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. identifies different visual search “targets?” Before we tell you the results, we’re going to break down the approach Marino took to answering these questions.

Where in the BrainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals.?
Let’s first examine the question of where -- where are the neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles in the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. that are responsible for identifying visual search targets? We can answer this question by guessing what the activity of a brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. area that identifies search targets would look like and then looking for brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. areas with activity that matches our hypothesis. Like we mentioned before, neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles in this area would respond or “turn on” when what you are looking at matches what you are looking for. In our earlier example about getting to your Uber, we would guess that a visual search area would turn on when you were looking at your wallet, or your keys, but not the coffee cup. However, in a different situation--let’s say making coffee--what you are looking for is different. This time, the visual search area would respond when you look at the coffee cup, but not the other items. Essentially, the visual search area should turn on when you are looking at the item you were searching for.

How?
Now let’s examine how the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. figures out whether what you’re looking at matches what you’re looking for. All information in the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. is represented as different patterns of neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles turning on - or “firing.” For example, a pattern in which all neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles fire could mean something different than a pattern in which only half the neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles fire. In order for the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. to determine whether what you’re looking for matches what you’re looking at, the pattern of neuronA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles firing in the visual search area must be different for matches versus non-matches. When your brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. can differentiate - or separate - the neuronA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles firing patterns for matches and non-matches, you will be able to distinguish between the two categories in the real world! So our question of how now becomes more specific - how does the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. separate patterns of firing for matches and non-matches? It turns out, there are many ways that the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. can separate firing patterns! Learning which ones the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. uses not only teaches us about how our brainsThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. work, but can also help us build better computer algorithms to perform search tasks- not just for finding keys on a table, but for, say, identifying faces in a crowd.

Although the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. has a lot of neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles, we’re going to think about different ways to separate firing patterns by pretending there are only two neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles in the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. area responsible for visual search. In this situation, one way for the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. to determine whether a firing pattern says “match” or “no match” is to make a simple rule that divides the patterns into two groups. One example of a rule could be “If neuronA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles 1 is firing more than neuronA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles 2, you are looking at a match. Otherwise, you are not”. This rule is shown in Fig 1, on the left. Notice how the rule makes it easy to draw a straight line that perfectly puts all matches on one side of the line, and all non-matches on the other. This type of neural firing is said to be linearly separable. It is linear something that can be represented as a straight line on a graph, and directly proportional changes in two related quantities because a simple, straight line can separate the two categories. This way of separating firing patterns is both very reliable and very easy for the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. to do! Other ways of separating might require many more complicated rules (an example of this is shown in Fig 1, right). Therefore, linearly separable neural firing is good candidate for how the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. might distinguish between search targets versus other objects.

With all this in mind, Marino Pagan and his PhD advisor Nicole Rust could make hypotheses about how the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. identifies different visual search “targets” and which area of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. is responsible for this: 1) the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals.’s “visual search area” would turn on when what you are looking for matches what you are looking at, and 2) neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles in this visual search area will have separable firing patterns for matches vs. non-matches. To test these hypotheses, they did one experiment to recreate the process we go through when looking for our keys.

Figure 1

Results
First, Marino trained monkeys to recognize specific, everyday objects (i.e. keys or wallet) in a sequence of images interspersed with other objects (i.e. the coffee cup). They then looked for (1) where in the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. there was activity specific to targets and (2) how this region separated its firing patterns for matches and non-matches (i.e. were they linearly separable). They narrowed their search to two regions of the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals.: the inferior temporal lobe and the perirhinal cortex. The inferior temporal lobe is a part of the visual system and is thought to be the first place that memory (i.e. what you are looking for) and visual information (i.e. what you are looking at) are combined². The perirhinal cortex receives information from the inferior temporal lobe and is necessary for good performance on visual search tasks³.

Marino first asked whether either brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. region had activity patterns that were selective for search targets. They found this selectivity to be much stronger in the perirhinal cortex than the inferior temporal lobe, suggesting that the where of visual search is primarily the perirhinal cortex (PRH).

To address how this selective activity arose, Marino then asked if neural firing in response to targets was more linearly separable in one brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. region compared to the other. After looking over many neuronsA nerve cell that uses electrical and chemical signals to send information to other cells including other neurons and muscles, they found that it was much easier to draw a simple line that separated targets from non-targets (similar to Fig 1, left) in perirhinal cortex compared to inferior temporal lobe. Together, Marino’s findings suggest that the perirhinal cortex codes information of the location of the search target, separated from other objects using , linearsomething that can be represented as a straight line on a graph, and directly proportional changes in two related quantities separability.

There are still many exciting questions to answer. What is the inferior temporal lobe doing to combine memory and visual information? How is the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. cell activity different in the inferior temporal lobe compared to perirhinal cortex? Do those differences contribute to the linearsomething that can be represented as a straight line on a graph, and directly proportional changes in two related quantities separability we see in the perirhinal cortex? Marino found that the answer was a bit more complicated. He found that the inferior temporal cortex may use non-linearsomething that can be represented as a straight line on a graph, and directly proportional changes in two related quantities separability, where flexible curves can separate visual and remembered information instead of rigid lines . The inferior temporal cortex then sends this information separable by flexible curves to the perirhinal cortex, which then may transform the information to again be separated by a line.

Conclusion
Like most problems in science, one experiment cannot fully and conclusively reveal everything there is to know about how we “search” with our eyes. However, the work of Marino Pagan and his mentor Nicole Rust takes important steps closer to this understanding, and adds valuable new information about where and how search targets are identified in the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals.. Not only does their work shine light on previously mysterious ways the brainThe brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. supports everyday actions like visual search but it also provides a foundation to engineer computers that can scan and find objects as quickly and flexibly as we do.

Figure 2

About the brief writer: Jeni Stiso

Jeni is a PhD Candidate in Dani Bassett’s lab. Jeni is interested in cognitive and computational neuroscience. She is interested in how changes in the electrical activity of the brain help people learn things.

Citations:

1. https://medium.com/deep-dimension/an-analysis-on-computer-vision-problems-6c68d56030c3

2. Chelazzi, L., & Desimone, R. (1993). A neural basis for visual search in IT. Nature. 363, pages 345–347.

3. Meunier, M., Bachevalier, J., Mishkin, M. & Murray, E.A. Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys. J. Neurosci. 13, 5418–5432 (1993).

To learn more about how the brain helps us quickly identify what we’re looking for, check out the full paper here.