2017: NakayamA

When we see objects in the world, what we actually perceive is much more than the two-dimensional images projected onto the eye. Our perception is three dimensional and reflects stable properties of objects and the scene, even though our vantage point and the retinal image constantly change as we move our eyes and travel through the environment. One major challenge for the visual system is to reconstruct a stable perception of the three-dimensional layout of surfaces based on the ever-changing retinal images they create.

Ken Nakayama has focused on addressing this important question. Using psychophysical tests in human subjects, he has made groundbreaking discoveries about the representation of surfaces. His research career also has spanned a wide range of other topics, including attention, visually guided motor behavior, motion processing, and face recognition.

For his seminal contributions to the field of visual neuroscience, Nakayama has been named the recipient of the 2017 Golden Brain Award from the Berkeley, California-based Minerva Foundation. The award, now in its 33rd year, recognizes outstanding contributions in vision and brain research.

“I believe that I was selected for this award because I have identified processes that are going on in our visual system that were poorly understood yet important. I note that my work is grounded in perception and that it does not rely on the usual physical measurements from the brain itself,” said Nakayama, professor emeritus of psychology at Harvard University. “I take it as a strong endorsement that psychological research itself is an essential part of brain science."

As an undergraduate at Haverford College, Nakayama was initially attracted to psychoanalysis, which claimed to understand the deep recesses of the mind. Not satisfied by this approach, he was lucky to meet famous and inspiring neuroscientists through one of his undergraduate courses. This led him to pursue research in physiological psychology as a graduate student. Nakayama performed neurophysiological experiments in which he recorded from single visual neurons as a PhD candidate at the University of California, Los Angeles, and later as a postdoctoral fellow at the University of California, Berkeley.

One of Nakayama’s mentors was his postdoctoral adviser Horace Barlow, a fellow of Trinity College, Cambridge, and a legendary vision scientist himself, who has shaped the way we think about how seeing depends on the underlying machinery of vision. The approaches he pioneered provide the foundation for much of contemporary visual neuroscience. “Just his example as a great scientist was all the mentoring I needed,” Nakayama said.

After this neurophysiological training, Nakayama shifted his focus to human psychophysics at the Smith-Kettlewell Eye Research Institute, where he remained for almost 20 years. “I was inspired by the elegant work of others in this area, where relatively simple experiments could yield surprising benefits.”

In 1990, Nakayama joined the Psychology Department at Harvard University and served as department chair from 2012 to 2016. In 2001, he founded the Vision Sciences Society, which hosts an annual meeting that brings together, in one forum, scientists from the broad range of disciplines that contribute to vision science, including visual psychophysics, neuroscience, computational vision and cognitive psychology. From 2001 to 2005, he also served as president of the Vision Sciences Society, which recently initiated the Ken Nakayama Medal for Excellence in Vision Science. 

Most recently, he has initiated research in two new and distinct areas. First, he has written many papers with colleagues on visually-guided action and action in relation to the actions of others. Second, he has initiated a citizen science project, for which almost two million individuals have freely volunteered their time as subjects.

“Professor Nakayama is arguably the most influential vision scientist alive today, in terms of his own discoveries, the students he has trained, and the institutions he has built,” said Nancy Kanwisher, the Walter A. Rosenblith Professor of Cognitive Neuroscience in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology and a former Golden Brain Award recipient. “He is a people person who loves interacting with other scientists and mentoring young scientists. He is beloved in the field.”

According to Kanwisher, Nakayama’s discoveries came not from methodical hypothesis testing per se, but more from exploration, intuition, and a deep feel for how vision works. “Nakayama has made fundamental discoveries about the mental processes that enable us to see,” she said. “He has made a lifetime of contributions to our understanding of mid-level vision, including work on visual search, visual attention, surface perception, stereo vision and its role in image segmentation, and face perception.”

Beyond these research areas, he has also worked in more distant fields, co-editing a book titled “The Science of Social Vision,” and co-inventing systems and methods for machine learning enhanced by human measurements. Nakayama has also served on the editorial boards of several key journals. In addition, he has been elected to the American Academy of Arts and Sciences and the Society of Experimental Psychologists. In 2017, he was honored with the 2017 Edgar D. Tillyer Award from the Optical Society of America.

In collaboration with Shinsuke Shimojo, Zijiang He, and Barbara Gillam, Nakayama used perceptual demonstrations to reveal the existence of an intermediate level of visual processing interposed between the retinal image and higher functions. “I believe my greatest scientific achievement was to develop and argue for a whole new level of visual processing, which we called surface processing,” he said.

This concept was not entirely new, but it was very far from being worked out. In a seminal paper published in Science in 1992, Nakayama and Shimojo advocated a conceptual shift for understanding surface perception. In particular, they highlighted the important role of visual experience during locomotion, that is, learning to map fluctuating retinal images to the stable, physical layout of surfaces. “Our contribution was to study this empirically and systematically and put it into a larger theoretical framework,” Nakayama said. “Our conclusion was that surface processing is pervasive and an obligate intermediate stage of visual processing.”

According to Doris Tsao, a professor of biology at the California Institute of Technology and a former Golden Brain Award recipient, a huge missing gap in vision research is the question of how objects first arise. Scientists understand quite a bit about early feature processing and high-level processes of object identification, but the stuff that happens in the middle remains a mystery, she explained.

“Ken, perhaps more than anyone else, has shone a light on these processes, revealing that very early on the brain constructs a surface representation, a representation in which bits of visual information belonging to a contiguous 3D surface are grouped together and selected as a unit by attention,” Tsao said. “Carrying on Ken's work, to figure out how surface representation is implemented by neural circuits, is one of the most exciting challenges in systems neuroscience.”