Fermüller Receives NSF Grant to Advance AI Research on the Learning of Complex Symbolic Signals

Cornelia Fermüller, an associate research scientist in the University of Maryland Institute for Advanced Computer Studies, has been awarded a $500K grant to explore how humans and animals learn complex events like music and language acquisition, visual action recognition, and visual navigation—and then create artificial intelligence (AI)-based systems with similar capabilities.

The three-year grant from the National Science Foundation will establish the Research Coordination Network (RCN), bringing together experts in neuroscience, cognitive science, applied mathematics, computer science, and engineering.

Fermüller says the roots of the RCN lie in neuromorphic engineering—an emerging interdisciplinary field that takes its inspiration from biology, physics, mathematics, computer science and engineering. Members of the RCN will design hardware and physical models of neural and sensory systems, and also implement learning algorithms using neuromorphic sensors and computing hardware.

Fermüller is principal investigator on the project. Shihab Shamma, a UMD professor of electrical and computer engineering with an appointment in the Institute for Systems Research, and Ralph Etienne-Cummings, a professor of electrical and computer engineering at Johns Hopkins University, are co-principal investigators.

The researchers say that understanding how humans learn complex events is necessary for further advancements in AI and machine learning.

“AI—nowadays equated with deep learning—has made tremendous progress in recent years,” Fermüller says. “We can now understand simple events. For example, in computer vision we know how we can learn to recognize objects and simple actions. This project will help us take the next step.”

A clearer understanding of how humans process music and language acquisition is part of that next step, Fermüller says. She gives the example of determining how different aspects of music are recorded and encoded in the human brain, such as recognizing, rhythm, pitch and timbre.

“Acquisition and performance of events in these domains can be ideal vehicles to study learning because they exhibit a wide range of skill levels, have highly developed and well-understood structures, and are culturally rich and ubiquitous, making them accessible for extensive experimentation,” Fermüller says. “However, the neurobiology of such learning and skill has been very challenging to study, as it requires an immense range of skills and expertise. The RCN affords us a rare opportunity to engage several experts to collaborate in a multidisciplinary effort.”

The researchers also plan to develop new software and software toolboxes used in robotics and other systems.

This includes software for analyzing from vision what humans are doing so that robots can collaborate with humans, electroencephalogram (EEG) decoding toolboxes to read electrical activity in the brain, and creating software for agile drones that navigate autonomously.

Fermüller notes that network members will participate in a yearly three-week workshop that will develop and test new tools and ideas, stimulate new collaborations, and educate students on unique interdisciplinary skills.

“This RCN seeks bio-inspired solutions, and there is no better way than bringing people from different disciplines together to exchange ideas and facilitate collaboration,” she says.

—Story by Melissa Brachfeld