Characterizing animal behavior through audio and video signal processing

TitleCharacterizing animal behavior through audio and video signal processing
Publication TypeJournal Article
Year of Publication2007
AuthorsValente, D., Wang H., Andrews P., Mitra P. P., Saar S., Tchernichovski O., Golani I., and Benjamini Y.
JournalIEEE MultiMedia
ISBN Number1070-986X
EndNote Rec Number11818
Keywordsand automation., Animal behavior, animal behavior characteristics, audio processing, audio signal processing, behavioral neuroscientists community, behavioural sciences, brain, brain function, Cities and towns, computer vision, data analysis, Educational institutions, genetic algorithms, genetic properties, image processing, Laboratories, multimedia system, multimedia systems, neural nets, neural network properties, neurophysiology, neuroscience, Organisms, phenotypic description, Pixel, quantitative neuroethology, signal analysis, Springs, Time measurement, Video compression, video signal processing

This article presents two instances in which multimedia systems and processing have elucidated animal behavior and have been central in developing quantitative descriptions. These examples demonstrate multimedia systems' utility and necessity in developing a complete phenotypic description. We hope that this article will spur interest in this subject in the multimedia community, so more advanced processing techniques will enter the field of quantitative neuroethology. You might have noticed that in our two examples, there was nothing very multimodal about the media techniques used. Both of these systems are transparently unimodal. This speaks to the limited crossover between the multimedia community and the behavioral neuroscientists (or neuroethologists). These examples did show, however, that the neuroscientific community can benefit greatly from incorporating multimedia techniques into their experiments and data analysis. As the walls between these disciplines begin to fall, experimental setups that are truly multimedia will likely appear. Such systems will allow complete phenotypic descriptions of animals in ethologically relevant settings, along with methods for analyzing, manipulating, annotating, and storing the resulting data. Combining these phenotypic descriptions with the corresponding genetic and neural network properties will facilitate the connection of these organization levels and lead to a more thorough understanding of brain functioning.