Video data is ubiquitous and information rich. The massive content of videos is however not available to automated search. Despite the tremendous progress of machine learning in static image analysis, extracting information from videos is still a challenge. The same actions encoded in videos can be visually very different and can be occluded by other actions and objects in a video. The goal of the project is building new computational methods to extract and quantify meaningful knowledge from video recordings.
We will address current challenges in video analysis by studying recordings of animal behavior – of a moving worm C. elegans and of honey bees in a hive. Focusing on these examples, recorded in similar conditions, will allow us to avoid challenges of visual variability of recordings, and distill the problem to accurately detecting repetitive and meaningful behaviors. We will build methods that robustly detect behaviors we know and recognize, such as bees cleaning the hive or worm coiling. Additionally, our methods will also point to behaviors that were previously not recognized. In collaboration with researchers at the University Hospital Cologne and University Colorado Boulder, USA we will use our methods to study questions C. elegans and honey bee biology and behavior. Both organisms are widely used in biological research however their behavior is not yet fully understood.
