The Why behind scenes – WhyBehindScenes

The project WhyBehindScenes aims at developing new methods for automatically understanding the storyline in videos, and in particular the why behind the scenes in edited videos (films and TV shows). This will be investigated in two directions: first, by automatically understanding the storyline by focusing on the timeline, parts that are crucial for a plot, and correlation of scenes; second, by identifying the filmmaker’s decisions and integrating them in the video analysis by extracting the film’s signature (the style of the direction, the emotion a scene is conveying) and by creating novel sequences with specific directorial style. WhyBehindScenes targets to build systems capable of converting videos to books, by including not only the audio-visual information present in scenes but also the motivation, intention or even emotion behind events.