Computer Vision #1 – Optical Flow

Spanish version
Turns out that computer vision not only works with static images but also with video processing. I am plenty conscious that this post performs a step forward in terms of difficulties, but it is said that the best way to learn concepts is attempting to explain them and, since I want to clarify some details related to optical flow, here we are! Are you ready? Go!

One of the up-to-date challenges that computer vision has to face is “activities or movements recognition”. Turns out that the development of this kind of methods is interesting (for instance) to make smart devices capable of interacting with humans according to their mood or, a more realistic example, to be able to detect traffic accidents, acts of vandalism and so on, with a security camera automatically.

Today I would like to describe a basic concept which is used as an elemental key in more complex algorithms: Optical Flow. Some studies assure that the key in activity recognition  is being able to analyse the optical flow of a set of points of interest or, in other words, its velocity field. A traditional example which exemplify this statement is the one which describes a dark room with a person within it. If that person had light spots evenly distributed through his arms, only by observing this points movement  we would be almost completely capable of knowing what this person is doing.

For those who are not interested in the maths, the next gif shows the expected result (once again, you can download the Matlab code used to generate the results presented here: GitHub). As it can be seen, we are able to associate to each pixel an arrow which points out the direction of the trajectory that it is being described by the pixel.

There are several approaches to calculate the optical flow. One of the most popular alternatives is Lucas Kanade method, which will be described in a second.

First of all, we need a mathematical representation for videos. Videos will be represented like images but only with an extra dimension, the time. . Given two consecutive instances and , we want to calculate the optical flow, more concretely a velocity field, which represents the difference between the image at time and time .

For each pixel , we look for the changing velocity . which move the target pixel from its location at time to its new position at time . Thus, the next expression is true.

It we track a specific pixel through the time, we can write:

For small displacement:

So that, Taylor series can be applied:

Obviously, here we have a “little” setback, we have two unknown variables and only one equation. Lucas Kanade method assumes that the neighbourhood of the pixel evaluated (defined by the pixels ) has the same velocity components. By doing this assumption we could write:

The system of equations can be expressed as where

             

Obviously, now the system of equations is overdetermined since we have more equations than unknown variables. Lucas Kanade proposes to solve the system with less square by the next relation:

Finally, you should know that this method works specially well in regions with geometry similar to corners’ and… that is!! It has not been too traumatic right? I hope you enjoy the content, and feel free to comment and enrich my ignorance :).

Take care!.