This is a pretty classic computer vision problem that combines detection, tracking, filtering, recognition, and logical parsing together into one system whose objective is to make sense of the comings and goings of people or objects within a scene. It is one of the few of the modules that requires some nominal level of system integration to get running properly. Matlab has Simulink code that does this for the case of abandoned object detection, which is documented online, so you can see one expected outcome of a surveillance system.

/* I also found this might be useful too: http://studentdavestutorials.weebly.com/particle-filter-with-matlab-code.html This website covers areas such as Bayes rule, Kalman filter and particle filter with short videos and Matlab implementation. The tracker parts(Kalman filter and particle filter) may be included to learning modules. */

1. Background estimation and subtraction.
2. Target tracking
3. Target modeling
4. Target recognition

The sequence below introduces one aspect of surveillance systems at a time. They direct you to Matlab code that sometimes implements multiple steps at a time. It is recommended that you implement each one individually to get a sense for what role it plays in the entire system, rather than just copy/paste the whole system.

Week #1: Setup, Data, and Basics

Explore the datasets available and select a couple of videos to use as the starting point for processing, testing, and debugging the surveillance system to be created as part of this learning module.

Meanwhile, check out this review of three simple background modeling methods. You are not advised to use the code provided when implementing the same in future activities. There are more efficient ways to do the same without resorting to as many for loops, plus the implementation needs to be packaged up for use by the overall system. Plus, the activities below utilize existing Matlab libraries to the extent possible or desirable.

On a related note, the sequence below introduces one aspect of surveillance systems at a time. They direct you to Matlab code that sometimes implements multiple steps at a time. It is recommended that you implement each one individually to get a sense for what role it plays in the entire system, rather than just copy/paste the whole system.

Week #2: Background Modelling
Matlab has implementation of the mixtures of Gaussians adaptive background estimation algorithm.

Week #3: Foreground Object Extraction
Extracting the foreground objects is really performing blob extraction from the binary foreground image.

Week #4: Optimization-Based Data Association
Performing detection does provide a means to identify objects of interest versus the prevailing background image. However if we are interested in maintaining the identity of the objects, additional processing and logic is required. The simplest scheme simply considers the spatio-temporal history of the targets and tries to link the current detected objects to the previously detected objects. This form of data association is known as the assignment problem.

Two algorithms: Hungarian or Munkres' algorithm and Jonker-Volgenant Algorithm.

The net result should be similar to this Matlab demo on multiple object tracking, which also implements far more than what is described. You should strip the extras.

Week #5: Adding Temporal Dynamics via a Kalman Filter

We can do a better job handling things like occlusions, as well as improve the data association, by adding in a temporal filter. A simple, powerful method is to recursively estimate and correct the moving target dynamics through a Kalman filter. Matlab has a page on how to do so for object tracking along with a more general intro page to additional documentation.

• If you are interested in the details of the Kalman filter and their connection to code, this Matlab file exchange code tutorial may be of assistance.

Week #1: Differentiating People

Gaussian mixture model for targets using Expectation Maximization algorithm.

Week #2: Appearance-Based Data Association

Comparing models.

Week #3: Re-Identification

Using comparison to know when same person has re-entered.

TBD.

Sample videos from past teams:

• Youtube Channel for Team DT from ECE4580, Fall 2014.

Presentations by researchers in computer vision

• A good example of how to present your results. See the visuals that the Discrete-Continuous Optimization algorithm author uses. Plus this one.
• Another example.
• .. and another example.

Online talks

• Oral defense of Guang Shu @ UCF.

Advertisement Videos of companies that provide surveillance algorithms as a service:

• LightHaus Logic

• Other peoples tips on parameter selection for MOG foreground detector.
• Implement on a live feed? Does it run fast enough, or is that not necessary?

ECE4580 Learning Modules