University of Houston

Multiple pedestrian tracking in unconstrained environments is an important task that has received considerable attention from the computer vision community in the past two decades. A number of approaches that address this issue have been proposed. Accurate multiple pedestrian tracking can greatly improve the performance of activity recognition and analysis of high level events through a surveillance system. However, the complexity of human motion poses several challenges to the accuracy and precision of a tracking system. Traditional approaches to pedestrian tracking build a motion prediction model to track the target. With improvements in object detection methods, recent approaches replace the motion prediction stage track targets be selecting among the outputs of a detector.

To incorporate the merit of traditional and recent approaches, we have developed a novel approach for multi-target tracking using an ensemble framework that optimally chooses target tracking results from that of independent trackers and a detector at each time step. The compound model is designed to select the best candidate scored by a function integrating detection confidence, appearance affinity, and smoothness constraints imposed using geometry and motion information. Parameters of our association score function are discriminatively trained with a max-margin framework. Optimal selection is achieved through a hierarchical data association step that progressively associates candidates to targets.

To further improve the tracking performance we focus on the design of a novel motion prediction model. Most of the existing methods assume that each pedestrian walks independently and the tracker predicts the location based on an underlying motion model, such as a constant velocity or an autoregressive model. However, human interaction behavior is known to play an important role in human motion. We present a novel tracking approach utilizing human collision avoidance behavior, which is motivated by the human vision system. The model predicts human motion based on modeling of perceived information. An attention map is designed to mimic human reasoning that integrates both spatial and temporal information. This perception based motion model is integrated into a data association based tracking framework with appearance and motion features. An enhanced tracker is also developed that models human group behavior using a hierarchical group structures. The groups a re identified by a bottom-up social group discovery method. The inter- and intra-group structures are modeled as a two-layer graph and tracking is posed as optimization of the integrated structure. The target appearance is modeled using HOG features, and the tracking solution is obtained via dynamic programming. Finally, we propose another novel tracking method to unify multiple human behavior. To investigate the effects of potential multiple social behaviors, we present an algorithm that decomposes the combined social behaviors into multiple basic interaction modes, such as attraction, repulsion, and no interaction. We integrate these multiple social interaction modes into an interactive Markov Chain Monte Carlo tracker and demonstrate how the developed method translates into a more informed motion prediction, resulting in robust tracking performance.

We test our methods on videos from unconstrained outdoor environments and evaluate it against common multi-object trackers.