An Adaptive Vision System for Tracking Soccer Players from Variable Camera Settings

Gedikli, Suat; Bandouch, Jan; Hoyningen-Huene, Nico von; Kirchlechner, Bernhard; Beetz, Michael; Gedikli, Suat; Bandouch, Jan; Hoyningen-Huene, Nico von; Kirchlechner, Bernhard; Beetz, Michael

by S Gedikli, J Bandouch, Nvon Hoyningen-Huene, B Kirchlechner and M Beetz

Abstract:

In this paper we present ASpoGAMo, a vision system capable of estimating motion trajectories of soccer players taped on video. The system performs well in a multitude of application scenarios because of its adaptivity to various camera setups, such as single or multiple camera settings, static or dynamic ones. Furthermore, ASpoGAMo can directly process image streams taken from TV broadcast, and extract all valuable information despite scene interruptions and cuts between different cameras. The system achieves a high level of robustness through the use of modelbased vision algorithms for camera estimation and player recognition and a probabilistic multi-player tracking framework capable of dealing with occlusion situations typical in team-sports. The continuous interplay between these submodules is adding to both the reliability and the efficiency of the overall system.

Reference:

An Adaptive Vision System for Tracking Soccer Players from Variable Camera Settings (S Gedikli, J Bandouch, Nvon Hoyningen-Huene, B Kirchlechner and M Beetz), In Proceedings of the 5th International Conference on Computer Vision Systems (ICVS), 2007.

Bibtex Entry:

@inproceedings{gedikli_adaptive_2007,
 author = {S Gedikli and J Bandouch and Nvon Hoyningen-Huene and B Kirchlechner and M Beetz},
 title = {An Adaptive Vision System for Tracking Soccer Players from Variable
	Camera Settings},
 booktitle = {Proceedings of the 5th International Conference on Computer Vision
	Systems ({ICVS)}},
 year = {2007},
 abstract = {In this paper we present {ASpoGAMo}, a vision system capable of estimating
	motion trajectories of soccer players taped on video. The system
	performs well in a multitude of application scenarios because of
	its adaptivity to various camera setups, such as single or multiple
	camera settings, static or dynamic ones. Furthermore, {ASpoGAMo}
	can directly process image streams taken from {TV} broadcast, and
	extract all valuable information despite scene interruptions and
	cuts between different cameras. The system achieves a high level
	of robustness through the use of modelbased vision algorithms for
	camera estimation and player recognition and a probabilistic multi-player
	tracking framework capable of dealing with occlusion situations typical
	in team-sports. The continuous interplay between these submodules
	is adding to both the reliability and the efficiency of the overall
	system.},
 keywords = {soccer},
}

by S Gedikli, J Bandouch, Nvon Hoyningen-Huene, B Kirchlechner and M Beetz

Abstract:

Reference:

Bibtex Entry:

@inproceedings{gedikli_adaptive_2007,
 author = {S Gedikli and J Bandouch and Nvon Hoyningen-Huene and B Kirchlechner and M Beetz},
 title = {An Adaptive Vision System for Tracking Soccer Players from Variable
	Camera Settings},
 booktitle = {Proceedings of the 5th International Conference on Computer Vision
	Systems ({ICVS)}},
 year = {2007},
 abstract = {In this paper we present {ASpoGAMo}, a vision system capable of estimating
	motion trajectories of soccer players taped on video. The system
	performs well in a multitude of application scenarios because of
	its adaptivity to various camera setups, such as single or multiple
	camera settings, static or dynamic ones. Furthermore, {ASpoGAMo}
	can directly process image streams taken from {TV} broadcast, and
	extract all valuable information despite scene interruptions and
	cuts between different cameras. The system achieves a high level
	of robustness through the use of modelbased vision algorithms for
	camera estimation and player recognition and a probabilistic multi-player
	tracking framework capable of dealing with occlusion situations typical
	in team-sports. The continuous interplay between these submodules
	is adding to both the reliability and the efficiency of the overall
	system.},
 keywords = {soccer},
}

Automatic Camera-based Analysis of Soccer Matches

The aim of this project is to develop a system for a robust analysis of soccer matches that is able to extract relevant events (such as passes, shots on goal, offsides, etc.) and to quantify and assess tactical behavior of players and teams. In contrast to the elaborate and costly setups that are used in the commercial top leagues and tournaments, our approach focus on the analysis of monocular recordings (such as TV broadcasts or non-professional off-the-shelf camera recordings). Hence, our system is intended to provide amongst others high-level analysis for scouts, coaches and fans of non-commercial soccer clubs and to investigate old TV records of historical matches. One of the main challenges in this context is to cope with difficult conditions such as moving and zooming cameras, bad image quality, adverse weather conditions, and inhomogenous lightning.

Cut and Transition Detection

Our approach for video shot detection is based on multi temporal distance images (MTDIs), formed by chi-square based similarity measures that are calculated pairwise within a floating window of video frames. By using MTDI-based boundary detectors, various cuts and transitions in various shapes (dissolves, overlayed effects, fades, and others) can be determined. Long view shots are of particular interest in the context of soccer videos, because they show scenes with the most useful information. Hence, our algorithm is designed to detect such shots with high accuracy.

Pixel-wise Grass Classification

In order to support player and field tracking, we have developed a new approach for detecting players and field lines in monocular TV video data that involves determining the field area and grass colors. This is carried out by considering contextual knowledge, together with a new method for color segmentation that selects polyhedrons in a frame-wise manner within the RGB cube. The resulting algorithms are able to detect non-long-view images and process data in real-time.

Camera Calibration and Tracking

To generate player and ball trajectories for further high-level analysis, image coordinates have to be transformed to pitch coordinates. Our approach is based on homographies that map the image plane to the field plane. To cope with the calibration of moving and zooming cameras (e.g. in TV records) changes of the camera view are tracked. For this purpose we rely on our approach for robust field line detection. It is supplemented by a tracking of salient points in the grass, which is important in cases where no field lines are visible.

Player Tracking

The tracking of players in monocular soccer videos is a challenging task because of numerous difficulties that can occur especially in TV broadcasts, such as camera motions, severe occlusion of players, or inhomogeneous lightning conditions. Our new robust method for multi-player tracking is based on finding local maxima on a confidence map. This map represents an ensemble of visual evidences, such as colors of the team outfits, responses of a HOG human detector, and grass regions in images. This combination of features allows for a robust online tracking procedure that does not require any further information about the camera calibration or other user input.

Check out some demonstration videos:

Demo Video ISSIA-CNR Cam3

Demo Video VS-PETS

Pose Estimation

For a detailed event detection an accurate pose estimation of the players can be very helpful (e.g. to detect a shot). Our novel approach to articulated human pose estimation in monocular images is based on a pixel-wise classification of body parts and an inverse kinematic technique for skeleton fitting. Instead of employing an exhaustive template matching, we efficiently detect body parts using a random forest, which assigns body part classes to each pixel in the image. Additionally, we use another random forest with similar structure to estimate the orientation of a person in an image. This information supports the body part classification as well as the initialization of the skeleton fitter. Our pose estimation method tries to iteratively match the joints of a 3D skeleton model with the estimated 2D positions of the body part detection and results in both a 3D pose and a back-projected 2D pose.

Ball Detection and Tracking

Usually in soccer videos, the ball is the smallest and fastest object on the playing field. Hence, the detection and tracking of the ball is a challenging task, especially if things are complicated by additional difficulties such as a moving camera. For a robust detection and tracking of the ball the optical evidence is not sufficient and other hints such as the player trajectories and detected events have to be incorporated.