Automatic Camera-based Analysis of Soccer Matches

The aim of this project is to develop a system for a robust analysis of soccer matches that is able to extract relevant events (such as passes, shots on goal, offsides, etc.) and to quantify and assess tactical behavior of players and teams. In contrast to the elaborate and costly setups that are used in the commercial top leagues and tournaments, our approach focus on the analysis of monocular recordings (such as TV broadcasts or non-professional off-the-shelf camera recordings). Hence, our system is intended to provide amongst others high-level analysis for scouts, coaches and fans of non-commercial soccer clubs and to investigate old TV records of historical matches. One of the main challenges in this context is to cope with difficult conditions such as moving and zooming cameras, bad image quality, adverse weather conditions, and inhomogenous lightning.

Cut and Transition Detection

Our approach for video shot detection is based on multi temporal distance images (MTDIs), formed by chi-square based similarity measures that are calculated pairwise within a floating window of video frames. By using MTDI-based boundary detectors, various cuts and transitions in various shapes (dissolves, overlayed effects, fades, and others) can be determined. Long view shots are of particular interest in the context of soccer videos, because they show scenes with the most useful information. Hence, our algorithm is designed to detect such shots with high accuracy.

Pixel-wise Grass Classification

In order to support player and field tracking, we have developed a new approach for detecting players and field lines in monocular TV video data that involves determining the field area and grass colors. This is carried out by considering contextual knowledge, together with a new method for color segmentation that selects polyhedrons in a frame-wise manner within the RGB cube. The resulting algorithms are able to detect non-long-view images and process data in real-time.

Camera Calibration and Tracking

To generate player and ball trajectories for further high-level analysis, image coordinates have to be transformed to pitch coordinates. Our approach is based on homographies that map the image plane to the field plane. To cope with the calibration of moving and zooming cameras (e.g. in TV records) changes of the camera view are tracked. For this purpose we rely on our approach for robust field line detection. It is supplemented by a tracking of salient points in the grass, which is important in cases where no field lines are visible.

Player Tracking

The tracking of players in monocular soccer videos is a challenging task because of numerous difficulties that can occur especially in TV broadcasts, such as camera motions, severe occlusion of players, or inhomogeneous lightning conditions. Our new robust method for multi-player tracking is based on finding local maxima on a confidence map. This map represents an ensemble of visual evidences, such as colors of the team outfits, responses of a HOG human detector, and grass regions in images. This combination of features allows for a robust online tracking procedure that does not require any further information about the camera calibration or other user input.

Check out some demonstration videos:

Demo Video ISSIA-CNR Cam3

Demo Video VS-PETS

Pose Estimation

For a detailed event detection an accurate pose estimation of the players can be very helpful (e.g. to detect a shot). Our novel approach to articulated human pose estimation in monocular images is based on a pixel-wise classification of body parts and an inverse kinematic technique for skeleton fitting. Instead of employing an exhaustive template matching, we efficiently detect body parts using a random forest, which assigns body part classes to each pixel in the image. Additionally, we use another random forest with similar structure to estimate the orientation of a person in an image. This information supports the body part classification as well as the initialization of the skeleton fitter. Our pose estimation method tries to iteratively match the joints of a 3D skeleton model with the estimated 2D positions of the body part detection and results in both a 3D pose and a back-projected 2D pose.

Ball Detection and Tracking

Usually in soccer videos, the ball is the smallest and fastest object on the playing field. Hence, the detection and tracking of the ball is a challenging task, especially if things are complicated by additional difficulties such as a moving camera. For a robust detection and tracking of the ball the optical evidence is not sufficient and other hints such as the player trajectories and detected events have to be incorporated.

Event Detection

The detection of events like pass, offside, throw-in, and kickoff is enabled by the detection of ball possessions. Detected events are used to improve tracking tasks.

SoccerTEST

The web-based project is available at the SoccerTEST project page: SoccerTEST


Export as PDF, XML, TEX or BIB
Journal Articles
2015
[]Real-Time Segmentation Methods for Monocular Soccer Videos (M Hoernig, M Herrmann and B Radig), In Pattern Recogntion and Image Analysis, To appear, 2015.  [bibtex]
2014
[] Online Multi-player Tracking in Monocular Soccer Videos (M Herrmann, M Hoernig and B Radig), In AASRI Procedia, volume 8, 2014. (2014 AASRI Conference on Sports Engineering and Computer Science (SECS 2014)) [bibtex] [pdf] [pdf]
2009
[]ASpoGAMo: Automated Sports Game Analysis Models (M Beetz, Nvon Hoyningen-Huene, B Kirchlechner, S Gedikli, F Siles and Mand LM Durus), In International Journal of Computer Science in Sport, volume 8, 2009.  [bibtex]
2005
[]Computerized Real-Time Analysis of Football Games (M Beetz, B Kirchlechner and M Lames), In IEEE Pervasive Computing, volume 4, 2005.  [bibtex]
Conference and Workshop Papers
2014
[]Multi Temporal Distance Images for Shot Detection in Soccer Games (M Hoernig, M Herrmann and B Radig), In EUSIPCO 2014 (22nd European Signal Processing Conference 2014) (EUSIPCO 2014), 2014.  [bibtex] [pdf]
[]Human Body Part Classification in Monocular Soccer Images (A Bigontina, M Herrmann, M Hoernig and B Radig), In 9-th Open German-Russian Workshop on Pattern Recognition and Image Understanding, 2014.  [bibtex] [pdf]
2013
[]Real Time Soccer Field Analysis from Monocular TV Video Data (M Hoernig, M Herrmann and B Radig), In 11th International Conference on Pattern Recognition and Image Analysis (PRIA-11-2013), The Russian Academy of Sciences, volume 2, 2013.  [bibtex] [pdf]
2009
[]Rao-Blackwellized Resampling Particle Filter for Real-Time Player Tracking in Sports (Nvon Hoyningen-Huene and M Beetz), In Fourth International Conference on Computer Vision Theory and Applications (VISAPP) (A Ranchordas, H Araujo, eds.), INSTICC press, volume 1, 2009.  [bibtex]
2007
[]An Adaptive Vision System for Tracking Soccer Players from Variable Camera Settings (S Gedikli, J Bandouch, Nvon Hoyningen-Huene, B Kirchlechner and M Beetz), In Proceedings of the 5th International Conference on Computer Vision Systems (ICVS), 2007.  [bibtex]
[]Visually Tracking Football Games Based on TV Broadcasts (M Beetz, S Gedikli, J Bandouch, B Kirchlechner, Nvon Hoyningen-Huene and A Perzylo), In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI), 2007.  [bibtex]
2006
[]Camera-based Observation of Football Games for Analyzing Multi-agent Activities (M Beetz, J Bandouch, S Gedikli, Nvon Hoyningen-Huene, B Kirchlechner and A Maldonado), In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2006.  [bibtex]
2004
[]Motion and Episode Models for (Simulated) Football Games: Acquisition, Representation, and Use (M Beetz, S Flossmann and T Stammeier), In 3rd International Joint Conference on Autonomous Agents & Multi Agent Systems (AAMAS), 2004.  [bibtex]
[]Watching Football with the Eyes of Experts: Integrated Intelligent Systems for the Automatic Analysis of (Simulated) Football Games (M. Beetz, F. Fischer, S. Flossmann, B. and UA. Kirchlechner and C. Holzer), In 5th Annual Conference dvs-Section Computer Science in Sport, 2004.  [bibtex]
PhD Thesis
2014
[] Automated Semantic Annotation of Football Games from TV Broadcast (F Siles Canales), PhD thesis, Technische Universität München, 2014.  [bibtex] [pdf]
[] Ball Tracking and Action Recognition of Soccer Players in TV Broadcast Videos (M Durus), PhD thesis, Technische Universität München, 2014.  [bibtex] [pdf]
2011
[]Real-time Tracking of Player Identities in Team Sports (N von Hoyningen-Huene), PhD thesis, Technische Universität München, 2011.  [bibtex]
2009
[]Continual and Robust Estimation of Camera Parameters in Broadcasted Sports Games (S Gedikli), PhD thesis, Technische Universität München, 2009.  [bibtex]

Powered by bibtexbrowser
Export as PDF, XML, TEX or BIB