The aim of this project is to develop a system for a robust analysis of soccer matches that is able to extract relevant events (such as passes, shots on goal, offsides, etc.) and to quantify and assess tactical behavior of players and teams. In contrast to the elaborate and costly setups that are used in the commercial top leagues and tournaments, our approach focus on the analysis of monocular recordings (such as TV broadcasts or non-professional off-the-shelf camera recordings). Hence, our system is intended to provide amongst others high-level analysis for scouts, coaches and fans of non-commercial soccer clubs and to investigate old TV records of historical matches. One of the main challenges in this context is to cope with difficult conditions such as moving and zooming cameras, bad image quality, adverse weather conditions, and inhomogenous lightning.

Our approach for video shot detection is based on multi temporal distance images (MTDIs), formed by chi-square based similarity measures that are calculated pairwise within a floating window of video frames. By using MTDI-based boundary detectors, various cuts and transitions in various shapes (dissolves, overlayed effects, fades, and others) can be determined. Long view shots are of particular interest in the context of soccer videos, because they show scenes with the most useful information. Hence, our algorithm is designed to detect such shots with high accuracy.

In order to support player and field tracking, we have developed a new approach for detecting players and field lines in monocular TV video data that involves determining the field area and grass colors. This is carried out by considering contextual knowledge, together with a new method for color segmentation that selects polyhedrons in a frame-wise manner within the RGB cube. The resulting algorithms are able to detect non-long-view images and process data in real-time.

To generate player and ball trajectories for further high-level analysis, image coordinates have to be transformed to pitch coordinates. Our approach is based on homographies that map the image plane to the field plane. To cope with the calibration of moving and zooming cameras (e.g. in TV records) changes of the camera view are tracked. For this purpose we rely on our approach for robust field line detection. It is supplemented by a tracking of salient points in the grass, which is important in cases where no field lines are visible.

The tracking of players in monocular soccer videos is a challenging task because of numerous difficulties that can occur especially in TV broadcasts, such as camera motions, severe occlusion of players, or inhomogeneous lightning conditions. Our new robust method for multi-player tracking is based on finding local maxima on a confidence map. This map represents an ensemble of visual evidences, such as colors of the team outfits, responses of a HOG human detector, and grass regions in images. This combination of features allows for a robust online tracking procedure that does not require any further information about the camera calibration or other user input.

Check out some demonstration videos:

Demo Video ISSIA-CNR Cam3

Demo Video VS-PETS

For a detailed event detection an accurate pose estimation of the players can be very helpful (e.g. to detect a shot). Our novel approach to articulated human pose estimation in monocular images is based on a pixel-wise classification of body parts and an inverse kinematic technique for skeleton fitting. Instead of employing an exhaustive template matching, we efficiently detect body parts using a random forest, which assigns body part classes to each pixel in the image. Additionally, we use another random forest with similar structure to estimate the orientation of a person in an image. This information supports the body part classification as well as the initialization of the skeleton fitter. Our pose estimation method tries to iteratively match the joints of a 3D skeleton model with the estimated 2D positions of the body part detection and results in both a 3D pose and a back-projected 2D pose.

Usually in soccer videos, the ball is the smallest and fastest object on the playing field. Hence, the detection and tracking of the ball is a challenging task, especially if things are complicated by additional difficulties such as a moving camera. For a robust detection and tracking of the ball the optical evidence is not sufficient and other hints such as the player trajectories and detected events have to be incorporated.

The detection of events like pass, offside, throw-in, and kickoff is enabled by the detection of ball possessions. Detected events are used to improve tracking tasks.

The web-based project is available at the SoccerTEST project page: SoccerTEST