Direkt zum Inhalt springen
Image Understanding and Knowledge-Based Systems
TUM School of Computation, Information and Technology
Technical University of Munich

Technical University of Munich

Menu

Links

Informatik IX

Image Understanding and Knowledge-Based Systems

Boltzmannstrasse 3
85748 Garching

info@iuks.in.tum.de




EYEWATCHME - 3D Hand and object tracking for inside out activity analysis (bibtex)
EYEWATCHME - 3D Hand and object tracking for inside out activity analysis (bibtex)
by L Sun, U Klank and M Beetz
Abstract:
This paper investigates the inside-out recognition of everyday manipulation tasks using a gaze-directed camera, which is a camera that actively directs at the visual attention focus of the person wearing the camera. We present EYEWATCHME, an integrated vision and state estimation system that at the same time tracks the positions and the poses of the acting hands, the pose that the manipulated object, and the pose of the observing camera. Taken together, EYEWATCHME provides comprehensive data for learning predictive models of vision-guided manipulation that include the objects people are attending, the interaction of attention and reaching/grasping, and the segmentation of reaching and grasping using visual attention as evidence. Key technical contributions of this paper include an ego view hand tracking system that estimates 27 DOF hand poses. The hand tracking system is capable of detecting hands and estimating their poses despite substantial self-occlusion caused by the hand and occlusions caused by the manipulated object. EYEWATCHME can also cope with blurred images that are caused by rapid eye movements. The second key contribution is the of the integrated activity recognition system that simultaneously tracks the attention of the person, the hand poses, and the poses of the manipulated objects in terms of a global scene coordinates. We demonstrate the operation of EYEWATCHME in the context of kitchen tasks including filling a cup with water.
Reference:
EYEWATCHME - 3D Hand and object tracking for inside out activity analysis (L Sun, U Klank and M Beetz), In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009., 2009. 
Bibtex Entry:
@inproceedings{sun_eyewatchme_2009,
 author = {L Sun and U Klank and M Beetz},
 title = {{EYEWATCHME} - {3D} Hand and object tracking for inside out activity
	analysis},
 booktitle = {{IEEE} Computer Society Conference on Computer Vision and Pattern
	Recognition, 2009. {CVPR} 2009.},
 year = {2009},
 pages = {9--16},
 month = {jun},
 abstract = {This paper investigates the inside-out recognition of everyday manipulation
	tasks using a gaze-directed camera, which is a camera that actively
	directs at the visual attention focus of the person wearing the camera.
	We present {EYEWATCHME}, an integrated vision and state estimation
	system that at the same time tracks the positions and the poses of
	the acting hands, the pose that the manipulated object, and the pose
	of the observing camera. Taken together, {EYEWATCHME} provides comprehensive
	data for learning predictive models of vision-guided manipulation
	that include the objects people are attending, the interaction of
	attention and reaching/grasping, and the segmentation of reaching
	and grasping using visual attention as evidence. Key technical contributions
	of this paper include an ego view hand tracking system that estimates
	27 {DOF} hand poses. The hand tracking system is capable of detecting
	hands and estimating their poses despite substantial self-occlusion
	caused by the hand and occlusions caused by the manipulated object.
	{EYEWATCHME} can also cope with blurred images that are caused by
	rapid eye movements. The second key contribution is the of the integrated
	activity recognition system that simultaneously tracks the attention
	of the person, the hand poses, and the poses of the manipulated objects
	in terms of a global scene coordinates. We demonstrate the operation
	of {EYEWATCHME} in the context of kitchen tasks including filling
	a cup with water.},
 doi = {10.1109/CVPR.2009.5204358},
 keywords = {{3D} object tracking, blurred images, computer graphics, {EYEWATCHME},
	gaze-directed camera, grasping segmentation, human computer interaction,
	image restoration, image segmentation, image sensors, inside out
	activity analysis, integrated activity recognition system, object
	recognition, reaching segmentation, state estimation system, substantial
	self-occlusion, {tracking3D} hand tracking, vision-guided manipulation},
}
Powered by bibtexbrowser
Go Back

Rechte Seite

Informatik IX

Image Understanding and Knowledge-Based Systems

Boltzmannstrasse 3
85748 Garching

info@iuks.in.tum.de