D5.4 Report on evaluation of methods
- Main information
-
This document reports on the first evaluation of tools developed in the LIVE project for manual, semiautomatic and automatic annotation and extraction of knowledge in work package 5. We start this report with findings on the international TRECVID 2007 evaluation of LIVE tools for automatic shot boundary classification. The compressed domain shot boundary detector developed in the LIVE project showed the third best recognition performance of all 15 participating research groups in this competition. Despite the excellent results, the generalization of the performance from news and documentary data used in TRECVID 2007 to more difficult sports data produced by the LIVE streams of Olympia 2008 remains difficult. Only further evaluations on labelled data stemming from Olympia 2004 and the upcoming Olympia 2008 event will show how suitable the developed technology is for extracting information automatically from sports broadcasts – a domain, for which neither standard international benchmarks nor any international competition exist. The detection of gradual transitions in sports video must still be considered unsolved and need further research. However, the evaluation results of TRECVID show the potential of the developed technology and their maturity. The next section of this document deals with the performance of different face recognition methods which are developed in the LIVE project to identify athletes and other important persons in the video stream automatically. We measure the performance in rather controlled optimal situations, benchmarked on the Bochum gallery, but also on a “worse-case” gallery with rather mixed content. The result is promising but uncontrolled environment and incorrect feature correspondence lead to poor results – especially if more advanced P2D-HMMs face recognition technology is applied. Hence, component face detectors have been developed in the project in order to improve the correspondence search in pose estimation before any identification can be performed. We report in this document on the performance of several face component detectors for eyes, nose and mouth locations developed in the course of the project to improve face pose estimation and recognition. Despite the fact that the performance of individual face component detectors is quite high when evaluated on a test set stemming from the same database, generalization of the facial recognition algorithms to other more uncontrolled galleries remains a challenge. However, as the integration of the face component detectors in the face recognition framework is still lacking, no sound evaluation can be performed. We will report in an upcoming report D 5.7 on the results of our research and how the different algorithms perform on Olympia 2008 sports data during the field trial. No referenced Knowhow defined WP5: Detection, Extraction and Annotation of Knowledge. IAIS Christian Eckes, Mohammad Reza Khalilbeigi Khameneh, Sebastian Schabe, Wolfgang Hesseler 2008-06-16 22:21 Request for more detail
- Access and Use Rights
-
For LIVE project internal research purposes only. For LIVE project internal development purposes only.
- Protection Status
-
Copyright LIVE Consortium. Add to LIVE's Intellectual Property Watch List Add to LIVE's Intellectual Property Watch List Add to LIVE's Intellectual Property Watch List Closed, attachment is not public
This item is not available for public download. For further information click on Request for more details above