 |
|
|
|
Detection, Extraction and Annotation of Knowledge
|
This work area deals with the development of methods and tools to detect and extract information and metadata from audio-visual material. It has to be investigated how these methods support the annotation capabilities of the live staging process.
Media streams should be enriched with additional information and metadata to produce media objects which can be used for personalisation and offer semantic access. Expected results include; formal models and methods to exploit semantic knowledge on content by human annotation; tools and interfaces for the automatic detection and extraction of semantic knowledge in media archives and live produced audio-visual material for live staging, and a deeper understanding and tools for enhancing media objects with external knowledge, coming from different sources.
|
|
|
|
|
|
| |
Research Focus
|
|
The research issues that will be addressed by this work area include:
- It has to be understood, how the currently unused potential of the content producers can be exploited and integrated into the staging process, by the semi-automatic online annotation of live video material. Which requirements on a formal model for live human annotation are given?How can methods and modules of automatic metadata generation be integrated in the annotation workflow?
- It has to be investigated, which kind of interactive tools are needed for the semi-automatic annotation process. Which kind of link and recognition results must be available for the human annotator? How does one enable the annotator to select and add annotation results?
- How can huge existing media archives be made available for the live staging of media events? What kind of methods and tools are appropriate for accessing these archives, and which kind of content information is valuable for the staging process? How is semantic information extracted appropriately to the 'live' character of the event? How can multilingual content be handled?
- Which kinds of methods are adaptable for the detection and extraction of information in live produced material? How is the live created metadata successfully exploited for the live production?
- Methods and modules to analyse the video data, that will be provided. Especially for the annotation of camera data video processing is important. Depending on the exact requirements of the annotation functionality several methods are considered. A video segmentation and key frame selection module which is able to process simple cuts but also dissolves and wishes will be provided. Existing text annotations in the video are important information. Other video processing modules are fast and robust face recognition, scene recognition (close-ups, field, spectators, human activity analysis) and scene clustering, detection of logos, advertisings and flags in the video streams.
- For speech analysis methods and tools will be developed. The audio streams will be attached with relevant metadata. Here robust methods, like speech and speaker segmentation will be developed. These methods must be implemented in real-time to support live tagging. Also a speaker recognition module will be developed and trained for a defined set of speakers. To achieve high professional user acceptance an appropriate lexicon size must be chosen. The grammars and the language models must be trained for the defined sports domain. To process a huge archive the speech recognition system must support sub word units, like syllables, to avoid the problem of out-of-vocabulary words. It is likely that the acoustic models for the ORF data must be adapted and optimised.
- How can external information (text, intranet, internet …) be assigned to media objects? Which kind of information is useful? Which kind of a formal model is appropriate (rules, heuristics, meta-descriptions) for this task?
|
|
|
|
|
|
Expected Results
|
|
|
- Methods and tools to achieve an automatic metadata generation using speech and video analysis approaches.
- Database applications together with an interface for retrieval and fast access to the media archive.
- Report on formal models for the semi-automatic annotation process.
- Interactive tools for the semi-automatic annotation process and the linking with the database.
- Tools and interfaces for the connection of external information resources to media objects.
- Report on the implementation and the tests of the tools and interfaces
|
This site conforms to the following standards:
|
|
