User test, Human Annotation Tool (HAT), 7th and 8th August 2007 at the ORF studios in Vienna

The user test of the Human Annotation Tool (HAT) took place in Vienna on the 7th and 8th August 2007. The first day saw the expert evaluation and functional test of the HAT. On the second, the usability tests were performed with professional users from ORF sports, ORF Archive and persons not familiar with the broadcast environment.

The annotation was tested in four different ways:

• Testing of the Human Annotation Tool (HAT) with touch screen
• Testing of the HAT with mouse
• Testing of the annotation using speech recognition
• Testing of the annotation in a conventional way (MAZ notes)

The intention was to see on the one hand how efficient annotation can be performed (quantitative), and on the other hand to test the quality of the annotation created (qualitative). Each test was filmed, with users thinking aloud. The users were also interviewed. The results of the user test of the Human Annotation Tool (HAT) will be available at the end of august.

The next user test will be the test of the Recommender System planned for the end of August at the ORF studios in Vienna.