Overview of Evaluation and Usability
With the technical advances and market growth in the field, the issues of evaluation and usability of spoken dialogue systems, unimodal as well as multimodal, are as crucial as ever. This chapter discusses those issues by reviewing a series of European an
- PDF / 1,659,380 Bytes
- 26 Pages / 594.72 x 841.68 pts Page_size
- 53 Downloads / 230 Views
Wolfgang Minker University of Vim, Department of Information Technology, Ulm/Donau,
Germany
[email protected]
Abstract
With the technical advances and market growth in the field, the issues of evaluation and usability of spoken dialogue systems, unimodal as well as multimodal, are as crucial as ever. This chapter discusses those issues by reviewing a series of European and US initiatives which have produced major results on evaluation and usability. Whereas significant progress has been made on unimodal spoken dialogue systems evaluation and usability, the emergence of, among others, multimodal, mobile, and non-task-oriented systems continues to pose entirely new challenges to research in evaluation and usability.
Keywords:
Multimodal systems; Spoken dialogue systems; Evaluation; Usability.
1.
Introduction
Spoken dialogue systems (SDSs) are proliferating in the market for a large variety of applications and in an increasing number of languages. As a major step forward, commercial SDSs have matured from technology-driven prototypes to business solutions. This means that systems can be copied, ported, localised, maintained, and modified to fit a range of customer and end-user needs This chapter is a modified version of the article entitled " Evaluation and Usability of Multimodal Spoken Language Dialogue Systems" published in Speech Communication, Vol. 43/1-2, pp. 33-54, Copyright (2004), reprinted with permission from Elsevier. 221 W. Minker, D. Bühler andL. Dybkjær (eds), Spoken Multimodal Human-Computer Dialogue in Mobile Environments, 221 -246 © 2005 Springer. Printed in the Netherlands
222
SPOKEN MULTIMODAL HUMAN-COMPUTER DIALOGUE
without fundamental innovation. This is what contributes to creating an emerging industry. At the same time, increasingly advanced SDSs are entering the market, drawing on experience from even more sophisticated research systems and continuous improvements in SDS technologies. Furthermore, in many research laboratories, focus is now on combining speech with other modalities, such as pen-based hand-writing and 2D gesture input, and graphics output, such as images, maps, lip movements, animated agents, or text (Wahlster et al., 2001; Bickmore and Cassell, 2002; Oviatt, 1997; Gustafson et al., 2000; Sturm et al., 2004; Oviatt et al., 2004; Whittaker and Walker, 2004). An additional dimension which influences development is the widening context of use. Mobile devices, in particular, such as mobile phones, in-car devices, PDAs and other small handheld computers open up a range of new application opportunities for unimodal as well as multimodal SDSs as witnessed by several chapters in this book. In this continually expanding field of unimodal and multimodal, mobile and non-mobile SDSs, many research issues still remain to be solved. Two issues of critical importance are evaluation and usability. Systems evaluation is crucial to ensure, e.g., system correctness, appropriateness, and adequacy, while usability is crucial to user acceptance. Many results are avail
Data Loading...
