Workshop Programme and Papers:

• Session 1: 9.00am -- 10.30am: 3 full papers
  M. Ablassmeier, G. Rigoll: A context-adaptive search engine concept and multimodal input strategies for automotive environments
  T. Hurtig, K. Jokinen: Modality fusion in a route navigation system
  M. Kon, T. Koshizen, H. Prendinger: A new user-machine interactive scheme using cross-modal computation for deep-interest
• Session 2: 11.00am -- 12.30pm: invited presentation; position paper; panel discussion
  C. Sidner (with C. Rich): Do dialogue systems really make a difference? Evaluating a dialogue system for use with consumer products
  D. Stevenson, M. Hutchins, et al: Multiple approaches to evaluating multi-modal collaborative systems
• Session 3: 2.00pm -- 3.30pm: 2 full papers; discussion
  S. Bayer, C. Doran, S. Condon, A. Gertner: Dialogue annotation as a correction task
  R. Porzel, A. Scheffler, R. Malaka: How entrainment increases dialogical effectiveness
• Session 4: 4.00pm -- 5.30pm: 2 extended position papers; 1 position paper; panel discussion
  T. Andrews, M. Broughton, D. Estival: Implementing an intelligent multimedia presentation planner using an agent-based architecture
  A. Marriott, A. Holic, D. Reid: The multi-modal Mentor system
  N. Richard, S. Yamada: Can an emotional agent motivate a user in achieving self-assigned tasks

Context

It is often claimed that such interfaces support more naturalistic and efficient styles of interaction. However, such claims are often accepted at face value; most metrics for evaluation focus on error and comparison to human performance of similar interactive tasks (e.g. see metrics for evaluating dialogue systems). Much work has recently been carried out in linguistics, psychology, and sociology identifying and investigating phenomena in human-human interaction that enables and enhance successful communication and collaborative task performance. However, it is not typically questioned whether implementing such phenomena carries the same value for human-machine interaction.

This workshop will address the issue of evaluating multimodal dialogue systems, and in particular the characteristics and interaction styles that are particularly effective for human-machine collaborative task performance. These may include features that are known to be effective and important in human-human interaction. Conversely, it may be the case that certain effective interaction design decisions (e.g. for overcoming speech-recognition error) are less .natural..

The workshop encourages participation by dialogue system and HCI researchers, interaction designers, as well as linguists, psychologists, and sociologists interested in human-human interaction, as well as evaluation of effective human-machine interaction.

Objectives

• What makes a multimodal dialogue interface effective, and how is this meaningfully measured for different task domains?
• What is the state of recent research in human-human interaction; how can such techniques impact the design of effective human-machine interfaces?
• What interaction strategies and techniques can be used to make human-machine multimodal dialogue more robust?

Topics

• Strategies for efficient interaction, engagement, and responsiveness, and their role in effectiveness;
• Methodologies and evaluation metrics for effective interaction;
• Results from human-human interaction regarding interaction for effective collaboration, and their impact on system design;
• Evaluation of effective human-system collaboration via interaction;
• Strategies for evaluating effective task performance in multimodal dialogue systems;
• Architectures and design for human-centered interactive systems;
• Strategies for effective information generation and presentation.

Workshop Format

Submission Instructions

Mail submissions to lcavedon@csse.unimelb.edu.au. Submissions should include:

1. A separate plain text cover page with title, authors and affiliation, abstract, a list of keywords, and an indication as to whether paper is a regular or position paper;
2. A pdf file of the paper (regular or position).

Contingent on quality of submissions, workshop proceedings will be published in either hardcopy or electronic format.

Timetable:

• November 27 2005: Due date for submission (extension)
• December 14 2005: Notification of acceptance
• December 23 2005: Final versions of accepted papers due
• January 29 2006: Workshop date

Organizing Committee

• Lawrence Cavedon, National ICT Australia and RMIT University, Australia
• Fang Chen, National ICT Australia
• Robert Dale, Macquarie University, Australia
• David Traum, Institute for Creative Technology (ICT) and University of Southern California, USA

Program Committee

• Elisabeth Andre, U. Ausberg, Germany
• Niels Ole Bernsen, U. Southern Denmark
• Herve Bourlard, IDIAP / EPFL, Switzerland
• Phil Cohen, OHSU, USA
• Nathalie Colineau, CSIRO, Australia
• Dominique Estival, DSTO, Australia
• Rod Farmer, U. Melbourne, Australia
• Christine Halverson, IBM Research, USA
• Lewis Johnson, ISI, USA
• Michael Johnston, AT&T Research, USA
• Kristiina Jokinen, U. Helsinki, Finland
• Barb Kelly, U. Melbourne, Australia
• Oliver Lemon, U. Edinburgh, UK
• Sharon Oviatt, OHSU, USA
• Cecile Paris, CSIRO, Australia
• Matthew Purver, Stanford U., USA
• Candy Sidner, MERL, USA

Inquiries