Multimodal communication - who owns the dialogue?

A 45 minute Hands-On by:

Dr. Charlie Cullen

Dublin Institute of Technology

About this Hands-On

The session will begin by considering some of the core assumptions of many user modelling methods - building a user persona to model system relationships. In our approach, we focus instead on the narrative of the scenario to tell the story of the communication that the system and user will engage in.

This narrative then forms the basis of an initial blank user model, using the premise ‘you are what you do’ rather than making any assumptions about the potential user(s). At this point, use cases are considered as a sequence of potential interactions as part of a wider concept of multimodal communication. The ownership of each interaction is the key to delivering effective multimodal content; who makes the decision (user or system) defines who is currently controlling the dialogue.

A data model derived from the use cases can then help to refine the blank user model, finding out more about the user based on their behaviours rather than their attributes. Therefore, a more effective understanding of the decision making process involved in multimodal content delivery can be made, avoiding some of the pitfalls of building systems based on assumptions of the potential communication the user may decide to have with them.

To illustrate this, I'll use a hands-on group session with a short scenario to model the potential interactions found in a set of simple use cases. These interaction diagrams quickly define both process and ownership within a communication, and are designed to allow decision making to be placed at the centre of every use case. We'll then discuss the interaction diagrams as a group to consider where information relating to the system (data model) and user (persona) can be derived. This is a rapid process that can be performed with pencil and paper to develop a far better understanding of the dialogue between user and system, also indicating where coding logic must model the system’s role in the proposed communication.

About the Speaker

Dr Charlie Cullen is Assistant Head of the School of Media and Head of the Multimodal Interaction Group at the Dublin Institute of Technology - a research team of 5 principal investigators and 12 doctoral researchers. He has obtained significant research funding at both national and European levels. He has patented and licensed research IP to several companies in the digital media industry and leads an industrially funded research programme in the area of spatial audio.

Dr Cullen has published widely in areas such as automated character animation, multimodal interfaces, location based services, data sonification and emotional speech analysis. He is currently pursuing a second PhD in multimodal blended learning. He has built research facilities such as the Multimodal Interaction Laboratory and lectures on various undergraduate and postgraduate modules within the school. In his spare time, he works in music and audio production.