Main Page

From DiET

DiET: Dialogue Experimentation Toolkit

The toolkit is a text-based experimental resource for carrying out experimental investigation of dialogue. The basic idea of the chat tool is that it allows a server to intercept participants' typed turns and selectively interfere with their content. This allows very fine-grained experimental control over what participants perceive the other participants to have typed.

The chat tool has a constantly expanding library of experiments that can be reconfigured. It also provides an extensive API that allows the programmer to design experimental interventions that are sensitive to the conversational context of the participants. Watch this video to see a demonstration of how the chat tool works.

Flowchart showing an experiment that introduces artificial clarification requests into the dialogue: The server generates a clarification request "home?" that appears to participantA to originate from participantB. Note that participantB is not aware of participantA's dialogue.

Possible kinds of experimental intervention include

• Introducing artificial feedback into the dialogue
  • artificial clarification requests, such as "What?" "so you mean?"
  • acknowledgments, such as "ok" "ok right"
• Blocking or transforming certain kinds of feedback
  • substituting "ok" with "hmm"
  • substituting "ok" with "ok, tell me more"
  • blocking "what?"
• Introducing artificial hesitations and disfluencies
• Masking the identity of the other participant
• Substition of synonyms / hypernyms / hyponyms
• Blocking or promoting alignment

:See library of experiments for more tasks and publications

Recorded data

The chat tool automatically records all the turns, recording the producers and recipients of each word, frequencies of use, recency, and other data, such as typing speed, number of edits (deletes, inserts), turn formulation time and overlap. This data can be used to generate experimental interventions that are sensitive to the conversational context. The chat tool also includes the Stanford Parser and a java implementation of WordNet.

Integration with SPSS / Excel / OpenOffice

All the data from the experiments is saved to a CSV file

Naturalistic interventions

The chat tool has functionality that allows "spoofing" of interlocutors' typing speed, mistypings that are sensitive to the keyboard layout, unique "txt" conventions, turn-taking phenomena.

Interfaces

The chat tool includes a variety of interfaces, including interfaces similar to MSN/ICQ/AIM, other interfaces similar to UNIX talk that display each character as they are typed, and custom interfaces that allow experimental control over turn-taking. We are currently prototyping a new interface that will allow teasing apart inter-individual from intra-individual language processing.

Immediate question no 1

"but don't the participants notice that something very strange is going on and realize that the server is manipulating the text?"

Surprisingly, no! Participants tolerate very high levels of disturbance..even quite unnatural looking clarification requests are treated as bona fide attempts to sustain mutual-intellgibility. We have carried out over 2500 interventions, and so far only one participant suspected that a turn might have been generated artificially. Of course, the interventions need to be piloted and designed judiciously.

Immediate question no 2

"but this isn't the same as real spoken dialogue"

Yes and no. Text-based dialogue exhibits disfluencies, hesitations, interruptions, pauses, fillers, repair, acknowledgments etc (obviously not intonation), all the co-ordination mechanisms that are required for any comprehensive account of dialogue. Text-based dialogue in commercial messenger applications is less sequentially coherent. However, this can be modified experimentally by the use of different chat client interfaces that either promote or interfere with coherence.

Program versions and download

The chat tool is programmed entirely in Java, and runs on all (Linux, Windows, Mac) platforms. It is released under a GPL license, and the latest version is available to download from sourceforge.