PhD Candidate, RMIT University
This pinboard looks at spoken conversational search - including IR, HCI and SDS
My are of research is on how we can present search engine results over a speech-only communication channel without overwhelming the user with information. I am also interested in how conversations can be structured between the user and the Spoken Conversation Search System. My focus is on the experimental design with multi method analysis as evaluation.
There is a lot of development around audio-only search systems such as Siri, Cortana or Google Now. These systems are great to answer factoid (very direct) questions. However, when the user has a more open question the system either reverts back to presenting search results on the screen or cannot handle these questions. We are investigating what the interactions are that people expect from these kind of systems and how we could integrate them in the search sessions. Theses systems will be beneficial for every day users but especially for users with a visual impairment.
I will spend the money for my travel to the SIGIR conference (www.sigir.org/sigir2017/) in Japan where I will be presenting several pieces of work. I am also volunteering at the conference with a program to welcome new PhD students to our information retrieval community. This program "PhD buddies" is set up by me and some other volunteers.
Abstract: Recently emerged intelligent assistants on smartphones and home electronics (e.g., Siri and Alexa) can be seen as novel hybrids of domain-specific task-oriented spoken dialogue systems and open-domain non-task-oriented ones. To realize such hybrid dialogue systems, this paper investigates determining whether or not a user is going to have a chat with the system. To address the lack of benchmark datasets for this task, we construct a new dataset consisting of 15; 160 utterances collected from the real log data of a commercial intelligent assistant (and will release the dataset to facilitate future research activity). In addition, we investigate using tweets and Web search queries for handling open-domain user utterances, which characterize the task of chat detection. Experiments demonstrated that, while simple supervised methods are effective, the use of the tweets and search queries further improves the F1-score from 86.21 to 87.53.
Pub.: 01 May '17, Pinned: 28 Jul '17
Abstract: Given the proliferation of 'intelligent' and 'socially-aware' digital assistants embodying everyday mobile technology - and the undeniable logic that utilising voice-activated controls and interfaces in cars reduces the visual and manual distraction of interacting with in-vehicle devices - it appears inevitable that next generation vehicles will be embodied by digital assistants and utilise spoken language as a method of interaction. From a design perspective, defining the language and interaction style that a digital driving assistant should adopt is contingent on the role that they play within the social fabric and context in which they are situated. We therefore conducted a qualitative, Wizard-of-Oz study to explore how drivers might interact linguistically with a natural language digital driving assistant. Twenty-five participants drove for 10 min in a medium-fidelity driving simulator while interacting with a state-of-the-art, high-functioning, conversational digital driving assistant. All exchanges were transcribed and analysed using recognised linguistic techniques, such as discourse and conversation analysis, normally reserved for interpersonal investigation. Language usage patterns demonstrate that interactions with the digital assistant were fundamentally social in nature, with participants affording the assistant equal social status and high-level cognitive processing capability. For example, participants were polite, actively controlled turn-taking during the conversation, and used back-channelling, fillers and hesitation, as they might in human communication. Furthermore, participants expected the digital assistant to understand and process complex requests mitigated with hedging words and expressions, and peppered with vague language and deictic references requiring shared contextual information and mutual understanding. Findings are presented in six themes which emerged during the analysis - formulating responses; turn-taking; back-channelling, fillers and hesitation; vague language; mitigating requests and politeness and praise. The results can be used to inform the design of future in-vehicle natural language systems, in particular to help manage the tension between designing for an engaging dialogue (important for technology acceptance) and designing for an effective dialogue (important to minimise distraction in a driving context).
Pub.: 16 May '17, Pinned: 28 Jul '17
Abstract: We address the challenge of extracting query biased audio summaries from podcasts to support users in making relevance decisions in spoken document search via an audio-only communication channel. We performed a crowdsourced experiment that demonstrates that transcripts of spoken documents created using Automated Speech Recognition (ASR), even with significant errors, are effective sources of document summaries or “snippets” for supporting users in making relevance judgments against a query. In particular, the results show that summaries generated from ASR transcripts are comparable, in utility and user-judged preference, to spoken summaries generated from error-free manual transcripts of the same collection. We also observed that content-based audio summaries are at least as preferred as synthesized summaries obtained from manually curated metadata, such as title and description. We describe a methodology for constructing a new test collection, which we have made publicly available.
Pub.: 28 Jun '17, Pinned: 28 Jul '17
Abstract: A critical determinant of message interactivity is the presence of contingency, that is, the messages we receive are contingent upon the messages we send, leading to a threaded loop of interdependent messages. While this "conversational ideal" is easily achieved in face-to-face and computer-mediated communications (CMC), imbuing contingency in human-computer interaction (HCI) is a challenge. We propose two interface features—interaction history and synchronous chat—for increasing perceptions of contingency, and therefore user engagement. We test it with a five-condition, between-participants experiment (N = 110) on a movie search site. Data suggest that interaction history can indeed heighten perceptions of contingency and dialogue, but is perceived as less interactive than chatting. However, the chat function does not appreciably increase perceived contingency or user engagement, both of which are shown to mediate the effects of message interactivity on attitudes toward the site. Theoretical implications for interactivity research and practical implications for interaction design are discussed.
Pub.: 13 Jun '16, Pinned: 28 Jul '17
Abstract: This article presents the Cogni-CISMeF project, which aims at improving the health information search engine CISMeF, by including a conversational agent that interacts with the user in natural language. To study the cognitive processes involved during information search, a bottom-up methodology was adopted. An experiment has been set up to obtain human dialogs related to such searches. The analysis of these dialogs underlines the establishment of a common ground and accommodation effects to the user. A model of artificial agent is proposed, that guides the user by proposing examples, assistance and choices.
Pub.: 30 Nov '09, Pinned: 28 Jul '17
Abstract: Online communities are valuable information sources where knowledge is accumulated by interactions between people. Search services provided by online community sites such as forums are often, however, quite poor. To address this, we investigate retrieval techniques that exploit the hierarchical thread structures in community sites. Since these structures are sometimes not explicit or accurately annotated, we introduce structure discovery techniques that use a variety of features to model relations between posts. We then make use of thread structures in retrieval experiments with two online forums and one email archive. Our results show that using thread structures that have been accurately annotated can lead to significant improvements in retrieval performance compared to strong baselines.
Pub.: 23 Apr '11, Pinned: 28 Jul '17