Research paper by Steven Ness, Paul Reimer, Justin Love, W. Andrew Schloss, George Tzanetakis

Indexed on: 21 Oct '11Published on: 21 Oct '11Published in: Journal on Multimodal User Interfaces


The study of periodic biological processes, such as when plants flower and birds arrive in the spring is known as Phenology. In recent years this field has gained interest from the scientific community because of the applicability of this data to the study of climate change and other ecological processes. In this paper we propose the use of tangible interfaces for interactive sonification with a specific example of a multimodal tangible interface consisting of a physical paper map and tracking of fiducial markers combined with a novel drawing interface. The designed interface enables one or more users to specify point queries with the map interface and to specify time queries with the drawing interface. This allows the user to explore both time and space while receiving immediate sonic feedback of their actions. This system can be used to study and explore the effects of climate change, both as tool to be used by scientists, and as a way to educate and involve members of the general public in a dynamic way in this research.