An accessible audiovisual memory game that can be played by anyone, including players with hearing or visual impairments
ShapeTones is an audio-visual memory game for iOS (iPhone/iPad). A sequence of 3 shapes and tones (we call them ShapeTones) is played, and the player tries to reproduce it with three taps. Tapping different areas of the screen triggers different ShapeTones.
The game starts with 3 ShapeTones. As the game evolves, more ShapeTones become available. When a new ShapeTone is added, a trial screen is shown to demonstrate where each ShapeTone is triggered. Some surprises happen along the way… As a one player game, the sequence is created automatically. As a two player game, one player creates the sequence, passes the device to the other player, who tries to repeat it. They then swap the roles.
The game can be downloaded from here.
ShapeTones is jointly developed in collaboration with Nuno Correia from Goldsmiths, University of London and Nick Bryan-Kinns, Tony Stockman and Fiore Martin from Queen Mary University of London.
(2016) Metatla O., Correia N. N., Martin F., Bryan-Kinns N., Stockman T., Tap the ShapeTones: Exploring the effects of crossmodal congruence in an audio-visual interface. To appear in the Proceedings of ACM SIGCHI 2016, San Jose, CA, USA.
There is growing interest in the application of crossmodal perception to interface design. However, most research has focused on task performance measures and often ignored user experience and engagement. We used ShapeTones to examine crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and audio-visual stimuli. Participants in a first study showed improved performance when using a visual congruent mapping that was cancelled by the addition of audio to the baseline conditions, and a subjective preference for the audio-visual stimulus that was not reflected in the objective data. Based on these findings, we designed the ShapeTones game app, an audio-visual memory game to examine the effects of crossmodal congruence on user experience and engagement. Results showed higher engagement levels with congruent displays with some reported preference for potential challenge and enjoyment that an incongruent display may support, particularly for increased task complexity.