Musicreatures iPhone App

Musicreatures is a novel mobile app that utilizes the multimodal interaction capabilities of smart phones in playful, real-time musicalization.

Musicreatures is an iPhone application, targeted for all users regardless of their prior musical background. With the application, the user can create music in a simple game-like environment mainly using motion gestures. The gestures are mapped to a visual view and the visual data is then musicalized. A number of musical restrictions are enforced in order to assist the user in achieving musically satisfying results and to avoid cacophonic outcomes.

The app offers two distinct modes: a standard mode for visually mediated musicalization of gestures and an improvisatory mode for more direct musicalization of motion gestures. In the standard mode, the user receives constant visual feedback. The states of the objects controlled by the user are communicated visually, which allows temporal flexibility in the musicalization of the user actions. For example, each user-controllable visual object—a set of circles—is musicalized as a looping rhythmic pattern, and the number of notes in a pattern corresponds to the number of circles represented visually. The user adjusts the number of notes contained in a pattern by steering the visual object and collecting distinct objects depicting notes and rests. A change in a rhythmic pattern triggers a re-distribution of the notes, and while the change is not audible before the next onset in the new pattern is played, it is immediately visible in the visual representation.

As a means to make the app more personally adjustable, the user initializes the app with another distinct type of input: a photo taken with the phone’s built-in camera. The musical ground rules, namely tempo and scale, are governed by this image, and the photo is also displayed as the background image within the app, thus providing the board on which the objects are visualized and from which they draw their musical characteristics. Download Musicreatures on the App Store or give the following examples a listen to find out what the app sounds like. The source code of Musicreatures is available at https://github.com/DiscoveryGroup/musicreatures

More about the news here.