Research team: Eleni-Ira Panourgia, Finbar Wheelaghan and Xue Yang
This research project centers around the development of a prototype that explores the simultaneous manipulation of three-dimensional digital forms and sound. The proposed multi-media artwork examines the aesthetic affordances of tight parameter couplings between digital three-dimensional objects and sound objects based on notions of process and user-machine interaction. It investigates how effective cohesion between visual, spatial and sonic might be established through changes perceived in parallel; what Michel Chion refers to as 'synchresis'. Inspired by Mike Blow's On the Simultaneous Perception of Sound and Three-Dimensional Objects and processual art, this prototype uses computer technology for forming and mediating a creative practice involving 3D animation, sound synthesis, digital signal processing and programming. Our practice-based approach entails the rendering of a three-dimensional digital object in Processing whose form changes over time according to specific actions. Spatial data is sent via Open Sound Protocol (OSC) to Max MSP in real time, where sound is synthesized and then manipulated. Sonic parameters such as amplitude, spectral density/width and timbre are controlled by select spatial parameters from the three-dimensional object. Sound processing is realized based on the changing of the three-dimensional object in time through basic actions such as splitting, distorting, cutting, shattering and rotating. We aim to use digital technology to look beyond basic synchronisation of sound and vision to a more complex cohesion of percepts, based on changes to sonic and visual parameters experienced concurrently.
More details on this project can be found in our article Digital Interactions: Sound and Three-dimensional Forms.