Excerpt from the premiere at Cresc. Biennale for New Music  "human_machine"

with Ensemble Modern, Enno Poppe-conductor

​

This is a multi-media work that combines original music for ensemble, live- electronics and AI-triggered visuals. With a core of acoustic sound augmented by live-electronics, we process resulting audio frequencies through a neural network to generate visual textures, morphing in time to the music. The idea stems from the theme “human_machine." Combining acoustic writing and live performance with emerging AI, we create a space to explore the relationship between art and technology. How does the artist respond to the digital realities? How does technology help us transcend physical limitations of our instrument(s) and what kind of art would that look like, the human and machine, not as polarities but as synergetic, co-evolving?

This unique visual tool that we implement for the project was designed by AI- specialist Nikolay Jetchev. It is a novel audio visualisation technique utilising artificial intelligence in the form of adversarial texture generation. The video uses a Periodic Spatial Generative Adversarial Network (PSGAN) model to generate textures smoothly morphing in time, carefully tuned to the music. Selecting input images with a suitable theme for training the PSGAN allows emphasis on the artists’ vision and represents a novel form of digital synesthesia. The artist can play flexibly with our tool and condition the noise on a moving signal in time, which would then lead to smooth transitions and animations between textures in time. With an appropriate audio descriptor we can map the distribution of audio samples to the distribution of textures. This tool opens a new pathway for collaboration between musicians and generative model visualisations, enabling the artists themselves to interact with the algorithm as it responds to aural stimuli.