accessible AI?
Can we demystify AI? What are the current tools for sonic machine learning and how accessible and creative are they? I created a music education programme for Brighter Sound, open to young women and people of marginalised genders to explore practical tools for machine learning. The programme was in the form of an experimentation lab, testing and critiquing tools, learning about key concepts in AI in order to create their own human-machine societal predictions, manifesting as their audiovisual ‘dream transmission’ artworks. We worked hybridly, undertaking creative training over four days at Seesaw, Manchester and interacting and experimenting online in our DWM Discord server. The group immersed themselves in human-machine collaboration and emotional technological states to explore the current state of AI music tools and creative potentials.
beginnings
We began by looking at broad technical and historical developments in the AI field, looking at exactly what machine learning is, what is latent space, a neural network?what are the limits and challenges of the field for artists and the current trends and types of model we can use creatively as beginners. We discussed popular myths and fears around AI and machine learning in popular culture (bias / surveillance / automation) and how creative AI can provide a positive space for alternative narratives and realities to manifest. Check out our ML library and our online Discord community above.
datasets & training
After theory and model introductions (pattern generation, timbre transfer and neural synthesis) it was time to get practical. We focused on how to construct a sonic dataset for training a model, with each participant thinking of a rationale for their material. How might they test the machine, what can we predict? We had some fascinating aural concepts to explore including dead technology in the future, elemental sonics and ritualistic waves. We used a range of google colab notebooks to generate sounds including Word2wave, Jukebox, PRiSMSampleRNN and plugins including MAWF and Magenta.
aura machine _ emf
To get closer to our machines and hidden energy forces, we soldered my DIY AURAMACHINE EMF circuit to detect and amplify the electromagnetic frequencies all around us. Learning about components, schematics and induction technology, we listened into our machines and the atmospheric electrical noises in the building. For some of the group it was the first time they had tried soldering, all machines worked and their technique got better and better. One of our group used their machine on the bus on the way home and nearly didn’t get off as they were enjoying listening to the sounds so much.
visual tools for machine learning
To create their ‘dream transmission’ final pieces, we explored some visual tools for machine learning using the brilliant community led pollinations platform, which curates the latest open source models via a simple to use interface for beginners. We looked at the history of imagenet and the problematic use of datasets and bias – and considered projects around resistance and reclaiming autonomy. How can we use AI to create alternative worlds and narratives, what do you want to see? What are the limitations of text to image methodologies? We considered the aesthetics and dominance of certain visual ML models such as CLIP, Dalle and VQGAN, looking at their architectural structures and the creative debates around their use.
dream transmissions_final pieces
It was time to showcase the groups work and transmit their visions via the net. Brighter Sound hosted a watch party on youtube and twitter. Watch the video below to see the participants introduce their final pieces. I am so pleased with the quality and depth to these works, well done dream team!
DREAMING WITH MACHINES ARTISTS 2022 WERE
Phia Sky, ‘Memory mess-mass’
Ella Kay, ‘Hybridised naturally’
Wendy Smith, ‘Dream transmission 1 & 2’
Adrien Chatelain, ‘Dream transmission 1 & 2’
RomanX, ‘Aetheriality’
final comments from DWM participants
The aim of Dreaming with Machines was to see how creative and accessible the current tools for sonic machine learning are for beginners. I wanted to provide a barrier free programme for participants, so you didn’t need to know how to code or have an expensive computer and we focused on open source platforms, training in the cloud and entry level interfaces in order to create. Here is what the participants thought of the experience…
Q: How was your experience of learning about ML and using the ML tools on the project?
“Really inspiring to explore not only how these tools can be used, but also deliberately abused. I’m someone who has slight reservations about the cultural implication of unhinged ML art generation with no human input – so the level of technical detail we went into regarding the processes made me feel more comfortable and gave me a good strong sense of creative control. I also enjoyed the fact that we touched on a bit of the history and background of these processes and the ethical implications of AI at large within society.”
“Super fun, there were some parts for me that were confusing like working with Google notebooks and stuff but it was super easy to get around, especially loved making the aura machine”
“Excellent. I appreciated the background teaching and especially the mini library. I found reading Artificial Intelligence[…] by Melanie Mitchell instructive and helped broaden my understanding between sessions. For the actual tools, the demonstration with the guides were very good, and the helping hand in person + assistance troubleshooting were great. Having the discord to bounce ideas off each other and share tips was super cool, very glad to have that as a resource.”
Q: Do you feel you will continue to use them in your music making?
“Yes, definitely! I got into a good creative stride of building my datasets, feeding them into neural synthesis, resampling the output within the DAW, editing them to make them a bit more palatable, and then finally sequencing them in MIDI using the Magenta Studio plugin. I’ll definitely be exploring this further”
“I have continued to use everything that was shown to us (just the free versions for now) for both my music development and adding visuals to my work. It all feels very accessible now and has given me a new exciting approach, especially when writing new stuff from scratch.”
“I intend to make some use of the tools discussed, however the one I’m most interested in, SampleRNN, sadly seems too low fidelity for my current production style. I do intend to integrate it in some way, and will attempt to train it at a higher sample rate.”
“It has been a freeing experience as an artist to learn how to produce my own visual material, because whenever I have worked with video I have had to rely on a visual artist to create the material. However, because of this residency I now have the skills to produce my own visual material. Also, I am currently developing a piece that will use the skills and tools I gained from this residency to compose a work for my PhD study. I also plan to learn how to use new tools, as this residency has built confidence in me to explore ML further.”
“I knew I wanted to utilise the possibilities of what machine learning could do for my future work. The PhD project I want to complete could lend itself extremely well to incorporating machine learning, so this project was absolutely perfect for an exciting introduction into how I can use machine learning in the future!”
Q: How did you find the format of working hybridly both in person and online? Any comments on the Discord tutorial resources?
“the amount of ground we covered in such a short time was really impressive – I think everyone came out the other end with their own preferred sets of ML tools and this showed in the variety of compositions we created.”
“The guide sheets to help with some of the more complicated processes, like the SampleRNN neural synthesis for example, were super well laid-out and easy to follow in our own time from home.”
“Coming from a slightly more technical background I think I took to SampleRNN and Colab quite well despite not having used them before.”
“I very much enjoyed the in person aspects, with self directed work happening based on the knowledge we gained in person. The ability to explore different additional tools online, talk to Vicky and troubleshoot was very beneficial. I think the way the hybrid working was managed – with primary learning and discussion in person paired with working on our own to complete tasks, finish implementing tools etc with discord was much better than having teaching sessions hosted online.”
“Discord also enabled residents to connect and share work with each other, this was fundamental to learn more about everyone’s creative practice and assisted with our bonding experience. All of the worksheets and resources were really dyslexia friendly and it was easy to access all the information in them, especially the step by step instructions for each ML tool.”
“I absolutely loved the use of discord and the materials and resources provided there! It’s not something I’ve used or done before, but having a hub for all things to do with machine learning and the residency was great and really inspiring. I enjoyed discussing topics, thoughts, opinions, anything to do with the residency and the concepts involved – it helped me feel more immersed in the topic but also helped me feel part of our residency group and helped me get to know my fellow composers more which just added to the enjoyment of the whole experience.”
AURA MACHINE 