Audio-reactive visuals
Week 8
Marko Řeháček
rehacek@mail.muni.cz
GENERATIVE DESIGN PROGRAMMING
p5 in VScode
GENERATIVE DESIGN PROGRAMMING
Install p5.vscode
p5 in VScode
1. Open Visual Studio Code
2. Open Extensions tab (Shift + Ctrl + X)
3. Search “p5”
4. Install “p5.vscode”
GENERATIVE DESIGN PROGRAMMING
1. Open Command Palette (Ctrl + Shift + P or View -> Command Palette)
2. Start typing “p5” and select “Create p5.js Project”
3. Select folder to put your project in
to open your sketch in a browser click the “Go Live” button
Installing libraries:
Command Palette -> “Install p5.js Contributor Library”
and install any library you want to add to your sketch
Create new p5 project
P5 in VScode
GENERATIVE DESIGN PROGRAMMING
● Live Preview from Microsoft
Allows you to see the sketch in the
● GitHub Copilot
Useful extensions
p5 in VScode
GENERATIVE DESIGN PROGRAMMING
Computer audition
GENERATIVE DESIGN PROGRAMMING
→
Pioneer in audio-visuals.
Matrix III
John Whitney, 1971
“Musical language, visual action, symbol, and
color. But are these elements united in true
complementarity? I do not know. But I do
know, that the union of color and tone is a very
special gift of computer technology.”
A Personal Search For the Complementarity
of Music and Visual Art (1992)
Recreating the visuals in
Processing.
GENERATIVE DESIGN PROGRAMMING
Sound
COMPUTER AUDITION
amplitude,
frequency
https://processing.org/tutorials/sound/#music-
and-sound-programming-in-the-arts
GENERATIVE DESIGN PROGRAMMING
❏ amplitude
❏ frequency
❏ beat
❏ pitch detection
❏ lyrics, pre-analyzed data about songs (look at Spotify API)
❏ music score – MIDI
What can we do?
COMPUTER AUDITION
GENERATIVE DESIGN PROGRAMMINGCOMPUTER AUDITION
Decomposing soundwaves
GENERATIVE DESIGN PROGRAMMING
https://alexanderchen.github.io/harmonics/
GENERATIVE DESIGN PROGRAMMINGCOMPUTER AUDITION
subbass 20-60 hertz
bass 60-250
mid 250-4k
high 4k-20k
Division of the audio spectrum
GENERATIVE DESIGN PROGRAMMING
FFT
COMPUTER AUDITION
❏ music to download (Nicolas Jaar)
❏ p5.sound library reference, most important p5.FFT
❏ analyze frequencies using fft.analyze([smoothing=0.8], [bins=1024])
❏ returns array of length bins, with each value representing the volume of frequencies
corresponding to that bin (frequency spectrum divided into 1024 parts)
❏ get the volume of bass using fft.getEnergy(“...”)
❏ try arguments "bass", "lowMid", "mid", "highMid", "treble"
❏ connect to microphone, if you have one
mic = new p5.AudioIn();
mic.start();
❏ detect beat
❏ demo, demo 2
❏ other examples
Code ->
GENERATIVE DESIGN PROGRAMMING
→
AUDIO-REACTIVE VISUALS
https://therewasaguy.github.io/p5-music-viz/demos/08_echonestPitchSegment/
GENERATIVE DESIGN PROGRAMMING
→ IMAGE
Sound of EARTH ⊕ SOUND OF SPACE
thedotisblack, 2017
The audiovisual is made with Processing based on an
audible collage constructed from various plasma wave
recordings of NASA's Voyager 1 and 2 spacecrafts. The
audible collage is part of a collection of works by
Brain/Mind Research that was inspired by these
audible-frequency plasma waves and arranged selections
from these recordings into a "musical" form. Launched in
1977 by NASA, Voyager 1 and 2 passed near different
planetary and moon bodies within our solar system.
The astrological symbol for the planet Earth is ⊕. It
represents the cardinal directions.
The audiovisual consists of one original "audio" file, cut
down to 3 minutes, and edited with a beginning and end
transition for a perfect loop.
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
Sonification
Sonification is the use of non-speech audio to convey information or perceptualize data (Wiki).
Listen to wikipedia:
http://listen.hatnote.com/
Listen to bitcoin:
https://www.bitlisten.com/
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
→
An audiovisual performance in which the speech,
shouts and songs produced by two abstract vocalists
are radically augmented in real-time by custom
interactive visualization software. The performance
touches on themes of abstract communication,
synaesthetic relationships, cartoon language, and
writing and scoring systems, within the context of a
sophisticated, playful, and virtuosic audiovisual
narrative.
LIVE PERFORMANCE
AUDIO-REACTIVE VISUALS
https://www.tmema.org/messa/messa.html
Messa di Voce (placing the voice)
Tmema, 2003
GENERATIVE DESIGN PROGRAMMING
→ AUDIO-VISUALS
Transfinite
RYOJI IKEDA
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
Party visuals
AUDIO-REACTIVE VISUALS
Party visuals
Joelle
https://vimeo.com/116097721 https://vimeo.com/68161863
GENERATIVE DESIGN PROGRAMMING
Algoraves
An algorave (from an algorithm and
rave) is an event where people dance
to music generated from algorithms,
often using live coding techniques.
AUDIO-REACTIVE VISUALS
An example of a live coding dev
environment for real-time visual
performance:
www.visor.live
GENERATIVE DESIGN PROGRAMMING
Take any of your previous sketches, select proper music, and make it audio-reactive.
Make it react
AUDIO-REACTIVE VISUALS
Make a calming thing.
Calming thing
GENERATIVE DESIGN PROGRAMMING
1. Open Command Palette (Ctrl + Shift + P or View -> Command Palette)
2. Start typing “p5” and select “Create p5.js Project”
3. Select folder to put your project in
to open your sketch in a browser click the “Go Live” button in bottom blue panel.
Installing libraries:
Command Palette -> “Install p5.js Contributor Library”
and install any library you want to add to your sketch
Create new p5 project
P5 in VScode