Audio-reactive visuals
Week 10
Marko Řeháček
rehacek@mail.muni.cz
GENERATIVE DESIGN PROGRAMMING
p5 in VScode
GENERATIVE DESIGN PROGRAMMING
Install p5.vscode
p5 in VScode
1. Open Visual Studio Code
2. Open Extensions tab (Shift + Ctrl + X)
3. Search “p5”
4. Install “p5.vscode”
GENERATIVE DESIGN PROGRAMMING
1. Open Command Palette (Ctrl + Shift + P or View -> Command Palette)
2. Start typing “p5” and select “Create p5.js Project”
3. Select folder to put your project in
to open your sketch in a browser click the “Go Live” button
Installing libraries:
Command Palette -> “Install p5.js Contributor Library”
and install any library you want to add to your sketch
Create new p5 project
P5 in VScode
GENERATIVE DESIGN PROGRAMMING
Live Preview from Microsoft
Allows you to see the sketch in the
GitHub Copilot
Useful extensions
p5 in VScode
GENERATIVE DESIGN PROGRAMMING
Computer audition
GENERATIVE DESIGN PROGRAMMING
→
Pioneer in audio-visuals.
Matrix III
John Whitney, 1971
“Musical language, visual action, symbol, and
color. But are these elements united in true
complementarity? I do not know. But I do
know, that the union of color and tone is a very
special gift of computer technology.”
A Personal Search For the Complementarity
of Music and Visual Art (1992)
Recreating the visuals in
Processing.
GENERATIVE DESIGN PROGRAMMING
Sound
COMPUTER AUDITION
amplitude,
frequency
https://processing.org/tutorials/sound/#music-
and-sound-programming-in-the-arts
GENERATIVE DESIGN PROGRAMMING
❏ amplitude
❏ frequency
❏ beat
❏ pitch detection
❏ lyrics, pre-analyzed data about songs (look at Spotify API)
❏ music score – MIDI
What can we do?
COMPUTER AUDITION
GENERATIVE DESIGN PROGRAMMINGCOMPUTER AUDITION
Decomposing soundwaves
GENERATIVE DESIGN PROGRAMMING
https://alexanderchen.github.io/harmonics/
GENERATIVE DESIGN PROGRAMMINGCOMPUTER AUDITION
subbass 20-60 hertz
bass 60-250
mid 250-4k
high 4k-20k
Division of the audio spectrum
GENERATIVE DESIGN PROGRAMMING
FFT
COMPUTER AUDITION
❏ music to download (Nicolas Jaar)
❏ p5.sound library reference, most important p5.FFT
❏ analyze frequencies using fft.analyze([smoothing=0.8], [bins=1024])
❏ returns array of length bins, with each value representing the volume of frequencies
corresponding to that bin (frequency spectrum divided into 1024 parts)
❏ get the volume of bass using fft.getEnergy(“...”)
❏ try arguments "bass", "lowMid", "mid", "highMid", "treble"
❏ connect to microphone, if you have one
mic = new p5.AudioIn();
mic.start();
❏ detect beat
❏ demo, demo 2
❏ other examples
Code ->
GENERATIVE DESIGN PROGRAMMING
→
AUDIO-REACTIVE VISUALS
https://therewasaguy.github.io/p5-music-viz/demos/08_echonestPitchSegment/
GENERATIVE DESIGN PROGRAMMING
→ IMAGE
Sound of EARTH ⊕ SOUND OF SPACE
thedotisblack, 2017
The audiovisual is made with Processing based on an
audible collage constructed from various plasma wave
recordings of NASA's Voyager 1 and 2 spacecrafts. The
audible collage is part of a collection of works by
Brain/Mind Research that was inspired by these
audible-frequency plasma waves and arranged selections
from these recordings into a "musical" form. Launched in
1977 by NASA, Voyager 1 and 2 passed near different
planetary and moon bodies within our solar system.
The astrological symbol for the planet Earth is ⊕. It
represents the cardinal directions.
The audiovisual consists of one original "audio" file, cut
down to 3 minutes, and edited with a beginning and end
transition for a perfect loop.
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
Sonification
Sonification is the use of non-speech audio to convey information or perceptualize data (Wiki).
Listen to wikipedia:
http://listen.hatnote.com/
Listen to bitcoin:
https://www.bitlisten.com/
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
→
An audiovisual performance in which the speech,
shouts and songs produced by two abstract vocalists
are radically augmented in real-time by custom
interactive visualization software. The performance
touches on themes of abstract communication,
synaesthetic relationships, cartoon language, and
writing and scoring systems, within the context of a
sophisticated, playful, and virtuosic audiovisual
narrative.
LIVE PERFORMANCE
AUDIO-REACTIVE VISUALS
https://www.tmema.org/messa/messa.html
Messa di Voce (placing the voice)
Tmema, 2003
GENERATIVE DESIGN PROGRAMMING
→ AUDIO-VISUALS
Transfinite
RYOJI IKEDA
AUDIO-REACTIVE VISUALS
GENERATIVE DESIGN PROGRAMMING
Party visuals
AUDIO-REACTIVE VISUALS
Party visuals
Joelle
https://vimeo.com/116097721 https://vimeo.com/68161863
GENERATIVE DESIGN PROGRAMMING
Algoraves
An algorave (from an algorithm and
rave) is an event where people dance
to music generated from algorithms,
often using live coding techniques.
AUDIO-REACTIVE VISUALS
An example of a live coding dev
environment for real-time visual
performance:
www.visor.live
GENERATIVE DESIGN PROGRAMMING
Take any of your previous sketches, select proper music, and make it audio-reactive.
Make it react
AUDIO-REACTIVE VISUALS
Make a calming thing.
Calming thing
GENERATIVE DESIGN PROGRAMMING
1. Open Command Palette (Ctrl + Shift + P or View -> Command Palette)
2. Start typing “p5” and select “Create p5.js Project”
3. Select folder to put your project in
to open your sketch in a browser click the “Go Live” button in bottom blue panel.
Installing libraries:
Command Palette -> “Install p5.js Contributor Library”
and install any library you want to add to your sketch
Create new p5 project
P5 in VScode