Glacier Sounds
Overlapping loops of varying duration to represent natural cycles.
In October I collaborated with Wade Kavanaugh and Stephen P. Nguyen to compose and perform the sounds of a glacier for their installation at the Gem theatre in Bethel, Maine. The glacier was made from paper.
Wade and Stephen:
A time-lapse video of the project:
A time-lapse video of a similar project they did in Minnesota 2005:
The approach was to take a series of ambient loops and organize them by duration. The longer loops would represent the slow movement of time. Shorter loops would represent events like avalanches. One-shot samples would represent quick events, like the cracking of ice.
It took several iterations to produce something slow and boring enough to be convincing. I used samples from the Ron MacLeod’s Cyclic Waves library from Cycling 74 https://www.ableton.com/en/packs/cyclic-waves/. Samples were pitched down to imply largeness.
Each vertical column in an Ableton Live set represents a time-frame of waves. That is, the far left column contains quick events and the far right column contains long cycle events. Left to right, the columns have gradually increasing cycle durations. I used a Push controller to trigger samples in real time as people walked through the theatre to see the glacier.
The theatre speakers were arranged in stereo but from front to back. Since the glacier was also arranged along the same axis, a slow auto-panning effect sent sounds drifting off into the distance, or vice versa. Visually and sonically there was a sense that the space extended beyond the walls of the theatre.
In the “control room” above the theatre… using Push to trigger samples and a Korg NanoKontrol to set panning positions of each track:
The performance lasted about 45 minutes. Occasionally the cracking of ice would startle people in the room. There were kids crawling around underneath the paper glacier. Afterwards we just let the sounds play on their own. A short excerpt:
Photographs by Rebecca Zicarelli.
Presets in Max for Live
How to use the Max preset object inside of M4L.
There is some confusion about how to use Max presets in a M4L device. The method described here lets you save and recall presets with a device inside of a Live set, without additional files or dialog boxes. It uses pattrstorage. It works automatically with the Live UI objects.
It also works with other Max UI objects by connecting them to pattr objects.
Its based on an article by Gregory Taylor: https://cycling74.com/2011/05/19/max-for-live-tutorial-adding-pattr-presets-to-your-live-session/
Download
https://github.com/tkzic/max-for-live-projects
Folder: presets
Patch: aaa-preset3.amxd
How it works:
Instructions are included inside the patch. You will need to add objects and then set attributes for those objects in the inspector. For best results, set the inspector values after adding each object
Write the patch in this order:
A1. Add UI objects.
For each UI object:
- check link-to-scripting name
- set long and short names to actual name of param
A2 (optional) Add non Live (ie., Max UI objects)
For each object, connect the middle outlet of a pattr object (with a parameter name as an argument) to the left inlet of the UI object. For example:
Then in inspector for each UI object:
- check parameter-mode-enable
- check inital-enable
B. Add a pattrstorage object.
Give the object a name argument, for example: pattrstorage zoo. The name can be anything, its not important. Then in the inspector for pattrstorage:
- check parameter-mode enable
- check Auto-update-parameter Initial-value
- check initial-value
- change short-name to match long name
C. Add an autopattr object
D. Add a preset object
In the inspector for the preset object:
- assign pattrstorage object name from step B. (zoo) to pattrstorage attribute
Notes
The preset numbers go from 1-n. They can be fed directly into the pattrstorage object – for example if you wanted to use an external controller
You can name the presets (slotnames). See the pattrstorage help file
You can interpolate between presets. See pattrstorage help file
Adding new UI objects after presets have been stored
If you add a new UI object to the patch after pattrstorage is set up, you will need to re-save the presets with the correct setting of the new UI object. Or you can edit the pattrstorage data.
Ableton Push Development Kit
The Live Object Model in Max for Live
Introduction to getting, setting, and observing Live parameters with Max for Live.
download
https://github.com/tkzic/max-for-live-projects
folder: lom
patch: aaa-lom-examples
tutorial
This example device shows several ways of working with Ableton Live parameters in a M4L patch. It can be a confusing process. And there are many different ways to accomplish the same result.
The examples here will use the LOM (Live Object Model) directly, and via builtin LiveAPI abstractions and choosers – available from the context menu that appears when you <ctrl> click inside of an open patcher window.
live.observer
The snippet above shows how to continuously monitor Live’s tempo.
- live.this.device sends out a bang when the device loads
- the “path live_set” message tells live.path to get an id number for the current set. This id is sent to the right inlet of live.observer, telling it we want to observe the current Live set
- the “property tempo” message asks live.observer for the current tempo value
- If the tempo changes it will be automatically updated
live.object (set)
The snippet above shows how to set Live’s tempo.
- Get the Live set path id using the same method as shown for live.observer
- the “set tempo” message sends a tempo value to live.object
live.object (get)
The snippet above shows how to get Live’s tempo.
- Get the Live set path id using the same method as shown for live.observer
- the “get tempo” message requests the current tempo value from live.object
live.object (call)
The snippet above shows how to start or stop the Live transport by calling functions.
- Get the Live set path id using the same method as shown for live.observer
- the “call start_playing” message tells live.object to start the Live transport. “start_playing” is the name of a function builtin to the Live set.
LiveAPI abstractions
The LiveAPI abstractions provide convenient shortcuts to working with Live parameters. Copy them into your patch by <ctrl> clicking inside an open (unlocked) patcher window – and selecting “Paste from -> LiveAPI abstractions”
Observing the Live transport
The snippet above shows how to monitor whether the Live transport is running
- paste the abstraction into your patch as explained above
Selecting the master track
The snippet above shows how to select the Master track
- paste the abstraction into your patch as explained above
LiveAPI choosers
The LiveAPI choosers provide convenient shortcuts to selecting Live parameters from menu objects. Copy them into your patch by <ctrl> clicking inside an open (unlocked) patcher window – and selecting “Paste from -> LiveAPI choosers”
Setting device parameters remotely with live.remote
The snippet above shows how to remotely set the volume on the master track
- paste the chooser into your patch as explained above
- connect the left outlet of “substitute” to the right inlet of live.remote
- send values to live.remote to change the selected parameter
Patches for Cycling 74 “Programming in Max for Live” videos
These are my versions of some of the patches. Not Cycling 74’s. If you find ‘official’ versions please let me know. The list isn’t complete. The simpler (beginning) patches are not included, but the interesting ones are.
download
https://github.com/tkzic/max-for-live-projects
folder: c74-video-tutorials
Sub-folders:
- API-step-sequencer
- dub-delay
- poly-synth
- velocity-sequencer
- wobble-bass
Instructions
In Live, create new Max for Live devices, as instructed in the videos – and then copy and paste from these patches into your new patches.
Warping acapella tracks
In Ableton Live
By Thavius Beck at Dubspot
Geisterwelt
Spectral slider plugin for Ableton Live
By Adam Rokhsar at Utami
http://makeyourselftransparent.tumblr.com
http://youtu.be/r-ZpwGgkGFI
Hearing voices
A presentation for Berklee BTOT 2015 http://www.berklee.edu/faculty
(KITT dashboard by Dave Metlesits)
The voice was the first musical instrument. Humans are not the only source of musical voices. Machines have voices. Animals too.
Topics
- synthesizing voices (formant synthesis, text to speech, Vocaloid)
- processing voices (pitch-shifting, time-stretching, vocoding, filtering, harmonizing),
- voices of the natural world
- fictional languages and animals
- accents
- speech and music recognition
- processing voices as pictures
- removing music from speech
- removing voices
Voices
We instantly recognize people and animals by their voices. As an artist we work to develop our own voice. Voices contain information beyond words. Think of R2D2 or Chewbacca.
There is also information between words: “Palin Biden Silences” David Tinapple, 2008: http://vimeo.com/38876967
Synthesizing voices
The vocal spectrum
What’s in a voice?
- Formant synthesis in Max by Mark Durham: https://reactivemusic.net/?p=9294 (singing vowels with formants)
- Formant synthesis Tutorial by Jordan Smith: https://reactivemusic.net/?p=9290 (making consonants with noise)
Singing chords
Humans acting like synthesizers.
- Singing chords: Lalah Hathaway https://www.youtube.com/watch?v=c5AdOZtRdfE (0:30)
- Tuvan throat singing: https://www.youtube.com/watch?v=5wHbIWH_NGc (near the end of the video)
- Polyphonic overtone singing: Anna-Maria Hefele https://www.youtube.com/watch?v=vC9Qh709gas
More about formants
- Formants (Wikipedia) http://en.wikipedia.org/wiki/Formant
- Rooms have resonances: “I am sitting in a Room” by Alvin Lucier
- Singer’s formant (2800-3400Hz).
Text to speech
Teaching machines to talk.
- phonemes (unit of sound)
- diphones (combination of phonemes) (Mac OS “Macintalk 3 pro”)
- morphemes (unit of meaning)
- prosody (musical quality of speech)
Methods
- articulatory (anatomical model)
- formant (additive synthesis) (speak and spell)
- concatentative (building blocks) (Mac Os)
Try the ‘say’ command (in Mac OS terminal), for example: say hello
More about text to speech
- History of speech synthesis http://research.spa.aalto.fi/publications/theses/lemmetty_mst/chap5.html (Helsinki University of Technology 1999)
- Speech synthesizers, 2014 https://reactivemusic.net/?p=18141
- Speech synthesis web API https://reactivemusic.net/?p=18138
Vocoders
Combining the energy of voice with musical instruments (convolution)
- Peter Frampton “talkbox”: https://www.youtube.com/watch?v=EqYDQPN_nXQ (about 5:42) – Where is the exciting audience noise in this video?
- Ableton Live example: Local file: Max/MSP: examples/effects/classic-vocoder-folder/classic_vocoder.maxpat
- Max vocoder tutorial (In the frequency domain), by dude837 – Sam Tarakajian https://reactivemusic.net/?p=17362 (local file: dude837/4-vocoder/robot-master.maxpat
More about vocoders
- How vocoders work, by Craig Anderton: https://reactivemusic.net/?p=17218
- Wikipedia: http://en.wikipedia.org/wiki/Vocoder. Engineers conserving information to reduce bandwith
- Heterodyne filter: https://reactivemusic.net/?p=17338 – digital emulation of an analog filter bank.
- Max/MSP: examples/effects/classic-vocoder-folder/classic_vocoder.maxpat
Vocaloid
By Yamaha
(text + notation = singing)
- Vocaloid website: http://www.vocaloid.com/en/
- Hatsune Miku: https://reactivemusic.net/?p=6891
Demo tracks: https://www.youtube.com/watch?v=QWkHypp3kuQ
- Vocaloid tutorial
- #1 https://www.youtube.com/watch?v=vcJDTDBWTrw (entering notes and lyrics – 1:25)
- #2 https://www.youtube.com/watch?v=qpGwgIyMGOk (raw sound – 0:42)
- #5 https://www.youtube.com/watch?v=YEAuL6Q2j-0 (with phrasing, vibrato, etc.,- 1:00)
Vocaloop device http://vocaloop.jp/ demo: https://www.youtube.com/watch?v=xLpX2M7I6og#t=24
Processing voices
Transformation
Pitch transposing a baby https://reactivemusic.net/?p=2458
Real time pitch shifting
Autotune: “T-Pain effect” ,(I-am-T-Pain bySmule), “Lollipop” by Lil’ Wayne. “Woods” by Bon Iver https://www.youtube.com/watch?v=1_cePGP6lbU
Autotuna in Max 7
by Matthew Davidson
Local file: max-teaching-examples/autotuna-test.maxpat
InstantDecomposer in Pure Data (Pd)
by Katja Vetter
http://www.katjaas.nl/slicejockey/slicejockey.html
Autocorrelation: (helmholtz~ Pd external) “Helmholtz finds the pitch” http://www.katjaas.nl/helmholtz/helmholtz.html
(^^ is input pitch, preset #9 is normal)
- local file: InstantDecomposer version: tkzic/pdweekend2014/IDecTouch/IDecTouch.pd
- local file: slicejockey2test2/slicejockey2test2.pd
Phasors and Granular synthesis
Disassembling time into very small pieces
- sorting noise; http://youtu.be/kPRA0W1kECg
- Phasors: https://reactivemusic.net/?p=17353
Time-stretching
Adapted from Andy Farnell, “Designing Sound”
https://reactivemusic.net/?p=11385 Download these patches from: https://github.com/tkzic/max-projects folder: granular-timestretch
- Basic granular synthesis: graintest3.maxpat
- Time-stretching: timestretch5.maxpat
More about phasors and granular synthesis
- Shepard tone upward glissando by Chris Dobrian: https://reactivemusic.net/?p=17255
- “Falling Falling” (Visual Shepard tone) https://reactivemusic.net/?p=17251
- Ableton Live – granulator (Robert Henke)
Phase vocoder
…coming soon
Sonographic sound processing
Changing sound into pictures and back into sound
by Tadej Droljc
https://reactivemusic.net/?p=16887
(Example of 3d speech processing at 4:12)
local file: SSP-dissertation/4 – Max/MSP/Jitter Patch of PV With Spectrogram as a Spectral Data Storage and User Interface/basic_patch.maxpat
Try recording a short passage, then set bound mode to 4, and click autorotate
Speech to text
Understanding the meaning of speech
The Google Speech API
A conversation with a robot in Max
https://reactivemusic.net/?p=9834
Google speech uses neural networks, statistics, and large quantities of data.
More about speech to text
- Real time German/English translator (Microsoft) http://digg.com/video/heres-microsoft-demoing-their-breakthrough-in-real-time-translated-conversation
- Skype translator – Spanish/English: http://www.skype.com/en/translator-preview/
- Dragon Naturally Speaking (Nuance) accidentally converts music to poetry
Voices of the natural world
Changes in the environment reflected by sound
- Bernie Krause: “Soundscapes”
Fictional languages and animals
“You can talk to the animals…”
- Derek Abbot’s animal noise page: http://www.eleceng.adelaide.edu.au/Personal/dabbott/animal.html
- Quack project http://www.quack-project.com/table.cgi
- Fictional language dialog by Naila Burney: https://reactivemusic.net/?p=7242
Pig creatures example: http://vimeo.com/64543087
- 0:00 Neutral
- 0:32 Single morphemes – neutral mode
- 0:37 Series, with unifying sounds and breaths
- 1:02 Neutral, layered
- 1:12 Sad
- 1:26 Angry
- 1:44 More Angry
- 2:11 Happy
What about Jar Jar Binks?
Accents
The sound changes but the words remain the same.
The Speech accent archive https://reactivemusic.net/?p=9436
Finding and removing music in speech
We are always singing.
Jamming with speech
- Drummer jams with a speed-talking auctioneer: https://reactivemusic.net/?p=7140
- Guitarist imitates crying politician: http://digg.com/video/guitarist-plays-along-to-sobbing-japanese-politician
Removing music from speech
SMS-tools
by Xavier Serra and UPF
Harmonic Model Plus Residual (HPR) – Build a spectrogram using STFT, then identify where there is strong correlation to a tonal harmonic structure (music). This is the harmonic model of the sound. Subtract it from the original spectrogram to get the residual (noise).
Settings for above example:
- Window size: 1800 (SR / f0 * lobeWidth) 44100 / 200 * 8 = 1764
- FFT size: 2048
- Mag threshold: -90
- Max harmonics: 30
- f0 min: 150
- f0 max: 200
feature detection
- time dependent
- Low level features: harmonicity, amplitude, fundamental frequency
- high level features: mood, genre, danceability
Acoustic Brainz: (typical analysis page) https://reactivemusic.net/?p=17641
Essentia (open source feature detection tools) https://github.com/MTG/essentia
Freesound (vast library of sounds): https://www.freesound.org – look at “similar sounds”
Removing voices from music
A sad thought
phase cancellation encryption
This method was used to send secret messages during world war 2. Its now used in cell phones to get rid of echo. Its also used in noise canceling headphones.
https://reactivemusic.net/?p=8879
max-projects/phase-cancellation/phase-cancellation-example.maxpat
Center channel subtraction
What is not left and not right?
Ableton Live – utility/difference device: https://reactivemusic.net/?p=1498 (Allison Krause example)
Local file: Ableton-teaching-examples/vocal-eliminator
More experiments
- Synthesizing laughter
- Bobby McFerrin: (pentatonic scale) http://www.ted.com/talks/bobby_mcferrin_hacks_your_brain_with_music.html
- Alphabet vocals
- jii lighter https://reactivemusic.net/?p=6970
- Sesame St – Joan LaBarbara: http://www.youtube.com/watch?v=y819U6jBDog
- Warping acapella tracks https://reactivemusic.net/?p=18046
Questions
- Why do most people not like the recorded sound of their voice?
- Can voice be used as a controller?
- (Imitone: http://imitone.com)
- Mari Kimura
- How do you recognize voices?
- Does speech recognition work with singing?
- How does the Google Speech API know the difference between music and speech?
- How can we listen to ultrasonic animal sounds?
- What about animal translators?
tempo dial in Max for Live
How would you make the dial respond to tempo changes?
download
https://github.com/tkzic/max-for-live-projects
folder: tempo
device: tempo-test.amxd
instructions
- drag the tempo.amxd file into a midi track
- turn the dial and watch the tempo change