Category: ideas

Spotify segment analysis player in Max

Echo Nest API audio analysis data is now provided by Spotify. This project is part of the internet-sensors project: https://reactivemusic.net/?p=5859  and updates the 2013 Echo Nest project described here: https://reactivemusic.net/?p=6296

 

The original analyzer document by Tristan Jehan can be found here (for the time being):  https://web.archive.org/web/20160528174915/http://developer.echonest.com/docs/v4/_static/AnalyzeDocumentation.pdf

This implementation uses node.js for Max instead of Ruby to access the API. You will need set up a developer account with Spotify and request API credentials. See below.

Other than that, the synthesis code in Max has not changed.  Some of the following background information and video is from the original version. ..

What if you used that data to reconstruct music by driving a sequencer in Max? The analysis is a series of time based quanta called segments. Each segment provides information about timing, timbre, and pitch – roughly corresponding to rhythm, harmony, and melody.

spotify-synth1.maxpat

download

https://github.com/tkzic/internet-sensors

folder: spotify2

files

main Max patch
  • spotify-synth1.maxpat
abstractions and other files
  • polyvoice-sine.maxpat
  • polyvoice2.maxpat
node.js code
  • spot1.js
node folders and infrastructure
  • /node_modules
  • package-lock.json
  • package.json
dependencies:
  • You will need to install node.js
  • the node package manager will do the rest – see below.

Note: Your best bet is to just download the repository, leave everything in place, and run it from the existing folder

authentication

You will need to sign up for a developer account at Spotify and get an API key. https://developer.spotify.com/documentation/general/guides/authorization-guide/

Edit spot1.js replacing the cliendID and clientSecret with your spotify credentials

node for max install instructions (first time only)

  •  Open the Max patch: spotify-synth1.maxpat
  •  Scroll the patch over to the far right side until you see this green panel:

  • Click the [script npm init] message – this initializes the node infrastructure in the current folder
  • Then click each of the 2 script npm install messages –  this installs the necessary libraries

Instructions

  •  Open the Max patch: spotify-synth1.maxpat
  •  Click the green [script start] message
  • Click the Speaker icon to start audio
  • Click the first dot in the preset object to set the mixer settings to something reasonable
  • open the Max Console window so you can see the Spotify API data
  • From the 2 menus at the top of the screen select an Artist and Title that match, for example: Albert Ayler and “Witches and Devils”
  • Click the [analyze] button – the console window should fill with interest data about your selection.
  • Click [play]
  • Note: if you hear a lot of clicks and pops, reduce the audio sample rate to 44.1 KHz.
Alternative search method:

Enter an Artist and Song title for analysis, in the text boxes. Then press the buttons for title and artist. Then press the /analyze button. If it works you will get prompts from the terminal window, the Max window, and you should see the time in seconds in upper right corner of the patch.

troubleshooting

If there are problems with the analysis, its most likely due to one of the following:

  • artist or title spelled incorrectly
  • song is not available
  • song is too long
  • API is busy
Mixer controls

The Mixer channels from Left to right are:

  • bass
  • synth (left)
  • synth (right)
  • random octave synth
  • timbre synth
  • master volume
  • gain trim
  • HPF cutoff frequency
You can also adjust the reverb decay time and the playback rate. Normal playback rate is 1.

programming notes

Best results happen with slow abstract material, like the Miles (Wayne Shorter) piece above. The bass is not really happening. Lines all sound pretty much the same. I’m thinking it might be possible to derive a bass line from the pitch data by doing a chordal analysis of the analysis.

Here are screenshots of the Max sub-patches (the main screen is in the video above)

Timbre (percussion synth) – plays filtered noise:

Random octave synth:

Here’s a Coltrane piece, using roughly the same configuration but with sine oscillators for everything:

There are issues with clicks on the envelopes and the patch is kind of a mess but it plays!

Several modules respond to the API data:

  • tone synthesiszer (pitch data)
  • harmonic (random octave) synthesizer (pitch data)
  • filtered noise (timbre data)
  • bass synthesizer (key and mode data)
  • envelope generator (loudness data)

Since the key/mode data is global for the track, bass notes are probable guesses. This method doesn’t work for material with strong root motion or a variety of harmonic content. It’s essentially the same approach I use when asked to play bass at an open mic night.

additional notes

Now that this project is running again. I plan to write additional synthesizers that follow more of the spirit of the data. For example, distinguishing strong pitches from noise.

Also would like to make use of  the [section] data as well as the rhythmic analysis. There is an amazing amount of potential here.

Analog video synthesis

Generative art in motion.

ck

https://vimeo.com/cskonopka

By Christopher Konopka

Background

In the past year, Chris has published nearly 2500 improvised video pieces.

13522998_945476555985_3590046569317439705_o

You may be familiar with analog modular audio synthesis. The hardware to produce video looks nearly identical – a maze of patch cords and dials.

Television

13709835_10154366721684231_6046749253184273850_n

Analog video is television. A CRT (cathode ray tube) resynthesizes video information by demodulating signals from a camera. Vintage televisions have dials to adjust color and vertical sync. When you turn the dials you are synthesizing analog video. Distortion, filtering, and feedback – either at the source (camera) or the destination (tv screen) – offer up an infinite variety of images.

Analog vs. Digital

Today all media is digital. Like the screen you are looking at. The difference with analog is in how it’s produced. Boundaries are less definite. Lines curve. Colors waver. Feedback looks like flames. Every frame is a painting.

Patterns

Images can be generated electronically using modules – without a camera.

Filters

Like with audio sampling, anything is a source. Movies, Youtube, live television, even Felix the Cat.

Feedback

When you aim a guitar at an amplifier it screams. Tilt it away slightly and the screaming subsides. In between there’s sweet spot. The same is true with cameras and screens. Feedback results when output is mixed with input.

Radio

Analog shortwave radio signals are distorted by the atmosphere in a manner similar to video filtering.

A studio in Bethel, Maine.

image1

An improvised collaboration between Chris and Tom Zicarelli using shortwave radio processed with audio effects.

Live Performance

Gem

A recent screen test at the Gem Theatre in Bethel, Maine. Source material is a time lapse film of a glacier installation – produced at the same theatre – by Wade Kavanaugh and Steven Nguyen. https://www.youtube.com/watch?v=6c36Y-Dcj30  The film was re-synthesized using analog video and feedback. Soundtrack by Tom Zicarelli.

Big screen equals mind bending experience.

Note: previous clip excerpted from this 15 minute jam: https://vimeo.com/177843310

TAL

The patterns in this clip appear to be three dimensional. They are not.

From a show that happened somewhere in the known universe:

Alto

Improvised analog video with the band “Alto”. Patterns reminiscent of magical textiles.

More about analog video synthesis

 

Teufelsberg

An acoustically interesting dome in an abandoned spy station at the highest point in Berlin.

dome

We ran into flutist Thomas Von Der Brücke in the woods around Teufelsberg. Thomas has often recorded and performed in the dome. He invited us to come along for a tour.

thomas

Thomas on Flute…

Thomas on vocals and flute, with me on percussion

 

Photographs by Rebecca Zicarelli

Glacier Sounds

Overlapping loops of varying duration to represent natural cycles.

glacier1

In October I collaborated with Wade Kavanaugh and Stephen P. Nguyen to compose and perform the sounds of a glacier for their installation at the Gem theatre in Bethel, Maine. The glacier was made from paper.

Wade and Stephen:

wadeandsteven

A time-lapse video of the project:

A time-lapse video of a similar project they did in Minnesota 2005:

The approach was to take a series of ambient loops and organize them by duration. The longer loops would represent the slow movement of time. Shorter loops would represent events like avalanches. One-shot samples would represent quick events, like the cracking of ice.

It took several iterations to produce something slow and boring enough to be convincing. I used samples from the Ron MacLeod’s Cyclic Waves library from Cycling 74 https://www.ableton.com/en/packs/cyclic-waves/. Samples were pitched down to imply largeness.

Screen Shot 2015-12-21 at 1.09.59 AM

Each vertical column in an Ableton Live set represents a time-frame of waves. That is, the far left column contains quick events and the far right column contains long cycle events. Left to right, the columns have gradually increasing cycle durations.  I used a Push controller to trigger samples in real time as people walked through the theatre to see the glacier.

The theatre speakers were arranged in stereo but from front to back. Since the glacier was also arranged along the same axis, a slow auto-panning effect sent sounds drifting off into the distance, or vice versa. Visually and sonically there was a sense that the space extended beyond the walls of the theatre.

In the “control room” above the theatre… using Push to trigger samples and a Korg NanoKontrol to set panning positions of each track:

glacer2

The performance lasted about 45 minutes. Occasionally the cracking of ice would startle people in the room. There were kids crawling around underneath the paper glacier. Afterwards we just let the sounds play on their own. A short excerpt:

 

Photographs by Rebecca Zicarelli.

Motorized potentiometers

And other ways to remotely control existing dials.

Suggestions from robotshop.com: http://www.robotshop.com/forum/post-p101569

 

Motorized potentiometers

from p3america: http://www.p3america.com/motorized_potentiometers_text.htm?gclid=CJSetfSYusUCFdOQHwodPEUAaw

Screen Shot 2015-05-11 at 1.38.41 PM

 

From Online Controls: http://www.onlinecontrols.com/mpots.htm

 Motorized faders

Sparkfun motorized fader (like the kind used in DAW control surfaces) https://www.sparkfun.com/products/10976

10734-01

Servo control and sensing

from electronics stack exchange: http://electronics.stackexchange.com/questions/60272/servo-controlled-potentiometer-using-rotary-position-sensor

X4YLv

notes

The Potentiometer Handbook by Carl David Todd: https://www.bourns.com/pdfs/onlinepotentiometerhandbook.pdf

Google search: servo to turn a knob

Shower temperature control from SmithyTech: http://smithytech.com/?p=5

Arduino HVAC Servo Thermostat/Controller by tikka308: http://www.instructables.com/id/Arduino-HVAC-Servo-ThermostatController/

What kind of motor would I need to turn this central heating valve? http://electronics.stackexchange.com/questions/97269/what-kind-of-motor-would-i-need-to-turn-this-central-heating-valve – observations about the difficulty of using robots to turn knobs

Sous vide cooker with feedback control: http://www.nerdkits.com/videos/sous-vide/

servobox01_t

Homemade Electric Telescope Focuser: http://emediadesigns.com/focuser/

 

Use your Raspberry Pi to move parts of a robot or control anything that can rotate – by Rob Zwetsloot http://www.linuxuser.co.uk/tutorials/control-servos-with-a-raspberry-pi