Sonification of Mass Ave buses, from Nubian to Harvard.
Updated for Max8 and Catalina
This patch requests data from MBTA API to get the current location of buses – using the Max js object. Latitude and Longitude data is mapped to oscillator pitch. Data is polled every 10 seconds, but it seems like the results might be more interesting to poll at a slower rate, because the updates don’t seem that frequent. And buses tend to stop a lot.
This project uses version 3 of the API. There are quality issues with the realtime data. For example, there are bus stops not associated with the route. The direction_id and stop_sequence data from the buses is often wrong. Also, buses that are not in service are not removed from the vehicle list or indicated as such.
The patch uses a [multislider] object to graph the position of the buses along the route – but due to the data problems described above, the positions don’t always reflect the current latitude/longitude coordinates or the bus stop name.
You will need to replace the API key in the message object at the top of the patch with your own key. Or you can probably just remove it. The key distributed with the patch is fake. You can request your own developer API key from MBTA. It’s free.
instructions
Open mbta.maxpat
Open the Max console window so you can see what’s happening with the data
click on the yellow [getstops] message to get the current bus stop data
Toggle the metro (at the top of the patch) to start polling
Turn on the audio (click speaker icon) and turn up the gain
Note: there will be more buses running during rush hours in Boston. Try experimenting with the polling rate and ramp length in the poly-oscillator patch. Also, you can experiment with the pitch range.
The Soundcloud client-id is embedded in scnode.js – you will need to edit this file to replace the worthless client-id with your own. To get a client ID you will first need a Soundcloud account. Then register an app at: http://soundcloud.com/you/apps
first time instructions
Open the Max patch: sc.maxpat
In the green panel, click on [script npm init]
In the green panel , click on [script install soundcloud-api-client]
instructions
Open the Max patch sc.maxpat
open the max.console window so you can see the API data
click [script start]
click the speaker icon to start audio
type something into the search box and press <enter> or click the button to the left to search for what is already in the box.
select a track from the result menu, wait for it to download and start playing
This implementation uses node.js for Max instead of Ruby to access the API. You will need set up a developer account with Spotify and request API credentials. See below.
Other than that, the synthesis code in Max has not changed. Some of the following background information and video is from the original version. ..
What if you used that data to reconstruct music by driving a sequencer in Max? The analysis is a series of time based quanta called segments. Each segment provides information about timing, timbre, and pitch – roughly corresponding to rhythm, harmony, and melody.
Edit spot1.js replacing the cliendID and clientSecret with your spotify credentials
node for max install instructions (first time only)
Open the Max patch: spotify-synth1.maxpat
Scroll the patch over to the far right side until you see this green panel:
Click the [script npm init] message – this initializes the node infrastructure in the current folder
Then click each of the 2 script npm install messages – this installs the necessary libraries
Instructions
Open the Max patch: spotify-synth1.maxpat
Click the green [script start] message
Click the Speaker icon to start audio
Click the first dot in the preset object to set the mixer settings to something reasonable
open the Max Console window so you can see the Spotify API data
From the 2 menus at the top of the screen select an Artist and Title that match, for example: Albert Ayler and “Witches and Devils”
Click the [analyze] button – the console window should fill with interest data about your selection.
Click [play]
Note: if you hear a lot of clicks and pops, reduce the audio sample rate to 44.1 KHz.
Alternative search method:
Enter an Artist and Song title for analysis, in the text boxes. Then press the buttons for title and artist. Then press the /analyze button. If it works you will get prompts from the terminal window, the Max window, and you should see the time in seconds in upper right corner of the patch.
troubleshooting
If there are problems with the analysis, its most likely due to one of the following:
artist or title spelled incorrectly
song is not available
song is too long
API is busy
Mixer controls
The Mixer channels from Left to right are:
bass
synth (left)
synth (right)
random octave synth
timbre synth
master volume
gain trim
HPF cutoff frequency
You can also adjust the reverb decay time and the playback rate. Normal playback rate is 1.
programming notes
Best results happen with slow abstract material, like the Miles (Wayne Shorter) piece above. The bass is not really happening. Lines all sound pretty much the same. I’m thinking it might be possible to derive a bass line from the pitch data by doing a chordal analysis of the analysis.
Here are screenshots of the Max sub-patches (the main screen is in the video above)
Timbre (percussion synth) – plays filtered noise:
Random octave synth:
Here’s a Coltrane piece, using roughly the same configuration but with sine oscillators for everything:
There are issues with clicks on the envelopes and the patch is kind of a mess but it plays!
Several modules respond to the API data:
tone synthesiszer (pitch data)
harmonic (random octave) synthesizer (pitch data)
filtered noise (timbre data)
bass synthesizer (key and mode data)
envelope generator (loudness data)
Since the key/mode data is global for the track, bass notes are probable guesses. This method doesn’t work for material with strong root motion or a variety of harmonic content. It’s essentially the same approach I use when asked to play bass at an open mic night.
additional notes
Now that this project is running again. I plan to write additional synthesizers that follow more of the spirit of the data. For example, distinguishing strong pitches from noise.
Also would like to make use of the [section] data as well as the rhythmic analysis. There is an amazing amount of potential here.
Note: You can either run a local web server, using nodeserver.js and index.html – in the WebAudio folder – or use the online version of WAP as described here.
Note: If the patch doesn’t exist you can paste it in by opening the file osctest/delay-thing.json in a text editor and copying the text. Then click paste in WAP and paste in the text. Then make sure to save it in WAP by typing in the name “delay-thing” and clicking the save-as: button
3. In a terminal window, go to the osctest/ folder and start the server by typing:
ruby ./wapOSCserver.rb
4. In WAP, Press the OSC button – the ruby server should acknowledge with the message: “WebSocket connection open”
5. Open the Max patch: wapOSCtester.maxpat
6. In WAP press the play button on the Oscillator module (you should hear sounds)
7. In the Max patch drag the slider on the left to control the oscillator pitch. You should hear the sound change and see the sliders move in WAP.
Notes on Web Sockets connection:
The html side of the connection is done in js/socketsOSC.js
look at the function: connectOSC() – which gets triggered by the OSC button in index.html
Incoming messages from Max (via Ruby server) are parsed in parseOSCMessage() which figures out how to set appropriate values for the audio objects in the DOM.
There are currently no acknowledgement or error messages being returned (at least I don’t remember doing this)
