Tag: OSC

hardware control of webaudio API

programming

This is a tweak of a demo program by Boris Smus from http://webaudioapi.com

The audio is generated in the web browser using the webaudio API. Oscillator frequency is being controlled by an iPod touch running touchOSC.

A ruby program is running in the background. It starts 2 servers, one to handle Osc messages from the iPod. and a websockets server which passes Osc messages to the web client in the browser, which parses the data and sets oscillator frequency.

local source files in tkzic/web-audio-api/webaudioapi/content/posts/oscillator-ws

instructions

(these are for my local files – source has not been published yet) Please contact me directly for more information about this code. Or have a look at a very similar example at: https://github.com/tkzic/WebAudio

1. run the node webserver in tkzic/

node nodeserver.js

(it will run on localhost port 8081 – for example http://127.0.0.1:8081)

2. In Chrome web browser, run: http://192.168.1.104:8081/web-audio-api/webaudioapi/content/posts/oscillator-ws/index.html

The following screen will appear:

3. From a terminal prompt run this ruby script (in tkzic/web-audio-api/webaudioapi/content/posts/oscillator-ws)

./wapOSCserver.rb

4. In webpage in Chrome, click the sockets button – the ruby server should open a socket connection, also click the play/pause button to start the oscillator

5. In touchOSC point the host to the ip address of the laptop (ie., 192.168.1.104) running on default ports, (out 8000, in 9000)

6. In touchOSC run the ‘simple’ layout and move fader1 to change the pitch of the oscillator

Note: You will probably also want to open the developer javascript console in Chrome to see what is going on.

next step

Need to clean up the ruby server code to handle error conditions, ie., network interruptions, missing files, browser incompatibility, etc.,

 

The sound of a new machine

Using internet ping data to control a synthesizer in Max

This project uses ‘ping’ times to about 40 Google domains, like google.ca, google.de, etc., to control pitch and amplitude of a 20 voice droning synthesizer.

Imagine working in a Google control center. A soothing low pitched drone fills the room. Then Suddenly you hear an slowly rising pitch. You check the monitors – Google Paraguay is experiencing network failure. You light a cigarette and wait for things to calm down.

The server is a ruby script which handles the http: requests using the Mashape ping-uin API and sends messages to Max using OSC

The synth has a weird clustering drone like effect like some kind of alien life force.

The patch design is kind of embarrassing. Its obvious I forgot how to use [poly~]. Maybe by the time you read this, we’ll have addressed this.  Hey billions of patch cords look cool.

Here’s an example of the Mashape API in curl

curl --include --request GET 'https://igor-zachetly-ping-uin.p.mashape.com/pinguin.php?address=google.ca' \
  --header 'X-Mashape-Authorization: YOUR-MASHAPE-API-KEY'

Here’s a list of Google domains

http://en.wikipedia.org/wiki/List_of_Google_domains

download

https://github.com/tkzic/internet-sensors

folder: ping

files

main Max patch
  • sound-of-a-new-machine2.maxpat
abstractions and other files
  • google.txt (list of domains for [coll] object
ruby
  • domain-ping.rb

authorization

  • Register with mashape http://mashape.com to get an API-key for ping-uin
  • Then edit domain-ping.rb to enter your mashape API-key.

instructions

  • Open the Max patch: sound-of-a-new-machine2.maxpat
  • Turn on audio. Turn up the gain.
  • From a terminal window type the following command
# ./domain-ping.rb


  •  In the Max patch, click the toggle box to start polling. It may take a minute to hear any sounds, while the oscillators are loading. Increase polling speed to 400 or so if you can’t wait.
  • Another reason you might not hear anything interesting is if the clip threshold is too low. Watch the incoming ping times and set the clip threshold above the average level.
  • Adjust the pitch multiplier to your desired pitch range.
  • When you’ve had enough, type <ctrl-c> in the terminal window to stop the server.

note: Occasionally the server program will time-out when its launched. Try launching again, or edit it and increase the timeout value.

 

Internet sensors projects

overview

A series of projects that use Internet API’s for interactive media projects.

Projects have been tested on Mac OS 10.9 (Mavericks). All but one of the projects use Max/MSP 6.1.7 from: http://cycling74.com. The other project uses Pure Data 0.42.5 (extended) from: http://puredata.info. Other required programs are listed in the documentation for individual projects.

download

internet-sensors is on Github at:  https://github.com/tkzic/internet-sensors

Each project is in a separate folder.

authorization

Some projects require passwords and API-keys from providers.

For example, for the ‘Twitter streaming API in Max’ project you’ll need to set up a Twitter application from your account to get authorization credentials.

For projects that need authorization usually you’ll just need to modify the patches/source code with your user information – as directed in the instructions.

help

The API’s used in the projects change fairly often. So there’s no guarantee they’ll work. If you find problems or have ideas – please post to them to the github repository. Or email me at tkzic@megalink.net.

projects

1. Twitter streaming API in Max (FM, php, curl, geocoding, [aka.speech], Soundflower (optional), Morse code, OSC, data recorder, Twitter v1.1 API, Twitter Apps, Oauth)

http://reactivemusic.net/?p=5786

2. Sending tweets from Max using curl ([sprintf], [aka.shell], xively.com API, zapier.com API, JSON, javascript Twitter v1.1 API, Oauth)

http://reactivemusic.net/?p=5447

 

3. Send and receive tweets in Max using ruby (ruby, API, JSON, javascript Twitter v1.1 API, OSC, Oauth)

http://reactivemusic.net/?p=7013

New! – use the project above to send tweets from using a Fisher Price “Little Tikes” piano: http://reactivemusic.net/?p=6993

4. Speech to text in Max (Google speech API, JSON, javascript, sox, Twitter v1.1 API, Oauth)

Note: Send Tweets using speech as well.

http://reactivemusic.net/?p=4690

5. A conversation with a robot in Max (Google speech API, sox, JSON,  pandorabots API, python, [aka.speech]

http://reactivemusic.net/?p=9834

7. Playing bird calls in Max (xeno-canto API, [jit.uldl], [jit.qt.movie])

http://reactivemusic.net/?p=4225

8. Soundcloud API in Max (JSON, javascript, curl, [aka.shell], [jit.qt.movie])

http://reactivemusic.net/?p=5413

9. Real time train map using Max and node.js (XML, JSON, OSC, data recorder, web sockets, Irish Rail API)

http://reactivemusic.net/?p=5477

10. stock market music in Max (OSC, netcat,  php, mysql, html, javascript, Yahoo API, linux)

http://reactivemusic.net/?p=12029

11. Using weather forecast data to drive weather sounds in Pure Data (ruby, OSC, JSON, openweathermap API, “Designing Sound” by Andy Farnell)

http://reactivemusic.net/?p=5846

12. Using ping times to control oscilators in Max (Mashape ping-uin API, ruby, OSC, JSON)

http://reactivemusic.net/?p=5945

13. Echonest Segment analysis player – sonification of audio analysis data from The Echo Nest (echonest API, ruby, Osc, Max/MSP)

http://reactivemusic.net/?p=6296

14. Quadcopter AR_drone – Fly a quadcopter using Max – with streaming Web video. ( node.js, AR_drone, Google Chrome, Osc, Max/MSP)

http://reactivemusic.net/?p=6635

15. Adding markers to Google Maps in Max – ( node.js, ruby, Google Chrome, Osc, Max/MSP, websockets, Google Maps API, Jquery, javascript)

http://reactivemusic.net/?p=11412

16. Max data recorder –  Record and play back streams of data simultaneously at various rates

http://reactivemusic.net/?p=8053

 

17. Web Audio version of Google domain ping machine –  The sound of an even newer machine using Web Audio API, web sockets. Max, Ruby, Osc, Google API, and Mashape

Note: This project is not in internet-sensors. You will find it on github in the tkzic/WebAudio project: https://github.com/tkzic/WebAudio

Instructions: http://reactivemusic.net/?p=6243

 

16. MBTA bus data in Max –  Sonification of Mass Ave buses, from Harvard to Dudley

http://reactivemusic.net/?p=17524

Screen Shot 2014-11-11 at 3.26.16 PM

 

Using wind forecast data to generate wind sounds with Pd

This project receives wind data from the U.S. Coastal Observing Research and Development Center (CORDC) at Scripps Institute of Oceanography  the openweathermap.org API

http://cordc.ucsd.edu/projects/models/coamps/api/

http://openweathermap.org/API

It uses the data to set the ‘wind speed’ of Andy Farnell’s wind sound patches from “Designing Sound”

http://aspress.co.uk/sd/index.php

Error 4/26/2014

Getting a parsing error on the ruby script. Will be debugging shortly!

<span style="color: rgb(0, 0, 0);">/Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `initialize': A JSON text must at least contain two octets! (JSON::ParserError)</span>
<span style="color: rgb(0, 0, 0);">from /Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `new'</span>
<span style="color: rgb(0, 0, 0);">from /Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `parse'</span>
<span style="color: rgb(0, 0, 0);">from ./wind-open-forecast.rb:92:in `&lt;main&gt;'</span>

———————————————–

The http: requests are managed by a ruby server script and sent via OSC to Pure Data.

Two of Andy Farnell’s patches are also running (with very slight mods): wind4a.pd, and thunder4a.pd

From pd, you can select which city to get data from, and control the rate at which the data is replayed from the ruby server. The forecast cycle is about 3 days long.

From the pd patch you can select the rate of data playback, and select which city to use for data (Santa Cruz or San Diego).

Note: the wind speed expected by Andy Farnell’s wind patch is between 0 and about 0.7

download

https://github.com/tkzic/internet-sensors

folder: pd-weather

files

main Pd patches
  • wind-open-machine.pd
  • thunder4a.pd
  • wind4a.pd
pd abstractions

designingSound/ folder:

Add this folder to the path list in Pd before the system files because the distance.pd abstraction has the same name as a system file used for map distances.

  • distance.pd
  • fcpan.pd
  • strike-pattern.pd
  • strike-sound.pd
  • udly.pd
ruby
  • wind-open-forecast.rb
required gems include:
  • osc-ruby
  • patron
  • json

authorization

  • none required

instructions

1. Open all three pd patches
  • wind-open-machine.pd
  • thunder4a.pd
  • wind4a.pd
2. Turn audio on
3. start the ruby script from a terminal window by typing

./wind-open-forecast.rb
(NOTE: Sometimes the ruby script will time-out when you first launch it. Just run it again.)

4. You can tweak various parameters to scale the effect of the wind data on the sounds

revision history

5/19/2014

currently getting this error in the ruby server – need to rewrite:

  • update 3-25-2013 If you are running this  project and getting errors from the ruby script –   CORDC is not producing wind data – so please use this workaround – which uses the openweathermap.org API

 

sending Tweets from Max using ruby, xively.com, and zapier.com

This is a ruby version of the Max tweetCurl5 patch (which tweets via xively.com) described here:

http://reactivemusic.net/?p=5447

In this version, the Max patch communicates via OSC to a background server running in ruby. An advantage of this method is that both the patch and the server are  compact and easy to understand. The Max patch does things in a Max way. And likewise with the ruby script.

Here’s a screen shot of the Max patch:

 files

Max

  • ruby-max-tweet.maxpat

ruby

  • ruby-max-tweet.rb

The ruby script requires installation of the following gems

  • patron
  • osc-ruby

For example:

# gem install patron

authorization

  • The xively.com feed id and api-key are embedded in ruby script
  • To get this project to work you’ll need a Twitter account. And you’ll need to set up a device (feed) at xively.com and a ‘zap’ at zapier.com as directed in this post. It explains how to send tweets using triggers. http://reactivemusic.net/?p=6903

instructions

  • Open the Max patch: ruby-max-tweet
  • In a terminal window run the ruby script:
  • # ./ruby-max-tweet.rb
  • In the Max patch, type in a tweet. Press the green button to send.
  • When you have Tweeted enough, end the ruby server program by typing <ctrl-c>

download

The files for this project can be downloaded from the intenet-sensors archive at github

https://github.com/tkzic/internet-sensors

Processing, Twitter, OSC, and Max

A variation on the Twitter mood-lamp program.

note 6/2014 – this may not work due to changes in oauth

local files:

  • Processing: /documents/processing/osc_max_testing
  • Max: tkzic/max teaching examples/processing-osc

It grabs a ‘feed’ or any URL which returns a bunch of text. Then it does some analysis on the text, and using the results to send RGB data back to Max using OSC.

Twitter streaming from php to Max

update 6/2014 – This project is part of the Internet sensors projects: http://reactivemusic.net/?p=5859. Check the link for current versions.

original post

notes

Got a test patch running today which breaks out tweets (in php and curl) and sends them to Max via Osc.

(update) Have parsed data to remove  hyperlinks and Twitter symbols.

It took some tweaking of global variables in php – and probably would be better written using classes (as in this example: http://stackoverflow.com/questions/1397234/php-curl-read-incrementally – see post from GZipp.

Max patch: tkzic/max teaching examples/twitter-php-streamer1.maxpat

php code: twitterStreamMax.php

<?php

// max-osc-play.php
//
//	collection of php OSC code from Max stock-market thing
//

include 'udp.php';		// udp data sending stuff

$DESTINATION = 'localhost';
$SENDPORT = '7400';
$RECVPORT = '7401';

//////////////////////////////////////////////////////////////////////////////////////////

	$USERNAME = 'username';
	$PASSWORD = 'password';
	$QUERY    = 'cats';		// the hashtag # is optional

	// these variables are defined as global so they can be used inside the write callback function
	global $osc;
	global $kount;

	// initialize OSC
	$osc = new OSCClient();  // OSC object
	$osc->set_destination($DESTINATION, $SENDPORT);

	// This amazing program uses curl to access the Twitter streaming API and breaks the data
	// into individual tweets which can be saved in a database, sent out via OSC, or whatever
	//

	/**
	 * Called every time a chunk of data is read, this will be a json encoded message
	 * 
	 * @param resource $handle The curl handle
	 * @param string   $data   The data chunk (json message)
	 */
	function writeCallback($handle, $data)
	{
	    /*
	    echo "-----------------------------------------------------------\n";
	    echo $data;
	    echo "-----------------------------------------------------------\n";
	    */

		$maxdata = "/tweet" ;				// header - begin   
		global $kount;					// test counter
		global $osc;						// osc object

	    $json = json_decode($data);
	    if (isset($json->user) && isset($json->text)) {

			// here we have a single tweet
	        echo "@{$json->user->screen_name}: {$json->text}\n\n";

			// do some cleaning up...
			// remove URL's
			$s = $json->text;		// raw tweet text

			// ok now need to do the same thing below for URL,s RT's @'s etc., 
			// and then remove redundant spaces	
			/* example
			Depending on how greedy you'd like to be, you could do something like:

			$pg_url = preg_replace("/[^a-zA-Z 0-9]+/", " ", $pg_url);

			This will replace anything that isn't a letter, number or space

			*/		

			// display all hashtags and their indices
			foreach( $json->entities->hashtags as $obj )
			{
			  echo "#:{$obj->text}\n";		// display hashtag
			  // get rid of the hashtag
			 	// note: this gets rid of all hashtags, which could obscure the meaning of the tweet, if
				// the hashtag is used inside a sentence like: "my #cat is purple" - would be changed to: "my is purple"
				// so we could use some intelligent parsing here...

			//  $s = str_replace("#{$obj->text}", "", $s );

			// this is a more benign approach, which leaves the word but removes the #

			$s = str_replace("#{$obj->text}", "{$obj->text}", $s );

			}

			foreach( $json->entities->urls as $obj )
			{
			  echo "U:{$obj->url}\n";		// display url			
			  $s = str_replace("{$obj->url}", "", $s );   // get rid of the url		
			}

			foreach( $json->entities->user_mentions as $obj )
			{
				echo "@:{$obj->screen_name}\n";		// display 			
				$s = str_replace("RT @{$obj->screen_name}:", "", $s );   // get rid of re-tweets
				$s = str_replace("@{$obj->screen_name}:", "", $s );   // get rid of other user mentions
				$s = str_replace("@{$obj->screen_name}", "", $s );   // get rid of other user mentions		
			}

			// $s = str_replace("RT ", "", $s );   // get rid of RT's (re-tweet indicators)

			// $s = preg_replace( '/[^[:print:]]/', '',$s); // remove non printable characters

			$s = htmlspecialchars_decode($s);		// decode stuff like &gt;

			$s = preg_replace('/[\x00-\x08\x0B\x0C\x0E-\x1F\x80-\x9F]/u', '', $s); // get rid of unicode junk

			$s = preg_replace('/[^(\x20-\x7F)]*/','', $s);		// get rid of other non printable stuff

			$s = preg_replace('!\s+!', ' ', $s);	// remove redundant white space

			echo "revised tweet: {$s}\n";

			$maxdata = "/tweet " . "{$json->text}";
			// $maxdata = $maxdata . " " . $kount++;
		   	$osc->send(new OSCMessage($maxdata));

	    }

	    return strlen($data);
	}

// initialize OSC 

// initialize curl

	$ch = curl_init();

	curl_setopt($ch, CURLOPT_URL, 'https://stream.twitter.com/1/statuses/filter.json?track=' . urlencode($QUERY));
	curl_setopt($ch, CURLOPT_USERPWD, "$USERNAME:$PASSWORD");
	curl_setopt($ch, CURLOPT_WRITEFUNCTION, 'writeCallback');
	curl_setopt($ch, CURLOPT_TIMEOUT, 20); // disconnect after 20 seconds for testing
	curl_setopt($ch, CURLOPT_VERBOSE, 1);  // debugging
	curl_setopt($ch, CURLOPT_ENCODING,  'gzip, deflate'); // req'd to get gzip
	curl_setopt($ch, CURLOPT_USERAGENT, 'tstreamer/1.0'); // req'd to get gzip

	curl_exec($ch); // commence streaming

	$info = curl_getinfo($ch);

	var_dump($info);

?>

Osc in php

The Osc code from the stock market music project http://reactivemusic.net/?p=12029 is not really doing Osc.

But… it works well going from php->max. In the other direction its using a kluge of nc and an alarm clock shell program – to receive messages from Max in UDP, but its really kind of horrible – so I’m going to look again for an OSC library in php.

update 2/2013 This is hard to believe, but I haven’t yet found a real OSC libraries for php. Apparently php is so uncool, that nobody wants to write for it anymore. Anyway, the code above, works unidirectionally, so its of some use for existing php code.

Local files are max-php-osc-tester.maxpat and max-osc-play.php in tkzic/api

Analysis that might help with parsing:

from Captain Caveman

http://forum.cockos.com/showthread.php?t=99076