curl examples

update 11/02/2014 – in process of updating and checking examples – but we know for sure that the Twitter examples are broken.

notes

This post is a collection of many examples of using curl (from a terminal command line) to make API requests. The ones marked with* will run without passwords or api-keys. The responses will generally be in XML, JSON, or HTML

Some of these curl examples will have API keys, user names, and passwords. To run them, you’ll need to replace the keys with your own.

Also, when running from the command line, ampersands ‘&’ will need to be escaped by preceding them with a backslash, like \&

Other characters may need to be escaped, like:

%23 = #

%40 = @

Or you can wrap the entire URL is quotes like this:

curl "dict://dict.org/d:penguin"
tips for finding curl examples

Look in the address bar of your web browser. Copy the entire URL and run it in curl. It will respond with the ‘text’ version of the web page.

displaying JSON on the command line

The respond from curl is raw data. To format JSON data, pipe the output through python, like this:

curl dict://dict.org/d:dog | python -mjson.tool

examples:

dict.org*
curl dict://dict.org/d:antelope
Twitter search (broken)

Twtitter API now requires authentication and will not run from a single curl command, except by using widgets. See this post: https://reactivemusic.net/?p=17428

openweathermap.com*
curl http://api.openweathermap.org/data/2.1/forecast/city/524901

See this post for details: https://reactivemusic.net/?p=6265

vine.com*
curl https://api.vineapp.com/timelines/popular
mashape.com API

Here’s an example of a typical API call which gets the weather in San Francisco. Note I have obscured the API-key.

curl -H "X-Mashape-Authorization: YOUR-MASHAPE-KEY" "https://george-vustrey-weather.p.mashape.com/api.php?_method=getForecasts&location=San%20Francisco"

Here’s the result:

[{"day_of_week":"Wed","high":"63","low":"47","condition":"Mostly Clear"},{"day_of_week":"Thu","high":"66","low":"50","condition":"Partly Cloudy"}]

[Note] more examples in tkzic/mashape

dictionaryapi.com

Here is an example of a request to the Thesaurus API

curl http://www.dictionaryapi.com/api/v1/references/thesaurus/xml/cheese?key=ee2466d2-07a0-40af-b959-abcdeb125f0ca

 

*pandorabots.com

Here are two examples I just got to work using curl

curl -X POST  --data "botid=b0dafd24ee35a477&input=hello" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Where are you?" http://www.pandorabots.com/pandora/talk-xml
Here is the result for the second question
<result status="0" botid="b0dafd24ee35a477" custid="b3422b612633ac87"><input>Where are you?</input><that>I am in the computer at Pandorabots.com.</that></result>
*more pandorabots.com
curl -X POST  --data "botid=b0dafd24ee35a477&input=hello" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Where are you?" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Lets play some music today." http://www.pandorabots.com/pandora/talk-xml
Google speech API (may be broken – need to check the key)

This is an example of the curl command to run from the command line

the test.flac file is located in tkzic/speech – it contains a recording of the phrase “there are turtles wandering through the city”. If you click on the file it will play through audacity.

curl \
  --data-binary @test.flac \
  --header 'Content-type: audio/x-flac; rate=16000' \
  'https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&pfilter=2&lang=en-US&maxresults=6'
cosm.com (may be broken)

Today I was finally able to get this working. Reading a Cosm (Pachube) feed from curl and from Max. Here is an example that works in curl: (replace API-KEY with actual key)

curl http://api.cosm.com/v2/feeds/76490/datastreams/Power.xml?key=API-KEY

You can get JSON responses by leaving off the .xml extension or replacing it with .json

Its critical to use “key=…” not “X-ApiKey=…” like in the cosm documentation, or you will get permission errors from curl and Max.

Youtube.com v3 API

Link to documentation: https://developers.google.com/youtube/v3/docs/search/list

Here is an example (minus the api-key) which searches for videos about cats and returns a JSON response

curl "https://www.googleapis.com/youtube/v3/search?key=API-KEY&part=snippet&q=cats"
google.com custom search API

Use this method to have a custom search which searches the whole web…

http://stackoverflow.com/questions/4082966/google-web-search-api-deprecated-what-now

Here is an example of a search using this method with curl – note the API-KEY is removed

curl "https://www.googleapis.com/customsearch/v1?key=API-KEY&cx=012117491442732664551:egvalbpelhq&q=lectures"
Twitter streaming API (broken)
 google.com simple search API*

Not really applicable to curl, but it shows something.

curl -A Mozilla http://www.google.com/search?q=linux
coastal wind data API (broken 11/2014)
with a specific lat/lon you can get results along the west coast of us - here's san francisco. This query returns all the current time points for SF

curl http://cordc.ucsd.edu/js/COAMPS/query.php?ll=37.78,-122.44\&fmt=json > sf.json 

lets try for santa cruz

curl http://cordc.ucsd.edu/js/COAMPS/query.php?ll=36.99,-122.03\&fmt=json > sc.json

 

spellout.org API*
very cool api that does phonetic spellings - works!!!!

curl http://api.spellout.org/en/convert?text=some%20text\&alphabet=int-icao\&format=json
cosm.com API (PUT request)
This works as of 1/28/2013

curl -v --request PUT --data-binary @./abc.json --header "X-ApiKey: 96PqSh4rj7HzNif3rWms3SUhwaDFGUT0g" http://api.cosm.com/v2/feeds/98281.json

 

Internet sensors projects

overview

A series of projects that use Internet API’s for interactive media projects.

updated 2/14/2021.

Projects have been tested on Max8 and Mac OS Catalina – except where noted.  Other dependencies are are listed on individual project pages.

My goal is to show a variety of methods to get data to and from Max. API’s come and go, as do the libraries that support them.

download

internet-sensors is on Github at:  https://github.com/tkzic/internet-sensors

Each project is in a separate folder.

authorization

Some projects require passwords and API-keys from providers.

For example, for the ‘Twitter streaming API in Max’ project you’ll need to set up a Twitter application from your account to get authorization credentials.

For projects that need authorization usually you’ll just need to modify the patches/source code with your user information – as directed in the instructions.  The API keys embedded in the code will not work unless specifically mentioned, like with the Google speech API.

help

API’s used in the projects change fairly often. So there’s no guarantee they’ll work. If you find problems or have ideas – please post to them to the github repository. Or email me at [email protected].

projects

1. Twitter streaming API in Max (FM, php, curl, geocoding, [aka.speech], Soundflower (optional), Morse code, OSC, data recorder, Twitter v1.1 API, Twitter Apps, Oauth)

https://reactivemusic.net/?p=5786

2. Sending tweets from Max using curl ([sprintf], [aka.shell], xively.com API, zapier.com API, JSON, javascript Twitter v1.1 API, Oauth)

deprecated 2/11/2021 – old project link here: https://reactivemusic.net/?p=5447

3. Send and receive tweets in Max using ruby (ruby, API, JSON, javascript Twitter v1.1 API, OSC, Oauth)

New! – use the project above to send tweets from using a Fisher Price “Little Tikes” piano: https://reactivemusic.net/?p=6993

4. Speech to text in Max (Google speech API, JSON, javascript, sox, Twitter v1.1 API, Oauth)

Note: Send Tweets using speech as well.

https://reactivemusic.net/?p=4690

5. A conversation with a robot in Max (Google speech API, sox, JSON,  pandorabots API, python, [aka.speech]

https://reactivemusic.net/?p=9834

7. Playing bird calls in Max (xeno-canto API, [jit.uldl], [jit.qt.movie])

https://reactivemusic.net/?p=4225

8. Soundcloud API in Max (node.js)

https://reactivemusic.net/?p=20120

 

9. Real time train map using Max and node.js (XML, JSON, OSC, data recorder, web sockets, Irish Rail API)

https://reactivemusic.net/?p=5477

10. stock market music in Max (OSC, netcat,  php, mysql, html, javascript, Yahoo API, linux)

…updates in progress…

https://reactivemusic.net/?p=12029

11. Using weather forecast data to drive weather sounds in Pure Data (ruby, OSC, JSON, openweathermap API, “Designing Sound” by Andy Farnell)

https://reactivemusic.net/?p=5846

… updates in progress…

12. Using ping times to control oscilators in Max (Mashape ping-uin API, ruby, OSC, JSON)

https://reactivemusic.net/?p=5945

13. Spotify Segment analysis player – sonification of audio analysis data from Spotify (echo nest) API, node,  Max/MSP)

https://reactivemusic.net/?p=20096

14. Quadcopter AR_drone – Fly a quadcopter using Max – with streaming Web video. ( node.js, AR_drone, Google Chrome, Osc, Max/MSP)

deprecated 2/14/2021 – old project link: https://reactivemusic.net/?p=6635

15. Adding markers to Google Maps in Max – ( node.js, ruby, Google Chrome, Osc, Max/MSP, websockets, Google Maps API, Jquery, javascript)

deprecated 2/14/2021 – old project link: https://reactivemusic.net/?p=11412

16. Max data recorder –  Record and play back streams of data simultaneously at various rates

https://reactivemusic.net/?p=8053

17. MBTA bus data in Max –  Sonification of Mass Ave buses, from Harvard to Dudley

… updates in progress…

https://reactivemusic.net/?p=17524

Screen Shot 2014-11-11 at 3.26.16 PM

 

Using wind forecast data to generate wind sounds with Pd

This project receives wind data from the U.S. Coastal Observing Research and Development Center (CORDC) at Scripps Institute of Oceanography  the openweathermap.org API

http://cordc.ucsd.edu/projects/models/coamps/api/

http://openweathermap.org/API

It uses the data to set the ‘wind speed’ of Andy Farnell’s wind sound patches from “Designing Sound”

http://aspress.co.uk/sd/index.php

Error 4/26/2014

Getting a parsing error on the ruby script. Will be debugging shortly!

<span style="color: rgb(0, 0, 0);">/Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `initialize': A JSON text must at least contain two octets! (JSON::ParserError)</span>
<span style="color: rgb(0, 0, 0);">from /Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `new'</span>
<span style="color: rgb(0, 0, 0);">from /Users/tkzic/.rvm/gems/ruby-2.0.0-p353@global/gems/json-1.8.1/lib/json/common.rb:155:in `parse'</span>
<span style="color: rgb(0, 0, 0);">from ./wind-open-forecast.rb:92:in `&lt;main&gt;'</span>

———————————————–

The http: requests are managed by a ruby server script and sent via OSC to Pure Data.

Two of Andy Farnell’s patches are also running (with very slight mods): wind4a.pd, and thunder4a.pd

From pd, you can select which city to get data from, and control the rate at which the data is replayed from the ruby server. The forecast cycle is about 3 days long.

From the pd patch you can select the rate of data playback, and select which city to use for data (Santa Cruz or San Diego).

Note: the wind speed expected by Andy Farnell’s wind patch is between 0 and about 0.7

download

https://github.com/tkzic/internet-sensors

folder: pd-weather

files

main Pd patches
  • wind-open-machine.pd
  • thunder4a.pd
  • wind4a.pd
pd abstractions

designingSound/ folder:

Add this folder to the path list in Pd before the system files because the distance.pd abstraction has the same name as a system file used for map distances.

  • distance.pd
  • fcpan.pd
  • strike-pattern.pd
  • strike-sound.pd
  • udly.pd
ruby
  • wind-open-forecast.rb
required gems include:
  • osc-ruby
  • patron
  • json

authorization

  • none required

instructions

1. Open all three pd patches
  • wind-open-machine.pd
  • thunder4a.pd
  • wind4a.pd
2. Turn audio on
3. start the ruby script from a terminal window by typing

./wind-open-forecast.rb
(NOTE: Sometimes the ruby script will time-out when you first launch it. Just run it again.)

4. You can tweak various parameters to scale the effect of the wind data on the sounds

revision history

5/19/2014

currently getting this error in the ruby server – need to rewrite:

  • update 3-25-2013 If you are running this  project and getting errors from the ruby script –   CORDC is not producing wind data – so please use this workaround – which uses the openweathermap.org API

 

sending Tweets from Max using ruby, xively.com, and zapier.com

This is a ruby version of the Max tweetCurl5 patch (which tweets via xively.com) described here:

https://reactivemusic.net/?p=5447

In this version, the Max patch communicates via OSC to a background server running in ruby. An advantage of this method is that both the patch and the server are  compact and easy to understand. The Max patch does things in a Max way. And likewise with the ruby script.

Here’s a screen shot of the Max patch:

 files

Max

  • ruby-max-tweet.maxpat

ruby

  • ruby-max-tweet.rb

The ruby script requires installation of the following gems

  • patron
  • osc-ruby

For example:

# gem install patron

authorization

  • The xively.com feed id and api-key are embedded in ruby script
  • To get this project to work you’ll need a Twitter account. And you’ll need to set up a device (feed) at xively.com and a ‘zap’ at zapier.com as directed in this post. It explains how to send tweets using triggers. https://reactivemusic.net/?p=6903

instructions

  • Open the Max patch: ruby-max-tweet
  • In a terminal window run the ruby script:
  • # ./ruby-max-tweet.rb
  • In the Max patch, type in a tweet. Press the green button to send.
  • When you have Tweeted enough, end the ruby server program by typing <ctrl-c>

download

The files for this project can be downloaded from the intenet-sensors archive at github

https://github.com/tkzic/internet-sensors

Twitter streaming API in Max

World map and radio simulation

note August 3, 2022 –

This program is broken on Mac OS Monterey. The PHP code is throwing errors in the OSC library.  I’m not certain there is a reasonable workaround at this point and will be looking at replacing the php code with node.js or another more reliable platform. Also, as noted below – php is no longer installed in Mac os – so it requires homebrew or macports

features

  • Twitter streaming v1.1 API and Twitter Apps (using http requests and Oauth in php)
  • lat/lon conversion and map plotting in Max
  • sending data to Max using OSC in php
  • ‘speaking’ tweets using several voices (text to speech)
  • Using geo-coordinates to control an FM synthesizer
  • Converting Tweet text to Morse code
  • Using a data recorder to replay/save data streams (Max lists)

Compare to satellite photo of earth – note the pattern of lights.

download

https://github.com/tkzic/internet-sensors

folder: twitter-stream

files

Max
  • world3.maxpat (main patch)
  • data_recorder_list-tz.maxpat (abstraction for recording data)
  • data-recorder-wrapper.maxpat (abstraction for recording data)
  • worldMap.jpg
  • twitter-morning.txt (sample data – not required)
php
  • ctwitter_max3.php (main program)
  • ctwitter_stream_max3.php (twitter engine)
  • udp.php (Osc client)

3 August, 2022

(note: starting with mac os monterey, php is no longer included in mac os. You can install it with homebrew. See this post: https://www.ergonis.com/products/tips/install-php-on-macos.php

externals

[note]  The project displays Tweets without these externals, but you won’t hear any speech

authorization

In addition to having a Twitter account, you will need to set up a Twitter application from the developer site here:

https://dev.twitter.com/apps

Good instructions on how to do this can be found in this stackoverflow.com post under this heading: So you want to use the Twitter v1.1 API?

http://stackoverflow.com/questions/12916539/simplest-php-example-for-retrieving-user-timeline-with-twitter-api-version-1-1

When you get to step 5 – in the instructions – instead of writing your own code, just use a text editor to copy your access tokens into this php program which is provided:

  • ctwitter_max3.php

Replace the strings in this line of code by copying and pasting the appropriate ones from your Twitter application:

$t->login('consumer_key', 'consumer secret', 'access token', 'access secret');

 

So it will end up looking something like this:

$t->login('ZdzfNaeflihFydfOHeOA', 'eXzUOfhif4riifgRbCTnnSN0T7neYtg8dIWDC7j3bs', '205589709-5kRI1fllJvU94jjffeerSn9LrTajtxSrvO8', 'u5MuSxPseBemUIBWlMxEFaw899feedXA0eHlReCnQ');

Yeah – its cryptic…

instructions

1. open the Max Patch: world3.maxpat

2. in a terminal window run the php program: ctwitter_max3.php. [note] it runs forever. Press <ctrl-c> when you want to stop streaming Tweets.

php ./ctwitter_max3.php

3. Switch back to world3.maxpat to see dots populating the map

4. In Max, press the speaker icon (lower left) to turn on audio.

5. Activate  voice synth/morse code using the blue toggle (lower left)

6. Clear the map by pressing the blue message box: “clear, drawpict a 0 0”

7. Stop the Tweet stream by pressing <ctrl-c> in the terminal window

special voice fx

If you have Soundflower installed, the Mac OS speech synth output can be routed back to Max for audio processing. This is somewhat complicated, but shows how to process audio in Max from other sources.

  • In MacOS System Preferences, set audio output device to Soundflower 2ch
  • Turn up hardware volume control on your computer
  • In Max, Options | Audio Status, set input device to Soundflower 2ch
  • In world3.maxpat double click on [p audio engine] (lower left). Then in the audio-engine sub-patch activate the toggle, (lower right) for voice-fx

data recording

The built-in data recorder/playback is on the left side of world3.maxpat:

  • toggle ‘record’ (red toggle)  to start or stop data recording
  • Note that data will only be recorded when the php program is streaming Tweets in the terminal window (see above)
  • Press /play message or other transport controls to replay data
<span style="font-family: 'Helvetica Neue', Helvetica, Helvetica, Arial, sans-serif; font-size: 23px; font-weight: bold; line-height: 1.1;">
</span>
<span style="font-family: 'Helvetica Neue', Helvetica, Helvetica, Arial, sans-serif; font-size: 23px; font-weight: bold; line-height: 1.1;">revision history</span>
<span style="font-family: 'Helvetica Neue', Helvetica, Helvetica, Arial, sans-serif; font-size: 23px; font-weight: bold; line-height: 1.1;">
</span>

revision history

1/19/2021

Updates for Max8 and Catalina:

Replaced [aka.speech] external with Jeremy Bernstein’s [shell] external and the Mac OS command line ‘say’ command.

Reinstalled Java Development Kit for [mxj] object

I revised the php code for the Twitter streaming project, to use the coordinates of a corner of the city polygon bounding box. That seems to be more reliable than the geo coordinates which are absent from most Tweets.

  • updated 3/26/2014 – fixed runtime error in php server
  • updated 2/2/2014 – simplified user interface and updated audio engine
  • updated 9/2/2013 for Twitter v1.1 API with Oauth – note that older versions of this project are broken due to discontinued Twitter v1.0 API as of June 2013

Twitter search in Max

Here’s a basic pattern:

  1. Send search string to Twitter
  2. Get JSON result
  3. Parse result and do something with it.

From Max, there are many ways to do this:

searchtweet – mxj, twitter4j

by Gokce Kinayoglu

http://cycling74.com/toolbox/searchtweet-design-patches-that-respond-to-twitter-posts/

This patch demonstrates the Twitter search API. Its self contained within Max – using the [mxj search tweet] external. This object allows you to input

  •  keyword
  • maximum number of results

The response is:

  • username
  • Tweet text
  • date/time

This patch is a great way to get Tweets into Max. You can use a [metro] object to poll the API. There are no additional programs running outside of Max. 

The limitation is lack of flexibility. You don’t have access to any of the other parameters in the request response. For example, geographic data. Also, it can be difficult to install and maintain java [mxj] programs in Max.

Here’s a screenshot of a patch which takes the output of the above patch and sends it to the aka.speech object – which runs the Mac Os built in text-to-speech program

Thoughts on API’s and Max

After looking at hundreds of API’s over past months – models begin to emerge:

  • Visualization: Looking at data by filtering, analysis, or factors defining movement.
  • Synthesis: producing  feeds from sources, in combination – fusion
  • Transcoding: changing one type of signal into another

Or some combination of all three.

The best tools for getting data in and out of Max:

  • curl (or variants in client libraries)
  • JSON
  • Osc
  • string parsing outside of Max
  • database tools (or data recorder in Max)
  • basic data filtering and scaling tools in Max
  • for complex networked systems: node.js

 

Mashape API’s

https://www.mashape.com

notes