Tag: web development

curl examples

update 11/02/2014 – in process of updating and checking examples – but we know for sure that the Twitter examples are broken.

notes

This post is a collection of many examples of using curl (from a terminal command line) to make API requests. The ones marked with* will run without passwords or api-keys. The responses will generally be in XML, JSON, or HTML

Some of these curl examples will have API keys, user names, and passwords. To run them, you’ll need to replace the keys with your own.

Also, when running from the command line, ampersands ‘&’ will need to be escaped by preceding them with a backslash, like \&

Other characters may need to be escaped, like:

%23 = #

%40 = @

Or you can wrap the entire URL is quotes like this:

curl "dict://dict.org/d:penguin"
tips for finding curl examples

Look in the address bar of your web browser. Copy the entire URL and run it in curl. It will respond with the ‘text’ version of the web page.

displaying JSON on the command line

The respond from curl is raw data. To format JSON data, pipe the output through python, like this:

curl dict://dict.org/d:dog | python -mjson.tool

examples:

dict.org*
curl dict://dict.org/d:antelope
Twitter search (broken)

Twtitter API now requires authentication and will not run from a single curl command, except by using widgets. See this post: http://reactivemusic.net/?p=17428

openweathermap.com*
curl http://api.openweathermap.org/data/2.1/forecast/city/524901

See this post for details: http://reactivemusic.net/?p=6265

vine.com*
curl https://api.vineapp.com/timelines/popular
mashape.com API

Here’s an example of a typical API call which gets the weather in San Francisco. Note I have obscured the API-key.

curl -H "X-Mashape-Authorization: YOUR-MASHAPE-KEY" "https://george-vustrey-weather.p.mashape.com/api.php?_method=getForecasts&location=San%20Francisco"

Here’s the result:

[{"day_of_week":"Wed","high":"63","low":"47","condition":"Mostly Clear"},{"day_of_week":"Thu","high":"66","low":"50","condition":"Partly Cloudy"}]

[Note] more examples in tkzic/mashape

dictionaryapi.com

Here is an example of a request to the Thesaurus API

curl http://www.dictionaryapi.com/api/v1/references/thesaurus/xml/cheese?key=ee2466d2-07a0-40af-b959-abcdeb125f0ca

 

*pandorabots.com

Here are two examples I just got to work using curl

curl -X POST  --data "botid=b0dafd24ee35a477&input=hello" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Where are you?" http://www.pandorabots.com/pandora/talk-xml
Here is the result for the second question
<result status="0" botid="b0dafd24ee35a477" custid="b3422b612633ac87"><input>Where are you?</input><that>I am in the computer at Pandorabots.com.</that></result>
*more pandorabots.com
curl -X POST  --data "botid=b0dafd24ee35a477&input=hello" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Where are you?" http://www.pandorabots.com/pandora/talk-xml

curl -X POST  --data "botid=b0dafd24ee35a477&input=Lets play some music today." http://www.pandorabots.com/pandora/talk-xml
Google speech API (may be broken – need to check the key)

This is an example of the curl command to run from the command line

the test.flac file is located in tkzic/speech – it contains a recording of the phrase “there are turtles wandering through the city”. If you click on the file it will play through audacity.

curl \
  --data-binary @test.flac \
  --header 'Content-type: audio/x-flac; rate=16000' \
  'https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&pfilter=2&lang=en-US&maxresults=6'
cosm.com (may be broken)

Today I was finally able to get this working. Reading a Cosm (Pachube) feed from curl and from Max. Here is an example that works in curl: (replace API-KEY with actual key)

curl http://api.cosm.com/v2/feeds/76490/datastreams/Power.xml?key=API-KEY

You can get JSON responses by leaving off the .xml extension or replacing it with .json

Its critical to use “key=…” not “X-ApiKey=…” like in the cosm documentation, or you will get permission errors from curl and Max.

Youtube.com v3 API

Link to documentation: https://developers.google.com/youtube/v3/docs/search/list

Here is an example (minus the api-key) which searches for videos about cats and returns a JSON response

curl "https://www.googleapis.com/youtube/v3/search?key=API-KEY&part=snippet&q=cats"
google.com custom search API

Use this method to have a custom search which searches the whole web…

http://stackoverflow.com/questions/4082966/google-web-search-api-deprecated-what-now

Here is an example of a search using this method with curl – note the API-KEY is removed

curl "https://www.googleapis.com/customsearch/v1?key=API-KEY&cx=012117491442732664551:egvalbpelhq&q=lectures"
Twitter streaming API (broken)
 google.com simple search API*

Not really applicable to curl, but it shows something.

curl -A Mozilla http://www.google.com/search?q=linux
coastal wind data API (broken 11/2014)
with a specific lat/lon you can get results along the west coast of us - here's san francisco. This query returns all the current time points for SF

curl http://cordc.ucsd.edu/js/COAMPS/query.php?ll=37.78,-122.44\&fmt=json > sf.json 

lets try for santa cruz

curl http://cordc.ucsd.edu/js/COAMPS/query.php?ll=36.99,-122.03\&fmt=json > sc.json

 

spellout.org API*
very cool api that does phonetic spellings - works!!!!

curl http://api.spellout.org/en/convert?text=some%20text\&alphabet=int-icao\&format=json
cosm.com API (PUT request)
This works as of 1/28/2013

curl -v --request PUT --data-binary @./abc.json --header "X-ApiKey: 96PqSh4rj7HzNif3rWms3SUhwaDFGUT0g" http://api.cosm.com/v2/feeds/98281.json

 

Internet sensors projects

overview

A series of projects that use Internet API’s for interactive media projects.

Projects have been tested on Mac OS 10.9 (Mavericks). All but one of the projects use Max/MSP 6.1.7 from: http://cycling74.com. The other project uses Pure Data 0.42.5 (extended) from: http://puredata.info. Other required programs are listed in the documentation for individual projects.

download

internet-sensors is on Github at:  https://github.com/tkzic/internet-sensors

Each project is in a separate folder.

authorization

Some projects require passwords and API-keys from providers.

For example, for the ‘Twitter streaming API in Max’ project you’ll need to set up a Twitter application from your account to get authorization credentials.

For projects that need authorization usually you’ll just need to modify the patches/source code with your user information – as directed in the instructions.

help

The API’s used in the projects change fairly often. So there’s no guarantee they’ll work. If you find problems or have ideas – please post to them to the github repository. Or email me at tkzic@megalink.net.

projects

1. Twitter streaming API in Max (FM, php, curl, geocoding, [aka.speech], Soundflower (optional), Morse code, OSC, data recorder, Twitter v1.1 API, Twitter Apps, Oauth)

http://reactivemusic.net/?p=5786

2. Sending tweets from Max using curl ([sprintf], [aka.shell], xively.com API, zapier.com API, JSON, javascript Twitter v1.1 API, Oauth)

http://reactivemusic.net/?p=5447

 

3. Send and receive tweets in Max using ruby (ruby, API, JSON, javascript Twitter v1.1 API, OSC, Oauth)

http://reactivemusic.net/?p=7013

New! – use the project above to send tweets from using a Fisher Price “Little Tikes” piano: http://reactivemusic.net/?p=6993

4. Speech to text in Max (Google speech API, JSON, javascript, sox, Twitter v1.1 API, Oauth)

Note: Send Tweets using speech as well.

http://reactivemusic.net/?p=4690

5. A conversation with a robot in Max (Google speech API, sox, JSON,  pandorabots API, python, [aka.speech]

http://reactivemusic.net/?p=9834

7. Playing bird calls in Max (xeno-canto API, [jit.uldl], [jit.qt.movie])

http://reactivemusic.net/?p=4225

8. Soundcloud API in Max (JSON, javascript, curl, [aka.shell], [jit.qt.movie])

http://reactivemusic.net/?p=5413

9. Real time train map using Max and node.js (XML, JSON, OSC, data recorder, web sockets, Irish Rail API)

http://reactivemusic.net/?p=5477

10. stock market music in Max (OSC, netcat,  php, mysql, html, javascript, Yahoo API, linux)

http://reactivemusic.net/?p=12029

11. Using weather forecast data to drive weather sounds in Pure Data (ruby, OSC, JSON, openweathermap API, “Designing Sound” by Andy Farnell)

http://reactivemusic.net/?p=5846

12. Using ping times to control oscilators in Max (Mashape ping-uin API, ruby, OSC, JSON)

http://reactivemusic.net/?p=5945

13. Echonest Segment analysis player – sonification of audio analysis data from The Echo Nest (echonest API, ruby, Osc, Max/MSP)

http://reactivemusic.net/?p=6296

14. Quadcopter AR_drone – Fly a quadcopter using Max – with streaming Web video. ( node.js, AR_drone, Google Chrome, Osc, Max/MSP)

http://reactivemusic.net/?p=6635

15. Adding markers to Google Maps in Max – ( node.js, ruby, Google Chrome, Osc, Max/MSP, websockets, Google Maps API, Jquery, javascript)

http://reactivemusic.net/?p=11412

16. Max data recorder –  Record and play back streams of data simultaneously at various rates

http://reactivemusic.net/?p=8053

 

17. Web Audio version of Google domain ping machine –  The sound of an even newer machine using Web Audio API, web sockets. Max, Ruby, Osc, Google API, and Mashape

Note: This project is not in internet-sensors. You will find it on github in the tkzic/WebAudio project: https://github.com/tkzic/WebAudio

Instructions: http://reactivemusic.net/?p=6243

 

16. MBTA bus data in Max –  Sonification of Mass Ave buses, from Harvard to Dudley

http://reactivemusic.net/?p=17524

Screen Shot 2014-11-11 at 3.26.16 PM

 

Thoughts on API’s and Max

After looking at hundreds of API’s over past months – models begin to emerge:

  • Visualization: Looking at data by filtering, analysis, or factors defining movement.
  • Synthesis: producing  feeds from sources, in combination – fusion
  • Transcoding: changing one type of signal into another

Or some combination of all three.

The best tools for getting data in and out of Max:

  • curl (or variants in client libraries)
  • JSON
  • Osc
  • string parsing outside of Max
  • database tools (or data recorder in Max)
  • basic data filtering and scaling tools in Max
  • for complex networked systems: node.js

 

Mashape

An API factory.

https://www.mashape.com

Happened to look at mashape.com again today. Its grown considerably over the past year and now provides access to over 1300 API’s with open source client libraries and Oauth proxies.

Mashape is an API of API’s – a layer of abstraction that standardizes how you consume API’s.

Instructions for using the API: https://www.mashape.com/docs/consume/rest

Here’s an example of a typical API call which gets the weather in San Francisco. Note I have obscured the API-key.

curl -H "X-Mashape-Authorization: YOUR-MASHAPE-KEY" "https://george-vustrey-weather.p.mashape.com/api.php?_method=getForecasts&location=San%20Francisco"

Here’s the result:

[{"day_of_week":"Wed","high":"63","low":"47","condition":"Mostly Clear"},{"day_of_week":"Thu","high":"66","low":"50","condition":"Partly Cloudy"}]

[Note] examples in tkzic/mashape

The API description pages have curl examples.

 

Parsing Tweets

A method using regexp and php. Actually what this does is parse Tweets using regexp to reformat the text as html with links. A tutorial here:

http://saturnboy.com/2010/02/parsing-twitter-with-regexp/

This is a php library that breaks out hashtags, usernames, etc., but doesn’t really provide a way to isolate the remaining stuff. I have put it in tkzic/API –  there is an example php program provided.

https://github.com/mzsanford/twitter-text-php

hashtags – using regular expressions

http://stackoverflow.com/questions/11551065/parsing-tweets-to-extract-hashtags-in-r

twitter-text-rb : ruby gem which parses out usernames and hashtags

https://github.com/twitter/twitter-text-rb

 

Netgear Universal Wifi Adapter WNCE2001

This device works as a bridge to allow devices that have ethernet but not wifi, to become wifi devices.

To access the config page, hook up an ethernet cable, and goto: www.mywifiext.com

Here’s more documentation:

http://kb.netgear.com/app/answers/detail/a_id/14157/~/wnce2001-faqs

This device allows wifi access using the Arduino Ethernet Shield. This is currently the easiest solution for using touch-OSC.