Tag: interactive media

Speech to text in Max

Using the Google speech API

This project demonstrates the Google speech-API. It records speech in Max, process it using the Google API, and displays the result in a Max [message] object.

Note: The sequence of events with shell calls is handled by arbitrary delay objects. This could be cleaned up by looking for some kind of termination code to indicate the shell command has completed.

download

https://github.com/tkzic/internet-sensors

folder: google-speech

files

main patch
  • speech-to-google-text-api5.maxpat
abstractions and other files
  • JSON-google-speech.js (parses JSON response from Google API)
  • autorecord-buffer2.maxpat
  • autorecord-switch.maxpat

external Max objects

external programs

sox: sox audio conversion program must be in the computer’s executable file path, ie., /usr/bin – or you can rewrite the [sprintf] input to [aka.shell] with the actual path

get sox from: http://sox.sourceforge.net

note: this conversion may not be necessary with recent updates to Max and the Google speech API

authorization

  • none required – so far
This may be changing.
Insert here: how to get a speech-api key from Google 

instructions

  • Open Max patch: speech-to-google-text-api5
  • Turn on audio
  • Click on the toggle (next to the level meters), say something, then un-click the toggle. The translation will begin automatically

Note: Instructions for auto-record feature are in progress, but simply use + and – keys to activate/deactivate auto record.

Note: If you have a slow internet connection you may need to increase the number in the [delay 4000] object – in “call google-speech” sub patch.

send Tweets using speech

Max [send] and [receive] objects pass data from this project to other projects that send Tweets from Max. Just run the patches at the same time.

Also, check out how this project is integrated into the Pandorabots chatbot API project

http://reactivemusic.net/?p=9834

Or anything else. The Google translation is amazingly accurate.

revision history

  • 4/24/2016: need to have explicit path to sox, in the call-google-speech subpatch. In my Macports version the path is /usr/local/opt/bin/sox.
  • 5/11/2014: The newest version requires Max 6.1.7 (for JSON parsing). Also have updated to Google Speech API v2.
  • update 3/26/2014 to use auto-record features developed for chatbot conversations

Using the Google speech API

Speech recognition:

http://fennb.com/fast-free-speech-recognition-using-googles-in

Here’s a link to the manual for sox (audio converter)

sox.soundfourge.net/sox.html

This is an example of the curl command to run from the command line

curl \
  --data-binary @test.flac \
  --header 'Content-type: audio/x-flac; rate=16000' \
  'https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&pfilter=2&lang=en-US&maxresults=6'

dictionaries

notes

 

 

 

 

 

random word generators

Arduino with touchOSC and Max

Bi-directional communication from touchOSC to Arduino using an ethernet shield.

In this version, the Macbook is directly connected to the Arduino to provide a serial monitor for status updates. 

How it works: press a toggle, or move a fader, in touchOSC – it sends a message to the Arduino which lights up, or fades, an LED – then sends back an OSC message to touchOSC to light up the toggle button. (note: local feedback should be off for the toggle button in touchOSC. This is the default)

Arduino circuit
  • Use an ethernet shield. 
  • Connect ethernet cable. (I am using a Netgear WNCE2001 ethernet to wiFi adapter)
  • LED is connected to pin 5 and ground. The shorter lead connects to ground.

download

https://github.com/tkzic/max-projects

folder: arduino-osc

files
  • Arduino sketch: OSC_ethernet_test1/
  • touchOSC screen: simple (default) uses /1/fader1 and /1/toggle1
  • Max patch: arduino-osc-ethernet1.maxpat
Arduino files and libraries

***update 1/20/2016 there is a new sketch that uses the OSCuino library from CNMAT instead of ardosc. The sketches should be interchangeable. https://github.com/CNMAT/OSC . The sketch is in a folder called: OSCuino_tz and is based on work by Trippylightning at: http://trippylighting.com/teensy-arduino-ect/touchosc-and-arduino-oscuino/

Copy the OSC_ethernet_test1/ folder to Documents/Arduino. This puts it in the Arduino sketchbook.

The sketch uses: #include <ArdOSC.h>

Download ArdOSC from: https://github.com/recotana/ArdOSC

  1. After downloading, copy the ArdOSC-master folder to /Documents/Arduino/Libraries
  2. Rename the folder to ArdOSC

This post was the key to figuring out how to make this work: http://arduino.cc/forum/index.php?topic=137549.0

Instructions
  1. Connect Arduino to Macbook via USB.
  2. Open the Arduino serial monitor to initialize the ethernet connection and display the IP address.
touchOSC
  1. In touchOSC or Max, set the target IP to the one just displayed in the Arduino serial monitor
  2. From touchOSC (or Max) send on port 8000, receive on port 9000.
  3. Use the default touchOSC layout (simple)
  4. Use /fader1 and /toggle1 to control the LED
Max
  1. Open arduino-osc-ethernet1.maxpat
  2. Set ip address in [udpsend] to the one just displayed in the Arduino serial monitor
  3. Have some fun
Fixed IP address

update 1/2016: A version of the Arduino sketch that uses a fixed IP instead of DHCP is located in the folder: OSC_ethernet_fixedIP/

The IP is set to 192.168.1.177 but you can change it to any valid address on your network.

Arduino sketch
// generic Arduino OSC program 
// works from Max or touchOSC
//
// plug LED into pin 5 (and gnd)
//
// requires ethernet shield
//
// use serial monitor to get the ip address
//
// use these OSC commands (will work from first page of touchOSC simple layout
//
// /1/fader1
// /1/toggle1
//
#include <SPI.h>
#include <Ethernet.h>
#include <ArdOSC.h>

byte mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0x0B, 0xCE }; //physical mac address
OSCServer server;
OSCClient client;
int serverPort = 8000; //Touch OSC Port (outgoing)
int destPort = 9000; //Touch OSC Port (incoming)
int ledPin = 5; 
int flag=0;
void setup(){
pinMode(2, OUTPUT);
 Serial.begin(9600); 
 Serial.println("DNS and DHCP-based OSC server");
 // start the Ethernet connection:
 if (Ethernet.begin(mac) == 0) {
 Serial.println("Failed to configure Ethernet using DHCP");
 // no point in carrying on, so do nothing forevermore:
 while(true);
 }
 // print your local IP address:
 Serial.print("Arduino IP address: ");
 for (byte thisByte = 0; thisByte < 4; thisByte++) {
 // print the value of each byte of the IP address:
 Serial.print(Ethernet.localIP()[thisByte], DEC);
 Serial.print("."); 
 }
 Serial.println();
 Serial.println();
//start the OSCserver
 server.begin(serverPort);
//add OSC callback function. One function is needed for every TouchOSC interface element that is to send/receive OSC commands.
 server.addCallback("/1/toggle1", &funcOnOff);
 server.addCallback("/1/fader1", &funcFader);
}
void loop(){
if(server.aviableCheck()>0){
 // Serial.println("alive! ");
 } 
}
//When the button on the TouchOSC inteface is pressed, a message is sent from the iDevice
//to the Arduino to switch (togle) the LED on the Arduino on/off
//then a messeage is sent bak from the Arduino to the iDevice to toggle the buttom on/off
void funcOnOff(OSCMessage *_mes){
 float value = _mes->getArgFloat(0); //TouchOSC expects float values
//create new osc message
 OSCMessage newMes;
//set destination ip address & port no
 newMes.setAddress(_mes->getIpAddress(),destPort);
 newMes.beginMessage("/1/toggle1");
Serial.println(value);
 if(value < 1.0) {
 digitalWrite(ledPin, LOW);
 }
 else{
 digitalWrite(ledPin, HIGH);
 }
newMes.addArgFloat(value);
//send osc message
 //
 // turn local feedback off on touch-osc control to test this
 client.send(&newMes);
}
// new callback for fader - using same comments
//When the button on the TouchOSC inteface is pressed, a message is sent from the iDevice
//to the Arduino to switch (togle) the LED on the Arduino on/off
//then a messeage is sent bak from the Arduino to the iDevice to toggle the buttom on/off
void funcFader(OSCMessage *_mes){
 float value = _mes->getArgFloat(0); //TouchOSC expects float values
//create new osc message
 OSCMessage newMes;
//set destination ip address & port no
 newMes.setAddress(_mes->getIpAddress(),destPort);
 newMes.beginMessage("/1/fader1");
Serial.println(value);
 int ledValue = value * 255.0;
 analogWrite(ledPin, ledValue);
newMes.addArgFloat(value);
//send osc message
 //
 // turn local feedback off on touch-osc control to test this
 client.send(&newMes);

}


 

 

Sending Tweets from Arduino through Pachube.com

http://www.tigoe.com/pcomp/code/arduinowiring/1135/#more-1135

from Tom Igoe

(update) I have got this working, exactly as described in the Igoe post – The code is in EthernetPachubeTweeter_tz1.

Essentially, anything that originates from the Arduino is sent to a feed in Pachube. That feed has a datastream which has a trigger which tweets any new data which arrives.

The next thing to try is figuring out whether this can be done as a single line http: request in curl, and therefore, from Max – or any other source

(update) – this is slightly broken – check out the post about converting cosm to xively http://reactivemusic.net/?p=6843]