Posts

Posted on

More conversations with robots in Max

Using Google speech API and Pandorabots API

(updated 1/21/2024)

all of these changes are local – for now.

replace path to sox with /opt/homebrew/bin/sox in [p call-google-speech]

Also had to write a new python script to convert xml to json. Its in the subfolder /xml2json/xml4json.py

The program came from this link: https://www.geeksforgeeks.org/python-xml-to-json/

Also inside [p call-pandorabots] the path for this python program had to be explicit to the full path on the computer. this will vary depending on your python installation.

Also, note that you must install a dependency with pip:

pip install xmltodict

After all that I was actually able to have a conversation. These bots seem primitive, but loveable, now compared to chatGPT. Guess its time for a new project.

Also the voice selection for speech synth is still not connected

(updated 1/21/2021)

This project is an extension to the speech-to-text project: https://reactivemusic.net/?p=4690 You might want to try running that project first to get the Google speech API running.

features

  • Everything runs in one Max patch
  • menu selection of chat bots and voices (currently disabled)
  • filtering of non speakable text (like HTML tags)
  • python script now runs under current directory of patch using relative path
  • refinements to recording and chatbot engines

download

https://github.com/tkzic/internet-sensors

folder: google-speech

files
main Max patch
  • robot-conversation7.maxpat
abstractions and other files
  • clean-html.js
  • xml2json/xml2json.py
  • JSON-google-speech.js
  • JSON-pandorabot.js
  • ms-counter.maxpat (timer for recording messages)
  • pandorabots.txt
Max external objects
external programs:

sox: sox audio conversion program must be in the computer’s executable file path, ie., /usr/bin – or you can rewrite the [sprintf] input to [aka.shell] with the actual path. In our case we installed sox using Macports. The executable path is /opt/local/bin/sox – which is built into a message object in the subpatcher [call-google-speech]

get sox from: http://sox.sourceforge.net

Instructions

  • Open robot-converstaion7.maxpat and turn on audio
  • select chatbot as destination
  • Press the spacebar to start recording.
  • Ask a question.
  • Press the spacebar to stop recording. 

notes

Need to fix the selection of voices.

revision history

  • 1/21/2021: complete rewrite for Max8 and Catalina
  • 4/24/2016: need to have explicit path to sox, in the call-google-speech subpatch. In my Macports version the path is /usr/local/opt/bin/sox.
  • 6/6/2014: re-added missing pandorabots.txt (list of chatbots) – also noticed that pandorabots.com was not available. May need to look for another site.
  • 5/11/2014: The newest version requires Max 6.1.7 (for JSON parsing). Also have updated to Google Speech API v2.
  • Note: Instructions for getting a real key from Google – which will need to be inserted into the patch.  http://www.chromium.org/developers/how-tos/api-keys – so far we have been getting by with common keys from a github site (see notes in next link)

Also please see these notes about how to modify the patch with your key – until this gets resolved: https://reactivemusic.net/?p=11035

Posted on

ep-4yy13 DSP – week 2

new composition tools

from various artists

tools that make tools

 

Posted on

Notes: Chatbots in Conversation

update 6/2014 – Now part of the Internet sensors projects: https://reactivemusic.net/?p=5859

original post

They can talk with each other… sort of.

Last spring I made a project that lets you talk with chatbots using speech recognition and synthesis. https://reactivemusic.net/?p=4710.

Yesterday I managed to get two instances of this program, running on two computers, using two chatbots,  to talk with each other, through the air. Technical issues remain (see below). But there were moments of real interaction.

In the original project, a human pressed button in Max to start and stop recording speech. This has been automated. The program detects and records speech, using audio level sensing. The auto-recording sensor turns on a switch when the level hits a threshold, and turns off after a period of silence. Threshold level and duration of silence can be adjusted by the user.  There is also a feedback gate that shuts off auto-record while the computer is converting speech to text, and ‘speaking’ a reply.

technical issues

  • The Google speech API has difficulty with some of the voices used by the Mac OS speech synthesizer. We’ll need to experiment to find which voices produce accurate results.
  • The overall levels produced by the builtin Macbook speakers is not quite enough to achieve clear communication. The auto-recorder missed the onset of speech sometimes. One solution would be to insert a click to trigger the recorder, just before the speech synthesizer begins the actual speech. Or to use external speakers, or a secondary “wired” connection.
  • It would be nice to have menus of chatbots and voices. Also to automate the start of a new conversation thread.
  • The button to start the audio detector had to be operated by key-press because pushing the trackpad on a MacBook makes too much noise and always triggers the audio level detector.
  • Occasionally a chat bot would deliver a long response, or one containing a web address. These were problematic for recognition and synthesis.

local files

  • tkzic/internetsensors/speech-to-google-text-api3.maxpat
  • tkzic/internetsensors/pandorabots-api2.maxpat
  • tkzic/internetsensors/text-to-speech3.maxpat