Category: Max/MSP

Posted on

rtl-sdr: Pd, Max and Mac

Notes on compiling rtl_sdr in Mac OS – writing Max and Pd Externals.

 

update 3/31/2014

Today I got the Pd external running – using essentially same source code as Max. There is occasional weirdness going on with audio clicks when starting/stopping the radio, but other than that it seems fine and it runs. wooHOO. More to follow…

 

update 3/28/2014

Now have set up a skeleton for Pd, called rtlfmz~ (inside the Pd application bundle) which does absolutely nothing but compiles all of the project files and calls a function in rtl_fm.  Next step will be to port the actual Max external code and do the conversion.

updates 3/38/2014 before working on Pd version

There is now a fairly solid max external (rtlfmz~) using a recent version for rtl_fm. Also there is a simple Makefile that compiles local version of rtl_fm3.c in:

tkzic/rtl-sdr-new/rtl-sdr/rtl-fm3

There are very minor changes to rtl_fm.c (for includes) and also a local version of librtlsdr.a (librtlsdr32.a) that is 32 bit.

The current state of the Max external does both pre-demodulated and raw IQ output, but you can only run one copy  of the object due to excess use of global variables and my uncertainty over how to run multiple devices, threading, etc.,  – but we’ll go with it an try porting to Pd now.

update 2/27 – converted to using new version of rtl_fm

I had been using an older version of rtl_fm –

renamed external to rtlfmz~ and now using new version as (rtl_fm3.c) in the project

In addition to recopying the librtlsdr.a – I also recopied all of the include files and added two new files

convenience.c convenience.h

There is a different method of threading and reading which I haven’t looked at yet, but it is now doing what it did before inside Max, which is read FM for a few seconds and write audio data to a file

plan: Set up a circular buffer accessible to the output thread and to the max perform function. – then see if we can get it to run for a few seconds.

The main thing to think about is how to let the processing happen in another thread while returning control to max. Then there really should be a way to interrupt processing from max.

There needs to be a max instance variable that tells whether the radio is running or not. Then, when its time to stop – you just do all the cleanup stuff that is at the end of the rtl_fm main() function.

 

update 2/18/2014 – Max external

Now have rtl_fm function within the plussztz~ external

It detects, opens device, demodulates about 30 seconds of FM, and writes audio at 44.1kHz. to a file /tmp/radio.bin – which can be played by the play command

Changes to code included:

  • removing exit calls
  • changing printf to post()
  • removing signal interrupt handling
remaining to do:
  • Need a way to get the device to read in the background – so its not blocking the Max thread
  • Need a way to stop/restrart the device – because if we aren’t, the continual sync_reads will waste a lot of cpu cycles.

update 2/15/2014 – Max external

Have now successfully  compiled a test external in Max 6.1.4 – name is plussztz~ and it includes the rtl_fm code.  Made 2 changes so far:

  • in build settings, set architectures to i386
  • in rtl_fm2, changed <include> libusb.h to “include” libusb.h

original post

Today I was able to write a simple makefile to compile the rtl_fm app using the libusb and librtlsdr dynamic libraries.

Pd requires i386 architecture for externals (i386) so I then compiled the app using static libraries and the i386 architecture.

libusb-1.0 already had a 32 bit version in /usr/local/lib/libusb32-1.0.a (note that this version also requires compiling these frameworks:

-framework foundation -framework iokit

For librtlsdr, I rebuilt, using cmake with the following flags:

cmake ../ -DCMAKE_OSX_ARCHITECTURES=i386 -DINSTALL_UDEV_RULES=ON

But did not install the result. See this link for details on building with cmake: http://sdr.osmocom.org/trac/wiki/rtl-sdr

This produced a 32 bit static version of librtlsdr.a that could be used for building the app.

See this Stack Overflow post for more on cmake and architectures: http://stackoverflow.com/questions/5334095/cmake-multiarchitecture-compilation

local files:

currently local version of this test is in: tkzic/rtl-sdr-tz/rtl_fm2/

The default makefile builds the 32 bit architecture.

Next:

  • try to move the makefile into Xcode
  • try compiling rtl_sdr or rtl_fm as a simple max object – the fm app might be better to start with since it gives an audio signal output.
  • then try in pd

 

 

 

Posted on

More conversations with robots in Max

Using Google speech API and Pandorabots API

(updated 1/21/2024)

all of these changes are local – for now.

replace path to sox with /opt/homebrew/bin/sox in [p call-google-speech]

Also had to write a new python script to convert xml to json. Its in the subfolder /xml2json/xml4json.py

The program came from this link: https://www.geeksforgeeks.org/python-xml-to-json/

Also inside [p call-pandorabots] the path for this python program had to be explicit to the full path on the computer. this will vary depending on your python installation.

Also, note that you must install a dependency with pip:

pip install xmltodict

After all that I was actually able to have a conversation. These bots seem primitive, but loveable, now compared to chatGPT. Guess its time for a new project.

Also the voice selection for speech synth is still not connected

(updated 1/21/2021)

This project is an extension to the speech-to-text project: https://reactivemusic.net/?p=4690 You might want to try running that project first to get the Google speech API running.

features

  • Everything runs in one Max patch
  • menu selection of chat bots and voices (currently disabled)
  • filtering of non speakable text (like HTML tags)
  • python script now runs under current directory of patch using relative path
  • refinements to recording and chatbot engines

download

https://github.com/tkzic/internet-sensors

folder: google-speech

files
main Max patch
  • robot-conversation7.maxpat
abstractions and other files
  • clean-html.js
  • xml2json/xml2json.py
  • JSON-google-speech.js
  • JSON-pandorabot.js
  • ms-counter.maxpat (timer for recording messages)
  • pandorabots.txt
Max external objects
external programs:

sox: sox audio conversion program must be in the computer’s executable file path, ie., /usr/bin – or you can rewrite the [sprintf] input to [aka.shell] with the actual path. In our case we installed sox using Macports. The executable path is /opt/local/bin/sox – which is built into a message object in the subpatcher [call-google-speech]

get sox from: http://sox.sourceforge.net

Instructions

  • Open robot-converstaion7.maxpat and turn on audio
  • select chatbot as destination
  • Press the spacebar to start recording.
  • Ask a question.
  • Press the spacebar to stop recording. 

notes

Need to fix the selection of voices.

revision history

  • 1/21/2021: complete rewrite for Max8 and Catalina
  • 4/24/2016: need to have explicit path to sox, in the call-google-speech subpatch. In my Macports version the path is /usr/local/opt/bin/sox.
  • 6/6/2014: re-added missing pandorabots.txt (list of chatbots) – also noticed that pandorabots.com was not available. May need to look for another site.
  • 5/11/2014: The newest version requires Max 6.1.7 (for JSON parsing). Also have updated to Google Speech API v2.
  • Note: Instructions for getting a real key from Google – which will need to be inserted into the patch.  http://www.chromium.org/developers/how-tos/api-keys – so far we have been getting by with common keys from a github site (see notes in next link)

Also please see these notes about how to modify the patch with your key – until this gets resolved: https://reactivemusic.net/?p=11035

Posted on

Notes: Chatbots in Conversation

update 6/2014 – Now part of the Internet sensors projects: https://reactivemusic.net/?p=5859

original post

They can talk with each other… sort of.

Last spring I made a project that lets you talk with chatbots using speech recognition and synthesis. https://reactivemusic.net/?p=4710.

Yesterday I managed to get two instances of this program, running on two computers, using two chatbots,  to talk with each other, through the air. Technical issues remain (see below). But there were moments of real interaction.

In the original project, a human pressed button in Max to start and stop recording speech. This has been automated. The program detects and records speech, using audio level sensing. The auto-recording sensor turns on a switch when the level hits a threshold, and turns off after a period of silence. Threshold level and duration of silence can be adjusted by the user.  There is also a feedback gate that shuts off auto-record while the computer is converting speech to text, and ‘speaking’ a reply.

technical issues

  • The Google speech API has difficulty with some of the voices used by the Mac OS speech synthesizer. We’ll need to experiment to find which voices produce accurate results.
  • The overall levels produced by the builtin Macbook speakers is not quite enough to achieve clear communication. The auto-recorder missed the onset of speech sometimes. One solution would be to insert a click to trigger the recorder, just before the speech synthesizer begins the actual speech. Or to use external speakers, or a secondary “wired” connection.
  • It would be nice to have menus of chatbots and voices. Also to automate the start of a new conversation thread.
  • The button to start the audio detector had to be operated by key-press because pushing the trackpad on a MacBook makes too much noise and always triggers the audio level detector.
  • Occasionally a chat bot would deliver a long response, or one containing a web address. These were problematic for recognition and synthesis.

local files

  • tkzic/internetsensors/speech-to-google-text-api3.maxpat
  • tkzic/internetsensors/pandorabots-api2.maxpat
  • tkzic/internetsensors/text-to-speech3.maxpat

 

Posted on

Pluggo fx matrix

A Max 4.6 patch that uses Pluggo to create random effect matrixes with random parameters and various routing options.

plugv4r6.pat is the patch that works

User guide

(in progress)

download

– not available yet

startup

Choose a data file from the menu in this panel

The data files contain patches – not Max patches, but banks of fx patches that define a configuration of fx saved and named by the user. I haven’t figured out just what is what yet. Select a patch and press the green button. If it worked you will see the patch name change in this text box:

If it doesn’t work, the drop down menu in this box will probably read ‘nothing’

To select a patch, use the drop down menu box, or the number box to the left to make a selection. Then press the green reload button just to the left… (the purple button is for saving the current patch)

After pressing the green button – you should see the fx rack modules reloading from top to bottom – they will turn yellow when loading – and you may see the Pluggo control panel appear.

note: you may need to load a patch twice – there is a bug in the sequence of events for reloading parameters

randomization

Channel randomization: There are 4 channels 0-3 which correspond to the individual fx in the rack, starting at the top. The number box selects which channel to randomize.

Global randomization: Randomize all channels

There are various randomization modes that you choose with the message boxes:

  • 0: randomly select a new plugin for the channel – using default preset (program)
  • 1: randomly select a new preset (program) for the current plugin
  • 2: randomly select new (reasonable) parameter settings for the current plugin
  • 3: randomize the plugin and the programs
  • 4: randomize the plugin and the parameters 
  • 5: randomize all the parameters for this plugin, reasonable or not. (this appears to not work)

Saving patch files

Enter the name of an xml file to save the new bank of patches to, and press the red button.

Note: the patch (xml) files are getting modified by the patch, even when they aren’t explicitly saved. Why is this?

 fx routing

Signals can be routed through the effects matrix in a variety of ways using the matrix control object. The radio buttons on the left side of the matrix control select the most common presets

The vertical lines represent inputs in the following order:

  • signal in
  • channel 0
  • channel 1
  • channel 2
  • channel 3
The horizontal lines represent outputs in the following order:
  • channel 0
  • channel 1
  • channel 2
  • channel 3
  • signal out
A red dot at any intersection makes a connection. Here are the default routings provided by pressing the radio buttons to the left of the matrix.
serial: in -> 0 -> 1 -> 2 -> 3 -> out
reverse serial: in -> 3 -> 2 -> 1 -> 0 -> out
parallel:
in -> 0 -> out
in -> 1 -> out
in -> 2 -> out
in -> 3 -> out
serial + parallel:
in -> 0 -> 1 -> out
in -> 2 -> 3 -> out
Bypass: in -> out
zigzag serial:
in -> 1 -> 2 -> 0 –> 3 -> out
alternate zigzag serial:
in -> 2 -> 3 -> 0 -> 1 -> out
reverse serial + parallel:
in -> 1 -> 0 -> out
in -> 3 -> 2 -> out

Midi plugins and the bypass button

The green indicator to the right of the channel meter indicates that the plugin is a Midi device

Midi devices receive Midi input and will block audio input in a serial routing. To bypass any plugin, click the red button to the left of the channel meter:

The green button reloads the plugin with the default preset. The brown button does nothing.

Mixer and Midi input

The mixer has 3 sets of stereo controls. From left to right, they are input, wet signal, dry signal. The radio buttons to the right of the sliders allow you to select the current channel – which will bring the plugin control panel for that channel into the foreground.

The 2 drop down menus to the right  of the radio buttons select the midi input devices.

The top menu selects the midi controller device. (bcr-2000)

The bottom menu selects the midi note input and performance device.

The letter assignments can be set in the Max midi-setup configuration.

notes

keyboard shortcuts

global randomization params

IO matrix

 

Posted on

return of Pluggo

Pluggo, running in Max 4.6, on a Macbook, inside a VirtualBox instance of Windows XP.

to be continued…

Notes:

update 1/26/2014 – audio input and Max search path

For audio input to work in a windows XP virtual box inside of Mac OS, the sample rate of the microphone in Mac OS (utilities/audio midi setup) must be set to 441000. I spent hours trying to figure this out. Then found this post: https://forums.virtualbox.org/viewtopic.php?f=8&t=56628

The strangest thing is that if you activate audio input in Max without setting the above sample rate, you will get no audio output either.

Also, note that switching default sound cards in the host OS can cause the sample rates to reset back to 96 kHz – requiring them to be reset again before using VirtualBox.

The second issue was that the [vst~] object wasn’t finding names passed with the plug message. Turned out to be a simple matter of setting the path to the plugin directory in the Max file preferences.

Almost forgot – I set a shared drive to be on the E: drive – which was the original location of the plug go project directory – this eliminated need for updates in the patch.

The Pluggo authorization worked.

I was able to use the Behringer UCA202 (audio device) just by plugging it in. Although I couldn’t use any sound cards that required drivers.

http://www.amazon.com/Behringer-Latency-U-Control-UCA202-Interface/dp/B000J0IIEQ

Note: I am running plugv4r6 (the version from 2006)

 original post 

Instructions for installing Windows XP to run max 4.6 in VirtualBox on mac OS 10.8

  • Download and install VirtualBox
  • create new VirtualBox instance (1 Gb of memory)
  • install Windows XP from CD
  • install Firefox (or some reasonable browser)
  • install “guest additions CD image” from device menu select  (inside the virtual machine)
  • install Max 4.6 from c74

For Midi devices:

  • virtual machine – settings -usb – add the midi device – then unplug and replug – windows should find and install
  • also activated windows over internet –
  • installed and tested Pluggo
  • activated drag and drop (doesn’t seem to work)
  • setup shared folders 
Posted on

Leap air piano in Max

Actually in this context, the word ‘piano’ is way too generous. This is a prototype from October, that uses screen mapping to separate left and right hands. It decodes gestures by looking for high velocity downward hand movements. The gestures are mapped to notes and chords based on X position. There is much de-bouncing and tweaking to get results.

Here’s a demonstration which is somewhat painful to listen to, but gets the point across.

local files:

tkzic/max teaching examples/

  • leap-scale-draw5.maxpat
  • leap-sender.maxpat