Tag: portfolio

Posted on

ep-4yy13 DSP – week 8

Voices

Examples

Questions

 

Posted on

ep-4yy13 DSP – week 7

frequency domain

transforming signals

  • DFT, FFT, STFT
  • The FFT produces a stream of complex numbers representing energy at frequencies across the spectrum
  • The length of the FFT determines the frequency resolution (number of bins)
  • Increasing length of FFT frame degrades resolution in time domain (rhythmic accuracy)
  • Amplitude = root of (r*r) + (i*i)  = magnitude
  • Phase = arctangent of i/r = angle

Practical Applications

  • Convolution/Deconvolution
  • Analysis
  • Spectral processing (pitch and timbre)
  • Amplitude processing: noise gates, crossovers
  • phase vocoder
  • radio

Examples

  • Max/MSP tutorials 25-26
  • Max/MSP Example DSP patches (in Extras | ExamplesOverview | MSP | FFT fun
    • convolution workshop
    • Forbidden planet
  • Fourier Filter (Vetter)
  • fft-tz2 (basics, SSB ring modulator)
  • fplanet-tz.maxpat: hacked version of forbidden-planet example which uses granular indexing to do spectral convolution and make spaceship sounds. To use patch: 1 ) turn on audio, 2) then press message boxes inside the green panel
  • fp-fft-tz.maxpat: pfft~ subpatch for above
  • fourierfilter (folder) containing fourierfiltertest.maxpat: Katja Vetter’s complex spectral filter example
  • Tristan Jehan’s frequency detector object
  • Little Tikes piano: https://reactivemusic.net/?p=6993
  • Helicopter frame rate video: http://www.youtube.com/watch?v=jQDjJRYmeWg
download example Max patches here:
Resources

Assignments

See notes from previous weeks: https://reactivemusic.net/?p=10109 

 

 

 

Posted on

ep-4yy13 DSP – week 6

Stop the experimental music

(click the picture)

from http://emoctv.tumblr.com

DSP – according to the Arctic Monkeys…

The time domain:

The frequency domain:

Samples, impulses, and convolution

(in the time domain)

  • Decomposition
  • Unit impulse (delta function)
  • convolution (from the input side and output side)
  • filters
Reference: http://www.dspguide.com “The Scientist and Engineer’s Guide to DSP”, By Steven Smith – chapters 6-7

Clocks

Time, under a microscope.

Granular synthesis

Audio under a microscope.

Andy Farnell, “Designing Sound” http://aspress.co.uk/sd/index.php

Chapter 16.7 Methods “Granular” p. 257

Chapter 13 “Shaping Sound” p. 205

Destroying information

Abstract is what remains after shedding details.

Example Max patches:

  • timestrech3.maxpat
  • nothingness.maxpat
[wpdm_file id=16]

Assignment:

See notes from last week. https://reactivemusic.net/?p=10059

Work through the convolution examples on your own. Its important to have a physical concept of signals, in various transformations. Become a wave. Have an out of body experience. Take a good look at yourself.

Posted on

ep-4yy13 DSP – week 5

transforming music into music

Examples

Notes

  • Solving problems
  • Exploration
  • Stories

Assignments

Mystery field recording: (email to me this week)

  • Record a very short sound clip (less than 15 seconds)
  • It should be something that you hear, not something you produce – for example, a fire-truck, a refrigerator, the wind…
  • Please don’t tell me where the sound came from. We will try to guess. When you send the file, just have your name on it. For example: field-recoding-keithMoon.mp3
  • Alternative: Record an impulse response in an interesting space. We will try to guess the space. The impulse can be anything, for example: hand clap, yelling “hello”, a trumpet.
  • Extra credit – transcribe your recorded event. For example, what chord or rhythms do the machines in a coffee shop produce?
  • Email a link or attachment to: [email protected]

Composition: Sound-byte (due March 17th)

  • The sound-byte is a short audio clip of speech.
  • The speech can come from anywhere. Something familiar, something famous, something unusual.
  • Every sound in the composition is derived only from the sound-byte. You can use any tool or method.
  • The sound-byte in its original form should occur somewhere in the piece
  • Duration: roughly 2-3 minutes?  That is up to you.

Music from the future:

Please send me a link to your future music piece – sometime before the end of the semester

Posted on

rtl-sdr: Pd, Max and Mac

Notes on compiling rtl_sdr in Mac OS – writing Max and Pd Externals.

 

update 3/31/2014

Today I got the Pd external running – using essentially same source code as Max. There is occasional weirdness going on with audio clicks when starting/stopping the radio, but other than that it seems fine and it runs. wooHOO. More to follow…

 

update 3/28/2014

Now have set up a skeleton for Pd, called rtlfmz~ (inside the Pd application bundle) which does absolutely nothing but compiles all of the project files and calls a function in rtl_fm.  Next step will be to port the actual Max external code and do the conversion.

updates 3/38/2014 before working on Pd version

There is now a fairly solid max external (rtlfmz~) using a recent version for rtl_fm. Also there is a simple Makefile that compiles local version of rtl_fm3.c in:

tkzic/rtl-sdr-new/rtl-sdr/rtl-fm3

There are very minor changes to rtl_fm.c (for includes) and also a local version of librtlsdr.a (librtlsdr32.a) that is 32 bit.

The current state of the Max external does both pre-demodulated and raw IQ output, but you can only run one copy  of the object due to excess use of global variables and my uncertainty over how to run multiple devices, threading, etc.,  – but we’ll go with it an try porting to Pd now.

update 2/27 – converted to using new version of rtl_fm

I had been using an older version of rtl_fm –

renamed external to rtlfmz~ and now using new version as (rtl_fm3.c) in the project

In addition to recopying the librtlsdr.a – I also recopied all of the include files and added two new files

convenience.c convenience.h

There is a different method of threading and reading which I haven’t looked at yet, but it is now doing what it did before inside Max, which is read FM for a few seconds and write audio data to a file

plan: Set up a circular buffer accessible to the output thread and to the max perform function. – then see if we can get it to run for a few seconds.

The main thing to think about is how to let the processing happen in another thread while returning control to max. Then there really should be a way to interrupt processing from max.

There needs to be a max instance variable that tells whether the radio is running or not. Then, when its time to stop – you just do all the cleanup stuff that is at the end of the rtl_fm main() function.

 

update 2/18/2014 – Max external

Now have rtl_fm function within the plussztz~ external

It detects, opens device, demodulates about 30 seconds of FM, and writes audio at 44.1kHz. to a file /tmp/radio.bin – which can be played by the play command

Changes to code included:

  • removing exit calls
  • changing printf to post()
  • removing signal interrupt handling
remaining to do:
  • Need a way to get the device to read in the background – so its not blocking the Max thread
  • Need a way to stop/restrart the device – because if we aren’t, the continual sync_reads will waste a lot of cpu cycles.

update 2/15/2014 – Max external

Have now successfully  compiled a test external in Max 6.1.4 – name is plussztz~ and it includes the rtl_fm code.  Made 2 changes so far:

  • in build settings, set architectures to i386
  • in rtl_fm2, changed <include> libusb.h to “include” libusb.h

original post

Today I was able to write a simple makefile to compile the rtl_fm app using the libusb and librtlsdr dynamic libraries.

Pd requires i386 architecture for externals (i386) so I then compiled the app using static libraries and the i386 architecture.

libusb-1.0 already had a 32 bit version in /usr/local/lib/libusb32-1.0.a (note that this version also requires compiling these frameworks:

-framework foundation -framework iokit

For librtlsdr, I rebuilt, using cmake with the following flags:

cmake ../ -DCMAKE_OSX_ARCHITECTURES=i386 -DINSTALL_UDEV_RULES=ON

But did not install the result. See this link for details on building with cmake: http://sdr.osmocom.org/trac/wiki/rtl-sdr

This produced a 32 bit static version of librtlsdr.a that could be used for building the app.

See this Stack Overflow post for more on cmake and architectures: http://stackoverflow.com/questions/5334095/cmake-multiarchitecture-compilation

local files:

currently local version of this test is in: tkzic/rtl-sdr-tz/rtl_fm2/

The default makefile builds the 32 bit architecture.

Next:

  • try to move the makefile into Xcode
  • try compiling rtl_sdr or rtl_fm as a simple max object – the fm app might be better to start with since it gives an audio signal output.
  • then try in pd

 

 

 

Posted on

Ubuntu in virtualbox

Ubuntu server guest, on Mac OS host.

instructions

http://www.lecloud.net/post/52224625343/the-ultimate-setup-guide-ubuntu-13-04-in-virtualbox

Skipped the SSH for now, also the shared folders are in /media/sf[folder-name] on the guest – will be trying again shortly.

In the meantime I installed a desktop version of Linux and the install was much easier – copy and paste works fine – and we were able to get sounds. now installing pd.

linux instructions for Pd: http://puredata.info/docs/faq/debian

After running into huge problems with copy/paste – and no audio, I was able to install 32 bit version of Ubuntu Studio. Audio works fine, but Jack doesn’t run – after a few hours, I’m abandoning this approach and will probably try running linux natively on something.

Posted on

Searching by image

Using Google’s search by image feature to return similar images

http://images.google.com/imghp?hl=en

With Google you can search by image. But it gets really interesting when you upload an image that is not available on the internet and look at the set of similar images returned. Or if you use a common image but just view the visually similar results. For example, here is a protein molecule (http://www.kurzweilai.net/images/ferritin.jpg)

Here are similar image results returned by Google.

You can also restrict the results to faces:

A few internet images to try:

Camera images (not on the Internet until they were posted here) These will give more interesting results. For example, the woman with the flower (using face matching) returns images of Erik Prince and Brad Pitt.
Posted on

More conversations with robots in Max

Using Google speech API and Pandorabots API

(updated 1/21/2024)

all of these changes are local – for now.

replace path to sox with /opt/homebrew/bin/sox in [p call-google-speech]

Also had to write a new python script to convert xml to json. Its in the subfolder /xml2json/xml4json.py

The program came from this link: https://www.geeksforgeeks.org/python-xml-to-json/

Also inside [p call-pandorabots] the path for this python program had to be explicit to the full path on the computer. this will vary depending on your python installation.

Also, note that you must install a dependency with pip:

pip install xmltodict

After all that I was actually able to have a conversation. These bots seem primitive, but loveable, now compared to chatGPT. Guess its time for a new project.

Also the voice selection for speech synth is still not connected

(updated 1/21/2021)

This project is an extension to the speech-to-text project: https://reactivemusic.net/?p=4690 You might want to try running that project first to get the Google speech API running.

features

  • Everything runs in one Max patch
  • menu selection of chat bots and voices (currently disabled)
  • filtering of non speakable text (like HTML tags)
  • python script now runs under current directory of patch using relative path
  • refinements to recording and chatbot engines

download

https://github.com/tkzic/internet-sensors

folder: google-speech

files
main Max patch
  • robot-conversation7.maxpat
abstractions and other files
  • clean-html.js
  • xml2json/xml2json.py
  • JSON-google-speech.js
  • JSON-pandorabot.js
  • ms-counter.maxpat (timer for recording messages)
  • pandorabots.txt
Max external objects
external programs:

sox: sox audio conversion program must be in the computer’s executable file path, ie., /usr/bin – or you can rewrite the [sprintf] input to [aka.shell] with the actual path. In our case we installed sox using Macports. The executable path is /opt/local/bin/sox – which is built into a message object in the subpatcher [call-google-speech]

get sox from: http://sox.sourceforge.net

Instructions

  • Open robot-converstaion7.maxpat and turn on audio
  • select chatbot as destination
  • Press the spacebar to start recording.
  • Ask a question.
  • Press the spacebar to stop recording. 

notes

Need to fix the selection of voices.

revision history

  • 1/21/2021: complete rewrite for Max8 and Catalina
  • 4/24/2016: need to have explicit path to sox, in the call-google-speech subpatch. In my Macports version the path is /usr/local/opt/bin/sox.
  • 6/6/2014: re-added missing pandorabots.txt (list of chatbots) – also noticed that pandorabots.com was not available. May need to look for another site.
  • 5/11/2014: The newest version requires Max 6.1.7 (for JSON parsing). Also have updated to Google Speech API v2.
  • Note: Instructions for getting a real key from Google – which will need to be inserted into the patch.  http://www.chromium.org/developers/how-tos/api-keys – so far we have been getting by with common keys from a github site (see notes in next link)

Also please see these notes about how to modify the patch with your key – until this gets resolved: https://reactivemusic.net/?p=11035

Posted on

ep-4yy13 DSP – week 2

new composition tools

from various artists

tools that make tools