Audio from Video

Using video to generate audio.

jit.peek~ outputs an audio signal from video matrix data.

(jit.peek~ help)

from cycling74.com

At any point in the the process, matrix data can be tapped to produce an audio signal. Analyzing video is very much like analyzing audio. For example, using envelope following and spectral analysis. Here are examples.

examples

1. jit.peek~-additivesynth.maxpat (in Max/MSP examples/jitter-examples/audio)

Draw a song

By Adam Florin and Joshua Kit Clayton

2. Frame subtraction

By Adam Rokshar

https://reactivemusic.net/?p=7005

3. Sound Emotion2 – using Macbook built-in camera

By Andreas Wittich

https://reactivemusic.net/?p=9225

4. jit.peek~-osctrack.maxpat (in Max/MSP examples/jitter-examples/audio)

Derives rhythmic audio data from video. (Using bball.mov)

By cycling74.com

5. Whispering Heights (in Max/MSP examples/jitter-examples/3rd-party/image-to-spectral-filter/whispering_heights.maxpat)

Similar to above but uses video to create a moving spectral filter.

6. Wolframatic.maxpat  (in Max/MSP examples/jitter-examples/other/Wolframatic/)

Fractal generator also generates audio.

By R. Luke Dubois

 

Blob update

Version 1.1 update to Blob now available in the iOS App store.

http://zerokidz.com/blob/home.html

Blob (by Bjorn Lindberg)  was converted in 2011 to run as a webApp using SenchaTouch 1.0 and Phonegap 0.94. Both of these frameworks have evolved to the point that the original code is no longer workable.

Here is an update to the Web app version: http://zerokidz.com/blob1.1/www/sencha-index.html

updates

  • Updated Sencha Touch containers and html canvas setup to same methods used in SecretSpot.
  • Improved algorithm for setting width/height of Blob window based on actual window dimensions – this makes it run better as a Web app on a desktop computer.
  • In process of updating icons and launch screens for Apple requirements.
  • screen size issues resolved by setting as universal app, and migrating launch images to asset manager – in Xcode –  and removing the status bar with settings in the plist file.
  • Lots of small things needed to upgrade to iOS 7.

bugs

Like SecretSpot, The native app runs only in portrait orientation.

notes

After this update has been submitted to Apple, I will try to convert it to run in current versions of Sencha Touch and Phonegap. There is relatively small amount of Sencha code in the app: A main window, toolbar, canvas window,  and a few customized buttons.

 

SecretSpot update

Version 1.1 update to SecretSpot now available in the iOS app store.

http://zerokidz.com/secretspot/Home.html

SecretSpot was written in 2011 as a webApp using SenchaTouch 1.0 and Phonegap 0.94. Both of these frameworks have evolved to the point that the original code is no longer workable. With iOS 6, the app version of secretSpot broke. Audio was only triggering with every other touch.

Instead of upgrading javascript code, I made a few changes to the phonegap library so it would compile in iOS 7, then added new icons and splash screen files – and resubmitted the app.

One change was to alter the way that launch images get loaded during webView initialization, to prevent annoying white flashes. This version also does an extra load of the launch image – instead of just going to a black background while the webView is loading. If this ends up being too slow, then it may be better to go directly black background following initial launch image. See this post: http://stackoverflow.com/questions/2531621/iphone-uiwebview-inital-white-view

bugs

The native app runs only in portrait orientation. This is something the app has had problems with before. Something has changed in the process of detecting current orientation. Its likely to be a Phonegap issue as the Web app version works fine.

I may have a look at this  bug – and apply updates to the Blob app as well. But the real solution is to upgrade to current versions of the frameworks.

notes

This is more a documentation thing: If you don’t touch the canvas when the app starts, then there will be no sound unless you restart the app.

 

maxradio updates

Upgrade to Mac OS Max version.

Windows, runtime, standalone, and documentation upgrades still in progress…

changes

This is an interim update to add new drivers.

  • driver for rtl-sdr
  • minor bug fixes

download

https://github.com/tkzic/maxradio

instructions

  • Connect a radio
  • Open maxsdr7a.maxpat 

note: 1/23/2015 – the repository was really messed up – missing externals, etc., have reloaded. The local source is now at tkzic/maxradio