Collision detection and state change with Leap Motion and Max/MSP.
Second version adding musical finger-painting during the state change.
The 3rd version features a “lack of hand motion” detector which, along with collision detection, fades sounds and triggers state changes.
The non-activity detector is from a Max patch by Mattyo. The spectral filter is from a Max patch by Katja Vetter. In the filter the X,Y, and Z position of hands control number-of-bands, position, and peak width.
How many musicians do you know that play with their eyes closed? Not many computer music apps allow this. Bloom is an exception. http://www.generativemusic.com/bloom.html
As an exercise, I tried to make something like Bloom, using Leap Motion. With your eyes closed you can accurately position your hand at the level of eyes, shoulders, hips, etc., And you can quickly move to a point directly outward from your nose, or shoulders. This is the basis of sobriety tests.
The interface, works with a hand motion like sprinkling seeds. Every time you open your hand, it triggers a note based on the height of your hand. It also triggers one of the “Bloom” circles at the XY position.
Note: If you don’t have a Leap Motion sensor, you can use a mouse.
If you are using Leap Motion, download the aka.leapmotion external – and add it to your Max file path in options | file preferences: http://akamatsu.org/aka/max/objects/
instructions
(if not using Leap Motion sensor, skip to step 4)
Plug in the Leap Motion sensor.
Open leap-finger-switch.maxpat and click the “start” toggle.
Wave your hand around – it should be detected and displayed
Open circlething.maxpat
If using mouse, just click in the black “circlething” window to play.
If using Leap Motion, click the message box to activate Leap Motion
Then open and close your hand, over the sensor to play
One method would be to use intermediaries for example – sending email using php (see this post https://reactivemusic.net/?p=4840) Then IFTTT could pick up the trigger – for example, from gmail. Commands could be embedded in the subject or body of message.
Or you could simply use Twitter as the intermediary – using existing internet sensors methods: (see this post: https://reactivemusic.net/?p=5859)
Would this work in reverse?
Other methods:
temboo: The temboo API should be accessible via Max using a ruby server. Temboo has many common points of interaction with IFTTT
This project demonstrates 2 reversible methods of encrypting audio
reciprocal (1/x)
frequency shifting
The reciprocal method is interesting because the reciprocal of a reciprocal is the original value. In other words you can reverse the process by doing it twice.
Frequency shifting allows you to ‘hide’ audio in the ultrasound range.
An example of sending data through the air, from one computer to another, using sound. The carrier frequency – 18Khz – just below ultrasound, is inaudible to most humans. The data protocol is Audio Frequency Shift Keying (AFSK) at 45 bits/second. It uses two tones – to encode 1’s and 0’s. This protocol was developed for radio teletype (rtty).
One computer sends the word “hello” to the other computer every 10 seconds.
The idea came from a paper by Michael Hanspach and Michael Goetz – “On Covert Acoustical Mesh Networks in Air” – http://www.jocm.us/uploadfile/2013/1125/20131125103803901.pdf. In the paper they explain that the concept was borrowed from current technology for underwater data networks. They warn of the vulnerability of “air-gapping” as a method of computer security.
Actually, after watching the video I realized its difficult to make exciting videos that feature sounds you can’t hear. Especially when you are whispering. Well, the pro camera crew should be knocking on the door at any moment.
in rtty-send12 set carrier frequency to something like 200
then toggle start/stop
You may want to increase the speed to 32
On the receive side, clear the text box
Make sure to set audio as described below
In audio settings try
io vector = 512,
signal vector = 32,
overdrive and audio-interrupt should be enabled,
SR=96 Khz.
Interesting discovery – or maybe a coincidence. Bit rates which are powers of 2 seem to work way better than arbitrary speeds. I wonder if its because the signal vector size is also a power of 2?
Going from a direct audio connection to an through-the-air connection led to a number of issues with filtering and and levels. The latest version of the Max patches have been organized into modules, like: encoder, transmitter, decoder, phasor-clock, etc., But they would benefit from some encapsulation.
It was interesting that throughput was better above 14Khz. Possibly due to less interference from environmental sounds – and less critical filtering. During the video I was able to talk (whisper), without interfering with data transfer. But if I squeaked a chair or tapped the desk, it would screw-up. Also, the builtin mic/speaker on MacBooks have response curves that are all over the place.
The next version will have sharper filters and an automatic level control (compressor). There’s difficult interaction in the detection process between filtering and timing. Up to this point I’ve been reluctant to use frequency domain filtering due to loss of timing resolution. Latency is ok though. But the other thing is that we don’t want filters which soften or distort the shapes of the pulses.
So one question is how high can you go – with the built-in mic and speaker. They are not rated above 20 Khz. but you never know?
This is a radio – but also a modem, in a literal sense of the word. You may be familiar with AM and FM. Have your tried using them backwards?
In this patch, an audio signal is modulated using SSB, AM, and FM then frequency shifted to about 1/4 the nyquist frequency. Then the process is reversed revealing the original audio.
notes
Have improved and simplified FM and AM detection. There is no frequency shift prior to detection. This eliminates some phase distortion.
Pd Version. (runs at 44.1 kHz, because I was too lazy to figure out how to do audio file playback which detects the SR of the audio file.
FM detection: The DC offset and makeup gain levels are sensitive to carrier frequency and SR. Will need to automate these corrections to prevent audio output problems.
This patch mixes two audio signals, then makes one disappear by reversing its phase. A similar method was used by U.S. military to encrypt radio transmissions during the 20th century. A phonograph record was the ‘key’ to unlock the encoded transmission. In practice it was difficult to synchronize the two recordings to achieve phase cancellation.
