The Max remote receiver project described in an earlier post: https://reactivemusic.net/?p=20965 now includes support for OpenWebRx and KiwiSDR sites.
When using OpenWebRx, you need to select a “profile” that supports the receive frequency range, before you can operate within that frequency. Its an unusual system that uses an offset frequency within the selected profile, instead of a global “actual frequency” like the other platforms.
The KiwiSDR sites, in general, have very good quality reception and audio.
The next step in this project will be to consolidate a directory of sites that are useful for remote reception in the ham bands for various parts of the world.
Mapping geocoded contest log data using node.js and openlayers.
The goal was to make something that looks like the Reverse Beacon Network map, only for contest log files. I use RBN for testing antennas now. That map display gives you a pretty good idea of your actual antenna pattern.
Code is written in node.js (javascript) and html.
Part 1: Read a Cabrillo log file containing QSO: records. Look up each callsign, get latitude and longitude, and rewrite the file as json data, tagged with geo coordinates. I originally tried getting the data from hamQTH but it was not current, so ended up using the qrz.com xml callsign lookup. For callsigns “not found” I used the qrz.com dxcc prefix lookup to get general coordinates for the country. There are still a few bad/missing data issues to resolve. Like European stations with coordinates at the South Pole.
Part 2: Tried various mapping frameworks – like leaflet, arcgis, and openlayers. Wanted to use a great-circle projection (azimuthal equidistant) like the big ARRL world map. And may still figure this out. But working with map projections and coordinate transforms is way worse than doing a Smith Chart. I ended up hacking a flight tracking example from openlayers.org and basically replacing airplanes with QSO’s. That is why the lines are animated from source to destination. Also added a layer for day/night, and QSO/time status display.
It probably makes sense to get rid of the flight animation and just display the entire path in sync with the QSO data – with color code for each band (K1KP) – and speed control on the time lapse, etc., So you can get a better sense of rate and propagation.
It would be cool to have a website where you could upload a log file and generate maps.
Note: this project is not yet available
Files
local files:
generating data:
internetsensors/cabrillomap
put the cabrillo data in testdata.cbr (use QSO: records only for now) should be sorted chronologically.
run: node index.js
the output file will be: geocab.json (which is used as input to the mapping program)
mapping
internetsensors/oltest
main.js = node source with ol mapping and data processing
index.html = web page for map
geocab.json = geocoded cabrillo json test data
to run, type: npm start
Then open: http://localhost:5173/ in a browser
Additional work / current issues
Some of the qrz.com callsign data has bad geo coordinates. In particular some of the records show a latitude of -89 and longitute -179 – need to check for these numbers and replace with dxcc coordinates.
There should be an argument on the node program to pass in the datafile. Also the program should clean up any non QSO: records, like the file header info and any X-QSO recs.
Also need to clean up the async/await stuff – currently there are several methods for handling state transitions.
mapping ideas:
As mentioned above, its probably a good idea to make a version of the code without the flight animation, and have various controls to stop/start the data playback to look at individual qso’s do speed control, etc.,
azimuthal equidistant projection: there are some links to examples in leaflet, and arcgis to handle complex projections. In documents, look at: “map links for projection stuff.txt”
leaflet test version:
in the internetsensors/cabrillomap folder there’s a test file: cbworld1.html that works using websockets when you run the index.js file to generate test data. It uses a leaflet map, but the lines don’t adapt to great circle polar paths.
arcgis
I believe the arcgis examples are in internetsensors/projected geometries
And: internetsensors/pe-gs-projection
The former is a a very nice world projection with some point markers. The latter is an example that shows how to switch out various projections in realtime.
Attempts to use the X API (formerly Twitter) for projects with Max/MSP have been disappointing at best. Most of the API is behind a paywall now. The cost is $5000 per month. to implement streaming API used in projects like this: https://reactivemusic.net/?p=5786
The free tier only allows basic tweeting and user lookup. Search is not available.
I was able to find only one node example that actually worked in the free tier. By “Coding with Ado”. The code requests a token from X, and then you enter a timestamped pin number to continue. Making it worthless for programs and bots. https://youtu.be/G5ZW5j5cwHk?si=vbAtGa0bQ3T_tga9
A local copy of the source code for this is in tkzic/nodetweet3/index.js
There service sits in the middle to handle X API calls. You are charged by the number of calls. It doesn’t offer streaming either, but you can simulate it by calling a search every few seconds.
Other social media options
There are API’s for other social sites like facebook, instagram, tiktok, etc.,
In the previous version, all of the markers for each train line were deleted and redrawn in a group at the polling interval. In this version the train markers move to their new location when the data is polled.
The project uses the MBTA JSON API, to query vehicle data for each train line. the geo coordinates of the trains are sent via websockets to the client map page.
This project is a work in progress and not available yet.
Files:
node server: internetsensors/mbtanode2/index.html
html map client: internetsensors/mbtanode2/mbtatrain2.html
Instructions
// to run, type node index.js at a command line.
// then open a web browser to: http://127.0.0.1:8124/
// type 'go' into the text box below the map, and press 'send' button
// in a few seconds the train markers will magically appear
Sonification of Mass Ave buses, from Nubian to Harvard.
Updated for Max8 and Catalina
This patch requests data from MBTA API to get the current location of buses – using the Max js object. Latitude and Longitude data is mapped to oscillator pitch. Data is polled every 10 seconds, but it seems like the results might be more interesting to poll at a slower rate, because the updates don’t seem that frequent. And buses tend to stop a lot.
This project uses version 3 of the API. There are quality issues with the realtime data. For example, there are bus stops not associated with the route. The direction_id and stop_sequence data from the buses is often wrong. Also, buses that are not in service are not removed from the vehicle list or indicated as such.
The patch uses a [multislider] object to graph the position of the buses along the route – but due to the data problems described above, the positions don’t always reflect the current latitude/longitude coordinates or the bus stop name.
You will need to replace the API key in the message object at the top of the patch with your own key. Or you can probably just remove it. The key distributed with the patch is fake. You can request your own developer API key from MBTA. It’s free.
instructions
Open mbta.maxpat
Open the Max console window so you can see what’s happening with the data
click on the yellow [getstops] message to get the current bus stop data
Toggle the metro (at the top of the patch) to start polling
Turn on the audio (click speaker icon) and turn up the gain
Note: there will be more buses running during rush hours in Boston. Try experimenting with the polling rate and ramp length in the poly-oscillator patch. Also, you can experiment with the pitch range.
Soundcloud deleted my API credentials and is not accepting new requests. So there is really no point to using this project unless you have active credentials.
console.error() function crashes node. -fixed locally
The Soundcloud client-id is embedded in scnode.js – you will need to edit this file to replace the worthless client-id with your own. To get a client ID you will first need a Soundcloud account. Then register an app at: http://soundcloud.com/you/apps
first time instructions
Open the Max patch: sc.maxpat
In the green panel, click on [script npm init]
In the green panel , click on [script install soundcloud-api-client]
instructions
Open the Max patch sc.maxpat
open the max.console window so you can see the API data
click [script start]
click the speaker icon to start audio
type something into the search box and press <enter> or click the button to the left to search for what is already in the box.
select a track from the result menu, wait for it to start playing
This implementation uses node.js for Max instead of Ruby to access the API. You will need set up a developer account with Spotify and request API credentials. See below.
Other than that, the synthesis code in Max has not changed. Some of the following background information and video is from the original version. ..
What if you used that data to reconstruct music by driving a sequencer in Max? The analysis is a series of time based quanta called segments. Each segment provides information about timing, timbre, and pitch – roughly corresponding to rhythm, harmony, and melody.
Edit spot1.js replacing the cliendID and clientSecret with your spotify credentials
node for max install instructions (first time only)
Open the Max patch: spotify-synth1.maxpat
Scroll the patch over to the far right side until you see this green panel:
Click the [script npm init] message – this initializes the node infrastructure in the current folder
Then click each of the 2 script npm install messages – this installs the necessary libraries
Instructions
Open the Max patch: spotify-synth1.maxpat
Click the green [script start] message
Click the Speaker icon to start audio
Click the first dot in the preset object to set the mixer settings to something reasonable
open the Max Console window so you can see the Spotify API data
From the 2 menus at the top of the screen select an Artist and Title that match, for example: Albert Ayler and “Witches and Devils”
Click the [analyze] button – the console window should fill with interest data about your selection.
Click [play]
Note: if you hear a lot of clicks and pops, reduce the audio sample rate to 44.1 KHz.
Alternative search method:
Enter an Artist and Song title for analysis, in the text boxes. Then press the buttons for title and artist. Then press the /analyze button. If it works you will get prompts from the terminal window, the Max window, and you should see the time in seconds in upper right corner of the patch.
troubleshooting
If there are problems with the analysis, its most likely due to one of the following:
artist or title spelled incorrectly
song is not available
song is too long
API is busy
Mixer controls
The Mixer channels from Left to right are:
bass
synth (left)
synth (right)
random octave synth
timbre synth
master volume
gain trim
HPF cutoff frequency
You can also adjust the reverb decay time and the playback rate. Normal playback rate is 1.
programming notes
Best results happen with slow abstract material, like the Miles (Wayne Shorter) piece above. The bass is not really happening. Lines all sound pretty much the same. I’m thinking it might be possible to derive a bass line from the pitch data by doing a chordal analysis of the analysis.
Here are screenshots of the Max sub-patches (the main screen is in the video above)
Timbre (percussion synth) – plays filtered noise:
Random octave synth:
Here’s a Coltrane piece, using roughly the same configuration but with sine oscillators for everything:
There are issues with clicks on the envelopes and the patch is kind of a mess but it plays!
Several modules respond to the API data:
tone synthesiszer (pitch data)
harmonic (random octave) synthesizer (pitch data)
filtered noise (timbre data)
bass synthesizer (key and mode data)
envelope generator (loudness data)
Since the key/mode data is global for the track, bass notes are probable guesses. This method doesn’t work for material with strong root motion or a variety of harmonic content. It’s essentially the same approach I use when asked to play bass at an open mic night.
additional notes
Now that this project is running again. I plan to write additional synthesizers that follow more of the spirit of the data. For example, distinguishing strong pitches from noise.
Also would like to make use of the [section] data as well as the rhythmic analysis. There is an amazing amount of potential here.
NOTE: There’s a problem with [shell] – it rejects input that is converted to a symbol using [tosymbol].
This can be fixed by using from symbol – or just eliminating [tosymbol] – it make affect the stderr-stdout redirection token, ie., “>” and other special characters but for now [shell] does not accept symbol input
aka.speech can be replaced using the “say” command in the shell. more details to follow about voice parameters.
‘say’ has similar params to aka.speech, eg., voice name and rate. There are voices for specific languages. This feature could be used, for example, to match the language from a Tweet to an appropriate voice
3. Twitter streaming API
I revised the php code for the Twitter streaming project, to use the coordinates of a corner of the city polygon bounding box. That seems to be more reliable than the geo coordinates which are absent from most Tweets.
There is a new API in the works – but its difficult to decipher the Twitter API docs because they have so many products and the documentation is obtuse.
Also it would be interesting to extract the “language” field and use it to select which voice to use in the speech synthesizer. Or even have an english translation option.
4. Echonest API
Echonest was absorbed into Spotify. The API is gone. But the Spotify API does have some of the feature detection and analysis code. But it doesn’t allow you to submit your own audio clips. There are also some efforts to preserve some of the Echonest stuff like the blog by Paul Lamere, and the remix code. Here are a few links I found to get started.
Inside the Ruby code, change “oauth_token” to “access_token” and “oauth_token_secret” to “access_token_secret”
The streaming client example code failed and I replaced it with a filter that looks for tweets from a twitter id. To get your twitter id, go to https://tweeterid.com/
This patch has been completely re-written. The old API was obsolete. This version uses [dict] and [maxurl] to format and execute the initial query. Then it uses [jit.uldl] to download the mp3 file with the bird-call audio. Interesting that [maxurl] would not download the file using the “download” URL. It only worked with a URL containing the actual file name.
9. ping
Needed to reinstall ruby gems using xcrun (see above)
seems to be a problem with mashape:
Could not resolve host: igor-zachetly-ping-uin.p.mashape.com (Patron::HostResolutionError)
[mashape was acquired by rapidapi.com – so will need to refactor the code in the ruby server.]
