Monday, April 15, 2013

First Demo

So today I prepped a CSV by hand to think through what I want the signal converter R file to do. You can find it here. In this first-pass test, I used the results of a PCA analysis on my first study, which has people donating to preserve threatened park land. However the PCA analysis was done by a friend who misunderstood the timing increment (thinking that we were capturing data every 1 second, instead of every 2) and so the results should be taken with a grain of salt. It resulted in the capture of 3 distinct neural networks, which roughly appeared to be attentional focus, visual processing, and the "default mode network" (what's lighting up when you've got the internal monologue going but aren't focusing on a task). I used the following equations to define pitch and velocity for each of the 3 networks:

Xi = signal at timepoint i
Xh = max signal
Xl = min signal
Pi = a given network's proportional contribution to the total signal strength of all 3 networks at timepoint i

Pitch: 128*[(Xi - Xl)/(Xh-Xl)]
Velocity: 128*Pi

These values (converted to integers) were dumped into a separate prepped CSV file to fit into the CSV-to-MIDI converter. This created a .mid file which I used Audacity to convert to a .midi file, then import into Reaper. Reaper split the tracks by channel, and I used Eastwest QL Symphonic Orchestra and Absynth 5 for the VSTs. The resulting file, which tracks attention with strings, visual processing with woodwinds, and the default mode network with a synth, can be found here.

You can hear really abrupt notes. After going into the Java file for the CSV-to-MIDI converter, I realized this is because note_on and note_off commands use the same Tick variable. So in future I will mod their converter to take both a duration variable where note_off occurs at (note_on + dur). I also found a trove of MIDI messages which I should be able to invoke in the Java file. This grants a specific way to do pitch bend, mod wheel, pedal, control change, etc. data interpretation which I would otherwise not be able to get at using their vanilla program.

No comments:

Post a Comment