Sunday, January 27, 2013

Expression, Part 2 (MIDI Aftertouch)


In my previous post I showed how to map the breath pressure measured at the time a note commences to MIDI note on velocity. This allows a wind controller to play synth patches that mimic percussive or plucked instruments in a fairly natural way - the harder you blow, the harder the attack sounds (assuming that the synthesizer patch is responsive to note on velocity).

Another expression parameter present in the MIDI spec is aftertouch, sometimes referred to as pressure. After pressing a key down, a player can "lean on" the note and the keyboard will send additional MIDI messages that tell a synthesizer how hard the key is being pressed. Synth patches that respond to aftertouch will often alter a filter frequency as the pressure on the key increases (usually brightening the sound), or apply more vibrato to the note, but that's just one of many possible things that can be altered in real time.

There are actually two types of aftertouch - monophonic aftertouch, sometimes called channel pressure, and polyphonic aftertouch. With monophonic aftertouch, the keyboard only sends one value for aftertouch, no matter how many keys are being held down. With polyphonic aftertouch, the keyboard sends one aftertouch value for each key being pressed. Since a wind instrument is inherently a monophonic (one note at a time) instrument, we'll only worry about monophonic aftertouch in this post.

This sketch builds on the sketch in the MIDI note on velocity sketch (and uses the same circuit built in this post), and brings back a couple of the concepts from the breath controller post. Notice how we keep track of the last time we sent an aftertouch value, and only send a new value if a specific amount of time has passed, to avoid sending unnecessary MIDI data.

I've highlighted in yellow the additions the sketch from the previous post (so anything not in yellow is unchanged). There really isn't a lot of new code, mostly because we only need to worry about sending aftertouch values when we know a note is on.


#define MIDI_CHANNEL 1
// The threshold level for sending a note on event. If the
// sensor is producing a level above this, we should be sounding
// a note.
#define NOTE_ON_THRESHOLD 80
// The maximum raw pressure value you can generate by
// blowing into the tube.
#define MAX_PRESSURE 500

// The three states of our state machine
// No note is sounding
#define NOTE_OFF 1
// We've observed a transition from below to above the
// threshold value. We wait a while to see how fast the
// breath velocity is increasing
#define RISE_TIME 10
// A note is sounding
#define NOTE_ON 3
// Send aftertouch data no more than every AT_INTERVAL
// milliseconds
#define AT_INTERVAL 70

// The five notes, from which we choose one at random
unsigned int notes[5] = {60, 62, 65, 67, 69};

// We keep track of which note is sounding, so we know
// which note to turn off when breath stops.
int noteSounding;
// The value read from the sensor
int sensorValue;
// The state of our state machine
int state;
// The time that we noticed the breath off -> on transition
unsigned long breath_on_time = 0L;
// The breath value at the time we observed the transition
int initial_breath_value;
// The aftertouch value we will send
int atVal;
// The last time we sent an aftertouch value
unsigned long atSendTime = 0L;


void setup() {
  state = NOTE_OFF;  // initialize state machine
}

int get_note() {
  return notes[random(0,4)];
}

int get_velocity(int initial, int final, unsigned long time_delta) {
  return map(final, NOTE_ON_THRESHOLD, MAX_PRESSURE, 0, 127);
}

void loop() {
  // read the input on analog pin 0
  sensorValue = analogRead(A0);
  if (state == NOTE_OFF) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Value has risen above threshold. Move to the RISE_TIME
      // state. Record time and initial breath value.
      breath_on_time = millis();
      initial_breath_value = sensorValue;
      state = RISE_TIME;  // Go to next state
    }
  } else if (state == RISE_TIME) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Has enough time passed for us to collect our second
      // sample?
      if (millis() - breath_on_time > RISE_TIME) {
        // Yes, so calculate MIDI note and velocity, then send a note on event
        noteSounding = get_note();
        int velocity = get_velocity(initial_breath_value, sensorValue, RISE_TIME);
        usbMIDI.sendNoteOn(noteSounding, velocity, MIDI_CHANNEL);
        state = NOTE_ON;
      }
    } else {
      // Value fell below threshold before RISE_TIME passed. Return to
      // NOTE_OFF state (e.g. we're ignoring a short blip of breath)
      state = NOTE_OFF;
    }
  } else if (state == NOTE_ON) {
    if (sensorValue < NOTE_ON_THRESHOLD) {
      // Value has fallen below threshold - turn the note off
      usbMIDI.sendNoteOff(noteSounding, 100, MIDI_CHANNEL);  
      state = NOTE_OFF;
    } else {
      // Is it time to send more aftertouch data?
      if (millis() - atSendTime > AT_INTERVAL) {
        // Map the sensor value to the aftertouch range 0-127
        atVal = map(sensorValue, NOTE_ON_THRESHOLD, 1023, 0, 127);
        usbMIDI.sendAfterTouch(atVal, MIDI_CHANNEL);
        atSendTime = millis();
      }
    }
  }
}


Here are a couple of recordings I made. In the first, I play some long notes that have a hard initial attack, then I immediately back off the breath, and blow harder and harder until I release the note (a fp plus a crescendo for you musicians).

In this first example, the synth is not responding to aftertouch, so you just hear a static note.

In the second example, I've modified the synth patch to apply vibrato to the patch, and the amount of vibrato depends on the aftertouch value.

Are you able to hear the vibrato (wavering sound) at the end of the notes?

What's Next


With the ability to add expressiveness via MIDI note velocity and aftertouch, we can make some pretty interesting sounds, but we're still mimicking a keyboard instrument. To realize the full potential of an electronic wind instrument, we're going to need to get synthesizers to respond to us in a way that makes sense for the way that a wind instrument works.

While I am by no means an expert in synth programming, I'll share what I know, and maybe we can come up with some interesting sounds.

10 comments:

  1. Hey Gordon,

    Can't wait to see the next of this tutorials man, the tutorials you make are great regarding wind synthesis. I hope you post soon the next update! Keep me updated please! I would seriously recommend your website to my friends from lower years to see this website for reference if they plan to make a project involving wind synthesis.

    I downloaded a software synthesizer that is very similar to garageband because im using Windows and not a mac. Waiting for the next of your tutorials so that I can go to the next step but they have been perfectly helpful for me

    Can you please include tutorial on how you let your button combinations from the arduino to communicate with the Garageband to play a certain note. Thanks!
    -Ryan

    ReplyDelete
  2. When I did my wind controller, I used value string for keeping CC values, instead of delays, for not flooding channel with the same CC values.

    When CC message is sent to computer, it's also stored in the string LAST_CC (for example).
    When the next loop runs, before sending CC from the breath controller, it's first compared to LAST_CC. If values are different, CC is sending to computer, and it's value writes in LAST_CC. Otherwise, nothing is sending.

    Hope it helps.

    ReplyDelete
  3. That is good advice, Alexander. I think, though, that it's still a good idea to limit the rate of CC messages even if you keep track of the last CC value sent. Imagine a continuous controller that is very "jittery" (like a pressure sensor). It might be the case that the current CC value is always different from the LAST_CC value.

    One might also average the last raw values from the pressure sensor and only send a new CC message if the new calculated average value differs. That could help with a noisy input signal.

    ReplyDelete
    Replies
    1. If you want do discuss the theme, you can write me an email to fundorin@gmail.com.
      It's been a year since I've build my controller, and it seems like I'm ready to build a new one now. It'll be more simple than previous one and will allow me to play only 3 chromatic octaves, instead of four that I have now. Anyway, I found myself playing the highest octave less frequently that the other ones and many woodwind VST's doesn't even contain four octaves sampled or synthesized.

      Delete
    2. BTW, I din't have any issues with "jitter". My pressure sensor works smooth, according to CC records in Reaper.

      Delete
    3. Sorry for flooding, but I've checked my sketch, and there's a smoothing applied to pressure sensor data, which I used earlier. Don't remember why it's commented in the last script version. Maybe, I've found, that bandwith is enough to send CC continuosly or some other occasion.

      Anyway, here's how the smoothing was done:

      // sensP = 0.9 * sensP + 0.1 * analogRead(sensPpin);

      It called sensP, cause I also have sensV, which is for drawing (2way sensor).

      P.S. Those CAPTCHAs are really freaking me out!

      Delete
  4. Sorry about the captchas, Alexander - I'm cheap and am using the free Blogger.

    Thanks for sharing your experiences. If I recall correctly from reading the datasheets, some pressure sensors may implement hysteresis on the chip, so the values you see may already be smoothed. Which pressure sensor are you using?

    ReplyDelete
  5. AFAIR it is MPXV5010DP from Freescale. It have two inputs for positive and negative pressure measuring. I using the second one as a calibration at power on, or when pressing the reset button. My controller have to of the. For exhale I use pressure input from the first sensor, and for inhale is the second input of the second sensor.

    ReplyDelete
  6. Interesting dual-sensor setup. Mine only has a single sensor with a single (blow) port, therefore my instrument doesn't have an inhale mode. What do you use your inhale mode for? It occurs to me that one could use it to put the instrument into a "meta" mode, where the controls perform some alternate function.

    ReplyDelete
    Replies
    1. Sorry for mistypes.

      I used two of them sensors to reduce number of the buttons. Exhale for C and inhale for D on the same button press.
      For what you called "meta", if I understand this term right, I used a separate switch. When it was turned on, all play buttons acted like octave\transpose shift, program change +\-, favorite programs, etc. This is the mode with two extra "shift" buttons. With a switch turned on, controller acted like a mixer with play, rec, stop, pause, undo, redo, track navigation.
      So, with a combination of the switch and two shifts, I had 4 extra modes to control everything by pressing play buttons. One is just a switch, second is switch+shift1, third is switch+shift2 and fourth is switch+both shifts. Actually, I couldn't find how to use the last mode, so it was like a reserve for future needs.
      My layout was based on a chromatic harmonica, where 4 holes (exhale, inhale and shift button) give you a full chromatic octave. I just switched holes to buttons, and placed octaves one above the other. Full four octaves in one small device.
      The only reason I see in adding the second sensor to my new controller is to be able to play melodies without taking pauses for taking a breath.

      Delete