Expression, Part 2 (MIDI Aftertouch)

In my previous post I showed how to map the breath pressure measured at the time a note commences to MIDI note on velocity. This allows a wind controller to play synth patches that mimic percussive or plucked instruments in a fairly natural way - the harder you blow, the harder the attack sounds (assuming that the synthesizer patch is responsive to note on velocity).

Another expression parameter present in the MIDI spec is aftertouch, sometimes referred to as pressure. After pressing a key down, a player can "lean on" the note and the keyboard will send additional MIDI messages that tell a synthesizer how hard the key is being pressed. Synth patches that respond to aftertouch will often alter a filter frequency as the pressure on the key increases (usually brightening the sound), or apply more vibrato to the note, but that's just one of many possible things that can be altered in real time.

There are actually two types of aftertouch - monophonic aftertouch, sometimes called channel pressure, and polyphonic aftertouch. With monophonic aftertouch, the keyboard only sends one value for aftertouch, no matter how many keys are being held down. With polyphonic aftertouch, the keyboard sends one aftertouch value for each key being pressed. Since a wind instrument is inherently a monophonic (one note at a time) instrument, we'll only worry about monophonic aftertouch in this post.

This sketch builds on the sketch in the MIDI note on velocity sketch (and uses the same circuit built in this post), and brings back a couple of the concepts from the breath controller post. Notice how we keep track of the last time we sent an aftertouch value, and only send a new value if a specific amount of time has passed, to avoid sending unnecessary MIDI data.

I've highlighted in yellow the additions the sketch from the previous post (so anything not in yellow is unchanged). There really isn't a lot of new code, mostly because we only need to worry about sending aftertouch values when we know a note is on.

#define MIDI_CHANNEL 1
// The threshold level for sending a note on event. If the
// sensor is producing a level above this, we should be sounding
// a note.
#define NOTE_ON_THRESHOLD 80
// The maximum raw pressure value you can generate by
// blowing into the tube.
#define MAX_PRESSURE 500

// The three states of our state machine
// No note is sounding
#define NOTE_OFF 1
// We've observed a transition from below to above the
// threshold value. We wait a while to see how fast the
// breath velocity is increasing
#define RISE_WAIT 2
// A note is sounding
#define NOTE_ON 3
// Send aftertouch data no more than every AT_INTERVAL
// milliseconds
#define AT_INTERVAL 70 

// We wait for 10 milliseconds of continuous breath

// pressure above NOTE+ON_THRESHOLD before we turn on

// the note, to de-glitch

#define RISE_TIME 10 

// The five notes, from which we choose one at random
unsigned int notes[5] = {60, 62, 65, 67, 69};

// We keep track of which note is sounding, so we know
// which note to turn off when breath stops.
int noteSounding;
// The value read from the sensor
int sensorValue;
// The state of our state machine
int state;
// The time that we noticed the breath off -> on transition
unsigned long breath_on_time = 0L;
// The breath value at the time we observed the transition
int initial_breath_value;
// The aftertouch value we will send
int atVal;
// The last time we sent an aftertouch value
unsigned long atSendTime = 0L;


void setup() {
  state = NOTE_OFF;  // initialize state machine
}

int get_note() {
  return notes[random(0,4)];
}

int get_velocity(int initial, int final, unsigned long time_delta) {
  return map(final, NOTE_ON_THRESHOLD, MAX_PRESSURE, 0, 127);
}

void loop() {
  // read the input on analog pin 0
  sensorValue = analogRead(A0);
  if (state == NOTE_OFF) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Value has risen above threshold. Move to the RISE_TIME
      // state. Record time and initial breath value.
      breath_on_time = millis();
      initial_breath_value = sensorValue;
      state = RISE_WAIT;  // Go to next state
    }
  } else if (state == RISE_WAIT) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Has enough time passed for us to collect our second
      // sample?
      if (millis() - breath_on_time > RISE_TIME) {
        // Yes, so calculate MIDI note and velocity, then send a note on event
        noteSounding = get_note();
        int velocity = get_velocity(initial_breath_value, sensorValue, RISE_TIME);
        usbMIDI.sendNoteOn(noteSounding, velocity, MIDI_CHANNEL);
        state = NOTE_ON;
      }
    } else {
      // Value fell below threshold before RISE_TIME passed. Return to
      // NOTE_OFF state (e.g. we're ignoring a short blip of breath)
      state = NOTE_OFF;
    }
  } else if (state == NOTE_ON) {
    if (sensorValue < NOTE_ON_THRESHOLD) {
      // Value has fallen below threshold - turn the note off
      usbMIDI.sendNoteOff(noteSounding, 100, MIDI_CHANNEL);  
      state = NOTE_OFF;
    } else {
      // Is it time to send more aftertouch data?
      if (millis() - atSendTime > AT_INTERVAL) {
        // Map the sensor value to the aftertouch range 0-127
        atVal = map(sensorValue, NOTE_ON_THRESHOLD, 1023, 0, 127);
        usbMIDI.sendAfterTouch(atVal, MIDI_CHANNEL);
        atSendTime = millis();
      }
    }
  }
}

Here are a couple of recordings I made. In the first, I play some long notes that have a hard initial attack, then I immediately back off the breath, and blow harder and harder until I release the note (a fp plus a crescendo for you musicians).

In this first example, the synth is not responding to aftertouch, so you just hear a static note.

In the second example, I've modified the synth patch to apply vibrato to the patch, and the amount of vibrato depends on the aftertouch value.

Are you able to hear the vibrato (wavering sound) at the end of the notes?

What's Next

With the ability to add expressiveness via MIDI note velocity and aftertouch, we can make some pretty interesting sounds, but we're still mimicking a keyboard instrument. To realize the full potential of an electronic wind instrument, we're going to need to get synthesizers to respond to us in a way that makes sense for the way that a wind instrument works.

While I am by no means an expert in synth programming, I'll share what I know, and maybe we can come up with some interesting sounds.