Thursday, February 25, 2016

Note Selection Basics


In my previous posts, I've focused on detecting note-on and note-off transitions from the breath controller, and on mapping the breath intensity to a MIDI continuous controller to control expressive parameters. But the circuits and code have never been able to play more than one note. In this post, I'll discuss how to read a set of finger inputs, like the keys on a woodwind instrument, or the valves of a trumpet.

To keep things simple, I'll start with 3 finger inputs, like found on a trumpet. For a B-flat trumpet, the 3 fingers can be used to play 7 different notes:


Index
Middle
Ring
Note Produced
Up
Up
Up
B-flat
Up
Down
Up
A
Down
Up
Up
A-flat
Down
Down
Up
G
Up
Down
Down
G-flat
Down
Up
Down
F
Down
Down
Down
E




Readers familiar with binary numbers may be saying, "wait, with 3 binary digits, you should be able to produce 2^3, or 8 distinct finger combinations." And you would be right, except that the fingering combination [Index=up, Middle=Up, Ring=Down] produces an "in-between-pitch" note in the lower registers of the trumpet, and is not used.

A Real Instrument


For those of you unfamiliar with how a trumpet works, most have 3 piston valves which re-route the flow of air through the instrument to change the effective length, and therefore the resonant frequency, of the instrument.

(Photo: Wikimedia Commons: https://upload.wikimedia.org/wikipedia/commons/9/9d/Trumpet_1.jpg)

A natural way to simulate the operation of a piston valve like this is to use a momentary switch, so let's wire something up. I've used a small tactile switch like the following:

(Photo: Wikimedia Commons: https://upload.wikimedia.org/wikipedia/commons/3/35/BUTA-06-X-STAN-01.jpg)

These tactile switches are widely used in consumer electronics, and are very inexpensive.e.g. $0.20 each. They are available from all the major parts suppliers.

I've wired up three of these on a breadboard, spaced approximately finger-width apart:



Now, to be clear, no actual trumpet player would find this thing pleasant to play, as the "feel" of the tactile switches is very different from the feel of an actual piston valve. But it will let us start to experiment with how we can simulate real-world musical instrument properties like trumpet valves and woodwind keys.

To connect these switches to your microcontroller, first figure out which pins are connected when the switch is pressed. If you are using the tactile switches like I am, there are 4 pins on each one, arranged in two pairs. If you find that the switch appears to be always pressed, you may need pull the switch out of the breadboard, rotate it 90 degrees, and re-insert it.

Once the switches are attached to the breadboard, connect one side of each switch to a common wire (this will attach to the ground on your microcontroller). The other side of each switch should be brought out on a separate wire. In total, there will be 4 connections to the microcontroller, like this:


Once you have these wired up, attach Common to the Ground (GND) pin on your microcontroller, and the Index, Middle, and Ring (finger) switch wires to three digital input pins. I've attached mine to digital inputs 2, 3, and 4 on my Teensy microcontroller. If you have yours attached to different input pins, you'll need to modify the code examples below accordingly.

One thing to be aware of is that we're pulling the digital inputs low when the buttons are pressed. So the "valve" will read HIGH when the button is not pressed and LOW when pressed.

Mapping Inputs to MIDI Notes

Now that we have our "trumpet valves" plugged into our microcontroller, how do we get it to send different MIDI notes to our synthesizers according to which switches are pressed?

One approach might be to use a bunch of if/then statements to decide which note to select. It might look like:

if (switch one pressed) {
    if (switch two pressed) {
        if (switch three pressed) {
            send E
        } else {
            send G
        }
    } else {
        // switch two is NOT pressed
        if (switch three pressed) {
            send F
        } else {
            send A-flat
        }
etc etc

As you can see, this is pretty hard to read, and it's pretty easy to make a mistake. Plus, imagine how horrifying this code would be if you were trying to build a woodwind controller, which could have as many as 14 or 15 switches.

A better approach is to use binary numbers to map any combination of the switch on/off states to a number. For our trumpet, we only need 3 bits, so all the possible key combinations, and the corresponding MIDI note, can be represented in a new column in our fingering table

Index
Middle
Ring
Note Produced
Binary (Decimal) Value
Up
Up
Up
B-flat
000 (0)
Up
Down
Up
A
010 (2)
Down
Up
Up
A-flat
011 (3)
Down
Down
Up
G
001 (1)
Up
Down
Down
G-flat
100 (4)
Down
Up
Down
F
101 (5)
Down
Down
Down
E
111 (7)

Now that ugly code for deciding which note to play can be replaced with:

if (value = 0) {
    send B-flat
} else if (value == 2) {
    send A
} else if (value == 3) {
    send A-flat
} else if (value == 1) {
    send G
} else if (value == 4) {
    send G-flat
} else if (value == 5) {
    send F
} else if (alue == 7) {
    send E
}
We'll actually use a C-language case statement in the sample code, and we'll express the numbers in binary instead of decimal, like this:

switch (value) {
    case 0:
        send B-flat;
        break;
    case 2:
        send A;
        break

and so on.

So let's go ahead and write the sketch for our trumpet. To do that, we'll replace the get_note() function with a new one:

int get_note() {
  // This routine reads the three buttons that are our "trumpet
  // valves", and returns a MIDI note.
  int value = (digitalRead(2) << 2) +
      (digitalRead(3) << 1) +
      digitalRead(4);
  // Since the digital input pins are pulled low when the
  // button is pressed, the "all valves open" value will
  // be 111 (binary) or 7 (decimal). Full table:
  // v1     v2     v3      note             value (decimal)
  // open   open   open    C                7
  // open   closed open    B                5
  // closed open   open    B-flat/A-sharp   3
  // closed closed open    A                1
  // open   closed closed  A-flat/G-sharp   4
  // closed open   closed  G                2
  // closed closed closed  G-flat/F-sharp   0
  // open   open   close   not used         6
  // 
#define BASE 60  // The base MIDI note value
  switch (value) {
    case 7:
      return BASE;
      break;
    case 5:
      return BASE - 1;
      break;
    case 3:
      return BASE - 2;
      break;
    case 1:
      return BASE - 3;
      break;
    case 4:
      return BASE - 4;
      break;
    case 2:
      return BASE - 5;
      break;
    case 0:
      return BASE - 6;
      break;
    default:
      // If invalid fingering, ignore and return the
      // currently sounding note.  
      return noteSounding;
  }
}

What's going on here?


int value = (digitalRead(2) << 2) +
      (digitalRead(3) << 1) +
      digitalRead(4);

Here, we're reading the values of the three buttons into a single integer value, using bit shift operators. Let's break it down:

First, we declare a variable named "value". For all practical purposes, that starts off as zero, and in binary that is:

00000000

which is 8 bits of binary zeros. 

(Yes, nerds, it's actually a 32-bit signed integer, but that's not important to this discussion)

When we do digitalRead(2) << 2

we're saying "read digital input two and shift the whole value to the left two bits. So if we read 00000001, we'd shift that two bits to the left, so we end up with:

00000100

Then we read digital input three. Suppose we also read a HIGH, and shift left one bit, that ends up being:

00000010

Finally we read input 4, but don't shift it at all. So if we get a HIGH on pins 4, we have:

00000001

Now, we add the following three binary numbers:

00000100
00000010
00000001

and we get:

00000111

So that means the three digital I/O pin we're using are giving the value 1. As I mentioned before, we're taking the digital I/O pins low with button presses, so this binary value 00000111 means no buttons are pressed.

In order to make pulling I/O pins low to detect button pressed, we also need to add pullups to the digitnal I/O pins. The Teensy has support for that built-in (otherwise we'd need to attach some resistors between the digital pin and the power rail). We'll add the following code (in blue) to the setup() function:

void setup() {
  state = NOTE_OFF;  // initialize state machine
  pinMode(2, INPUT_PULLUP);
  pinMode(3, INPUT_PULLUP);
  pinMode(4, INPUT_PULLUP);
}

We'll also need to change the main loop a little bit to handle the situation where the player chooses a new fingering but keeps blowing. In that case we need to turn off the old note and turn on the new one. So we'll add the following code in the block where we're handing the case where a note is on (new code in blue):

void loop() {
  // read the input on analog pin 0
  sensorValue = analogRead(A0);
  if (state == NOTE_OFF) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Value has risen above threshold. Move to the RISE_TIME
      // state. Record time and initial breath value.
      breath_on_time = millis();
      initial_breath_value = sensorValue;
      state = RISE_WAIT;  // Go to next state
    }
  } else if (state == RISE_WAIT) {
    if (sensorValue > NOTE_ON_THRESHOLD) {
      // Has enough time passed for us to collect our second
      // sample?
      if (millis() - breath_on_time > RISE_TIME) {
        // Yes, so calculate MIDI note and velocity, then send a note on event
        noteSounding = get_note();
        int velocity = get_velocity(initial_breath_value, sensorValue, RISE_TIME);
        usbMIDI.sendNoteOn(noteSounding, velocity, MIDI_CHANNEL);
        state = NOTE_ON;
      }
    } else {
      // Value fell below threshold before RISE_TIME passed. Return to
      // NOTE_OFF state (e.g. we're ignoring a short blip of breath)
      state = NOTE_OFF;
    }
  } else if (state == NOTE_ON) {
    if (sensorValue < NOTE_ON_THRESHOLD) {
      // Value has fallen below threshold - turn the note off
      usbMIDI.sendNoteOff(noteSounding, 100, MIDI_CHANNEL);  
      state = NOTE_OFF;
    } else {
      // Is it time to send more aftertouch data?
      if (millis() - atSendTime > AT_INTERVAL) {
        // Map the sensor value to the aftertouch range 0-127
        atVal = map(sensorValue, NOTE_ON_THRESHOLD, 1023, 0, 127);
        usbMIDI.sendAfterTouch(atVal, MIDI_CHANNEL);
        atSendTime = millis();
      }
    }
    int newNote = get_note();
    if (newNote != noteSounding) {
      // Player has moved to a new fingering while still blowing.
      // Send a note off for the current node and a note on for
      // the new note.
      usbMIDI.sendNoteOff(noteSounding, 100, MIDI_CHANNEL);
      noteSounding = newNote;
      int velocity = get_velocity(initial_breath_value, sensorValue, RISE_TIME);
      usbMIDI.sendNoteOn(noteSounding, velocity, MIDI_CHANNEL);
    }
  }
}

Here's a short video of the sketch in action:



Here's the entire sketch, available at https://github.com/ggood/NoteSelectionTutorial


Thanks for reading. If there's anything you would like me to address in future posts, leave a comment. Next up: how to play more than the seven notes this instrument is capable of playing.

10 comments:

  1. Thank you so much for posting this! I'm trying to make a MIDI wind controller in the style of a clarinet, and this was the aspect I had so much trouble wrapping my head around-- mapping the buttons to notes. I've been trying to get my project started for months and your posts are the only thing keeping the project in the realm of possibility, as there's almost no documentation on wind controllers anywhere else. I have some questions that if you have the time, I would really appreciate your advice on, given your specific know how of wind controllers. Is there an email I can contact you at?

    ReplyDelete
  2. I'm glad this is helping! My email address is at the top of the code sample in this post, although, if you wouldn't mind, can you post the questions here in the comments? That way everyone can see them and benefit from the discussion. Thanks!

    ReplyDelete
    Replies
    1. My issue might be specific to me but I'll give it a try-- I'm planning on making a MIDI wind controller in the style (and exact fingerings) of a b flat clarinet, and the project is definitely a lot more daunting than I thought it would be. I'm familiar with Python, but I haven't done coding in a while (I'm a music technology major, but I mostly compose so I'm unfamiliar with a lot of the hardware involved in the process).

      The base idea is essentially setting up a breadboard with tactile buttons as close to the layout of a clarinet as possible. The issue right off is that a clarinet has buttons on both sides of the instrument as well as the left and right sides (I know this is worded poorly, I mean that I would have to somehow find a doublesided breadboard that also has inputs on the sides? I could draw a diagram of what I mean if needed).

      My next fear is that there aren't enough inputs on the Teensy to accommodate the amount of buttons I want to use (I know that I won't be able to use every button on the clarinet due to the amount of fingerings, but I was hoping to have most of them).

      And then my other issue is the code itself, the code you provided looks incredibly in depth for just 3 buttons, and I'd be using at least 10. I can sort of understand the code you've provided (my slight background in coding combined with your explanations), but I worry that getting my theoretical MIDI clarinet to send MIDI on and off information for all the notes on the clarinet (~3 fingerings per note, over 3 registers), I'm worried it just isn't possible to accommodate all that, given my low level of knowledge in this subject.

      I know that's very wordy, sorry! I've been trying to make this project happen for months and I just can't wrap my mind around a lot of the technical things because my education is more focused on composition and live sound, and I don't have experience with arduinio and stuff like this.

      Delete
  3. Eric, thanks, that's a very clear statement of the challenges you're running into, and helps me understand what I should cover next. I'll go into more detail in a new blog post, but here are what I see as the major areas you need help with:

    - understanding the code, which is in a language you're not familiar with (C)
    - physical design. Have you considered using PVC tubing for the main structure of the instrument? You could attach single-sided breadboards to it, or just attach the switches directly to the pipe (hot glue?). I have a similar prototype instrument that I haven't written a blog post about - maybe I can document that next.
    - expansion beyond the number of digital inputs available on the microcontroller. You can use a multiplexer to accomplish that. There's a chip that costs just a few dollars that can expand to 16 inputs.

    I was thinking about making a prototype woodwind controller for my next post.

    Where do you go to school, BTW?

    -Gordon

    ReplyDelete
    Replies
    1. Hey Gordon, thanks for replying so fast!

      Yeah I definitely am having trouble understanding the language, I know some VERY basic things about C, but nothing substantial, I didn't even know you could do anything MIDI with it!
      For design my original idea was to take off all the keys on my old plastic Yamaha clarinet (I have it lying around after I upgraded to a Buffet), and use that as the physical base as what's more accurate to a clarinet than a clarinet? This is where my lack of circuitry know-how is evident though, I wasn't sure how connecting the buttons to the arduino could work if they're mounted on a clarinet body (or even if you could do that, is hot glue conductive?)

      This is a silly comparison, but is a multiplexer essentially a "power strip" of sorts, but for inputs?

      I appreciate your response because my knowledge on this is clearly lacking ^^;;
      And I go to the illustrious Plymouth State University for a degree in Music Technology.

      Delete
    2. Ok, Eric, I get it. Interesting idea about using the clarinet body as the core of the instrument. I think it would be good to build your prototypes out of cheap hardware-store materials, like PVC pipe, then do any final build on the one-and-only plastic clarinet. As far as I know, hot glue is not conductive, and would provide a good and cheap way to attach tactile switches to the prototype. And if it sucks, well, you didn't spend a lot of money. :-) And it should be fine to run long wires between the Arduino/Teensy and the body of the instrument. For this application you could run many feet of wire before you have problems.

      Here's my suggestion for learning C. Go though the tutorials for Arduino on the Arduino web site at https://www.arduino.cc/en/Tutorial/HomePage. Almost none of them have anything to do with music or MIDI, but it'll help you get familiar with C. And it will focus on the specific C features you'll need to know without getting into a lot of stuff that isn't really needed for programming on the Arduino or Teensy.

      After working through the basic Arduino tutorials, go back to my early posts in this series and understand the code there. I try to start with basics, then build on previous knowledge, so if you jump in at the end, it may be a little overwhelming.

      Your power strip analogy for the multiplexer is pretty close, but imagine a 6-outlet power strip with a 6-position switch at one end. That switch controls which one of the plugs is turned on, and all the others are turned off. Now imagine that the switch can be computer-controlled. That's pretty much how it works. So if you had 6 light bulbs hooked up to the strip, you could use the computer to make a light display with bulbs flashing in any order you want (only one can be on at a time), but you only need one AC power outlet, not 6.

      And, as you correctly note, it's for inputs, so now imagine it in reverse. So if you have 6 inputs you want to read, you can use a multiplexer to read input #1, store it in the computer memory, read input #2 and store it, and so on. Since the computer can do this really fast (hundreds of thousands of times per second), even though it doesn't *really* happen simultaneously, no human can tell the difference. That's how we can read all the keys "simultaneously" even though they are read one after another.

      Delete
  4. That's what I thought too. I have been trying to find info on clarinet wind controllers, but apart from some basic circuits I found years ago regarding the "Elect-RO-Clar clarinet" (the pages of which have vanished) I have not found much.

    I am planning to use a length of wooden broom handle, with micro-switches sitting in slots cut in the wood. These I would carefully multiplex into a row/column multiplexing arrangement. Not sure if this would need to be done in hardware, as surely most Arduino based microcontrollers should be able to scan what's effectively a keypad in software?

    Also, I found a lot of MIDI library software on the Arduino website. Some of this might help.

    But I look forward to your next instalment, especially if it's going to be about a woodwind wind controller.

    Thanks again,

    Simon B.

    ReplyDelete
  5. Hi Gordon! I'm back again with another question, this time concerning note selection using pushbuttons.

    Specifically, I'm having trouble consistently choosing the notes I want. The main problem lies in note changes involving two or more actions (releasing or pressing a button). When such a note change occur, an interim note is almost always sounded, as the players fingers can almost never perform all the actions simultaneously.

    For example, in transitioning from a C (000) to an A (110), there will always be either a Bb (100) or a B(010) in between, depending on which button is pressed first. This problem gets even worse with three-valve transitions, like low C to low C#.

    Do you have a solution for this problem? I have tried waiting a certain amount of time for the valve position to settle, but often times notes will just not sound at all, or the problem will persist regardless. Are you aware of any coding techniques for this particular issue?

    Thanks,

    David

    ReplyDelete
    Replies
    1. Hi David,

      I think your approach of waiting for the keys to settle is the right approach. Writing some code to do that is on my todo list. One idea might be to sample the keys at a high rate and put the readings in a queue. At a lower rate, examine the last readings, and take the majority.

      Another commenter, Yoe, solved the problem in his sketch here - you should take a look at his code (which is for a woodwind-style controller): https://github.com/Trasselfrisyr/MiniWI/tree/master/TeensieWI

      Delete
    2. Whoops, that should read "At a lower rate, examine the last N readings, and take the majority".

      Delete