Thursday, April 21, 2016

Building a Woodwind Controller Using the MPR121 Touch Sensor

The Akai EWI uses a set of touch-sensitive metal keys:


A key is actuated simply by touching it. The instrument detects the change in capacitance that results when your body touches the key. For more information on how this works, see https://en.wikipedia.org/wiki/Capacitive_sensing. And for some videos of people playing the EWI, look on YouTube. I especially like this one of the late Michael Brecker: https://www.youtube.com/watch?v=tPUBp9uTLIw

Can I Do This?


Is there a way to inexpensively build this into our own wind controllers? The answer is definitely yes.

NXP Semiconductors manufactures a very low-cost chip, the MPR121 ($1.95 in single quantities), that provides 12 separate touch inputs.
and Adafruit Industries has a nice breakout board that breaks out the tiny pins on the chip, and has some support circuitry that allows use with either 3.3v or 5v controllers.


Sparkfun also has a breakout board for the MPR121. It doesn't include the level shifters to allow 3.3/5v operation, but is less expensive.

I won't write a tutorial on how to wire up the sensor and use it, since the Adafruit tutorial does a great job of that. Instead, I'll show how we can incorporate it into a woodwind-style controller.

One thing that may occur to you is that there are only 12 inputs, but typical woodwinds have more than 12 keys. Is there a way to use more than one touch sensor chip in an instrument? Yes, there is. The MPR121 is an I2C device, which means you can attach more than one to the same two-wire communication bus, as long as they have different addresses. Both the Adafruit and Sparkfun breakouts include a way to set the address (up to 4 different addresses can be set), so it is possible to do touch-sensing on up to 48 keys. That should be enough.

Modifying get_note() For Touch Sensing

In my post, Note Selection Basics, I define a function named get_note() that read the switches that simulated our "trumpet valves" and returns the MIDI note to play. Let's replace that with a new function that can read the MPR121 and detect which of the 12 inputs is being touched.

First, a little background. Adafruit provides a library for the MPR121 that makes configuring and reading it very simple. To initialize the chip:

touchSensor.begin(0x5A);

(0x5A is the default I2C address of the Adafruit breakout)

Then, to read all of the pins:

uint16_t touchValue = touchSensor.touched();

This will return a 16 bit value where each of the lower 12 bits is a 1 if the key is touched, and a 0 if it is not. Which bit corresponds to which key depends on how I wired the brass washers to the touch sensor input pins. In my case, the octave keys are mapped to bits 0 and 1, and then the rest of the keys are:

LH index finger: bit 2
LH ring finger: bit 3
LH middle finger: bit 4
(bit 5 not used - it's an extra key on my instrument not currently used)
RH index finger: bit 6
RH ring finger: bit 7
RH middle finger: bit 8
RH pinkie: bit 9
(bits 10, 11 not used)

Table Lookup for Note Mapping

As I mentioned in the Note Selection Basics post, a C language case statement will start to get pretty ugly for an instrument with all these keys, so let's look into a table lookup approach. We'll build a table that has, in one column, the expected bit values read from the touched() function of the MPR121, and in the next column, the MIDI note to send.

In C, we can define a structure that holds one row of that table like this:

struct fmap_entry {
  uint16_t keys;
  uint8_t midi_note;
};


This is a chunk of memory that can hold a 16-bit value (named "keys", which will hold a bitmap of key values), and an 8-bit MIDI note value.

Next, we'll build an array of these:

#define FMAP_SIZE 33
struct fmap_entry fmap[FMAP_SIZE];

This defines an array (or table, if you prefer) of 33 rows of the structure we define above.

To map from a fingering to a MIDI note, we start at the beginning of this table and check to see if the fingering we just read from the sensors matches the value in the "keys" field. If it does, we've found the MIDI note and we're done. Otherwise, we skip to the next entry, and so on. In programming, this is called a linear search.

We actually need to initialize the "fmap" array with all of the key and note values. That's very verbose, so I'll omit it here and will just include it with the full code.

So here's the revised get_note() function. Since my instrument has an extra key I'm not using, there's a little code (in blue) that makes sure that even if the player touches that unused key, it won't change the value we read, so we don't need to have entries in the table for when that key is touched/not touched.

int get_note() {
  // This routine reads the touch-sensitive keys of the instrument and
  // maps the value read to a MIDI note. We use a lookup table that maps
  // valid combinations of keys to a note. If the lookup fails, this
  // routine returns -1 to indicate that the fingering was not valid.
  int ret = -1;  // Sentinel for unknown fingering
  uint16_t touchValue = touchSensor.touched();


  // Since we're not using the 4th finger of the left hand, mask off that key
  touchValue = touchValue & 0b1111111111011111;


  for (uint8_t i = 0; i < FMAP_SIZE; i++) {
    if (touchValue == fmap[i].keys) {
      ret = fmap[i].midi_note;

      break;
    }
  }

  return ret;
}


The for loop (in red) is where we do the linear search. We look at each keys value, and when we find a match, we stop looking and return the corresponding MIDI note.

If we don't find a match, then the player has their fingers in a non-supported position, and our function returns -1, which means to take no action. In other words, we ignore the glitch. That's not the only thing we could do. If, for example, we wanted to try to mimic how a real instrument behaves, we might send a pitch bend message to indicate that the pitch should be a little sharp or flat relative to the normal fingering. There are a lot of possibilities.

Ok, now that we've had a look at some code, is it possible to build something?

A Test Platform 

I've built a very hacky prototype woodwind-style controller as a platform to test these ideas. The body of the instrument is a 14" length of PVC plumbing pipe, 2" or so in diameter.  I was going for roughly the dimensions of an alto recorder, but chose a thicker tube to make running wires inside a little easier.

To make the touch-sensitive keys, I soldered lengths of wire to ten brass washers:



Then I drilled holes in the tube, ran the wires from the outside to the inside, and hot-glued the washers to the outside of the tube, about where my fingers would fall. I didn't arrange the keys to resemble any particular instrument, although they're somewhat like a recorder layout (albeit with one extra key for the left hand, oops). The Frankenstein-like contraption looks like this:


The white mouthpiece protruding from the top is something I made to use with my Blowchucks Controller for the Electro-Music Festival in 2014. It connects to a tube that runs down the length of the instrument and connects to the Freescale pressure sensor I've been using in all of my wind controller projects.

On the back, I added two octave keys, and strapped all of the electronics near the base. This means I only have a USB cable running from the instrument to my laptop.

In this close up of the electronics, you can see the MPR121 touch sensor breakout on the left. The green and white wires that attach to that board are the wires coming from the brass washers. The board to the right has the Teensy microcontroller mounted on the right, the the pressure sensor on the left. You can also see the power and ground (red/black) wires going to the touch sensor, as well as the two wires (yellow/white) that connect the touch sensor to the I2C bus on the microcontroller.


I'll do a proper video later, but for now, here's a quick audio demo of the instrument running the code below. You'll hear some glitches as I'm not a woodwind player and have a hard time getting multiple fingers to touch keys at exactly the same time. We can de-glitch that stuff in code, and I'll work on that for a future post.




The full code is posted below, and is also available at https://github.com/ggood/NoteSelectionTutorialRecorder


13 comments:

  1. You got me to really wanna try this! I've ordered the Adafruit MPR121 board, plus some cheap Aliexpress ones out of curiosity. Also ordered a couple of 4011 cmos nand gates to see if I can make some really really cheap resistive touch stuff. http://www.learningaboutelectronics.com/Articles/Touch-sensor-circuit-with-NAND-gate.php Don't even know if this will work with several keys at once, but we'll see :)

    ReplyDelete
  2. Hi, it's me again :) I built a playable prototype with the Adafruit MPR121 board during the weekend. Really cool stuff! Thanks for the inspiration :)
    https://www.instagram.com/p/BEzG2GLoVXD/?taken-by=trasselfrisyr
    https://www.instagram.com/p/BE4I0BzoVSn/?taken-by=trasselfrisyr
    https://github.com/Trasselfrisyr/MiniWI/blob/master/MiniWI-cap/MiniWI-cap.ino

    ReplyDelete
  3. Johan, excellent! I like your idea of using conductive tape (I think) for the touch sensor input. Much tidier than my prototype!

    ReplyDelete
  4. Thanks! Yeah, the tape looks tidy and it's easy to use, but it's a bit hard to feel where the fingers should go. I'll probably have to solve that somehow. I also wanna try this thing with metal inlays that I found on the Adafruit page. Looks really nice. https://learn.adafruit.com/metal-inlay-capacitive-touch-buttons

    ReplyDelete
  5. Made a project entry for my MiniWI on hackaday.io, check it out :)

    https://hackaday.io/project/11843-miniwi-woodwind-midi-controller

    ReplyDelete
  6. I've been meaning to do exactly this for years, thanks for the informational-- Hopefully I'll attempt it soon!

    ReplyDelete
  7. Yoe, thanks for sharing you hackaday project. I especially liked your "Song for Barry" video.

    I sure miss having Michael Brecker on this planet. At least we have YouTube videos like https://youtu.be/gBtJQyF1c_I

    ReplyDelete
  8. Hi! I've made some more progress with my wind controllers, and I just put together a simple usb midi version using just a Teensy LC and the pressure sensor that I sandwiched between two sheets of perspex with the aluminium tape electrodes on. Don't know if it was the higher speed of the Teensy LC or whatever, but I got a lot more stuck notes than with my previous controller (that also suffered from stuck notes now and then). First I thought it was the softsynths not keeping up, but after some troubleshooting I found what seems to be the problem (at least I haven't had any stuck notes after changing it). In the note on state, if the pressure has dropped and the state was set to note off, the code following the time to send cc (aftertouch/breath) check is always executed. So even if the state is set to off, a new note can be trigged and not registered as on. To fix this I moved the ending curly bracket for the "else" part of the threshold check to include the "new fingering" part. Hope this makes sense.

    ReplyDelete
    Replies
    1. Nice catch, Johan. I wonder why I never ran into stuck notes in my testing? Maybe because I'm not doing any de-bouncing like you are? In any case, I'll fix the code example when I have time. The fix, as I understand it, is to move lines 265-277 within the "else" clause that ends at line 264?

      Delete
    2. Yeh, it's probably a timing thing. On my MiniWI there was just the occasional stuck note, but on the TeensieWI it got sort of annoying, and the main loop is pretty much identical in both, as I just removed the function calls for the additional controls. The fix is exactly as you wrote it (line numbers are pretty neat to have, I just realized.. :)

      Delete
  9. Btw, check out my TeensieWI if you like, it's at https://github.com/Trasselfrisyr/MiniWI/tree/master/TeensieWI

    The built in touch sensing in the Teensy was surprisingly easy to use, though the documentation for it was sort of lacking...

    ReplyDelete
    Replies
    1. Very nice! And I see that you tackled the problem of de-bouncing (or, de-glitching, to be more precise). That was going to be the topic of one of my next posts.

      Have you experimented with the debounceDelay value? I suspect that larger values will make the instrument easier to play and would be appropriate for a beginner, but advanced players will want a shorter debounce delay. So making it easily changeable, e.g. via a potentiometer, seems like a nice feature to have.

      Delete
    2. Yes, I experimented with it a bit when I had just put the code in. Started with a higher value. With the higher lag precise tonguing to fingering changes gets hard, and trills get impossibly sluggish, but for someone just beginning to play wind instruments a higher value is probably useful. A potentiometer would be a great solution. I wonder if I've got an analog input left for it in my MiniWI... getting a bit cramped in there :) And the TeensieWI is so skinny, I don't think I can fit anything like that. Just 6mm of space between the plastic sheets. A drilled hole for a trimming screwdriver and a small trim pot with the legs bent to the sides, perhaps... I'll look into that :)

      https://www.instagram.com/p/BGpTv5HoVZF/?taken-by=trasselfrisyr
      https://www.instagram.com/p/BGpSw4coVV_/?taken-by=trasselfrisyr

      Delete