I've finally gotten back to working on my 3-D Trombone concept. It's been a while - work has gotten pretty crazy lately, but I've just gotta get back to this to exercise my creative outlets.
Here are a few photos of the electronics:
The large black unit in the center is the 3-position joystick I liberated from a Gametrak controller. You can see the joystick in the upper right. The circuit board in the upper left is just a wiring junction I built to make the electrical connections more secure. All the potentiometer connections and the power supply line come in/out on the ribbon cable you see (IDC connecors FTW!).
Here's another view where you can see some of the mechanics of the Game Trak controller. It's pretty simple, really - a gear system drives a potentiometer. The large enclosure houses a large coil spring that provides the return force for the z-axis.
There is also a breath sensor based on the same Freescale chip I used in my previous controller, built into a small project box and attached to the gametrak controller box:
I also have a controller that the player holds in the right hand. It consists of 5 monentary pushbutton switches used to select the overtone played. Here's a view of it:
The right hand controller and gametrak box connects to the same Arduino + MIDI Shield box I built for my other instrument (the new box is plug-compatible with the old one). Here's a view from the outside:
The controller electronics attach via the DB-25 connector. The switch on the top is used to select "load" mode, where the Arduino is able to download code from the IDE, and "run" mode where the serial pins are connected to the MIDI shield, which you can see in this photo of the inside:
Visible at the upper-left is the MIDI out port.
Ideas
I had an email conversation with Onyx Ashanti about the cool work he's doing on his Tron Beatjazz Controller. Onyx has some big goals for his work, and some of them could benefit my design, especially the hand-controllers he's working on - they'll have keys for each finger to operate, as well as an accelerometer. I think he's also incorporating some ThingMMaxM LEDs, which is very cool. ThingM's founder, Mike Kuniavsky, is a friend of mine and a former coworker at the University of Michigan.
The night after I had the email conversation, I had a dream that I was playing live at some sort of a club. I had an instrument sort of like my 3-D Trombone, but instead of the overtone controller, in my right hand I held a microphone, and to control the overtones, I moved my right hand up and down in the vertical axis. In my dream, I found it very natural, and was able to play melodies about as easily as on my trombone. I'm still not sure why I was holding a microphone in my dream.
I suspect this idea had been hanging around in my brain for a while, and my conversation with Onyx brought it to the front.
Naive Implementation
I took that idea and modified my 3-D Trombone Arduino sketch to read the Y-axis position of the gametrak controller and map the value, in a linear fashion, to an overtone.
In a nutshell, the result was unplayable. The problem is that, in order to maintain the "slide" position, the amount of "string" deployed out the gametrak controller has to be constant. So, in order to move up and down the harmonic series without changing slide position, you need to move your arm in an constant-radius arc, which is already pretty hard. But to make matters worse, the radius of the arc depends on which position you're in. So if you're in seventh position, the arc is several feet long, but if you're in first position, the arc is only one or two inches long. It produced some fun random effects flying up and down the harmonic series will glissando-ing in and out, but that would get old quick. Here's a picture of what I mean:
I have another idea about how to make this idea (vertical motion selects overtones, horizontal motion selects pitch) - a simple mapping of overtone to vertical position, pitch to horizontal position. Basically, a grid like this:
The X axis is the slide position - a trombone has seven of them, and the Y axis is the overtone. A good trombone player can play about 10 overtones, which corresponds to a range of about 3 octaves plus a few extra notes. So where a person playing an acoustic trombone would keep the slide in first position and move up and down the harmonic series with his/her embouchure, the player of the 3-D trombone would trade in the embouchure for up and down motion.
For those of you who may play trombone, here's how a B-flat major scale would look:
Implementation:
It turns out that it will be very hard to implement this "grid" idea using my current setup. The reason is that, due to the arrangement of the gametrak controller and the player (s/he holds it near the face), and the travel limits of the gametrak unit's joystick axes (about 90 degrees), it's not possible to cover the whole range of motion that might be interesting to a trombone player. So... back to the drawing board.
A more promising arrangement would be to use the Gametrak as it was intended to be used; base unit sitting on the floor. I'll investigate that and post on it in the future.
Over the past month or two, I've been slowly working on a refinement to the trombone controller, and I've finally got something that's physically stable and reasonably playable. Here's an update:
First, here's a photo of the prototype:
If the wooden handle (used to hold the instrument) looks like an axe handle to you... you'd be right. I'm still working on how the performer's left hand is involved, but for the time being, I've relegated it to a supporting role.
The major change between this prototype and the previous prototypes is the slide. The new prototype is essentially a copy of one half of Thomas Henriques Double Slide Controller. The basic construction is pretty simple, and involves:
A piece of 1/8" x 1 1/2" x 36" aluminum stock (Home Depot) - the substrate for the "trombone slide"
A 24" x 2" piece of plexiglass (Tap Plastics - a 15 minute walk from my house) - the bearing surfaces of the "slide"
A 5" x 3" piece of plexiglass, that holds:
Two 3/4" x 2 3/4" blocks of teflon, which I shaped with a router, and ride along the long plexiglass piece. These are equivalent to the outer parts of the "slide"
A repurposed joystick controller, which the player holds in his/her right hand. The controller handle is used to move the "slide", but also has 5 buttons that control overtone selection and a slide quantization mode (I'll explain that later). The buttons were epoxied into holes drilled into the joystick handle.
A project box that contains a Freescale pressure sensor, and all the wiring interconnects.
A "mouthpiece" that allows the performer to blow into the pressure sensor.
To set the stage, here's a little video that shows how the instrument is held, and how the slide moves:
Sensors
There are three types of sensors on the instrument:
Breath is detected via a Freescale pressure sensor. In the photo below, the mouthpiece (the clear plastic vinyl tubing that the player blows into) is connected to a box with some 1/4" tubing. The box contains the pressure sensor, along with the other wiring interconnects. At the top of the mouthpiece, there is a "T" connector that allows half of the airflow to exit (so the player feels like s/he is blowing through the instrument) and the rest goes to the pressure sensor. The Arduino code reads the sensor and produces MIDI Breath Controller data.
Slide Position is detected with a SpectraSymbol 500mm SoftPot linear potentiometer. The SoftPot is adhered to the aluminum stock using the adhesive backing provided with the SoftPot. The aluminum stock is screwed to a slightly wider piece of plexiglass. A mechanism slides along the edges of the plexiglass. The bearing surfaces are made of teflon block. I cut a groove in two pieces of the block using a router, and these grooves line up with and ride along the plexiglass, as shown below.
In the middle of the clear block, you can see what appears to be a setscrew. This is actually a stylus that is manufactured by SpectraSymbol. You can't see it in the photo, but the end of the screw is a small nylon stylus that rides on a small spring that provides a constant force. This makes the pressure on the SoftPot very consistent, which means the slide behavior is very predictable.
The Arduino code reads the slide position and computes the appropriate pitch bend to send. My most recent firmware also includes a mode where the slide positions are quantized - in effect, the slide "clicks into position" automatically. Although a glissando isn't possible with this setup, the instrument's notes are always in tune. The slide quantization mode can be toggled on and off using the thumb of the right hand (there was a spare button on the joystick handle that was perfect for that).
A future enhancement I'm considering is providing an LED on the instrument that lights up when the player has the slide in one of the seven positions (well, actually, within a certain range of the dead-on position). Trombonists are used to reaching out to touch the bell to gauge where 3rd position is; this LED would do the same thing, but for all seven positions.
Overtone Selection is accomplished via a set of four switches on the handle that the player uses to move the slide. Here's a picture of the handle:
The handle is a repurposed joystick, with some momentary switches epoxied into some (very crudely drilled) holes I made in the handle. The "trigger" button is actuated with the index finger, and the remaining buttons are operated with the second, third, and fourth fingers. By using a simple "chording" method, the player can select any one of eight overtones:
Overtone 0: off off off off Overtone 1: on off off off Overtone 2: on on off off Overtone 3: on on on off Overtone 4: on on on on Overtone 5: off on on on Overtone 6: off off on on Overtone 7: off off off on
Now, only 8 overtones is not really enough to make a trombone player feel at home (most accomplished players can produce at least 10), so I need to think about this some more, but 8 overtones does give me enough range to play stuff that's interesting.
Here's a short video of me playing a little improvisation on the instrument. I still have some work ahead of me to produce a patch that makes the instrument play well. The patch you're hearing is one I built for the ES2 FM synth that comes with Apple Logic Express.
I have to say that, while the instrument is still pretty glitchy, I'm starting to feel like it's possible to be expressive with it. I'm also feeling like the instrument is pretty consistent; it behaves predictably, which allows me to practice a musical passage, get it right, and then to be able to perform it in a repeatable fashion.
Finally, as a point of comparison addressing the expressiveness of the instrument, here's an improvisation in the same vein, but on my trombone.
After trying to place the overtone selector switches on the left hand, I've decided that's no good - it's too hard to hold the instrument with the left hand and also actuate the switches. So I've decided to copy the general layout of Thomas Henrique's Double Slide Controller and put the overtone selectors on the right hand.
To accomplish that, I'll need some sort of a handle I can put tactile switches on. After I thought about it for a while, I realized what I needed would look an awful lot like a joystick handle. So, after a quick trip to Weird Stuff Warehouse, I had two old joysticks ($8 total). Time for disassembly!
I picked up two different joysticks, both of which have a large surface area on the right where I should be able to mount buttons. Also, the joysticks have some existing buttons that could be used for other functions, like patch changing.
I started with the simpler, two-button joystick. Off with the bottom plate...
And expose the inside of the handle, to see what's in there. Pretty simple, really - two switches, and 3 wires out the bottom.
The base housed the X-Y potentiometers. I won't be needing them, so out they came, which released the handle.
Verifying the wiring of the switches - as expected, green is ground.
So that handle is ready to try out. I'll need to attach it to the new slide I'm building. More on that in a later post.
I also bought a Wingman joystick. This one is more complicated - there's a trigger, three pushbuttons, and a small thumb-actuated joystick. Those all could be interesting to use.
Getting the handle out required some... surgery. Let's just say the warranty is definitely voided now.
There are seven wires coming out the bottom. I tried a little "black box reverse engineering" to see what leads corresponded to what switch, but that proved inconclusive.
So I opened up the handle. And all the switches promptly fell out. Everything is held in place by the two halves of the handle. I eventually got everything back together, but it wasn't easy.
Next up: order some sample tactile switches from Mouser and find just the right one. I'll use the joystick's trigger for the index finger, so I'll need three additional switches for the other three fingers.
My latest experiment with overtone selection on the trombone controller is to use four momentary switches, played with the left hand, to select overtones. I used some Radio Shack lever switches and epoxied them to a 1/2" by 1/2" piece of scrap wood I had, then tie-wrapped it to the handle of the instrument (hey, I'm just prototyping).
The four switches are wired to pull Arduino digital pins 2, 3, 4, and 5 to ground when pressed, and I coded up my sketch to give the following overtones for the given switch selections:
Switch 0 is under the index finger, and switch 3 is under the pinky. Here's a short video showing how it is played:
In terms of playability, it feels pretty good. I can more or less play a scale and the fingers of the left hand will generally do the right thing.
Here's the sketch:
/*
Prototype sketch for a trombone-like MIDI controller based on the Arduino hardware.
Hardware:
- An set of four switches used to select an overtone. We use "chording" to allow the 4 switches to select overtones. I'm not sure what the most natural method of chording is, but let's try the following:
Switches 0-3 are wired to pull Arduino digital input pins 2-5 low when pressed.
- A "slide". Currently, this produces pitch bend information, and is implemented with a 500mm SpectraSymbol SoftPot linear resistance strip.
- A volume controller, implemented with a FreeScale pressure sensor. The player blows into a tube that goes to a "T" - one leg goes to the pressure sensor, and the other is open (a "dump tube") so that the player can put air through the instrument.
Feb 9, 2010 Gordon Good (velo27 yahoo com)
*/ #include <MidiUart.h> #include <Midi.h>
MidiClass Midi;
// If DEBUG == true, then the sketch will print to the serial port what // it would send on the MIDI bus. const boolean DEBUG = false; //const boolean DEBUG = true;
const int BREATH_PIN = 0; // Breath sensor on analog pin 0 const int SLIDE_LPOT_PIN = 1; // Slide sensor on analog pin 1
const int OT_SW_0_PIN = 2; // Overtone switch 0 const int OT_SW_1_PIN = 3; // Overtone switch 1 const int OT_SW_2_PIN = 4; // Overtone switch 2 const int OT_SW_3_PIN = 5; // Overtone switch 3
const int PANIC_PIN = 6; // MIDI all notes off momentary switch on digital I/O 6
// The overtone series this instrument will produce const int FUNDAMENTAL = 36; // MIDI note value of our fundamental const int OT_1 = 48; // First overtone (B flat) const int OT_2 = 55; // Second overtone (F) const int OT_3 = 60; // Third overtone (B flat) const int OT_4 = 64; // Fourth overtone (D) const int OT_5 = 67; // Fifth overtone (F) const int OT_6 = 70; // Sixth overtone (A flat - not in tune - need to tweak pitch bend) const int OT_7 = 72; // Seventh overtone (B flat) const int OT_8 = 74; // Eighth overtone (C) const int OT_9 = 76; // Ninth overtone (D) const int OT_NONE = -1; // No overtone key pressed (not possible with ribbon)
// All overtones for this instrument const int overtones[10] = {FUNDAMENTAL, OT_1, OT_2, OT_3, OT_4, OT_5, OT_6, OT_7, OT_8, OT_9}; // Switch values for given overtones. 0xff means that overtone can't be selected. const int overtone_sw_values[10] = {0xff, 0x00, 0x01, 0x03, 0x07, 0x0f, 0x0e, 0x0c, 0x08, 0xff};
const int MIDI_VOLUME_CC = 7; // The controller number for MIDI volume data const int MIDI_BREATH_CC = 2; // The controller number for MIDI breath controller data
long ccSendTime = 0; // Last time we sent continuous data (volume, pb); const int MIN_CC_INTERVAL = 10; // Send CC data no more often than this (in milliseconds); const int PB_SEND_THRESHOLD = 10; // Only send pitch bend if it's this much different than the current value const int VOLUME_SEND_THRESHOLD = 1; // Only send volume change if it's this much differnt that the current value const int NOTE_ON_VOLUME_THRESHOLD = 50; // Raw sensor value required to turn on a note
// If a value larger than this is read from a SoftPot, treat it as if the player is not touching it. // Note: for some reason, the two SoftPots interact, e.g. just actuating the slide pot gives me // no-touch values all above 1000, but when also touching the overtone pot, the values can go // as low as 999. I suspect I may be taxing the 5v supply line. const int LPOT_NO_TOUCH_VALUE = 1010;
int currentNote = -1; // The MIDI note currently sounding int currentPitchBend = 8192; // The current pitch bend int currentVolume = 0; // The current volume
/** * Enable a pin for analog input, and set its internal pullup. */ void enableAnalogInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin + 14, enablePullup ? HIGH : LOW); }
/** * Enable a pin for digital input, and set its internal pullup. */ void enableDigitalInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin, enablePullup ? HIGH : LOW); }
/** * Read the slide pot and return a pitch bend value. The values * returned are all bends down from the base pitch being played, * and are in the range 8192 (no bend) to 0 (maximum bend down). * This means that the synth patch needs to be adjusted to provide * a maximum pitch bend of seven semitones, if you want it to * behave like a trombone. * * Return -1 if the player is not touching the sensor. */ int getPitchBendFromLinearPot() { // Get the raw value from the linear pot int pbRawVal = analogRead(SLIDE_LPOT_PIN); if (pbRawVal > LPOT_NO_TOUCH_VALUE) { return -1; } else { return map(pbRawVal, 0, LPOT_NO_TOUCH_VALUE, 0, 16383 / 2); } }
int getPitchBend() { return getPitchBendFromLinearPot(); }
/** * Read the overtone switches and return the appropriate overtone. * If an invalid key combination is found, return -1. Note that * we invert the values from digitalRead, since these switches * pull to ground, so switch enabled = digital 0. */ int getOvertoneFromOvertoneSwitches() { unsigned char val = !digitalRead(OT_SW_3_PIN); val = val << 1 | !digitalRead(OT_SW_2_PIN); val = val << 1 | !digitalRead(OT_SW_1_PIN); val = val << 1 | !digitalRead(OT_SW_0_PIN); // now select the appropriate overtone for (int i = 0; i < sizeof(overtone_sw_values); i++) { if (val == overtone_sw_values[i]) { return i; } } return -1; }
int getMIDINote() { int ot = getOvertoneFromOvertoneSwitches(); if (-1 == ot) { return currentNote; } else { return overtones[ot]; } }
/** * Read the breath sensor and map it to a volume level. For now, * this maps to the range 0 - 127 so we can generate MIDI * continuous controller information. */ int getVolumeFromBreathSensor() { int volRawVal = analogRead(BREATH_PIN); if (volRawVal < NOTE_ON_VOLUME_THRESHOLD) { return 0; } else { return map(constrain(volRawVal, 30, 500), 30, 500, 0, 127); } }
int getVolume() { return getVolumeFromBreathSensor(); }
void allNotesOff() { for (int i = 0; i < 128; i++) { sendNoteOff(i, 0, 1, DEBUG); } }
void loop() {
if (digitalRead(PANIC_PIN) == 0) { allNotesOff(); }
int pb = getPitchBend(); int note = getMIDINote(); int volume = getVolume();
if ((-1 != currentNote) && (0 == volume)) { // Breath stopped, so send a note off sendNoteOff(currentNote, 0, 1, DEBUG); currentNote = -1; } else if ((-1 == currentNote) && (0 != volume) && (-1 != note)) { // No note was playing, and we have breath and a valid overtone, so send a note on sendNoteOn(note, 127, 1, DEBUG); currentNote = note; } else if ((-1 != currentNote) && (note != currentNote)) { // A note was playing, but the player has moved to a different note. // Turn off the old note and turn on the new one. sendNoteOff(currentNote, 0, 1, DEBUG); sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); sendNoteOn(note, 127, 1, DEBUG); currentNote = note; } else if (-1 != currentNote) { // Send updated breath controller and pitch bend values. if (millis() > ccSendTime + MIN_CC_INTERVAL) { sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); ccSendTime = millis(); } } delay(50); }
Tonight I experimented with placing a 100 mm linear pot on the handle of the "trombone" instrument, where the player holds it with the left hand. By touching the pot with one of the four fingers of the left hand, the player is able to select one of five partials (no fingers, one finger, ... four fingers). With this arrangement, I was able to play a decent taps:
I also found that playing trombonistically was a lot more natural with this arrangement. For example, if you're playing F (2nd overtone) and want to go up to G, on a trombone, you'd go from first to fourth position, and blow up to the next partial. On my instrument, you would go from first to fourth position, and put the next highest finger down. When I tried this, my body just sort of did it naturally, probably because the physical orientation of the overtone selection was the same in my brain (up).
Now, since there are only 4 fingers to work with (and possibly the thumb, if I can free it up from its job of keeping me from dropping the instrument), to cover the typical 8 partial + range of the instrument, to make this work, we may need to figure out some sort of "chording" for the fingers of the left hand. One thing that occurs to me right away is to use the one-finger-per-overtone approach for the lower partials, then bring the other fingers back into the picture, e.g. (fingers are numbered 1 = index, 4 - pinky)
If you're a trombonist, you see that this is missing a few more playable partials (probably another 4-6 semitones is required of the physical instrument).
I don't have any good answers for how to solve these problems yet, but I'm revisiting my assumption that left-hand control of the overtone series is a dead-end. If I follow up on this approach, I think a set of momentary switches would work out a lot better than the linear pot.
On that left-hand-is-a dead-end front, I built a prototype mouthpiece with a baffle that splits the airflow into two vertically separated streams. After the epoxy hardens, I plan to use this to investigate the feasibility of using embouchure "gestures" as overtone selectors. It may be a total bust, but if I can make it work, I think it might make the instrument a lot more playable.
Tonight, I wired up a Force Sensitive Resistor (FSR) so that it would select an overtone on my MIDI trombone. I put it where the performer grips the instrument so that it could be actuated by the performer's left thumb. Then, I tried to play it myself (by trying to move the slide and actuating the overtone selector). The results were disappointing. Switching between partials required a far more subtle gesture than I was able to produce.
Today I figured out that I was wrong in my last post - I didn't destroy the 500mm SoftPot. The output of the pot does float, which I discovered when I added another SoftPot to use as an overtone controller, and it behaved the same way. By enabling the internal pullups on my Arduino, I was able to solve that problem. It also got rid of the last external components - all the sensors plug directly into the Arduino now, and no external pullups or any other components are needed. Not that that was a goal, but it's nice when re-assembling the thing.
That means it's possible for me to detect when the player has let go of the slide or overtone sensor, and maintain the previous value. That makes it possible to move from one note to another on the slide without producing a glissando, and also allows jumping from one partial to another without sounding the partials in between.
I also added code to produce note on/off events based on the breath controller input. When the player blows above some threshold value, a note on is sent, and when s/he stops blowing, a note off is sent.
And, finally, I added a MIDI panic button, which sends note off events for all notes on the instrument's channel.
Here are some views of the prototype:
Top View
The breath tube is on top of the wooden post. A T connector routes half of the air to the black box on the horizontal bar, which houses the actual breath sensor. The second box, with the labels, just routes voltages and signals to/from the sensors. The grey cable at the bottom right is an Ethernet cable, which I use to connect the instrument to the Arduino/breadboard. The other inputs into the box are OT (overtone sensor), BR (breath sensor), and SL (slide), and are built using 1/8" stereo jacks to supply 5v, ground, and signal. Note to self: depending on how you wire the jacks, the 5v may short to ground while being plugged/unplugged. It was a bad choice to use them for this application.
Bottom View
The wooden rod protruding below is where the player holds the instrument. Immediately behind the handle is a 100mm SoftPot that the player uses to select a partial with the left thumb (a trombonist would use breath and embouchure to overblow a different partial). Sliding back and forth will run the instrument up and down the partial series, like this:
Top View
This view shows the "slide", a 500mm SoftPot. As described in a previous post, as the player moves his/her finger up and down, the instrument sends pitch bend values to the MIDI bus, allowing glissando, like this:
Thoughts and Future Direction
I don't think the instrument plays very "trombonistically" yet. Using a finger to actuate slide feels pretty unnatural, and it's very easy for the player's finger to slip off the SoftPot. It might work better to use a "stylus" that pressed on the pot, and allow the player to move a handle that hangs below the slide and moves the stylus along with it. That will probably behave more like a trombone slide.
I also find the overtone selector hard to use in a reliable fashion. There are some other possibilities I can explore, e.g. flex sensors, but I have some serious doubts that the whole idea of using the left hand to select overtones may not work out. On that front, I want to think about ways of allowing an "overblowing" gesture that is familiar to trombonists to be used for overtone selection.
Summary
Although the current prototype doesn't translate trombone gesrtures all that well, I think it is an interesting MIDI controller, and has some interesting expressive qualities that I haven't yet explored. I would like to spend some time programming some soft synths to take advantage of the instrument, and compose some examples using the controller.
Here's the sketch:
/*
Prototype sketch for a trombone-like MIDI controller based on the Arduino hardware.
Hardware:
- An overtone selector. A SpectraSymbol 100mm SoftPot linear resistance strip, actuated by the player's left thumb.
- A "slide". Currently, this produces pitch bend information, and is implemented with a 500mm SpectraSymbol SoftPot linear resistance strip.
- A volume controller, implemented with a FreeScale pressure sensor. The player blows into a tube that goes to a "T" - one leg goes to the pressure sensor, and the other is open (a "dump tube") so that the player can put air through the instrument.
Jan 17, 2010 Gordon Good (velo27 yahoo com)
*/ #include "Midi.h"
// If DEBUG == true, then the sketch will print to the serial port what // it would send on the MIDI bus. const boolean DEBUG = false; //const boolean DEBUG = true;
const int BREATH_PIN = 0; // Breath sensor on analog pin 0 const int SLIDE_LPOT_PIN = 1; // Slide sensor on analog pin 1 const int OT_LPOT_PIN = 2; // Overtone sensor on analog pin 2
const int PANIC_PIN = 2; // MIDI all notes off momentary switch on digital I/O 2
// The overtone series this instrument will produce const int FUNDAMENTAL = 36; // MIDI note value of our fundamental const int OT_1 = 48; // First overtone (B flat) const int OT_2 = 55; // Second overtone (F) const int OT_3 = 60; // Third overtone (B flat) const int OT_4 = 64; // Fourth overtone (D) const int OT_5 = 67; // Fifth overtone (F) const int OT_6 = 70; // Sixth overtone (A flat - not in tune - need to tweak pitch bend) const int OT_7 = 72; // Seventh overtone (B flat) const int OT_8 = 74; // Eighth overtone (C) const int OT_9 = 76; // Ninth overtone (D) const int OT_NONE = -1; // No overtone key pressed (not possible with ribbon) const int overtones[10] = {FUNDAMENTAL, OT_1, OT_2, OT_3, OT_4, OT_5, OT_6, OT_7, OT_8, OT_9};
const int MIDI_VOLUME_CC = 7; // The controller number for MIDI volume data const int MIDI_BREATH_CC = 2; // The controller number for MIDI breath controller data
long ccSendTime = 0; // Last time we sent continuous data (volume, pb); const int MIN_CC_INTERVAL = 10; // Send CC data no more often than this (in milliseconds); const int PB_SEND_THRESHOLD = 10; // Only send pitch bend if it's this much different than the current value const int VOLUME_SEND_THRESHOLD = 1; // Only send volume change if it's this much differnt that the current value const int NOTE_ON_VOLUME_THRESHOLD = 50; // Raw sensor value required to turn on a note
// If a value larger than this is read from a SoftPot, treat it as if the player is not touching it. // Note: for some reason, the two SoftPots interact, e.g. just actuating the slide pot gives me // no-touch values all above 1000, but when also touching the overtone pot, the values can go // as low as 999. I suspect I may be taxing the 5v supply line. const int LPOT_NO_TOUCH_VALUE = 990;
Midi midi(Serial);
int currentNote = -1; // The MIDI note currently sounding int currentPitchBend = 8192; // The current pitch bend int currentVolume = 0; // The current volume
/** * Enable a pin for analog input, and set its internal pullup. */ void enableAnalogInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin + 14, enablePullup ? HIGH : LOW); }
/** * Enable a pin for digital input, and set its internal pullup. */ void enableDigitalInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin, enablePullup ? HIGH : LOW); }
/** * Read the slide pot and return a pitch bend value. The values * returned are all bends down from the base pitch being played, * and are in the range 8192 (no bend) to 0 (maximum bend down). * This means that the synth patch needs to be adjusted to provide * a maximum pitch bend of seven semitones, if you want it to * behave like a trombone. * * Return -1 if the player is not touching the sensor. */ int getPitchBendFromLinearPot() { // Get the raw value from the linear pot int pbRawVal = analogRead(SLIDE_LPOT_PIN); if (pbRawVal > LPOT_NO_TOUCH_VALUE) { return -1; } else { returnmap(pbRawVal, 0, LPOT_NO_TOUCH_VALUE, 0, 16383 / 2); } }
int getPitchBend() { return getPitchBendFromLinearPot(); }
/** * Read the overtone pot and select the appropriate MIDI note from * the overtone table. Return -1 if the player is not touching the pot. */ int getOvertoneFromOvertoneLinearPot() { int val = analogRead(OT_LPOT_PIN); if (val > LPOT_NO_TOUCH_VALUE) { return -1; } else { returnmap(constrain(val, 0, 900), 0, 900, 9, 0); // Map to an overtone number } }
int getMIDINote() { int ot = getOvertoneFromOvertoneLinearPot(); if (-1 == ot) { return currentNote; } else { return overtones[ot]; } }
/** * Read the breath sensor and map it to a volume level. For now, * this maps to the range 0 - 127 so we can generate MIDI * continuous controller information. */ int getVolumeFromBreathSensor() { int volRawVal = analogRead(BREATH_PIN); if (volRawVal < NOTE_ON_VOLUME_THRESHOLD) { return 0; } else { returnmap(constrain(volRawVal, 30, 500), 30, 500, 0, 127); } }
int getVolume() { return getVolumeFromBreathSensor(); }
void allNotesOff() { for (int i = 0; i < 128; i++) { sendNoteOff(i, 0, 1, DEBUG); } }
voidloop() {
if (digitalRead(PANIC_PIN) == 0) { allNotesOff(); }
int pb = getPitchBend(); int note = getMIDINote(); int volume = getVolume();
if ((-1 != currentNote) && (0 == volume)) { // Breath stopped, so send a note off sendNoteOff(currentNote, 0, 1, DEBUG); currentNote = -1; } elseif ((-1 == currentNote) && (0 != volume) && (-1 != note)) { // No note was playing, and we have breath and a valid overtone, so send a note on sendNoteOn(note, 127, 1, DEBUG); if (note == -1) { Serial.println("OOPS 1"); } currentNote = note; } elseif ((-1 != currentNote) && (note != currentNote)) { // A note was playing, but the player has moved to a different note. // Turn off the old note and turn on the new one. sendNoteOff(currentNote, 0, 1, DEBUG); sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); sendNoteOn(note, 127, 1, DEBUG); if (note == -1) { Serial.println("OOPS 2"); } currentNote = note; } elseif (-1 != currentNote) { // Send updated breath controller and pitch bend values. if (millis() > ccSendTime + MIN_CC_INTERVAL) { sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); ccSendTime = millis(); } } delay(50); }
Tonight I made another baby step with the trombone controller. I got the slide sensor (see FAIL* below) and the breath controller both working, mounted on the instrument, and added an RJ-45 connector so I can hook the thing up to the Arduino via an Ethernet cable, unclamp it from the workbench, and pick it up and play it. So, for the first time, I was able to get a feel for how playable this thing will be.
I was pleasantly surprised. While there are a lot of glitches, I felt like I was able to be expressive on the instrument. Here's a video:
The whole air pressure system is made from garden drip irrigation tubing I had laying around. I really hope it's non-toxic.
*FAIL (or, how to blow up a $27 linear potentiometer and an $11 pressure sensor)
In my quest to make all the pieces of my new instrument modular, I decided that I would terminate each sensor in some sort of connector that could be plugged/unplugged to assemble new, unanticipated instruments. Well...
Don't use 1/8" stereo phono jacks for anything carrying Vcc. They short as they're being plugged/unplugged, which (thank you, Arduino designers) doesn't make the magic smoke come out of the Arduino, but it does make your Mac grumpy with the excessive current on the USB bus - but no Mac damage either (thank you, Mac designers).
Measure (er, check wiring) twice. Or maybe 5 or 6 times. I managed to wire 5v across the wrong two leads of my quite expensive SpectraSymbol SoftPot, and now when no pressure is applied, the output lead floats. Well, at least Mouser Electronics gets something out of this. :-(
Freescale Pressure Sensors don't care that your low-wattage soldering iron sucks and won't melt solder and you pulled out the big iron to solder the sensor to a perfboard. Moral of the story: socket everything in a prototype!
This weekend, I had two thoughts about the design of my trombone controller, while watching Media Lab's Podcast (LabCast) about the Chameleon Guitar.
Thought 1: While the instrument they built is very different from the one I'm building, what struck me was the statement:
"Computers have radically changed our lives over the last fifteen years. However, the design of the electric guitar has hardly changed in the last fifty. Is this because musicians respond more to tradition, than to innovation, or, is it because the right interface has yet to be developed? (emphasis mine). The project argues that combining the unique traditional values of established musical instruments with modern technology can open the door to a myriad of new opportunities.
This resonated well with me, and it's encouraged me to finally write down a "manifesto" that captures my ideas about musical instrument design (the last three points are more about open source).
The instruments I design should:
reward the mastery of the performer. It should be possible to become a virtuoso on one of the instruments I design.
translate gesture well for skilled performers. This might mean that an instrument is idiomatically "correct" for a performer familiar with a traditional instrument. Or it might mean that a musically skilled person can instinctually control my instrument in a way that is pleasing both to the performer and her audience.
be reproducible by others. This means that all software and hardware will be open source.
encourage further experimentation and extension.
not impose any restrictions on their use. I want performers of instruments of my design to be able to do anything they wish with the music they make.
Thought 2: There is a lot of value in making the instrument beautiful to look at and pleasing to touch.
This thought occurred to me while watching the section in the Media Lab video where they describe how they collaborated with a string instrument builder to build the neck and body shell of the instrument.
My first thoughts about my instrument's design centered around metals, which makes sense when you consider that I spend years blowing air into a hunk of brass. But once I saw the Media Lab video, it occurred to me that there's no really good reason that I couldn't build the thing out of wood. And, given that, there's no reason I couldn't find someone with woodworking skill to make the final product really beautiful, and furthermore, there's no reason I couldn't make an interesting prototype myself, using some simple tools.
So, this afternoon I went to Home Depot and bought a 6-foot long section of 1 x 2 pine, a 3/4" dowel, and a SurForm tool, and stopped at Fry's Electronics and bought a connector I thought would allow the SpectraSymbol sensor to plug in (soldering directly to the leads would certainly destroy it).
Then, this evening I spent about an hour forming the 1x2 into the "slide" of my trombone controller prototype out of that 1x2. The slide ends up looking more like the fingerboard of a string instrument than a trombone slide(see photo). The player places his/her index finger on the top, where the sensor is, and the thumb underneath. Here's a (not very good) photo of the result, clamped to my workbench for testing (click to view a larger image):
The SpectraSymbol sensor is great - only a very light pressure is required to maintain a reading, and it seems to have very little jitter when I run a sketch that prints the values on an analog input pin.
I then hooked up the set of momentary switches I use as a temporary overtone selector, and tried to play a major scale. It's glitchy, but clearly recognizable as an ascending/descending scale. I'll be addressing the glitchiness later in software.
Next up: get the breath sensor attached to the prototype, and then figure out how to let the player control the overtones with the left hand.
With my blow-tube from experiment 6 hooked up to this sensor, through a T connector with half the flow going to the sensor and half dumped, the zero to full response feels pretty natural to me (although I haven't played my trombone in a looooong time). The raw values read from the sensor are about double those of the sensor I used in Experiment 6.
Physical Interfacing
The sensor comes in a package designed for SMD mounting, so I straightened the legs and soldered it to some dual headers.
To hook up the air supply, I attached some heat shrink tubing (unsure of the diameter) to the input port and gently heated it to get a seal.
This tubing, it turns out, has an outer diameter very close to the inner diameter of 1/4" garden drip irrigation tubing, which is what I hooked up to my blow-tube. So sliding the irrigation tubing over the heat shrink and overlapping for a couple of inches makes a good-enough seal.
Electrical Interfacing
Easy:
+5V to pin 2
Ground to pin 3
pin 4 to analog input of Arduino. Raw input values from the Arduino ADC range from 58 (no blowing) to 1015 (blowing like I'm playing trombone on Varese's Integrales).
My initial experiment with using a pressure sensor for breath pressure detection didn't turn out so well, because the sensor I chose responded to a much higher pressure than I could produce. So I ordered a couple of lower pressure sensors to experiment with. The first one I'm trying is a Freescale MPX5010GSX. It costs more ($14 from Mouser) than the previous one I tried but it's not so expensive as to be unreasonable.
The sensor comes in a nice 6-pin package with leads that will plug right into a breadboard. Pin 1 has a little notch in it, so it's easy to identify. Pin 1 is the output, Pin 2 is ground, and pin 3 is a 5v supply voltage. Pins 4, 5, and 6 are labeled as V1, V2, Vex, but I'm not sure what those do. I left them floating.
To drive the input port, I created a "mouthpiece" of a 6-inch length of one-inch vinyl tubing (bought at the local Ace Hardware). I folded over one end and clamped it shut with a couple of clamps. Then I used an awl from an irrigation tubing kit to open a hole near the clamped-off end and attached a T-connector for 1/4" irrigation tubing. I attached one end of the T to the pressure sensor through a couple feet of 1/4" tubing (it fits perfectly over the sensor), and ran the other end of the T through about a foot of 1/4" tubing. This will be the "blow through" for the sensor, and by clamping it off partially, I can control how much air goes through the instrument.
Here's a picture:
The pressure sensor is on the breadboard at the bottom of the photograph. The tube to blow into is on the right, as is the dump tube. The orange clamps close off the end of the blow tube, forcing all of the air into the tubes that flow (a) to the pressure sensor and (b) to the dump tube.
I then wrote a small Arduino sketch that printed the values on analog port 2 every 100 milliseconds and ran it.
(Sketch)
/*
Print values received from a Freescale MPX5010GSX pressure sensor
Gordon Good (velo27 yahoo com)
*/ const int SENSOR_PIN = 2; // Sensor Vout on analog input 2
voidsetup() { Serial.begin(9600); }
voidloop() { int pVal = analogRead(SENSOR_PIN); Serial.println(pVal); delay(100); }
Results:
With the dump port completely closed off, and blowing my brains out, the highest reading I can get is about 1015, not much below the maximum sensor value of 1023. With the dump port wide open, the max value is about 650. Wide open feels pretty good to me in terms of how hard I'm blowing, so I think we're in the ballpark with this sensor.
So, in other words, a comfortable "loud blow" for me, with a comfortable amount of air dumped, produces a value of about 1/2 scale on a 10 kilopascal sensor. Assuming the response is linear, the pressure I'm producing is about 5 kilopascals.
For a long time now I've been thinking of how I could build an electronic wind instrument that would be easy for a trombone player to pick up. Although I've owned a WX-7 and have played around with the Akai EVI and EWI, I've never had the time/patience to get the finger chops needed to enjoy those instruments.
Once I found out about the Arduino and the various types of sensors that are available cheaply, it seemed like the right time to get serious about designing a trombone controller.
My current working plans are to use three sensors:
A breath sensor that controls articulation and volume.
A position sensor of some kind that can detect the slide position.
A set of switches, or a position sensor that controls which overtone is selected. I envision that this will be actuated with the player's left thumb, where the F-attachment lever is usually found.
In other words, pretty much like an Akai EVI/Steinerphone, except that we replace the valve buttons with a slide, and we replace the pitch rollers + drop-a-fourth button with the overtone selector.
There are a lot of different possibilities for what kind of sensors to use for each of these, and I'll need to do some experimentation to find what works best. But I don't need to figure that out before I can start working on the code - the Arduino lets me abstract away those issues. So, really, we have:
An analog input for the breath sensor.,
an analog input for the slide position, and
a set of switches for selecting the overtone.
Regarding the code: there are two main paths I might choose:
Make the whole thing a MIDI controller, and have the Arduino send MIDI to a synthesizer (or computer + softsynth, etc).
Use something like Max/MSP or pd to read the controllers directly and control synths implemented in software on the Mac/PC.
The advantages of #1 are:
You can take advantage of any existing synth patch, although you do have the problem every wind controller player has in finding/creating patches that respond well.
This approach is faster for me to implement, since I don't know Max/MSP.
The advantages of #2 are:
You have more control of how the synth responds to the instrument.
You aren't limited by MIDI's coarse-grained values (if those are in fact a problem).
The nice thing is that I'm not limited to one or the other approach. All you need is different firmware on the Arduino (heck, just get another one and plug into whichever one you want to use).
So I decided to start down the MIDI path. Just to get started, I built the following circuit to prove some concepts:
The "slide" is a Sharp Infrared sensor, the "breath controller" is a force sensing resistor, and the overtone selector is 5 separate momentary pushbutton switches.
I wrote a sketch that reads the values of each sensor/switch in a loop. For this first iteration, I decided to focus on making the overtone selector and the slide work well, and not worry so much about when note on/off messages are sent. So this thing is always "on", and a new note is only started when a different register button is pressed.
I also opted in this iteration to use pitch bend messages exclusively for pitch within a given overtone. In other words, the only note on values we ever send are the fundamental, the first overtone, the second overtone, etc. All the notes in between are produced by sending a downward pitch bend message. This requires that the synth be set so that maximum bend is six semitones, corresponding to the seven positions of the trombone slide.
On a breadboard, obviously, you can't twiddle all these controllers at once, so I first verified that the overtone selector worked properly. I then checked that tapping/holding/squeezing the force sensor produced something like a reasonable articulation and volume changes. Then I just set up the firmware to send a constant value instead of reading the force sensor (I only have two hands) and verified that the "slide" did something reasonable.
Here's a short sample:
In the first part, I'm moving up and down the overtone series with the slide stationary, and in the second part I "glissando" down from three diffrerent overtones.
In summary, this all looks promising. Next steps:
Find a pressure sensor to use for a breath controller. The first one I tried required far too much pressure.
Figure out how to physically make the slide, and experiment with different sensors. Some other ideas include using nichrome wire as a sensor, and position-sensing resistors (these are expensive in the size I need - about 2 feet long).
I'm mostly a software guy, so actually making the physical instrument will be a big challenge.
Here's the sketch.
/*
Prototype sketch for a trombone-like MIDI controller based on the Arduino hardware.
Gordon Good (velo27 yahoo com)
Hardware:
- An overtone selector. At this point, it's a series of discrete on-off pushbottons. This replaces the act of blowing overtones an a physical instrument.
- A "slide". Currently, this produces pitch bend information, and is implemented using an infrared distance sensor.
- A volume controller, intended to be actuated by the player's breath.
const int FSR_PIN = 0; const int IR_PIN = 1; const int OT_0_PIN = 2; const int OT_1_PIN = 3; const int OT_2_PIN = 4; const int OT_3_PIN = 5; const int OT_4_PIN = 6;
const int FUNDAMENTAL = 36; // MIDI note value of our fundamental const int OT_1 = 48; // First overtone const int OT_2 = 55; // Second overtone const int OT_3 = 60; // Third overtone const int OT_4 = 64; // Fourth overtone const int OT_NONE = -1; // No overtone key pressed
const int MIDI_VOLUME_CC = 7;
long ccSendTime = 0; // Last time we sent continuous data (volume, pb); const int MIN_CC_INTERVAL = 10; // Send CC data no more often than this (in milliseconds);
int getPitchBendFromForceSensor() { // Get the raw value from the force sensor int pbRawVal = analogRead(FSR_PIN); // Constrain it to values we're willing to handle int pbConstrainedVal = constrain(pbRawVal, 0, 1023); // Then map those values to the MIDI pitch bend range values. returnmap(pbConstrainedVal, 0, 1023, 0, 16383); }
int getPitchBendFromIRSensor() { // Get the raw value from the IR sensor int pbRawVal = analogRead(IR_PIN); // Constrain it to values we're willing to handle int pbConstrainedVal = constrain(pbRawVal, 50, 640); // Then map those values to the MIDI pitch bend range values. // NOTE: We stick our head in a hole here and assume the IR // sensor response is linear. In fact, it's not, so we need to // come back and improve this. The current implementation will // result in a trombone slide that is non-linear. returnmap(pbConstrainedVal, 640, 50, 16383 / 2, 0); }
int getNeutralPitchBend() { return 8192; }
int getPitchBend() { return getPitchBendFromIRSensor(); }
int getMIDINote() { // middle C = 60, so let's use two octaves below middle c as the fundamental of this instrument for now if (digitalRead(OT_4_PIN) == 1) { return OT_4; } elseif (digitalRead(OT_3_PIN) == 1) { return OT_3; } elseif (digitalRead(OT_2_PIN) == 1) { return OT_2; } elseif (digitalRead(OT_1_PIN) == 1) { return OT_1; } elseif (digitalRead(OT_0_PIN) == 1) { return FUNDAMENTAL; } else { // No overtone key pressed - return -1 so caller can know return OT_NONE; } }
int getFixedVolume() { return 127; }
int getVolumeFromFSR() { // Temporary code, since I don't have a good breath // sensor yet. However, initial tests with a force sensing resistor // indicate that it's possible to do decent articulation with the // following code. int volRawVal = analogRead(FSR_PIN); returnmap(volRawVal, 0, 970, 0, 127); }
I'm a software engineer with VMware in Palo Alto, CA, and a musician who likes to build crazy musical instruments from microcontrollers. The latest thing I'm working on sonifying air traffic control data.
You will also find a series of blog posts covering the basics of electronic wind instrument controllers - electronic instruments controlled by the performer's breath.
While I can't claim to be an expert in the field, I've tinkered with enough stuff over the past few years that I feel like I've got something to share. Take a look at my intro post:
If you make musical instruments from microcontrollers and have an interesting project to share, let me know and I'll share it here.
Looking forward to comments, and to others building crazy instruments.
