Tonight I coded the Marmonizer v2. For those of you with some music theory background, this harmonizer adds notes below the played pitch that spell a major triad in second inversion (a "6/4" chord). The algorithm just takes the input MIDI note, and outputs on the MIDI out connector the original note plus two other notes; one 4 semitones down, and another 5 semitones down.
In the examples below, I am playing (in a very hacky fashion, since I have no sax chops) a Yamaha WX-7 wind controller.
When you play a scale with this harmonization, it sounds like this:
And when you fiddle with the slide potentiometer we built for transposition in Marmonizer v1, you get something like this:
Here's the sketch:
/**
The Marmonizer
The Marmonizer is a MIDI harmonizer. It takes MIDI data on its input port and sends harmonized data on its output port. The types of harmonizations will eventually be user-programmable and extremely flexible.
Version 2:
Version 2 builds on version 1, which was a simple MIDI transposer, and implements a simple MIDI harmonizer. The harmonizarion ia simple; the input note is the top voice of a triad in second inversion (e.g. if the input note is E4, then the voices below are G3 and C4. For discussion of what those note names mean, see http://en.wikipedia.org/wiki/C_%28musical_note%29 The transposition pot is retained.
Limitations: the algorithm always "maps down" so we may roll notes off the deep end of the MIDI spec.
void loop() { while (MidiUart.avail()) { // Read the transposition pot, and map the value to a + or - one octave transposition transposition = map(analogRead(trPotPin), 0, 1023, -12, 12); Midi.handleByte(MidiUart.getc()); } }
In the spirit of Agile, I'm doing the minimal coding necessary to achieve a goal, so some of this might seem a bit silly to experienced programmers.
One of the things I've wanted to build for a long time is, for lack of a better term, a MIDI harmonizer. It would have:
A MIDI in
A MIDI out
Some number of controls, e.g. knobs, sliders, control surfaces, and inputs for foot controls
The idea is that any MIDI data presented to the input would be transformed by algorithms running on the box, producing some other set of MIDI output. A performer would be able to control the parameters of these transformations using the controls. Also, it would be possible to configure the box so that input parameters also affect the output in non-obvious ways.
The simplest application I can think of for such a device is a simple MIDI transposer, which is what I put together tonight. The breadboard for this experiment has a single slide potentiometer which controls the amount of transposition. If the pot is centered, no transposition is performed. At full travel one direction, the pitch is transposed up 12 semitones (one octave), and at the other end, the pitch is transposed down 12 semitones. Here's the sketch. It's based on the very excellent MidiDuino Library from Ruin & Wesen.
/**
The Marmonizer
The Marmonizer is a MIDI harmonizer. It takes MIDI data on its input port and sends harmonized data on its output port. The types of harmonizations will eventually be user-programmable and extremely flexible.
Version 1:
To prove some basic assumptions, the very first version is a simple MIDI transposer. The input note is transposed up or down, and the amount of transposition is controlled by a voltage applied to analog input 0, e.g. with a potentiometer.
This proves: - That we can do the transposition with reasonable latency - That we've got the Miduino library working properly
Note: this will probably leave dangling notes if the transposition is changed between a note on and the corresponding note off. The final code will have to account for user knob-twisting while playing, and make sure it turns off the right notes. Probably some sort of a map that relates a received note to all the note on messages it spawned.
Gordon Good (velo27 <at> yahoo <dot> com) Jan 24, 2010 */
void loop() { while (MidiUart.avail()) { // Read the transposition pot, and map the value to a + or - one octave transposition transposition = map(analogRead(trPotPin), 0, 1023, -12, 12); Midi.handleByte(MidiUart.getc()); } }
There are some stubs in there for passing through or acting on most of the other MIDI data types. They're not ever executed in this sketch.
Also, I built the MIDI interface detailed here: MIDI Shield. For now, I've got everything on a breadboard, but I will eventually make a real Arduino shield for it, maybe using one of the Adafruit Protoshields I have on order.
In the future, I'm planning to:
Write more interesting transformation algorithms, including complex harmonizations.
Allow the parameters of the harmonization to be controlled by the performer using knobs, sliders, foot pedals, randomness, etc.
Allow the parameters of the harmonization to be controlled by input parameters, e.g. note on velocity can select a different chord voicing.
Allow users to create new combinations of controller/input assignments, and save those as patches that can be recalled easily.
Today I figured out that I was wrong in my last post - I didn't destroy the 500mm SoftPot. The output of the pot does float, which I discovered when I added another SoftPot to use as an overtone controller, and it behaved the same way. By enabling the internal pullups on my Arduino, I was able to solve that problem. It also got rid of the last external components - all the sensors plug directly into the Arduino now, and no external pullups or any other components are needed. Not that that was a goal, but it's nice when re-assembling the thing.
That means it's possible for me to detect when the player has let go of the slide or overtone sensor, and maintain the previous value. That makes it possible to move from one note to another on the slide without producing a glissando, and also allows jumping from one partial to another without sounding the partials in between.
I also added code to produce note on/off events based on the breath controller input. When the player blows above some threshold value, a note on is sent, and when s/he stops blowing, a note off is sent.
And, finally, I added a MIDI panic button, which sends note off events for all notes on the instrument's channel.
Here are some views of the prototype:
Top View
The breath tube is on top of the wooden post. A T connector routes half of the air to the black box on the horizontal bar, which houses the actual breath sensor. The second box, with the labels, just routes voltages and signals to/from the sensors. The grey cable at the bottom right is an Ethernet cable, which I use to connect the instrument to the Arduino/breadboard. The other inputs into the box are OT (overtone sensor), BR (breath sensor), and SL (slide), and are built using 1/8" stereo jacks to supply 5v, ground, and signal. Note to self: depending on how you wire the jacks, the 5v may short to ground while being plugged/unplugged. It was a bad choice to use them for this application.
Bottom View
The wooden rod protruding below is where the player holds the instrument. Immediately behind the handle is a 100mm SoftPot that the player uses to select a partial with the left thumb (a trombonist would use breath and embouchure to overblow a different partial). Sliding back and forth will run the instrument up and down the partial series, like this:
Top View
This view shows the "slide", a 500mm SoftPot. As described in a previous post, as the player moves his/her finger up and down, the instrument sends pitch bend values to the MIDI bus, allowing glissando, like this:
Thoughts and Future Direction
I don't think the instrument plays very "trombonistically" yet. Using a finger to actuate slide feels pretty unnatural, and it's very easy for the player's finger to slip off the SoftPot. It might work better to use a "stylus" that pressed on the pot, and allow the player to move a handle that hangs below the slide and moves the stylus along with it. That will probably behave more like a trombone slide.
I also find the overtone selector hard to use in a reliable fashion. There are some other possibilities I can explore, e.g. flex sensors, but I have some serious doubts that the whole idea of using the left hand to select overtones may not work out. On that front, I want to think about ways of allowing an "overblowing" gesture that is familiar to trombonists to be used for overtone selection.
Summary
Although the current prototype doesn't translate trombone gesrtures all that well, I think it is an interesting MIDI controller, and has some interesting expressive qualities that I haven't yet explored. I would like to spend some time programming some soft synths to take advantage of the instrument, and compose some examples using the controller.
Here's the sketch:
/*
Prototype sketch for a trombone-like MIDI controller based on the Arduino hardware.
Hardware:
- An overtone selector. A SpectraSymbol 100mm SoftPot linear resistance strip, actuated by the player's left thumb.
- A "slide". Currently, this produces pitch bend information, and is implemented with a 500mm SpectraSymbol SoftPot linear resistance strip.
- A volume controller, implemented with a FreeScale pressure sensor. The player blows into a tube that goes to a "T" - one leg goes to the pressure sensor, and the other is open (a "dump tube") so that the player can put air through the instrument.
Jan 17, 2010 Gordon Good (velo27 yahoo com)
*/ #include "Midi.h"
// If DEBUG == true, then the sketch will print to the serial port what // it would send on the MIDI bus. const boolean DEBUG = false; //const boolean DEBUG = true;
const int BREATH_PIN = 0; // Breath sensor on analog pin 0 const int SLIDE_LPOT_PIN = 1; // Slide sensor on analog pin 1 const int OT_LPOT_PIN = 2; // Overtone sensor on analog pin 2
const int PANIC_PIN = 2; // MIDI all notes off momentary switch on digital I/O 2
// The overtone series this instrument will produce const int FUNDAMENTAL = 36; // MIDI note value of our fundamental const int OT_1 = 48; // First overtone (B flat) const int OT_2 = 55; // Second overtone (F) const int OT_3 = 60; // Third overtone (B flat) const int OT_4 = 64; // Fourth overtone (D) const int OT_5 = 67; // Fifth overtone (F) const int OT_6 = 70; // Sixth overtone (A flat - not in tune - need to tweak pitch bend) const int OT_7 = 72; // Seventh overtone (B flat) const int OT_8 = 74; // Eighth overtone (C) const int OT_9 = 76; // Ninth overtone (D) const int OT_NONE = -1; // No overtone key pressed (not possible with ribbon) const int overtones[10] = {FUNDAMENTAL, OT_1, OT_2, OT_3, OT_4, OT_5, OT_6, OT_7, OT_8, OT_9};
const int MIDI_VOLUME_CC = 7; // The controller number for MIDI volume data const int MIDI_BREATH_CC = 2; // The controller number for MIDI breath controller data
long ccSendTime = 0; // Last time we sent continuous data (volume, pb); const int MIN_CC_INTERVAL = 10; // Send CC data no more often than this (in milliseconds); const int PB_SEND_THRESHOLD = 10; // Only send pitch bend if it's this much different than the current value const int VOLUME_SEND_THRESHOLD = 1; // Only send volume change if it's this much differnt that the current value const int NOTE_ON_VOLUME_THRESHOLD = 50; // Raw sensor value required to turn on a note
// If a value larger than this is read from a SoftPot, treat it as if the player is not touching it. // Note: for some reason, the two SoftPots interact, e.g. just actuating the slide pot gives me // no-touch values all above 1000, but when also touching the overtone pot, the values can go // as low as 999. I suspect I may be taxing the 5v supply line. const int LPOT_NO_TOUCH_VALUE = 990;
Midi midi(Serial);
int currentNote = -1; // The MIDI note currently sounding int currentPitchBend = 8192; // The current pitch bend int currentVolume = 0; // The current volume
/** * Enable a pin for analog input, and set its internal pullup. */ void enableAnalogInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin + 14, enablePullup ? HIGH : LOW); }
/** * Enable a pin for digital input, and set its internal pullup. */ void enableDigitalInput(int pin, boolean enablePullup) { pinMode(pin, INPUT); digitalWrite(pin, enablePullup ? HIGH : LOW); }
/** * Read the slide pot and return a pitch bend value. The values * returned are all bends down from the base pitch being played, * and are in the range 8192 (no bend) to 0 (maximum bend down). * This means that the synth patch needs to be adjusted to provide * a maximum pitch bend of seven semitones, if you want it to * behave like a trombone. * * Return -1 if the player is not touching the sensor. */ int getPitchBendFromLinearPot() { // Get the raw value from the linear pot int pbRawVal = analogRead(SLIDE_LPOT_PIN); if (pbRawVal > LPOT_NO_TOUCH_VALUE) { return -1; } else { returnmap(pbRawVal, 0, LPOT_NO_TOUCH_VALUE, 0, 16383 / 2); } }
int getPitchBend() { return getPitchBendFromLinearPot(); }
/** * Read the overtone pot and select the appropriate MIDI note from * the overtone table. Return -1 if the player is not touching the pot. */ int getOvertoneFromOvertoneLinearPot() { int val = analogRead(OT_LPOT_PIN); if (val > LPOT_NO_TOUCH_VALUE) { return -1; } else { returnmap(constrain(val, 0, 900), 0, 900, 9, 0); // Map to an overtone number } }
int getMIDINote() { int ot = getOvertoneFromOvertoneLinearPot(); if (-1 == ot) { return currentNote; } else { return overtones[ot]; } }
/** * Read the breath sensor and map it to a volume level. For now, * this maps to the range 0 - 127 so we can generate MIDI * continuous controller information. */ int getVolumeFromBreathSensor() { int volRawVal = analogRead(BREATH_PIN); if (volRawVal < NOTE_ON_VOLUME_THRESHOLD) { return 0; } else { returnmap(constrain(volRawVal, 30, 500), 30, 500, 0, 127); } }
int getVolume() { return getVolumeFromBreathSensor(); }
void allNotesOff() { for (int i = 0; i < 128; i++) { sendNoteOff(i, 0, 1, DEBUG); } }
voidloop() {
if (digitalRead(PANIC_PIN) == 0) { allNotesOff(); }
int pb = getPitchBend(); int note = getMIDINote(); int volume = getVolume();
if ((-1 != currentNote) && (0 == volume)) { // Breath stopped, so send a note off sendNoteOff(currentNote, 0, 1, DEBUG); currentNote = -1; } elseif ((-1 == currentNote) && (0 != volume) && (-1 != note)) { // No note was playing, and we have breath and a valid overtone, so send a note on sendNoteOn(note, 127, 1, DEBUG); if (note == -1) { Serial.println("OOPS 1"); } currentNote = note; } elseif ((-1 != currentNote) && (note != currentNote)) { // A note was playing, but the player has moved to a different note. // Turn off the old note and turn on the new one. sendNoteOff(currentNote, 0, 1, DEBUG); sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); sendNoteOn(note, 127, 1, DEBUG); if (note == -1) { Serial.println("OOPS 2"); } currentNote = note; } elseif (-1 != currentNote) { // Send updated breath controller and pitch bend values. if (millis() > ccSendTime + MIN_CC_INTERVAL) { sendPitchBend(pb, DEBUG); sendBreathController(volume, 1, DEBUG); ccSendTime = millis(); } } delay(50); }
Tonight I made another baby step with the trombone controller. I got the slide sensor (see FAIL* below) and the breath controller both working, mounted on the instrument, and added an RJ-45 connector so I can hook the thing up to the Arduino via an Ethernet cable, unclamp it from the workbench, and pick it up and play it. So, for the first time, I was able to get a feel for how playable this thing will be.
I was pleasantly surprised. While there are a lot of glitches, I felt like I was able to be expressive on the instrument. Here's a video:
The whole air pressure system is made from garden drip irrigation tubing I had laying around. I really hope it's non-toxic.
*FAIL (or, how to blow up a $27 linear potentiometer and an $11 pressure sensor)
In my quest to make all the pieces of my new instrument modular, I decided that I would terminate each sensor in some sort of connector that could be plugged/unplugged to assemble new, unanticipated instruments. Well...
Don't use 1/8" stereo phono jacks for anything carrying Vcc. They short as they're being plugged/unplugged, which (thank you, Arduino designers) doesn't make the magic smoke come out of the Arduino, but it does make your Mac grumpy with the excessive current on the USB bus - but no Mac damage either (thank you, Mac designers).
Measure (er, check wiring) twice. Or maybe 5 or 6 times. I managed to wire 5v across the wrong two leads of my quite expensive SpectraSymbol SoftPot, and now when no pressure is applied, the output lead floats. Well, at least Mouser Electronics gets something out of this. :-(
Freescale Pressure Sensors don't care that your low-wattage soldering iron sucks and won't melt solder and you pulled out the big iron to solder the sensor to a perfboard. Moral of the story: socket everything in a prototype!
This weekend, I had two thoughts about the design of my trombone controller, while watching Media Lab's Podcast (LabCast) about the Chameleon Guitar.
Thought 1: While the instrument they built is very different from the one I'm building, what struck me was the statement:
"Computers have radically changed our lives over the last fifteen years. However, the design of the electric guitar has hardly changed in the last fifty. Is this because musicians respond more to tradition, than to innovation, or, is it because the right interface has yet to be developed? (emphasis mine). The project argues that combining the unique traditional values of established musical instruments with modern technology can open the door to a myriad of new opportunities.
This resonated well with me, and it's encouraged me to finally write down a "manifesto" that captures my ideas about musical instrument design (the last three points are more about open source).
The instruments I design should:
reward the mastery of the performer. It should be possible to become a virtuoso on one of the instruments I design.
translate gesture well for skilled performers. This might mean that an instrument is idiomatically "correct" for a performer familiar with a traditional instrument. Or it might mean that a musically skilled person can instinctually control my instrument in a way that is pleasing both to the performer and her audience.
be reproducible by others. This means that all software and hardware will be open source.
encourage further experimentation and extension.
not impose any restrictions on their use. I want performers of instruments of my design to be able to do anything they wish with the music they make.
Thought 2: There is a lot of value in making the instrument beautiful to look at and pleasing to touch.
This thought occurred to me while watching the section in the Media Lab video where they describe how they collaborated with a string instrument builder to build the neck and body shell of the instrument.
My first thoughts about my instrument's design centered around metals, which makes sense when you consider that I spend years blowing air into a hunk of brass. But once I saw the Media Lab video, it occurred to me that there's no really good reason that I couldn't build the thing out of wood. And, given that, there's no reason I couldn't find someone with woodworking skill to make the final product really beautiful, and furthermore, there's no reason I couldn't make an interesting prototype myself, using some simple tools.
So, this afternoon I went to Home Depot and bought a 6-foot long section of 1 x 2 pine, a 3/4" dowel, and a SurForm tool, and stopped at Fry's Electronics and bought a connector I thought would allow the SpectraSymbol sensor to plug in (soldering directly to the leads would certainly destroy it).
Then, this evening I spent about an hour forming the 1x2 into the "slide" of my trombone controller prototype out of that 1x2. The slide ends up looking more like the fingerboard of a string instrument than a trombone slide(see photo). The player places his/her index finger on the top, where the sensor is, and the thumb underneath. Here's a (not very good) photo of the result, clamped to my workbench for testing (click to view a larger image):
The SpectraSymbol sensor is great - only a very light pressure is required to maintain a reading, and it seems to have very little jitter when I run a sketch that prints the values on an analog input pin.
I then hooked up the set of momentary switches I use as a temporary overtone selector, and tried to play a major scale. It's glitchy, but clearly recognizable as an ascending/descending scale. I'll be addressing the glitchiness later in software.
Next up: get the breath sensor attached to the prototype, and then figure out how to let the player control the overtones with the left hand.
With my blow-tube from experiment 6 hooked up to this sensor, through a T connector with half the flow going to the sensor and half dumped, the zero to full response feels pretty natural to me (although I haven't played my trombone in a looooong time). The raw values read from the sensor are about double those of the sensor I used in Experiment 6.
Physical Interfacing
The sensor comes in a package designed for SMD mounting, so I straightened the legs and soldered it to some dual headers.
To hook up the air supply, I attached some heat shrink tubing (unsure of the diameter) to the input port and gently heated it to get a seal.
This tubing, it turns out, has an outer diameter very close to the inner diameter of 1/4" garden drip irrigation tubing, which is what I hooked up to my blow-tube. So sliding the irrigation tubing over the heat shrink and overlapping for a couple of inches makes a good-enough seal.
Electrical Interfacing
Easy:
+5V to pin 2
Ground to pin 3
pin 4 to analog input of Arduino. Raw input values from the Arduino ADC range from 58 (no blowing) to 1015 (blowing like I'm playing trombone on Varese's Integrales).
My initial experiment with using a pressure sensor for breath pressure detection didn't turn out so well, because the sensor I chose responded to a much higher pressure than I could produce. So I ordered a couple of lower pressure sensors to experiment with. The first one I'm trying is a Freescale MPX5010GSX. It costs more ($14 from Mouser) than the previous one I tried but it's not so expensive as to be unreasonable.
The sensor comes in a nice 6-pin package with leads that will plug right into a breadboard. Pin 1 has a little notch in it, so it's easy to identify. Pin 1 is the output, Pin 2 is ground, and pin 3 is a 5v supply voltage. Pins 4, 5, and 6 are labeled as V1, V2, Vex, but I'm not sure what those do. I left them floating.
To drive the input port, I created a "mouthpiece" of a 6-inch length of one-inch vinyl tubing (bought at the local Ace Hardware). I folded over one end and clamped it shut with a couple of clamps. Then I used an awl from an irrigation tubing kit to open a hole near the clamped-off end and attached a T-connector for 1/4" irrigation tubing. I attached one end of the T to the pressure sensor through a couple feet of 1/4" tubing (it fits perfectly over the sensor), and ran the other end of the T through about a foot of 1/4" tubing. This will be the "blow through" for the sensor, and by clamping it off partially, I can control how much air goes through the instrument.
Here's a picture:
The pressure sensor is on the breadboard at the bottom of the photograph. The tube to blow into is on the right, as is the dump tube. The orange clamps close off the end of the blow tube, forcing all of the air into the tubes that flow (a) to the pressure sensor and (b) to the dump tube.
I then wrote a small Arduino sketch that printed the values on analog port 2 every 100 milliseconds and ran it.
(Sketch)
/*
Print values received from a Freescale MPX5010GSX pressure sensor
Gordon Good (velo27 yahoo com)
*/ const int SENSOR_PIN = 2; // Sensor Vout on analog input 2
voidsetup() { Serial.begin(9600); }
voidloop() { int pVal = analogRead(SENSOR_PIN); Serial.println(pVal); delay(100); }
Results:
With the dump port completely closed off, and blowing my brains out, the highest reading I can get is about 1015, not much below the maximum sensor value of 1023. With the dump port wide open, the max value is about 650. Wide open feels pretty good to me in terms of how hard I'm blowing, so I think we're in the ballpark with this sensor.
So, in other words, a comfortable "loud blow" for me, with a comfortable amount of air dumped, produces a value of about 1/2 scale on a 10 kilopascal sensor. Assuming the response is linear, the pressure I'm producing is about 5 kilopascals.
I'm a software engineer with VMware in Palo Alto, CA, and a musician who likes to build crazy musical instruments from microcontrollers. The latest thing I'm working on sonifying air traffic control data.
You will also find a series of blog posts covering the basics of electronic wind instrument controllers - electronic instruments controlled by the performer's breath.
While I can't claim to be an expert in the field, I've tinkered with enough stuff over the past few years that I feel like I've got something to share. Take a look at my intro post:
If you make musical instruments from microcontrollers and have an interesting project to share, let me know and I'll share it here.
Looking forward to comments, and to others building crazy instruments.