The Marmonizer, Version 4

What is the Marmonizer?

The Marmonizer is a MIDI Harmonizer. To use it, you attach a MIDI instrument to the MIDI IN port, and attach a synthesizer, or a computer running a softsynth, to the MIDI OUT. When you play a note on the MIDI instrument, the Marmonizer sends that note, plus other notes, to its MIDI output.

I originally conceived of the Marmonizer as something that could be used by players of wind synths like the Yamaha WX-5 or the Akai EWI. But there's no reason it couldn't be used by players of other kinds of MIDI instruments. In the clips in this post, I'm using an older Yamaha WX-7.

There are a number of different harmonization algorithms that the Marmonizer knows how to produce. Some are quite simple. For example, one of the harmonizations produces a major triad in first inversion. If you play a C, the Marmonizer will send C, the G below it, and the E below that. If you play an E, the Marmonizer will send E, B, and G#. Here's a clip of this harmonization. You'll hear it unharmonized once, then with the harmonization.





Other algorithms are more complex. For example, there is a harmonization that sounds a raised-ninth chord (the played note becomes the raised ninth), and then sounds a bass note underneath, forming a slash chord. The actual bass note that sounds is randomly selected from one of four possibilities, which can produce some interesting voice leading when you play a melody.





For those of you who are familiar with Michael Brecker's EWI work, this idea of changing the bass note around comes from the patch he uses on Original Ray's from his debut album.

In this clip, I play a line unharmonized, then I play the same line in each of the 11 harmonizations the Marmonizer knows about, and finally the unharmonized line again.






There's also mode where the Marmonizer cycles through all the harmonizations it knows. Each new note gets a different harmonization. This can produce some pretty zany results when you drive an Asian percussion ensemble. In this clip, I'm just double-tounging a single note for a couple of bars, then a different note. Since the actual output notes are changing as the Marmonizer cycles through its 11 harmonizations, the result is pretty interesting.





Since this is all just code running on a microcontroller, the harmonizations can be more complicated than the ones mentioned above. As of now, I've only begun to think about all the possibilities, but some thoughts are:

• Allow the player to specify a key, and make the harmonizations make sense in that key.
• Select several different harmonizations and automatically cycle between them on each new note.
  
• Allow different notes to be steered to different MIDI channels. Probably the most useful configuration would be to sound the topmost voice on one MIDI channel, and the other voices on a different channel.
• Allow new harmonizations to be programmed by the user.
• Allow changing the harmonization by sending the unit a MIDI program change. For a live setup, the player could use a stomp box to select harmonizations. Possibly allow harmonizations to be grouped into banks (or maybe use a folder paradigm) to allow a performer to choose a set of harmonizations that work well together for a particular piece.
  
• Save/load harmonizations via MIDI system exclusive messages.
• Allow the player to control how many notes of the harmonization sound, perhaps via a knob or expression pedal. Or, make the number of notes sounding a function of the note on velocity.
• Adding new "algorithmic" harmonizations that give the player a high degree of control and reproducability.
  

Finally, here's a little improvisation:





The Hardware

The hardware is pretty simple: an Arduino microcontroller, some pushbutton switches, some toggle switches, and some potentiometers. The current version only utilizes one pushbutton, one toggle switch, and one pot, but future versions may enable more controls. It's just a prototype at this point.

For a final version, I'm investigating using Ruin & Wesen's Minicommand, which is an outrageously cool idea, and way more roadworthy than anything I'll ever be able to build.

The code:

/**

The Marmonizer
 
The Marmonizer is a MIDI harmonizer. It takes MIDI data on its input port and
sends harmonized data on its output port. The types of harmonizations will
eventually be user-programmable and extremely flexible.

Version 4:

Version 4 builds on version 3, which was a simple MIDI harmonizer with some
randomization. Version 4 introduces:

- multiple voicings (11 to be precise)
- allows the player to select which voicings are playing
- allows a cycle mode, where each new note on selects a new harmonization algorithm
- allows a split channel mode, where the top note in each harmonization
  goes to a primary MIDI channel, and all others go to a secondary channel
  (currently primary and secondary are fixed at 1 and 2, respectively)
- passes continuous controllers
- allows the player to control how many of the possible notes in a
  particular voicing are sounding (with a potentiometer).
- fixes a stuck note problem with v3

Limitations: the algorithm always "maps down" so we may roll notes off the
deep end of the MIDI spec.
 
 Gordon Good (velo27 <at> yahoo <dot> com)
 Mar 18, 2010 
 
 */
 
#include <MidiUart.h>
#include <Midi.h>

#include <Debounce.h>

MidiClass Midi;

int ledPin = 13;  // LED pin to blink for debugging
#define CYCLE_MODE_PIN 7 // Switch connected to this pin sets cycle mode (new harmonization on each note on)
#define SPLIT_CHANNEL_MODE_PIN 6 // Switch connected to this pin sets split channel mode

#define MAX_VOICES 5  // Maximum number of voices allowed in a harmonization
int nVoices = MAX_VOICES; // number of voices for current harmonization to sound

// A structure that represents a harmonization algorithm. It currently
// includes a name for the algorithm, and a pointer to a function
// that implements the algorithm.
typedef unsigned char* (*harmonizationAlgorithm)(byte); // a harmonizationAlgorithm knows how to harmonize any midi note
typedef struct {
  char *name;
  harmonizationAlgorithm algorithm;
} Harmonizer;

// Minimum and maximum values we read from potentiometers.
int POT_MIN = 0;
int POT_MAX = 1023;

boolean isCycleMode = false;  // If true, a new harmonization sounds on each note on event

// Digital input 2 cycles through the harmonizations if 
int PIN_HARMONIZATION_SELECT = 2;

// All available harmonizers
Harmonizer allHarmonizers[11] = {0};

int nHarmonizers = sizeof(allHarmonizers) / sizeof(Harmonizer); // the number of harmonizers

// The index of the current harmonizer
int harmonizationIndex = 0;

// These are the notes of the current harmonization.
unsigned char harmonization[MAX_VOICES] = {0};

// A structure that keeps track of a sounding note (note number, MIDI channel)
typedef struct {
  unsigned char note; // MIDI note number
  unsigned char channel; // MIDI channel
} SoundingNote;

// This array keeps track of all the cuurently sounding notes.
SoundingNote notesOn[128][MAX_VOICES] = {0}; 

// Boolean that tracks if we are sending the top note to one channel the the other
// notes to a different channel
boolean isSplitChannelMode = false;
#define PRIMARY_MIDI_OUT_CHANNEL 0 // Human-friendly name is channel 1
#define SECONDARY_MIDI_OUT_CHANNEL 1  // Human-friendly name is channel 2

// Instiantiate debouncers for the pushbutton switches.
Debounce debouncer_harmonization_select = Debounce(20, PIN_HARMONIZATION_SELECT);

/* ********** Harmonization Algorithms ********** */
// These are all the harmonization algorithms the program knows about

/*
 * A harmonization algorithm that sounds like Michael Brecker's
 * Oberheim XPander patch on "Original Ray's" from the album
 * "Michael Brecker" (MCA Records, 1987).
 */
unsigned char *breckerizeAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -4;
  harmonization[2] = -9;
  int rnd, sel;
  rnd = millis() % 4;
  sel = rnd % 4;
  sel = (sel + 1) % 4;
  if (0 == rnd) {
    harmonization[3] = -14;
  } else if (1 == rnd) {
    harmonization[3] = -15;
  } else if (2 == rnd) {
    harmonization[3] = -25;
  } else if (3 == rnd) {
    harmonization[3] = -23;
  } else {
    harmonization[3] = 0;
  }
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer breckerize = {
  "Breckerizer",
  breckerizeAlgorithm
};

/*
 * A tritone chord with the played note on top,
 * and a random note on the bottom.
 */
unsigned char *tritoneChordAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -5;
  harmonization[2] = -11;
  int rnd, sel;
  rnd = millis() % 4;
  sel = rnd % 4;
  sel = (sel + 1) % 4;
  if (0 == rnd) {
    harmonization[3] = -14;
  } else if (1 == rnd) {
    harmonization[3] = -15;
  } else if (2 == rnd) {
    harmonization[3] = -25;
  } else if (3 == rnd) {
    harmonization[3] = -23;
  } else {
    harmonization[3] = 0;
  }
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer tritoneChord = {
  "Tritone",
  tritoneChordAlgorithm,
};

/*
 * A major triad in first inversion with the played
 * note on top (only three note are played).
 */
unsigned char *majorTriadFirstInversionAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -5;
  harmonization[2] = -8;
  harmonization[3] = 0;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer majorTriadFirstInversion = {
  "MajTriad",
  majorTriadFirstInversionAlgorithm
};

/*
 * A series of stacked fourths.
 */
unsigned char *fourthsAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -5;
  harmonization[2] = -10;
  harmonization[3] = -15;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer fourths = {
  "Fourths",
  fourthsAlgorithm
};

/*
 * A series of stacked fifths.
 */
unsigned char *fifthsAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -7;
  harmonization[2] = -14;
  harmonization[3] = -21;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer fifths = {
  "Fifths",
  fifthsAlgorithm
};

/*
 * A fifths-based voicing (from Brian Good)
 */
unsigned char *feetAlgorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -7;
  harmonization[2] = -8;
  harmonization[3] = -15;
  harmonization[4] = -22;
  return harmonization;
}
Harmonizer feet = {
  "Feet",
  feetAlgorithm
};

/*
 * A Jon Hassell-style voicing (from Brian Good)
 */
unsigned char *hassell1Algorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -5;
  harmonization[2] = -7;
  harmonization[3] = 0;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer hassell1 = {
  "Hassell1",
  hassell1Algorithm
};

/*
 * Another Jon Hassell-style voicing (from Brian Good)
 */
unsigned char *hassell2Algorithm(byte note) {
  harmonization[0] = -2;
  harmonization[1] = -5;
  harmonization[2] = -7;
  harmonization[3] = 0;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer hassell2 = {
  "Hassell2",
  hassell2Algorithm
};

/*
 * A rootless Bill Evans-style voicing (from Brian Good)
 */
unsigned char *evans1Algorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -5;
  harmonization[2] = -7;
  harmonization[3] = -10;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer evans1 = {
  "Evans1",
  evans1Algorithm
};

/*
 * Another rootless Bill Evans-style voicing (from Brian Good)
 */
unsigned char *evans2Algorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -7;
  harmonization[2] = -9;
  harmonization[3] = 0;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer evans2 = {
  "Evans2",
  evans2Algorithm
};

/*
 * Yet another rootless Bill Evans-style voicing (from Brian Good)
 */
unsigned char *evans3Algorithm(byte note) {
  harmonization[0] = 0;
  harmonization[1] = -4;
  harmonization[2] = -5;
  harmonization[3] = -8;
  harmonization[4] = 0;
  return harmonization;
}
Harmonizer evans3 = {
  "Evans3",
  evans3Algorithm
};



/* ********** End Harmonization Algorithms ********** */

/*
 * Figure out which MIDI channel this note should go out on. Currently,
 * the top note of a voicing goes out on the primary channel, and all
 * other notes go out on the alternate channel.
 */
unsigned char determineMidiOutChannel(boolean isTopNote) {
  if (!isSplitChannelMode || isTopNote) {
    return PRIMARY_MIDI_OUT_CHANNEL;
  } else {
    return SECONDARY_MIDI_OUT_CHANNEL;
  }
}
      
/*
 * Handle a note on event. Map the note to its harmonizations, and turn
 * on those MIDI notes.
 */
void noteOnCallback(byte *msg) {  // or is it uint8_t?
  digitalWrite(ledPin, HIGH);
  unsigned char origNote = msg[1];
  unsigned char *harmonization = allHarmonizers[harmonizationIndex].algorithm(origNote);
  for (int i = 0; i < nVoices; i++) {
    unsigned char newNote = origNote + harmonization[i];
    unsigned char channel = 0;
    if (0 != newNote) {
      channel = determineMidiOutChannel(0 == i);
      MidiUart.sendNoteOn(MIDI_VOICE_CHANNEL(channel), newNote, msg[2]);
    }
    notesOn[origNote][i].note = newNote;
    notesOn[origNote][i].channel = channel;
  }
}

/*
 * Look up all the transposed notes for the given note
 * and turn them off.
 */
void noteOffCallback(byte *msg) {
  digitalWrite(ledPin, LOW);
  unsigned char note = msg[1];
  for (int i = 0; i < 4; i++) {
    unsigned char noteOff = notesOn[note][i].note;
    unsigned char channel = notesOn[note][i].channel;
    if (0 != noteOff) {
      MidiUart.sendNoteOff(MIDI_VOICE_CHANNEL(channel), noteOff, 0);
      notesOn[note][i].note =notesOn[note][i].channel = 0;
    }
  }
  if (isCycleMode) {
    harmonizationIndex = (harmonizationIndex + 1) % nHarmonizers;
  }
}

/*
 * Echo any received continuous controller data, e.g. breath controller,
 * to the primary output channel, and to the secondary output channel
 * if it is enabled.
 */
void continuousControllerCallback(byte *msg) {
  MidiUart.sendCC(MIDI_VOICE_CHANNEL(PRIMARY_MIDI_OUT_CHANNEL), msg[1], msg[2]);
  if (isSplitChannelMode) {
    MidiUart.sendCC(MIDI_VOICE_CHANNEL(SECONDARY_MIDI_OUT_CHANNEL), msg[1], msg[2]);
  }
}

void afterTouchCallback(byte *msg) {

}

void channelPressureCallback(byte *msg) {

}

void programChangeCallback(byte *msg) {

}

void pitchWheelCallback(byte *msg) {
  int16_t bend = msg[0] << 16 + msg[1];
  MidiUart.sendPitchBend(1, bend);
}

/*
 * Turn off all the notes that are on, and reharmonize them with
 * the new algorithm. This will eventually allow the user to
 * cycle to a new harmonization without sounding a new note.
 */
void reharmonize(int oldIndex, int newIndex) {
  // Not yet implemented. Before we can implement this,
  // we need to keep track of which MIDI channel the sounding
  // notes are on. We currently don't.
}

/*
 * Read the hardware attached to the Arduino, and set global state
 * accordingly.
 */
void readHardware() {
  // Read the pot that controls how many voices should sound
  nVoices = map(analogRead(0), POT_MIN, POT_MAX, 1, 4);
  isCycleMode = digitalRead(CYCLE_MODE_PIN);
  isSplitChannelMode = digitalRead(SPLIT_CHANNEL_MODE_PIN);
  // Read the button that increments the harmonization type
  if (debouncer_harmonization_select.update() && debouncer_harmonization_select.read() == HIGH) {
    harmonizationIndex = (harmonizationIndex + 1) % nHarmonizers;
  }
}

/*
 * Enable a digital pin for input, and set the pullup.
 */
void enableDigitalInput(int pin) {
  pinMode(pin, INPUT);
  digitalWrite(pin, HIGH);
}


void setup() {
  // Enable the MIDI library and register callbacks
  MidiUart.init();
  Midi.setOnNoteOnCallback(noteOnCallback);
  Midi.setOnNoteOffCallback(noteOffCallback);
  Midi.setOnControlChangeCallback(continuousControllerCallback);
  Midi.setOnAfterTouchCallback(afterTouchCallback);
  Midi.setOnChannelPressureCallback(channelPressureCallback);
  Midi.setOnProgramChangeCallback(programChangeCallback);
  Midi.setOnPitchWheelCallback(pitchWheelCallback);
  
  // Set analog ports for input
  pinMode(0, INPUT);
  pinMode(1, INPUT);
  pinMode(2, INPUT);
  
  // Set digital pins for input, enable pullups, set up debouncers
  enableDigitalInput(PIN_HARMONIZATION_SELECT); 
  enableDigitalInput(3); // Not used yet
  enableDigitalInput(4); // Not used yet
  enableDigitalInput(5); // Not used yet
  enableDigitalInput(SPLIT_CHANNEL_MODE_PIN);
  enableDigitalInput(CYCLE_MODE_PIN);
 
  // Initialize the available harmonizers
  allHarmonizers[0] = majorTriadFirstInversion;
  allHarmonizers[1] = fourths;
  allHarmonizers[2] = fifths;
  allHarmonizers[3] = tritoneChord;
  allHarmonizers[4] = breckerize;
  allHarmonizers[5] = feet;
  allHarmonizers[6] = hassell1;
  allHarmonizers[7] = hassell2;
  allHarmonizers[8] = evans1;
  allHarmonizers[9] = evans2;
  allHarmonizers[10] = evans3;
}

/*
 * Main loop. Read buttoins/switches/pots, update global state,
 * and handle any MIDI data that has arrived.
 */
void loop() {
  while (MidiUart.avail()) {
    readHardware();
    Midi.handleByte(MidiUart.getc());
  }
}

Marmonizer Progress

Here's a little video about my progress building the Marmonizer (a MIDI Harmonizer). I'll post the Arduino sketches soon.

The basic idea is that you plug a MIDI instrument into the MIDI in port, and a synthesizer (or a computer with softsynths) into the output. When you play a note into the Marmonizer, you get different, and probably more, notes out. It's really optimized for a monophonic instrument like the WX-7 wind controller I'm playing in the video (disclaimer: I'm a trombonist by training, so I have no sax chops).

In this video, the WX-7 is plugged into the MIDI in of the Marmonizer, and the Marmonizer's MIDI out is plugged into a MIDI interface that's plugged into my Mac, which is running Logic Express 9. The sounds you hear are a Logic ES-2 synth patch that I modified to respond to breath controller data.