DIY Vocoder: Robots Feelin' It

The vocoder.  It's undeniably attention-grabbing.  And, for many, it is also completely off-putting.  It is absolutely the anchovies-on-pizza of the synthesizer world.  I like anchovies.  But would I like a vocoder?  I needed to find out...so, I built my own!

What is a Vocoder?  A vocoder is an effect that makes a synthesizer sound like your voice.  You sing or talk into a microphone and you play your synthesizer.  The vocoder transfers certain qualities of your voice onto your synthesizer.  Like a wah pedal, equalizer, or filter, a vocoder doesn't make its own sound; it manipulates and changes the sound from another device, such as a synthesizer.

Setup.  As shown below, a vocoder needs a microphone and a synthesizer.  The two are plugged into the vocoder.  The output of the vocoder goes out to your PA or DAW.  Pretty easy.  It's what's inside the vocoder that is the magic.

Starting Simple: 1-Band Vocoder.  Before we get too deep, we should look at a simpler example.  Since a vocoder processes many frequency bands in parallel, lets start by looking a simple 1-band vocoder.  As I've illustrated below, a 1-band vocoder has two elements: a block to sense the instantaneous loudness of the voice ("RMS", which is an envelope follower) and a gain block (a VCA) to change the loudness of the synth in response to the voice's loudness.  This is the core unit from which we will build up a vocoder.

 

What Does It Sound Like?  This basic effect changes the overall loudness of the synthesizer.  When the voice is loud, it allows the synth to be loud.  When the voice is quiet, it forces the synth to be quiet.  It responds very quickly and very naturally.  While that is cool, it does not actually sound like the voice. Just changing the loudness is not enough to sound like your voice. 

Tracking Your Voice's Formants.  To add more voice-like qualities, a real vocoder looks at the loudness of many different frequency regions of your voice.  The different frequency regions are chosen to sense the different frequency regions associated with the different vowel sounds.  As you make the different vowel sounds, you change the shape of your mouth (and nasal passages) to enhance or attenuate the different harmonics contained in your voice.  This shaping is why an "A" sounds different from an "E" and from an "O"; the peaks in the frequency response (the "formants") are very different for the different vowel sounds.  A vocoder detects this frequency shaping and applies the same frequency shaping to the synthesizer's audio.  The result is a voice-like quality to the sound of the synth.

Multiband Vocoder.  Our 1-band vocoder changed the loudness of the overall signal -- all frequencies were affected equally.  To make the synth more voice like, we want to sense the frequency shaping (the formants) as it dynamically changes to make the different vowels.  Therefore, we need to break up the audio so that we can control the loudness of individual slices of the frequency spectrum.  To make this give us this finer control, we copy our 1-band vocoder many times in order to get a multiband vocoder.  The illustration below expands our 1-band vocoder into a 3-band vocoder.

Breaking Up the Audio into Frequency Bands.   The core of this system are still the green RMS blocks (envelope followers) and orange gain blocks (VCAs) as discussed before.  This system still controls loudness.  But, note that we precede each of these channels with a bandpass filter.  Therefore, each channel is controlling the loudness of just one frequency region.  In this simple 3-band example, the first filter might isolate the low frequencies so that that loudness of the low frequencies are made to be the same between the synth and the voice. The middle filter and last filter would do the same for the middle and high frequencies.  Now, we are impressing more of the voice's qualities onto the synth.  A real vocoder uses 8-16 of these channels, which gives it much better resolution, making the output even more voice-like.

Making My Own Vocoder.  In the old days, the bandpass filters, RMS blocks, and gain blocks would all be implemented by actual electronic circuits.  Today, however, it's much easier to write signal processing software to perform these functions.  That's what I did.   I used an open-source digital audio device from Blackaddr (the "Teensy Guitar Audio Pro", TGA) and wrote software to implement the vocoder.  The TGA uses a Teensy 3.6 as its processor, which can be programmed in the Arduino IDE, This is great because Arduino is what I use for many of my other synth hacks.   I've shared my Teensy/Arduino vocoder software on my GitHub here.

Filtering and Processing Speed.  Having up to 16 channels, and needing at least two filters per channel, a vocoder needs a lot of filters.  So, your choice of filter is important as it can consume a lot of the available processing power.  I used the multimode filter model that comes in the Teensy Audio Library  They implemented the filter as a time-domain IIR filter with fixed-point operations.  I ended up using two filters in series to sharpen the filter's response (though I'm not sure that was really necessary).  Even with the burden of the extra filtering, the Teensy 3.6 was fast enough to enable a 16-band vocoder.  With the double-filtering, that's 64 filters in total!  I was pleased.  

Filter Frequencies.  An important choice is to pick the frequency bands for your vocoder's filters.  I know that the frequencies should be tailored to the human voice, but I had no specific guidance on what frequencies to use.  After a bit of trial-and-error, I chose to center my first filter at 125 Hz and then I step the frequency upward by a factor of  1.319x for each subsequent filter.  This seemingly-bizarre value is partway between half-octave steps (1.414x) and third-octave steps (1.260x).  Using this step size, my filters end up being centered at: 125 Hz, 165 Hz, 218 Hz,...<etc>..., 4595 Hz, 6063 Hz and 8000 Hz.  To me, this felt like a good span for the human voice.

Make Your Own!  To be clear, when I wrote this vocoder, many of my choices were arbitrary.  Don't be afraid to make your own choices!  Choose a different type of bandpass filter.  Choose different filter frequencies.  Use FFTs instead of time-domain filters.  That's the beauty of hacking using open technology: you can try things out for yourself!  Go and have fun!

DIY Beat Box to MIDI Converter

It was the best spouse's workplace party ever.  I got to talk with a guy about how he could make his modular synth respond to his voice.  How cool is that!  I was inspired.  So, while I don't have a modular synth, I do have an 808-style drum machine.  Here is a joyous toast to you, the best spouse's workplace party ever: my voice-driven drum machine!

DIY Voice-to-MIDI Converter.  The key to this hack is making a device that can listen to my voice and can distinguish between my different voice sounds.  When I make a low-pitched sound ("boom"), it should issue a MIDI command for a kick drum.  When I make a high-pitched sizzly sound ("tsst") it should command a hi-hat.  And when I make a mid-frequency sound ("kuh"), it should command a snare/clap sound.  I have a mic for my voice.  I have a MIDI-compatible drum machine for drum sounds.  What I needed was a voice-to-MIDI converter.  That is what I made.

Hardware.  For this voice-to-MIDI converter, I used an open-source audio processing device called a "Teensy Guitar Audio Pro" (TGA) from Blackaddr.  As the name implies, it is intended for guitar.  But, knowing that dynamic microphones share many qualities with guitar pick-ups, I knew that the device would work just fine with my old, cheap dynamic mic.  The TGA has tons of hardware features, but what's important for this hack is that it has a MIDI output.  What hackable guitar processor includes MIDI?  This is what makes the TGA the perfect tool for this hack.

Teensy Guitar Adapter Processes the Mic Signal to Generate MID Commands

Programmable.  The processor in the TGA is an open-source device that is programmable through the Arduino IDE.  This is perfect for me since many of my hacks are built around Arduino.  I have an old version of the TGA, which is built around a Teensy 3.6, which is 180 MHz processor with floating point support.  Newer versions of the TGA bump that up to a Teensy 4.1, which is 600 MHz!  Either way, the Teensy is great because there's an Audio-processing library and an active community to learn from.  The fine folks at Blackaddr also provide a bunch of example programs to help with TGA-specific features.

The Teensy Guitar Adapter is Powered by a Teensy 3.6

How Are the Sounds Different?  The main challenge for me was designing the audio processing.  How do I get the Teensy to automatically detect which sound I was making with my voice?  The first step is always to figure out what you, as the human, to understand what makes the sounds different.  A good way to do this is to make recordings of the three sounds ("boom", "tsst", and "kuh") and to compare the frequency spectrum of each.  

Frequency Spectra.  Using the TGA as an audio interface for my computer (thanks to Teensy's USB Audio features!), I plugged in my microphone and recorded myself beat boxing.  I then manually sliced up the recording and excerpted the first 50 msec of each sound sample.  I pulled the excerpts into Matlab and computed the average spectrum for all the kick sound ("boom"), hi-hat sounds ("tsst") and clap sounds ("kuh") that I made.  You can see the average spectra below.

Lows, Mids, and Highs.  After normalizing all the excerpts to the same loudness (I don't want to throw off my voice classifier just because I happened to be quite or loud), the plot of the spectra shows what is the same and what is different about my three voice sounds.  In their normalized form, the amount of low frequency sound is similar.  But, the mids and highs are quite different:

• The "boom" sound for the kick has little mids and little highs. 
• The "tsst" for hi-hats have little mids but lots of highs.  
• The "kuh" for the clap has lots of mids and lots of highs.

Classify Based on Mids and Highs.  So, the mids and highs are the key.  The figure below plots measurement for each individual beatbox sound that I recorded.  I plotted each excerpt's mid-frequency energy on the horizontal axis versus its high-frequency energy on the vertical axis.  Notice how the three types of sounds nicely separate from each other!  This plot is the key to making the voice classifier. 

All The Pieces of the Voice Classifier.  Having figured out how to distinguish between the three voice sounds, we know what we need to do.  Once we detect that a voice sound is preset at all (such as by simply looking for the overall loundess of my voice), here are three steps of the classifier:

1. Measure the mid and high frequency energy (via some sort of filters)
2. Compare the measured mids and highs to the 2D plot above classify the sound
3. Issue a MIDI command for the sound we want

Implementing the Classifier via Filters.  Based on the average spectra, it looks like I want a lowpass filter with a cutoff around 1200 Hz, a bandpass filter passing 1200 Hz to 3000 Hz, and a highpass filter with a cutoff at 3000 Hz.   Given the filter types available in the Teensy Audio library, I used the state-variable filter because it offers lowpass, bandpass, and highpass outputs.  To get a sharper frequency response, each "filter" is actually three of these filters in series.

Threshold Detection.  With the signals filtered into three streams (low, mid, high), I wrote a simple audio class to compute the RMS envelope of each signal.  After extracting he envelop, I compare the level to a threshold to detect when the sound is loud enough that we can assume that my voice is present.  That is a "detection".  There is a voice sound that needs to be classified into kick, hihat, or clap.

Classify.  Having run my three filters, I have three measured values for each detection: the low-frequency loudness, the mid-frequency loudness, and the high-frequency loudness.  I normalize for the overall loudness by dividing each value by the sum of the three values.  Then, for the mid and high frequency values, I compare them to my 2D plot shown earlier.  Where do they fall in this plot?  Depending upon where it falls, I issue a MIDI "note on" for the kick (note 36), hi-hat (note 42), or clap (note 39).

Tuning  Of course, the development of this hack did not go as smoothly as was implied here.  It took a lot of experimentation and tuning.  The final code is here on my GitHub. 

Having Fun. Once I got it working, it was fun to make deep 808 kicks with my voice.  It was fun to use my same voice sounds to trigger other drum sounds (like the cowbell!).  And, since the system simply outputs MIDI notes, I used it to drive my synthesizer.  If I had a sampler, it would have been fun to trigger silly sounds on the sampler using silly sounds from my voice.  If you try this yourself, let me know what fun you have with it!

DIY MIDI Breath Controller

Breath controllers are a difficult subject to bring up.  They have a bit of a bad reputation...perhaps because of synthesized saxophone sounds.  Breath controllers often induce people to roll their eyes dismissively.  It's kinda like how many people respond to keytars.  Are they awesome?  Or are they cheesy?  How can you know, if you've never played one?  Well, a buddy of mine started sending me links about DIY breath controllers (see Mixtela's MK1 and MK2).  He knows I'm a sucker for DIY.  So, with this as motiviation, I pulled together my own plan and made my own DIY MIDI breath controller.

Breath Controller?  The idea of a breath controller is that you use your pressure from your breath to control some aspect of your synthesizer.  The harder you blow, you make your synth louder, or you open its filter more, or you bend its pitch.  Some breath controllers are shaped like wind instruments and include both breath sensitivity and buttons (keys) to change notes.  Other breath controllers (such as mine) are simpler and just have the breath-sensitive part.  With only the breath-sensitive part, you control sound parameters on our synth, like having a breath-controlled mod wheel.

My Approach.  I'm choosing to have my breath controller use MIDI messages to control my synth.  Therefore, I'll need to (1) choose a pressure sensor, (2) connect it to some sort of electronics to read the signal, and (3) write some software to send MIDI messages to my synth.  Let's do it!

The Shopping List.   First, let's go shopping.  There are lots of ways to build this device.  Below are the components that I used:

• Pressure Sensor: Honeywell NSCDRRN0001NDUNV.  This is a differential pressure sensor whose output voltage changes as pressure (positive or negative!) is applied.  The pressure range of this sensor is +/- 1 inch of water (about +/- 250 Pa).  It can be bought it from Mouser, but it's not cheap.  Cheaper sensors will probably work, too. 
• Flexible Tubing: I used Tygon clear tubing.  To fit onto the sensor's snout, you need a short length of 1/16" ID tubing (here), but most of your length should be 1/8" ID (here).  You want that larger diameter so that you can push enough air through it to be comfortable.  Bigger tubing might be even better.
• Electronics:  I used an Arduino Uno because I had one around.  You can probably use any hobbyist microcontroller board (including the Redboard or Metro) but you want one that'll mate to your MIDI Interface.
• MIDI Interface:  I used the Sparkfun MIDI Shield.  I like that it has built-in pots so that I can adjust things while playing.  Note that it does not come with the header pins, so be sure to buy some...you should definitely buy stackable headers.
• Prototyping Supplies:  I did my prototyping on a solderless breadboard.  I also used some little jumper wires that are typical in the Arduino world.   
• Power:  During development, I powered the system from my computer's USB port.  But, when I was done, I wanted to be free from the computer.  So, I've been using USB power wall-wart like this one along with a standard USB cable.

Wiring It Up.  After assembling the MIDI shield and plugging it on top of the Arduino, I needed to figure out how to wire up the pressure sensor. After digging through the datasheet for the pressure sensor, I found the information I needed (copied it below).  It has three types of connections: power pin, ground pin, and two output voltage pins (and you can ignore one of them, if you want).  Easy.

Using the solderless breadboard and my jumper wires, I connected the sensor to the Arduino via the MIDI Shield's stackable headers:

• Sensor Pin 1 to Arduino 5V
• Sensor Pin 2 to Arduino analog input A2
• Sensor Pin 3 to Arduino analog input A3
• Sensor Pin 4 to Arduino ground

After adding the flexible tubing to the snout of the pressure sensor, the hardware is ready.

Arduino Software.  Once it is assembled, you need to write a software to command the Arduino to read the sensor and to issue MIDI commands.  There are lots of choices here, but at it's core, it is three steps:

• An "analogRead" to measure the output of the pressure sensor
• Arithmetic to convert the measurment to a valid MIDI value (usually 0-127)
• A few "Serial.write" commands to send the MIDI message to the synth

In practice, things are modestly more complicated.  Really, though, the primary question is what MIDI message you want to send.  Everything depends upon that. 

Choose Your MIDI Command!  There are many ways to command your synth via MIDI.  You need to find out what MIDI commands your synth can respond to and what is easy for you to configure (three good choices are shown below).  On my synth, aftertouch is very easy to set up without any menu diving.  So, I chose to have my breath controller send aftertouch MIDI messages.  Later, I made it more complicated, but aftertouch is still what I use most of the time.  My code is available on my GitHub here.

From midi.org

Initial Disappointment.  Once I got my system working, I had to learn how to play a breath controller with my synth.  I quickly become bored because all I could make it do were relative slow filter sweeps or unusable pitch bends.  There was no way to give it articulation because it was simply about lung pressure...and my chest does not have much dexterity for quick motions.  Disappointment.

The Key is Airflow.   After some messing around, I had the idea to poke a hole in my tubing so that some air could flow through the tube and out the hole.  Once I did that, everything changed.  I quickly found that I could use very quick motions like tongue-stops on the tip of the tube (in my mouth).  Suddenly, it could respond quickly, like any other wind instrument.  Without the hole, tongue-stops had no effect because the sealed tube kept the pressure high.  With the hole, however, the air vents out when the tongue stops its end.  Therefore, the sensor sees a quick change in pressure and articulation is achieved.  It's so much more varied and expressive this way.  The vent hole in the tube is the key.

It's Fun.  With this modification, I found myself having fun with the breath controller.  Yes, I could still do slow swells on sustained pads.  But, now I could also articulate fast riffs, giving them a new kind of bounce.  Using the breath controller on repetitive arps, I could literally "breathe" life into them via breath-modulating the filter.  And, for single note work, the breath controller is great for contouring one's phrases in a way that is very difficult to do any other way.  So, while I wouldn't use a breath controller in every situation, it is a good time.  I recommend trying it.  And maybe even making it yourself!

DIY Ribbon Controller - CV

In sharing my work on a MIDI-enabled ribbon controller, a couple of the good folks at Muff Wiggler wanted more.  They wanted to see a CV-enabled ribbon.  So, as I'm always ready for more hacking, today I present for your enjoyment: my home-brew CV ribbon controller.

Switching from MIDI to CV:  This CV-enabled ribbon builds directly upon my MIDI ribbon.  It uses literally the same ribbon, which I built (like everyone) using a $20 soft-pot sensor.  Like my MIDI ribbon, This CV ribbon also uses an Arduino to sense ribbon and to generate the commands for the synth.  The main thing that is different with the CV ribbon is that I needed to add a digital-to-analog converter (DAC) to generate the CV signals that'll drive my old synths.

The only difference between the MIDI ribbon and the CV ribbon is how the Arduino outputs its commands.  The MIDI ribbon uses an Arduino "MIDI Shield" to send the MIDI messages whereas the CV Ribbon uses a DAC to generate the CV voltages.

Using a DAC:  A DAC is a device for generating arbitrary analog voltage signals.  You send it commands (such from an Arduino) and its output jumps to the voltage level that you desire.  They're very handy devices for synth hacking.  My experience is with the MCP4922, which is a 2-output, 12-bit DAC.  After refreshing my memory using its datasheet, I decided to wire the DAC to the Arduino as shown in the schematic.

The ribbon (left)  is wired to the Arduino (left-center).  The Arduino is wired to the DAC (right-center).  The DAC is wired to the CV outputs (right).  Easy.

Wiring the DAC:  From this schematic, I used a solderless breadboard to wire up my DAC.  I chose to use an Arduino Micro because it easily fits into the breadboard (unlike an Arduino Uno).  Wiriting software for the Micro is the same as for the Uno, so it's a trivial swap.  The wiring diagram below (go Fritzing!) shows my plan for wiring the DAC to the Arduino.  I added the headphone jacks to make it easy to get the signal in from the ribbon and to get the CV signals out to the synth.  For breadboarding, I like using this jack with this breakout from Sparkfun.

My wiring plan for the Arduino Micro and the MCP4922 DAC.  Power is supplied from USB connection to the Arduino.

Real-World Wiring:  Of course, once real wires are involved, it never looks as pretty as the plan.  The picture below shows my actual breadboard with its actual wires.  All the connections are the same, but the routing is different.  Also, I moved where I put the headphone jacks: the ribbon input is the one on the bottom right and the two output jacks are dangling off the back (one is black, the other beige).  Note my use of the blue rubber band to keep the output jacks somewhat contained.

The real-world wiring is never as nice as the plan.

Pitch and Trigger CV Signals:  For this hack, I'm generating three signals to control my synth: a pitch CV, a trigger (a.k.a. gate) signal, and a filter CV.  The pitch CV controls the pitch of the synth.  It's a signal that can take any value between zero volts (a low pitched signal) and 5V (a high pitched signal).  The trigger signal, by contrast, is simply low (0V) or high (5V).  It tells the synth whether the note is "on" or "off".  Since it is just a LOW/HIGH signal, it is generated by one of the Arduino's digital pins, not the DAC.

Filter CV:  The last signal that the system generates -- the filter CV -- controls the cutoff frequency of the filter.  Like the pitch CV, it can be any value between 0-5V, and so I use the 2nd channel of the DAC to generate it.  I added this capability to the system (like I did with the MIDI version) so that the filter opens up a bit as you play higher notes.  This is the same kind of interaction you'd get with the keyboard if you had the "keyboard tracking" turned up about half-way.  Adding this behavior made the ribbon feel more natural and more engaging.

In response to the ribbon, the Arduino+DAC generates three signals: pitch CV, trigger CV, and filter. CV 

Arduino Software:  With the hardware assembled, and with my CV signals decided, I wrote the software for the Arduino.  I reused my software for the MIDI ribbon and simply added the functions to drive the MCP4922 DAC.  My code is available on my GitHub here.  Having both the MIDI and CV functionality in one program makes it more complicated to read, but it gets the job done.

The Korg Mono/Poly has good flexibility for interfacing to external gear.  Here, my circuit is creating CV signals that I'm injecting to control the filter cutoff ("VCF fcM IN", on the left), to control the gating of the note ("TRIG IN"), and to control the pitch of the note ("CV IN").

Putting it All Together:  With the software written, and with the CV signals injected into the correct locations on the back of the synth (see pic above), I was ready to get to the business of playing.  Of course, it didn't work the first time,but after some debugging (should the Trigger signal be set high or low on the Mono/Poly?), I got it to work pretty well.  Interestingly, I found myself playing the ribbon differently when attached to my Mono/Poly than when it was connected to my Prophet 6.  I think that it had something to do with the CV vs MIDI.  MIDI is designed for discrete pitch instruments (like keyboards).  I feel that the continuous pitch of a ribbon is just better suited to CV.

My old Mono/Poly, with the magnet-backed ribbon just above the keybed, and with my magnet-backed breadboard controller in the upper-left.

Next Steps:  Now that I have a DAC working, it opens all sorts of possibilities.  A DAC could be used to generate audio signals directly, not just CV signals.  By directly synthesizing its own audio, I could make the ribbon a stand-alone instrument.  Yes, using these electronics to create a wavetable ribbon synth could very, very interesting.

DIY MIDI Ribbon Controller

In a previous age of the world (2011), I built myself a ribbon controller.  I used it to drive my Korg Mono/Poly via CV.  And it was good.  But, that was before "synthhacker" first awoke under the stars on the shores of Cuivienen.  Has its glory been forever lost?  No!  The ribbon lives again!  And it has been updated!  Behold:

The Overall Setup:  This ribbon is just a controller -- it controls a synthesizer.  It doesn't make noise itself.  Those black-and-white keys that are built into the synthesizer are also just a controller.  They tell the brain of the synth to make noise at fixed pitches.  My ribbon controller uses the synth's MIDI interface to tell the synth to make noise across a continuous spectrum of pitches.  More bendy.  More swoopy.  Fun!  My controller has two parts: the ribbon itself, and some electronics (Arduino +  MIDI Shield) to do the ribbon-to-MIDI conversion.

The Ribbon Parts:  The ribbon itself, I built a long time ago.  But, as stores on the internet never forget what you've ordered, I easily went back and looked it up.  The core of the ribbon controller is a 500mm-long "soft pot" (Sparkfun).  This is the part that senses where you touch.  Since it is soft and floppy, however, I also bought a 2-ft long strip of plastic (McMaster) to act as a backbone.  For the electrical connection to the ribbon, I used a headphone jack (Sparkfun) and a small perfboard (Sparkfun).  The only real innovation with my ribbon controller (there are lots of DIY ribbons out there) is that I also added a magnetic strip along its back (McMaster).  Since most of my synths  have a metal front panel, the magnetic strip is a great way of keeping the ribbon in place!

Components of the Ribbon Controller

The soft pot is on the top.  The magnetic strip (partly unrolled) is in my hand.

Assembling the Ribbon:  I started with the plastic strip.  Its edges were surprisingly shart, so I got out some sandpaper to smooth the edges.  Now it's much nicer to touch.  After cleaning off the sandy bits, I spray-painted the whole thing black.  When fully dry, I attached the magnetic strip and the soft pot, both via their adhesive backing.  Then, I carefully soldered the ribbon's terminals (don't melt the soft pot!) to the perfboard and then to the audio connector.  I wired the connector so that the "top" of the pot was to the tip, "bottom" of the pot was to the sleeve, and the "wiper" of the pot was to the ring.  Once it was all soldered together, I epoxied the perfboard to the plastic strip to make it all nice and solid.

Headphone jack is connected to the perfboard, which is connected to the soft pot.  The whole thing is epoxied to the plastic strip, for strength.

Arduino-MIDI Electronics:  The ribbon is just a potentiometer -- it needs electronics to interface to a MIDI synth.  After my previous project making an MIDI-to-trigger converter, I realized that I could re-use that exact same hardware for this ribbon-to-MIDI converter.  Repeating the recipe list from that project, my parts include an Arduino Uno, a MIDI Shield* from Sparkfun, some stackable headers, and a 3.5 mm stereo headphone jack (or maybe one like this).

(* Note: I used the old Sparkfun MIDI Shield but, presumably, the new model linked above works well, too).

Components for the "Brain" of the Ribbon Controller

Assembling the Electronics:  I soldered together the MIDI shield, including soldering on the stackable headers.  I then soldered some wires to the audio jack and soldered those wires (being sneaky) to the header pins sticking down from the MIDI shield (detailed soldering pics here).  Once it was all assembled, I connected the ribbon to the MIDI shield using a standard 3.5 mm audio cable.  I also connected a USB cable to my laptop so that I could power it and program it.  The photo below also shows the MIDI cable connected, but we're not quite ready for that, yet.

Connecting my ribbon controller to the MIDI Shield (the Arduino is under the MIDI shield).

The Ribbon is a Potentiometer:  Touching the ribbon (the "soft pot") is like turning the knob on a potentiometer to a particular spot.  If I touch it near the low end, it'll show a low resistance.  If I touch it near the high end, it'll have a high resistance.  The job of the Arduino, therefore, is to sense the resistance of the ribbon so that it knows what note to play.  In theory, pretty easy.  But, to get some details on how this works, I hooked used my setup to collect some actual data.

The Ribbon's Response via Arduino:  To measure the response of the ribbon with an Arduino, I used the setup described above, which has the wiper of the pot connected to one of the Arduino's analog input pins (A3).  The bottom of the pot is tied to ground.  I activated the input pin's pullup resistor (via pinMode(A3, INPUT_PULLUP)) and started reading ribbon values via AnalogRead(A3).  I printed the values to the serial port so that I could see them on the P and log them to a file.  Sliding my finger to different octave points on the ribbon, I made the plot below,

The AnalogRead() Values Generated by the Arduino when Touching the Ribbon at Different Spots.

Values When Not Touching the Ribbon:  The first thing to notice is what happens when *not* touching the ribbon: the ribbon shows a very high value.  The high value results from the fact that the wiper is floating when you're not touching the ribbon.  As a result, the pull-up resistor on the analog input pin is able to pull the value all the way to the maximum value (which is 1023 on the Arduino Uno).  So, it's easy to know when you're touching the ribbon versus non touching the ribbon -- just look to see if the value is near 1023.

Values When Touching the Ribbon:  The middle of the graph shows the values when I touch the ribbon between its bottom (C1) and its top (C4), Looking at the detailed values, the lowest value seen in my ribbon (C1) is about 12 whereas the highest value (C4) is about 339.  So, I know that I have to program the Arduino to map values between 13-339 to musical notes between C1-C4.

Touching the Ribbon To Record the Arduino's AnalogRead() Value at Different Locations.

Software:  After seeing how the ribbon responds, I wrote my Arduino software.  It's basically a big loop that reads the ribbon value, converts it to a MIDI note number, and sends those MIDI commands to my synth.  While it seems conceptually simple to do an AnalogRead() for the ribbon followed by a Serial.write() to send the corresponding MIDI message, it does actually take some thought to get it to respond the way that you'd like.  Also, to make the ribbon pitches change continuously (and not be quantized like with the piano keys), you need to compute and transmit all the pitch bend commands.  It takes some work to get it right.  If you're interested, my Arduino code is available on my GitHub here.

Linearizing the Response:  One tricky spot is if you want the pitch to track linearly with your finger position on the ribbon.  In my case, I found that the notes were too closely-spaced at the top of the ribbon and too widely-spaced at the bottom.  I didn't like that.  The key to linearizing the response is to recognize that the ribbon and the Arduino's pull-up resistor actually form a voltage divider.  Once you do the math on that voltage divider, you program the Arduino to better deduce the note you want given the apparent voltage at the soft pot's wiper.  It's not perfect, but it's much better than before.

I was lucky at how well the ribbon fits on the face of the synth.  I took the extra effort to make the software fit well, too.

Bend Range:  The ribbon is fun because you can use your finger to sweep through all the pitches in between the normal notes.  Since the MIDI messages assume a piano-like keyboard, you do these "in between" pitches by commanding a note to one of the standard pitches followed by commanding a pitch bend to get the in-between pitch that you actually want.  Ideally, you could set the synth so that its allowable amount of bend would span the whole ribbon.  Unfortunately, on the Prophet 6, the max bend amount is +/- 1 octave.  As a result, with my setup, I get a brief note discontinuity when I try to continuously slide across more than 1 octave.  This is a limitation of the synth, not my ribbon controller. (If the synth's software were open source, I could fix that!)

Wobbling Pitch (ie, Pitch "Noise"):  Another subtlety that I uncovered is related to the fact that one's ear is very sensitive to changes in pitch.  The issue is that any noise in the measurement of the ribbon value will directly translate into wobble (instability) in the pitch commands sent to the synth.  When I was powering my system from the laptop and when the laptop was running on battery, my system sounded great.  But, when I plugged the computer into AC power, the pitch wobbled a lot.  It was quite unpleasant.  My solution was to add a 33 nF capacitor between the ribbon's analog input pin and ground and to add some filtering in software (revised code is here).

Future Work:  It's pretty fun to have my ribbon controller working with my MIDI synth.  Sure, I don't like that there is still a note discontinuity that happens when I try to exceed the 12-step max bend allowed by the Prophet 6, but it's still quite fun.  (Maybe I can sweet talk someone at Dave Smith Instruments to help me out on that one remaining issue :).  Looking for other ways to improve the ribbon, I'm considering adding a DAC so that the the Arduino (or Teensy!) can directly create its own audio without needing to command an external synthesizer.  That'd be fun.  How about a ribbon wavetable synth!  Or, how about a ribbon FM synth?  Fun!

Follow-Up:  I also made a version of this ribbon that outputs CV signals.  Now it can drive my old pre-MIDI synths like my Korg Mono/Poly.  Check it out here!

Polysix OTA Overdrive

I still chase my dream -- to get a decent electric piano feeling from my Korg Polysix.  Adding velocity sensitivity got me a long way there, but the sound itself needs to be dirtier, with more compression, more grit.  I tried adding diodes to the signal path to generate some distortion, but it sounded bad.  Too fizzy.  Now, I've modified my Polysix to overdrive one of its OTA circuits.  Having a soft onset that eventually leads to a deep, warm saturation, I think that it worked out pretty well!

OTA Overdrive:  After the failure of my experiment with diodes, I looked for other places in the Polysix circuit where I could generate distortion in other ways.  I noticed that the Polysix has a number of LM13600 chips, which are operational transconductance amplifiers, or OTAs.  Seeing them there reminded that a number of newer synths (e.g. Moog Sub Phatty) say that they offer "OTA Overdrive" as a means of user-controllable distortion.   I don't know how these synths overdrive their OTAs, but I was certainly game to figure out how to overdrive mine.

Pre-OTA Attenuation:  A key requirement of the LM13600 is that the input signals have to be very small.  If the inputs are too strong, the out signal will become distorted.  Since I want distortion, this looks like a good place to make my modifications.  Looking at the Polysix schematic, I see voltage dividers in front of every LM13600 to cut the signal down.  One example is shown below, which shows the elements around IC20.  The signal comes in on the right and exits on the left.  The voltage divider in front of IC20 is formed by R155 (10 kOhm) and R163 (100 ohm).  This cuts the input signal voltage by a factor of 100.  That's 40 dB of pre-OTA attenuation!

Adding a resistor in parallel to R155 reduces the attenuation prior to IC20 enabling the LM13600 OTA to be overdriven

Reducting the Pre-OTA Attenuation:  If I modify the circuit to have less pre-OTA attenuation (ie, to allow the signal to be stronger), I will likely overdrive this OTA.  I can easily reduce the attenuation by adding a resistor in parallel with R155, which will reduce the effect of the voltage divider.  For example, putting a 1 kOhm resistor in parallel to R155 will result in the signal being cut down by only a factor of 10, instead of the factor of 100.  This means that the signal is attenuated by only 20 dB instead of the 40 dB.  In effect, I'm slamming the OTA with a 20 dB stronger signal.  This seems like an easy path to overdriving IC20.  That's what I want.

Connecting the Resistor:  Looking inside the synth, I first looked for IC20 on the KLM-368 PCB.  I found R155 nearby.  To easily try different resistors in parallel to R155, I attached clip leads on either side of R155.  See the pics below.  I also attached my oscilloscope to C75 so that I could visualize the output as well as hear it.

Clipping in on either side of R155.

Adding a resistor, via the clip leads, to be in parallel with R155.

Trying a Range of Resistors:  As you saw in the video at the top of the post, I tried a range of different resistor values.  For gritty electric piano sounds, I liked the 3.3K and 1.1K values the best because the effect was subtle...it simply warmed up the signal and only added some grit when I played hard.  At the other end of the spectrum of resistor values (ie, when I shorted across R155), I unexpectedly enjoyed the unpredictable chaos of the heavily saturated distortion.  It was really fun!

Seeing the Effect:  The video demonstrated what this modification sounds like.  The figure below shows what it *looked* like.  The figure shows the output of IC20 for different resistor values across R155.  As you can see, with no resistor (which is the same as a very large resistor), the output signal is a nice sawtooth, but it is relatively small.  Then, as I add a large resistor (3.3 kOhm) the signal gets quite a bit stronger due to the extra gain.  As I make the resistor smaller (1.1 kOhm), the signal gets stronger.  Continuing to make the resistor stronger (500 ohm and on), the shape of the waveform starts to deviate from a sawtooth -- the top and bottom are becoming rounded.  This is signature of the OTA being overdriven.  It's a softer, more rounded, distortion than seen in my diode mod from my last post.  Finally, when I short across R155, the distortion becomes so heavy that the signal is a square wave.  Under certain contditions, as seen in the video, that can be cool in its own way.

Output of IC20 recorded for different resistors placed in parallel with R155. Note that decreasing resistance results in increased gain, which eventually leads to overdriving IC20, causing the sawtooth to become distorted.  Note that, for these images, I played two notes at the same pitch and waited for the two voices to naturally phase into alignment.

More Than Just Gain:  Clearly, the main effect of changing the resistor is to boost the level of the signal going into the OTA, which then overloads the OTA and causes the output signal to be distorted,  But, gain and distortion is not the resistor's only effect.  Looking back at the Polysix schematic, I see that R156 and C80 are also in parallel with R155.  This resistor and capacitor act to boost the treble frequencies.  By adding my own resistor around R155, I will also be reducing the effect of this treble boost.  So, maybe the "warmth" that I felt was actually a result of changing the frequency response, and not necessarily due to overdriving the OTA.  To confirm this theory, let's analyze the circuit for different resistor values and see what happens.

Circuit Simulation:  To simulate this part of the circuit, I used 5Spice Analysis, which is a SPICE-based circuit simulation program with a nice graphical interface for Windows.  Like most graphical versions of SPICE, you start by drawing the schematic of the circuit that you want to simulate.  The screenshot below shows the schematic that I made to represent the elements leading into the IC20 OTA.  For this simulation, the input signal is generated by "SigIn" on the right.  "TPv1" on the left represents the output signal, which would normally go to IC20.  In between the elements of the Polysix's voltage divider, the R-C treble boost, and my added overdrive-inducing resistor.  Now I can run the simulation and see the expected frequency response.

Using "5Spice Analysis" to model the frequency response of the pre-OTA voltage divider network.

Expected Response, Unmodified Polysix:  The graph below shows the modeled response of the circuit.  On the bottom (in blue) is the the circuit response with no added resistor around R155.  As expected, the signal is 40 dB down, which is what we expect based on the 10 kOhm / 100 Ohm voltage divider.  Also note, however, that the treble frequencies are boosted due to the effect of R156 and C80.  At 5 kHz, the signal is boosted by about 5 dB.  While not a huge boost, this would definitely sharpen the sound.

Expected Response of the pre-OTA voltage divider network.  The different lines show the effect of different values for the resistor that I placed in parallel to R155.

Expected Response, Modified Polysix:  When the parallel resistor is added, I get the black traces shown in the figure above.  The primary effect is that the signal level goes up.  The 1.1 kOhm case, for example, shows that the signal is at -20 dB instead of at -40 dB.  This is the 20 dB gain that I mentioned earlier.  A secondary effect of adding the parallel resistor, however, is that the boost to treble frequencies becomes much less.  Perhaps this is the increased "warmth" that I perceived.  Hmm.  I think that this requires a little more exploration.  Maybe there are additional modifications that I can make to shape the frequency response to make the OTA overdrive sound even better!

Next Step:  I liked the sound and feel of this OTA overdrive much more than the diode distortion.  I think that, if I can figure out how to make this OTA overdrive controllable from the Polysix's front panel, I'll enjoy having this modification.  So, that's my next step: figure out how to have a controllable amount of overdrive for IC20.  Stay tuned!

Follow-Up:  How to make this mod controllable without clipping in new resistors?  In this follow-up post, I start designing this mod to use an Arduino and a digipot.  

Polysix Drive - Diode Distortion

I recently modified my Korg Polysix to have velocity sensitivity.  Now that it is responsive to the dynamics in my playing, my Polysix makes much more convincing electric piano and clav sounds.  The problem, however, is that those classic EP and clav riffs from the classic records from the 70s were often played through a guitar amp, which means that we're used to hearing them with the warm compression and light overdrive of tube guitar amps.  By comparison, the EP and clav sounds from my Polysix are too clean and too dynamic.  In this post, I start the process of finding the right "drive" mod to bring some of that compression and overdrive to my Polysix.  I'm going to start with some simple diode-based distortion.

A simple distortion generator: an LED, or a diode, or both!

Classic Diode Distortion:  Being a guitarist, I know that it is very difficult to simulate the sound and feel of a tube amp.  So, instead of heading down that challenging path, I'll look to the well-known overdrive and distortion tricks used in guitar pedals.  A very common technique used for guitar distortion is to use a pair of opposite-facing diodes in parallel.  This is how distortion pedals such as the Ibanez Tubescreamer and the ProCo Rat achieve their classic sounds.  Maybe it'll work well in my Polysix, too!

Putting Diodes in the Polysix:  Below is an excerpt from the Polysix schematic.  It shows the last VCA and the last amplifier prior to the audio signal being sent to the output jack.  In the blue region, it shows that the make-up gain after the VCA is simply an op-amp configured as an inverting amplifier.  Since the Tubescreamer also uses an inverting op-amp, I chose to follow the Tubescreamer's distortion approach of adding a pair of diodes into the op-amp's feedback loop.  I just need to clip in a pair of diodes?  That's it?  Easy.

Schematic of the last VCA and last amplifier in the Polysix.  I've chosen this location to add my "Diode Distortion Mod", which is simply a pair of clipping diodes in the feedback loop of the last amplifier.

In Parallel with R196:  In the modified schematic above, you can see that the pair of diodes are added in parallel with the other elements in the feedback loop.  To enable a quick test, I simply attached the diodes using clip leads on either end of R196.  The other ends of the clip leads (not shown) are clipped onto the diodes.  The only tricky part is to make sure that the diodes face in opposite direction.  I used my handheld multi-meter to determine which direction was "forward" for each diode, and then I swapped one of them around backwards.

Using small clip leads to attach my diodes (not shown) in parallel with R196.

Looking at the Output:  With the diodes hooked up, I used my oscilloscope to look at how the diodes affected the audio signal from the op-amp.  The pictures below show some screenshots from the oscilloscope.  To make a strong signal, I set the Polysix to play two voices at the same pitch.  Then, I set the Polysix's "Attenuator" dial to maximum volume (ie, "+10dB").  Now the signal is strong enough to really show the effect of the diodes.

Screenshots of my oscilloscope when recording the output of the op-amp.
I tried different combinations of diodes in the op-amp's feedback loop.

Clipping of the Waveform:  As can be seen in the top-left screenshot ("No Mods"), I've got a normal-looking sawtooth waveform.  When I add just the LED, I see that the top of each sawtooth has been clipped to a maximum voltage of 1.6V (ie, the LED's forward voltage drop).  When I swap out the LED for a backwards silicon diode, it's the bottom of the sawtooth that has been limited (clipped at 0.6V, which is the diode's forward voltage drop).  Finally, when I have both the red LED with the backwards silicon diode in circuit, both the top and bottom of the waveform is clipped.  So, the diodes act to distort the signal by limiting its dynamic range.  Pretty sensible.

Demo:  OK, these oscilloscope views are interesting and all...but how does it SOUND?!?  That's a good question.  Below is a short demo that I pulled together.  It shows the baseline Polysix with no distortion mod, the Polysix with LED distortion only, and the Polysix with the LED + diode.

Not What I Wanted:  By the end, my conclusion was that I didn't like the results.  My first reaction was that the distortion was too fizzy/fuzzy.  But, beyond just the fizzy sound, the diode distortion didn't have the right feel when playing it.  I had been hoping that the distortion would provide a sense of dynamic compression, like a guitar amp feels.  But, instead of feeling that kind of compression, I actually felt like the signal was being hard limited...which, of course, is exactly what diodes do.  I should not have been surprised at this disappointment.

Moving Forward:  Yes, I could have spent more time refining this diode distortion modification to try to address my criticisms.  For example, to reduce the fizzy/fuzzy sound, the Tubescreamer includes additional capacitors to reduce the high frequency sizzle.  I could have tried that.  But, in truth, this experiment reminded me of why I don't use diode-based distortion pedals with my guitar...I never end up liking the sound.  So, for my Polysix, I'm going to leave behind the diodes and try some other approaches.  Let the synth hacking continue!

Update: I've removed the diode distortion and, instead, modded the Polysix to overdrive one of its OTAs.  I like that sound much better!

Korg Polysix with Velocity Sensitivity

In truth, a Korg Polysix has rather limited features.  But, because it is an analog synth, it is open to modifications.  And so I've dived in head first.  I've already added aftertouch and portamento, controllable detuning, sustain pedal, and Moog-style legato triggering.  My most recent modifications began a few weeks ago, when I decided that I wanted my Polysix be velocity sensitive.  After a bunch of explorations and trials, I've finally pulled together all of the pieces.  That's today's story.  But first, here's a demo!  (Note, the bass distorts my laptop speakers, too.  Go for headphones.)

The Pieces:  The addition of velocity sensitivity to my Polysix builds upon many of the previous mods that I've done.  The core enabler of this mod was the fact that I had already replaced the stock Polysix keybed (which does not transmit velocity information) with a new Fatar keybed that is both velocity and aftertouch sensitive.  It's sweet.  As shown below, in my modified Polysix, the data from the new keybed is fed to an Arduino Mega, which assigns each new note to one of the six voices in the Polysix.  To modulate that voice in response to the note's velocity, I also feed the velocity information to my new "Velocity Processor", which then does the actual velocity-based modulation.

My highly-modified Polysix.  The newly-added "Velocity Processor" is highlighted in yellow.

Velocity-Based Modulation:  So what kind of velocity sensitivity did I implement?  I chose to modulate the intensity of the filter envelope.  As you heard in the demo, the intensity of the filter opening and closing gives a strong sense of articulation and dynamics.  In a stock Polysix, the intensity is the same for all voices (as set by the "EG Int" knob).  My goal was to dynamically change the envelope intensity based on how hard I hit the key.  As shown below, I chose to attenuate the filter envelope based on the note velocity.  At maximum velocity (MIDI 127), the envelope would be at its full intensity (as set by the "EG Int" knob, as before).  At lower velocity values, the intensity would be progressively smaller.  As a result, notes that I press hard should have a strong intensity and notes that I press lightly will soft intensity.

Scaling the VCF Envelope Based on the Note Velocity

Hardware Approach: After studying the Polysix schematic, and after figuring out how the VCF envelope multiplexing and how its associated addressing and synchronization works, I settled on the plan that I would manipulate the VCF envelopes in the multiplexed portion of its circuit.  Being multiplexed, I only have one signal line to manipulate, instead of six individual signals (one for each voice).  Sure, the signal on that one multiplexed line is changing very quickly (all six envelopes are time-sliced onto a single line), but I've already proved that a Teensy 3.1 plus a digital potentiometer can keep up just fine.  So, my hardware plan was settled -- I'd insert my Velcoity Processor right after the VCF Multiplexer (IC24) on the Polysix's KLM-366 PCB.

Building It:  In the Polysix, I was happy to see that IC24 is socketed, which means that you can pull out the chip and insert all sorts of fun new circuitry via the now-empty socket.  And so that's exactly what I did.  I took a piece of protoboard, soldered on the Teensy 3.1, the digipot, and IC24 itself.  More details on the build are here.  I then inserted the assembled protoboard into the empty IC24 socket.  As you can see below, it's not too pretty.  It's not complicated, it's just messy.  Someone with more experience would certainly do this more cleanly.

My "Velocity Processor" Installed in my Polysix.  Lots of wires.  Someone smarter than me could surely do it more cleanly.

Arduino and Teensy Software:  Hardware alone does not make this modification work.  I also needed to write some software.  For example, I needed to program the Teensy (my "Velocity Processor") to keep up with the multiplexing circuitry and to drive the digipot to properly attenuate the envelope signal.  I also needed to expand the programming on the Arduino Mega (my replacement for the Polysix's "Key Assigner CPU") to properly route the velocity data from the keybed to the Velocity Processor.  While this routing of the velocity data sounds straight-forward, the need to support the Arpeggitor modes made it more challenging (note that the velocity-sensitive arpeggiator leads to fun effects, like being able to make some notes feel accented...definitely a cool effect).  In the end, the software writing went pretty smoothly.  My latest code for both the Arduino and the Teensy are on my GitHub here.

First Trials and Tweaking:  Once I got it all wired up and programmed, I started to play it.  The velocity sensitivity really makes it more expressive.  Velocity sensitivity also quickly exposes bad keyboard technique!  I need practice!  But, even with my poor technique, I felt that the velocity response wasn't quite right.  So, I spent some time remapping the velocity values to create a custom response curve.  Furthermore, to get a good electric piano/clav sound, I felt that I needed a little grit and compression in the sound.  So, I added a drive circuit, which will be the subject of another post.

Playing It:  Once I got it adjusted to my liking, I've had a lot of fun adjusting all of my traditional Polysix patches to best include the velocity capability.  I started with the piano and clav sounds (as seen at the top of this post) because they our ears most expect velocity dynamics with these kinds of sound.  Interestingly, though, even organ sounds (which are traditionally not velocity sensitive) can benefit from velocity dynamics.  Hitting the keys a little harder, gives the organ more "pop!".  You can here it this little carnie organ vamp:

Moving Forward:  My next step is to continue to refine the interaction of the velocity with the other synth's features.  For example, I'd like a way to turn off the velocity sensitivity so that the EG Int will work as it does in a stock Polysix.  To do this, I simply need to wire up a switch, or re-purpose one of the switches already on the back of the synth.  Also, I need to further tweak (and then write-up!) my drive circuit.  It is critical for the velocity-sensitive piano/clav sounds, but I've found fun uses with other sounds, too.  Finally, I also want to improve how the velocity interacts with the Unison and Chord Mem modes of the synth.  It doesn't feel quite right yet.  So there's lot's to do!  But first, I've got some playing to do.  :)

Update: I tried adding diode-based distortion to add some grit to my EP/clav sounds...but I didn't like it very mcuh.

Update: I tried overdriving one of the Polysix's OTAs to add some grit and I liked it a lot!