Using math to blend musical notes seamlessly

From News.mit.edu (Sept. 27):

In music, “portamento” is a term that’s been used for hundreds of years, referring to the effect of gliding a note at one pitch into a note of a lower or higher pitch. But only instruments that can continuously vary in pitch — such as the human voice, string instruments, and trombones — can pull off the effect.

Now an MIT student has invented a novel algorithm that produces a portamento effect between any two audio signals in real-time. In experiments, the algorithm seamlessly merged various audio clips, such as a piano note gliding into a human voice, and one song blending into another. His paper describing the algorithm won the “best student paper” award at the recent International Conference on Digital Audio Effects.

The algorithm relies on “optimal transport,” a geometry-based framework that determines the most efficient ways to move objects — or data points — between multiple origin and destination configurations. Formulated in the 1700s, the framework has been applied to supply chains, fluid dynamics, image alignment, 3-D modeling, computer graphics, and more.

In work that originated in a class project, Trevor Henderson, now a graduate student in computer science, applied optimal transport to interpolating audio signals — or blending one signal into another. The algorithm first breaks the audio signals into brief segments. Then, it finds the optimal way to move the pitches in  each segment to pitches in the other signal, to produce the smooth glide of the portamento effect. The algorithm also includes specialized techniques to maintain the fidelity of the audio signal as it transitions. [read more]

Not bad. Interesting.