My silly hack, written and presented at Music Hack Day Boston 2010 last weekend.

 

1. A problem

For an increasing number of us, the same device we use to play music also handles email and GPS directions. Listen with me for a few moments of my drive.

2. A solution: “In 500 feet...”

Let’s throw a ridiculous amount of technology at it!

We analyze the phonemes of the speech and map each syllable to a MIDI note.

 

We’ll use Yamaha’s Vocaloid speech synthesizer available as part of the Canoris API to sing the GPS directions, overlaying the lyrics.

 

3. Another problem: “Keep going straight”

So now we’re listening to the Byrds.

As speech technology improves, this could get confusing.

 

4. Another solution: Musixmatch lyrics lookup

We’ll look for songs that contain “dangerous” phrases, and just not play them.

Python code listing

5. Technique

The current iteration required hand tweaking to get the timing right, but this is the roadmap:

Extract melody from channel 4 of karaoke MIDI file (.kar)

If no MIDI source available, synchronize syllables to Echo Nest beats, and use root of chord for corresponding timestamp from analysis.

In the future, the same technique could be used to change the lyrics for an entire song.

Stay tuned for more information and source code.

blog comments powered by Disqus