For those following the progress of this project, I'd hoped that I'd have a final version of my synthetic singer program completed back in March. However, it had a bad case of the mumbles, and I headed back to the drawing board to rewrite it. It's been eating my spare time since then.
I spent far too much time trying to get synthesized plosives and fricatives working. Then I listened to earlier versions of the program, and realized how much better they sounded using samples instead. So all that work got tossed out, and I ended up writing the code from scratch... again.
Just the other day, I got the code to where it is again reading MusicXML files and generating .wav files. There's still a lot of work to do, but the end is (hopefully) in sight.
I've been using Twinkle, Twinkle, Little Star" as my demo song, and here's the most current version, warts and all. For example, I haven't yet figured out why it can't say "world" correctly.
As a change of pace, I decided to do my own version of Daisy Bell, one of the first examples of computer generated singing. It's still missing some phonemes, so I've cheated in spots. For example, the /G/ is actually /D SH/. The lyrics were automatically converted to phonemes, but I did some replacement by hand because the allophonic replacement code isn't working yet.
I also added some compression and reverb because everything sounds better with reverb. Just to let you know, it doesn't sound quite this good out of the box... But it sounds exactly as bad .
Anyway, here's synSinger singing "Daisy", as well as the 1961 version by Max Mathews, John Kelly, and Carol Lochbaum, which I found on Perry Cook's website. To make comparison easier, they're in the same key and tempo:
I've always been curious about what software was used to generate the computer performance 55 years ago. It turns out that the data was hand-coded into the computer. That explains why I was never able to find any references to the text-to-singing program... It never existed!
As always, comments (positive or negative) are always appreciated.
I'll confess - that's one thing I hadn't considered.
But that reminds me that I do need to add another feature - using another sound source instead of a glottal pitch. Remember the cellos voice on the MacInTalk?
I've never heard that original recording, absolutely insane that that was 1961... I also can't believe that you coded this yourself, that's crazy!! Amazing work. Your version is sounding very good. Keep us updated on how you progress with "world"!
Wow. Amazing you coded that yourself. I can't even begin grasping how you did that. Like how do you pick the right sample for a specific part of a word, or did you sample sounds and complete words? Individual letters can sound different when used in combination with other letters...
There are a couple different ways that vocal synthesis can be approached. The method that I'm using is called "formant synthesis", and is one of the oldest techniques that's been used for computer synthesis.
In English, there are approximately 40 distinct "sounds" that make up the language, are referred to as "phonemes".
There are different phonetic systems, but one of the simplest for American English is the "Arpabet", which uses plain text characters to represent phonemes. For example, the word "dictionary" would be written:
I use the CMU Dictionary to convert English into phonemes. If a word isn't found in the dictionary, I fall back to a public domain program called "Reciter" which guesses how to pronounce the word.
Phonemes are turned into sound by simulating the human vocal tract electronically. Before explaining that, let give give a (very simplified) explanation of how we create vocal sounds.
As air passes through the glottal folds, the folds vibrate and create sound. By controlling the tension (which in turn controls the length of the folds' opening), we can raise and lower the pitch we create. This pitch is called the fundamental frequency (F0), which we hear at the pitch of the voice.
This pitched glottal pulse (which resembled a kazoo sound) passes through our mouth. We use our tongue to create one or more resonating chambers that reinforce specific frequencies in the glottal pitch. These reinforced frequencies are called "resonances", and are what distinguishes one phonemes from another.
For example, (borrowing from the SoftVoice website), here are a number of vowel sounds, and the frequencies of resonance for the "average" male speaker in Hz:
In the phoneme /IY/ (as in beet), the first formant (F1) is at 270Hz, the second (F2) is at 2300Hz, and the third (F3) is at 3000Hz. Again, these formants don't alter the fundamental pitch, and remain fixed no matter what pitch you're singing.
Some phonemes are obviously more complex than that. For example, the phonemes /IY/ and /UW/ are diphthongs, and consist of two distinct targets. But I'm digressing...
To do this electronically, I generate a waveform that approximates a glottal pulse at the desired pitch, and pass it through a series of bandpass filters - one for each formant frequeny - to resonate at the desired frequencies. The output is a rough approximation of the sound.
Changing the pitch of the glottal pulse changes the pitch that's being sung. Changing the resonating filters to new values changes the phoneme that's being sung.
Some sounds (like the frication in the /F/ or the plosive in the /T/) are created by means different than described above. I used to synthesize them, but I now use digital samples because they give better results.
Very little work that I've done are my own ideas. I've borrowed heavily from the published work of Dennis Klatt, who wrote one of the first text-to-speech computer programs.
If you're curious, I'd highly recommend downloading this Formant Synthesis Demo. Click and drag in the area marked F1/F2 (formants one and two) and you'll get a good idea how this works.
Thanks for you answer, David. It did bring me more understanding of the subject of vocal synthesis. I will visit the sites you mentioned. Please keep us informed about your progress with this project. I find it very interesting.
The various parameters of vibrato - depth, speed, minimum note length and delay before start - can be specified.
I've also worked on clearing up a number of phonemes, including the /ERL/ in "world", although the main problem with that word is the trailing /D/.
This version of synSinger is written in Lua, and isn't particularly fast - it renders audio about about half the speed of the song. I still haven't been able to spend time to figure out how to create a stand-alone executable. I need to spend some time with the squish documentation.
There are still instances where it will "squelch" when parameters change to quickly, and some of the phonemes still need more attention. But for the most part, the output seems to be fairly acceptable, although not always intelligible. Truth be told, it's only incrementally better than prior versions.
I've also created a "female" voice for synSinger by mapping phonemes formants from average male phoneme space to average female phoneme space. It also modifies some other parameters, such as raising the pitch up an octave (so "she" doesn't sound like a chain smoker), adding more breath noise, altering the glottal pulse, and modifying the formant bandwidths based on a shorter larynx. But it still sounds a bit cheesy, because female voices aren't something formant synthesis does that well:
Ci siamo dati da fare e abbiamo aggiunto oltre 50 nuove funzionalità e una straordinaria raccolta di nuovi contenuti, tra cui 222 RealTracks, nuovi RealStyles, MIDI SuperTracks, Instrumental Studies, "Songs with Vocals" Artist Performance Sets, Playable RealTracks Set 3, Playable RealDrums Set 2, due nuovi set di "RealDrums Stems", XPro Styles PAK 6, Xtra Styles PAK 17 e altro ancora!
Band-in-a-Box® 2024 apporte plus de 50 fonctions nouvelles ainsi qu'une importante de contenus nouveaux à savoir : 222 RealTracks, des RealStyles nouveaux, des SuperTracks MIDI, des Etudes d'Instruments, des Prestations d'Artistes, des "Morceaux avec Choeurs", un Set 3 de Tracks Jouables, un Set 2 de RealDrums Jouables, deux nouveaux Sets de "RealDrums Stems", des Styles XPro PAK 6, des Xtra Styles PAK 17 et bien plus encore!
Video: Making a Song with Band-in-a-Box®, ChatGPT, and Synth V
Take your Band-in-a-Box® project to a whole new level when you incorporate ChatGPT and Synth V to add lyrics and vocals to your song!
We wanted to demonstrate how this is done with our video, where we show you how to go from nothing to a finished "radio ready" modern pop song by combining the features of Band-in-a-Box®, ChatGPT, and Synth V!
User Video: Next-Level AI Music Editing with ACE Studio and Band-in-a-Box®
The Bob Doyle Media YouTube channel is known for demonstrating how you can creatively incorporate AI into your projects - from your song projects to avatar building to face swapping, and more!
His latest video, Next-Level AI Music Editing with ACE Studio and Band-in-a-Box, he explains in detail how you can use the Melodist feature in Band-in-a-Box with ACE Studio. Follow along as he goes from "nothing" to "something" with his Band-in-a-Box MIDI Melodist track, using ACE Studio to turn it into a vocal track (or tracks, you'll see) by adding lyrics for those notes that will trigger some amazing AI vocals!
Wir waren fleißig und haben über 50 neue Funktionen und eine erstaunliche Sammlung neuer Inhalte hinzugefügt, darunter 222 RealTracks, neue RealStyles, MIDI SuperTracks, Instrumental Studies, "Songs with Vocals" Artist Performance Sets, abspielbare RealTracks Set 3, abspielbare RealDrums Set 2, zwei neue Sets von "RealDrums Stems", XPro Styles PAK 6, Xtra Styles PAK 17 und mehr!
Add updated printing options, enhanced tracks settings, smoother use of MGU and SGU (BB files) within PowerTracks, and more with the latest PowerTracks Pro Audio 2024 update!
One of our representatives will be happy to help you over the phone. Our hours of operation are from
6:00AM to 6:00PM PST (GMT -8) Monday thru Friday, and 8:00AM to 4:00PM PST Saturday. We are closed Sunday. You can also send us your questions via email.
One of our representatives will be happy to help you on our Live Chat or by email. Our hours of operation are from
6:00AM to 6:00PM PST (GMT -8) Monday thru Friday; 8:00AM to 4:00PM PST (GMT -8) Saturday; Closed Sunday.