For those following the progress of this project, I'd hoped that I'd have a final version of my synthetic singer program completed back in March. However, it had a bad case of the mumbles, and I headed back to the drawing board to rewrite it. It's been eating my spare time since then.
I spent far too much time trying to get synthesized plosives and fricatives working. Then I listened to earlier versions of the program, and realized how much better they sounded using samples instead. So all that work got tossed out, and I ended up writing the code from scratch... again.
Just the other day, I got the code to where it is again reading MusicXML files and generating .wav files. There's still a lot of work to do, but the end is (hopefully) in sight.
I've been using Twinkle, Twinkle, Little Star" as my demo song, and here's the most current version, warts and all. For example, I haven't yet figured out why it can't say "world" correctly.
As a change of pace, I decided to do my own version of Daisy Bell, one of the first examples of computer generated singing. It's still missing some phonemes, so I've cheated in spots. For example, the /G/ is actually /D SH/. The lyrics were automatically converted to phonemes, but I did some replacement by hand because the allophonic replacement code isn't working yet.
I also added some compression and reverb because everything sounds better with reverb. Just to let you know, it doesn't sound quite this good out of the box... But it sounds exactly as bad .
Anyway, here's synSinger singing "Daisy", as well as the 1961 version by Max Mathews, John Kelly, and Carol Lochbaum, which I found on Perry Cook's website. To make comparison easier, they're in the same key and tempo:
I've always been curious about what software was used to generate the computer performance 55 years ago. It turns out that the data was hand-coded into the computer. That explains why I was never able to find any references to the text-to-singing program... It never existed!
As always, comments (positive or negative) are always appreciated.
I'll confess - that's one thing I hadn't considered.
But that reminds me that I do need to add another feature - using another sound source instead of a glottal pitch. Remember the cellos voice on the MacInTalk?
I've never heard that original recording, absolutely insane that that was 1961... I also can't believe that you coded this yourself, that's crazy!! Amazing work. Your version is sounding very good. Keep us updated on how you progress with "world"!
Wow. Amazing you coded that yourself. I can't even begin grasping how you did that. Like how do you pick the right sample for a specific part of a word, or did you sample sounds and complete words? Individual letters can sound different when used in combination with other letters...
There are a couple different ways that vocal synthesis can be approached. The method that I'm using is called "formant synthesis", and is one of the oldest techniques that's been used for computer synthesis.
In English, there are approximately 40 distinct "sounds" that make up the language, are referred to as "phonemes".
There are different phonetic systems, but one of the simplest for American English is the "Arpabet", which uses plain text characters to represent phonemes. For example, the word "dictionary" would be written:
I use the CMU Dictionary to convert English into phonemes. If a word isn't found in the dictionary, I fall back to a public domain program called "Reciter" which guesses how to pronounce the word.
Phonemes are turned into sound by simulating the human vocal tract electronically. Before explaining that, let give give a (very simplified) explanation of how we create vocal sounds.
As air passes through the glottal folds, the folds vibrate and create sound. By controlling the tension (which in turn controls the length of the folds' opening), we can raise and lower the pitch we create. This pitch is called the fundamental frequency (F0), which we hear at the pitch of the voice.
This pitched glottal pulse (which resembled a kazoo sound) passes through our mouth. We use our tongue to create one or more resonating chambers that reinforce specific frequencies in the glottal pitch. These reinforced frequencies are called "resonances", and are what distinguishes one phonemes from another.
For example, (borrowing from the SoftVoice website), here are a number of vowel sounds, and the frequencies of resonance for the "average" male speaker in Hz:
In the phoneme /IY/ (as in beet), the first formant (F1) is at 270Hz, the second (F2) is at 2300Hz, and the third (F3) is at 3000Hz. Again, these formants don't alter the fundamental pitch, and remain fixed no matter what pitch you're singing.
Some phonemes are obviously more complex than that. For example, the phonemes /IY/ and /UW/ are diphthongs, and consist of two distinct targets. But I'm digressing...
To do this electronically, I generate a waveform that approximates a glottal pulse at the desired pitch, and pass it through a series of bandpass filters - one for each formant frequeny - to resonate at the desired frequencies. The output is a rough approximation of the sound.
Changing the pitch of the glottal pulse changes the pitch that's being sung. Changing the resonating filters to new values changes the phoneme that's being sung.
Some sounds (like the frication in the /F/ or the plosive in the /T/) are created by means different than described above. I used to synthesize them, but I now use digital samples because they give better results.
Very little work that I've done are my own ideas. I've borrowed heavily from the published work of Dennis Klatt, who wrote one of the first text-to-speech computer programs.
If you're curious, I'd highly recommend downloading this Formant Synthesis Demo. Click and drag in the area marked F1/F2 (formants one and two) and you'll get a good idea how this works.
Thanks for you answer, David. It did bring me more understanding of the subject of vocal synthesis. I will visit the sites you mentioned. Please keep us informed about your progress with this project. I find it very interesting.
The various parameters of vibrato - depth, speed, minimum note length and delay before start - can be specified.
I've also worked on clearing up a number of phonemes, including the /ERL/ in "world", although the main problem with that word is the trailing /D/.
This version of synSinger is written in Lua, and isn't particularly fast - it renders audio about about half the speed of the song. I still haven't been able to spend time to figure out how to create a stand-alone executable. I need to spend some time with the squish documentation.
There are still instances where it will "squelch" when parameters change to quickly, and some of the phonemes still need more attention. But for the most part, the output seems to be fairly acceptable, although not always intelligible. Truth be told, it's only incrementally better than prior versions.
I've also created a "female" voice for synSinger by mapping phonemes formants from average male phoneme space to average female phoneme space. It also modifies some other parameters, such as raising the pitch up an octave (so "she" doesn't sound like a chain smoker), adding more breath noise, altering the glottal pulse, and modifying the formant bandwidths based on a shorter larynx. But it still sounds a bit cheesy, because female voices aren't something formant synthesis does that well:
User Video: Next-Level AI Music Editing with ACE Studio and Band-in-a-Box®
The Bob Doyle Media YouTube channel is known for demonstrating how you can creatively incorporate AI into your projects - from your song projects to avatar building to face swapping, and more!
His latest video, Next-Level AI Music Editing with ACE Studio and Band-in-a-Box, he explains in detail how you can use the Melodist feature in Band-in-a-Box with ACE Studio. Follow along as he goes from "nothing" to "something" with his Band-in-a-Box MIDI Melodist track, using ACE Studio to turn it into a vocal track (or tracks, you'll see) by adding lyrics for those notes that will trigger some amazing AI vocals!
Wir waren fleißig und haben über 50 neue Funktionen und eine erstaunliche Sammlung neuer Inhalte hinzugefügt, darunter 222 RealTracks, neue RealStyles, MIDI SuperTracks, Instrumental Studies, "Songs with Vocals" Artist Performance Sets, abspielbare RealTracks Set 3, abspielbare RealDrums Set 2, zwei neue Sets von "RealDrums Stems", XPro Styles PAK 6, Xtra Styles PAK 17 und mehr!
Add updated printing options, enhanced tracks settings, smoother use of MGU and SGU (BB files) within PowerTracks, and more with the latest PowerTracks Pro Audio 2024 update!
Download and install this to your RealBand 2024 for updated print options, streamlined loading and saving of .SGU & MGU (BB) files, and to add a number of program adjustments that address user-reported bugs and concerns.
Did you know... not only can you download your Band-in-a-Box® Pro, MegaPAK, or PlusPAK purchase - you can also choose to add a flash drive backup copy with the installation files for only $15? It even comes with a Band-in-a-Box® keychain!
For the larger Band-in-a-Box® packages (UltraPAK, UltraPAK+, Audiophile Edition), the hard drive backup copy is available for only $25. This will include a preinstalled and ready to use program, along with your installation files.
Backup copies are offered during the checkout process on our website.
Already purchased your e-delivery version, and now you wish you had a backup copy? It's not too late! If your purchase was for the current version of Band-in-a-Box®, you can still reach out to our team directly to place your backup copy order!
Note: the Band-in-a-Box® keychain is only included with flash drive backup copies, and cannot be purchased separately.
Handy flash drive tip: Always try plugging in a USB device the wrong way first? If your flash drive (or other USB plug) doesn't have a symbol to indicate which way is up, look for the side with a seam on the metal connector (it only has a line across one side) - that's the side that either faces down or to the left, depending on your port placement.
Update your Band-in-a-Box® 2024 for Windows® Today!
Update your Band-in-a-Box® 2024 for Windows for free with build 1111!
With this update, there's more control when saving images from the Print Preview window, we've added defaults to the MultiPicker for sorting and font size, updated printing options, updated RealTracks and other content, and addressed user-reported issues with the StylePicker, MIDI Soloists, key signature changes, and more!
A few excerpts:
"The Tracks view is possibly the single most powerful addition in 2024 and opens up a new way to edit and generate accompaniments. Combined with the new MultiPicker Library Window, it makes BIAB nearly perfect as an 'intelligent' composer/arranger program."
"MIDI SuperTracks partial generation showing six variations – each time the section is generated it can be instantly auditioned, re-generated or backed out to a previous generation – and you can do this with any track type. This is MAJOR! This takes musical experimentation and honing an arrangement to a new level, and faster than ever."
"Band in a Box continues to be an expansive musical tool-set for both novice and experienced musicians to experiment, compose, arrange and mix songs, as well as an extensive educational resource. It is huge, with hundreds of functions, more than any one person is likely to ever use. Yet, so is any DAW that I have used. BIAB can do some things that no DAW does, and this year BIAB has more DAW-like functions than ever."
One of our representatives will be happy to help you over the phone. Our hours of operation are from
6:00AM to 6:00PM PST (GMT -8) Monday thru Friday, and 8:00AM to 4:00PM PST Saturday. We are closed Sunday. You can also send us your questions via email.
One of our representatives will be happy to help you on our Live Chat or by email. Our hours of operation are from
6:00AM to 6:00PM PST (GMT -8) Monday thru Friday; 8:00AM to 4:00PM PST (GMT -8) Saturday; Closed Sunday.
.
Dansk
Deutsch - Windows
Español
Français - Windows
Italiano - Windows
Nederlands
Polski
Português
Svenskt
русский
Japanese
Simplified Chinese
Traditional Chinese
Korean