You need to be logged in to post
Options
Index
#204722 - 05/29/13 06:16 PM [User Showcase] Loch Lomond (Synthetic Vocals)
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
When I posted a song a while back recorded using Sinsy, Don Gaynor had expressed hope that with this sort of technology he'd be able to sing again. While the fine folk at DynaVox haven't gotten back to him yet, I thought I might have a go at solving the problem.

You may have noticed I've been a bit absent from the board since then. blush

Here's the result: Loch Lomond (with harmony)

It's "state of the art" 1980's formant synthesis. It's not the technology I'd intended to use, but that's a rather long story. I'm still working out the bugs in the code. It's not ready for end users, but it's finally "singing."

There's a bit of post-processing on the vocals. I used the PG Vinyl plugin to reduce some of the popping on consonants, and a low-pass filter to kill some of the high-end noise. I've also run things through EZ Mix to add some reverb and compression.

Most of the phonemes are fairly acceptable, but there's an /H/ and /L/ that seem to have gone missing. I've only tweaked one of the phonemes. At 0:15 there's what sounds like a breath noise. It's actually a bug, so I lowered the volume there. (I intend to add support for breaths at some point).

The accompaniment is PJONPBA.STY (PopBalladPiano & Ac.Guitar[85RS]). I guessed at the chords from the sheet music, so there may be some clinkers in there. The harmony is just thirds above the melody, so there may be some clinkers there, too.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#204723 - 05/29/13 06:35 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 12/10/03
Posts: 8410
Loc: Oklahoma, USA
Don Gaynor Offline
Veteran

Registered: 12/10/03
Posts: 8410
Loc: Oklahoma, USA
I love it David! Thanki, mate!

Now you may understand the difficulty I ran into trying to overcome inertia and reluctance to change on the part of Corporations, especially from ootsiders. Very frustrating and disheartening. It's extremely difficult to keep on slugging when we see no progress. I have several dead-end projects that I am stymied on including "The iPod Project" (qv).


Edited by Don Gaynor (05/29/13 06:48 PM)
Edit Reason: To make a semblance of cents to Canadian readers and moderators.
_________________________
Donny

SoundClick: http://www.soundclick.com/bands/default.cfm?bandID=1209254&content=music
SoundCloud: https://soundcloud.com/donald-gaynor
BIAB 2019 w/current updates.

Top
#204730 - 05/29/13 07:21 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
Hopefully, I can shape it into something useful. I've taken a couple of days off work in an effort to get this into shape, and I've just found a major portion of the code that needs to be rewritten.

Once this is stable, I still need to set up some sort of front end for it. The current plan is to have it read MusicXML files. I've got some code I've written for a different project I should be able to re-purpose.

I also need to write some code to do dictionary lookup. I've got a nice hyphenated phonetic dictionary, so hopefully the majority of that work is already done.

But... I still need to stabilize this code.

If you're really lucky, this will give incentive to DynaVox to finish up their project, and you'll get some real voice synthesis. wink
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#204740 - 05/29/13 09:48 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 12/06/05
Posts: 4047
Loc: 80 Mi North of Sacramento CA
seeker Offline
Veteran

Registered: 12/06/05
Posts: 4047
Loc: 80 Mi North of Sacramento CA
David,

This effort you have made into this cutting edge technology is outstanding.
I've spent thousands of hours since the 70's working on coding projects that
had never been done before. Can really appreciate the efforts you have put into
this.

Hang in there and continuing good efforts on your "projects". Note didn't say
good luck. Patience and stamina, be with you.
_________________________
FrankB

Down The Street vs2015 12-03-2014
Win7, AMD QuadA8-5500,16GB,2TeraHD, Komplete 10
PG Ultra Plus 2016,Alesis 12USB, Sonar Platinum

Top
#204748 - 05/30/13 01:42 AM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: seeker]
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
Originally Posted By: seeker
This effort you have made into this cutting edge technology is outstanding.


Yes, cutting edge 1980's technology at it's best. wink

This is about four generations removed from current voice synthesis technology. You might recall S.A.M. (Software Automatic Mouth), which was the basis of MacInTalk. If you play with the demo on that page (decompiled from assembly into C, and then converted into JavaScript!), you can hear the familial resemblance.

Interestingly, the company that put out S.A.M. is still in business as SoftVoice, Inc., and it was their demo of Twinkle, Twinkle Little Star that convinced me that while formant synthesis might not create realistic results, it might be "good enough" for my purposes. Since they've been doing this for the last 30 years, I think their example is probably as good as this technology gets.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#204794 - 05/30/13 12:34 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 06/22/12
Posts: 2868
Loc: Karlsruhe, Germany
boehm Offline
Veteran

Registered: 06/22/12
Posts: 2868
Loc: Karlsruhe, Germany
Hi David,

a milestone in your efforts. Following your blog I know
what a lot of work this was and still is.

Guenter
_________________________
http://www.21d.de/
Latest Song: What You Desire

Top
#204807 - 05/30/13 02:59 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
The next step it to make this usable.

I'd like to write a UI that displays the music on a staff, and integrates with the CMU phonetic dictionary. Nothing terribly complex - just good enough to "get the job done." It'll output a .wav file, which BiaB can load.

The rest is up to Don. whistle
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#204822 - 05/30/13 04:49 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 08/10/12
Posts: 7576
Loc: Florida
floyd jane Offline
Veteran

Registered: 08/10/12
Posts: 7576
Loc: Florida
David,

I commend you on the effort that this must require. Standing ovation for that, alone. Bravo.

floyd

Top
#204856 - 05/30/13 09:27 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 05/30/08
Posts: 4495
Sundance Offline
Veteran

Registered: 05/30/08
Posts: 4495
Kudos David. What a thoughtful thing to do - not to mention all the time and effort involved. I'm sure it will bring Don and others lots of fun in making their music when you're done.

Top
#204913 - 05/31/13 12:25 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
Thanks! smile
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#204957 - 05/31/13 05:54 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 10/31/08
Posts: 15784
Loc: Australia
Noel96 Offline
Veteran

Registered: 10/31/08
Posts: 15784
Loc: Australia
Hi David,

This is amazing! I can't even begin to comprehend the amount of effort that must have been involved. You realize that you'll have to change your signature's saying now! "Loch Lomond" is all about 'voice control' at it's absolute cleverest smile

Regards,
Noel
_________________________
LINKS TO MY BIAB/RB SONGS


Top
#205047 - 06/01/13 12:01 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 10/25/08
Posts: 7733
Loc: Winston-Salem, NC
Pat Marr Offline
Veteran

Registered: 10/25/08
Posts: 7733
Loc: Winston-Salem, NC
David,

I've got to be honest and say that my (very) first impression of this was that the timing of your presentation couldn't have been worse, considering the fact that Guenter has recently posted some excellent projects using a more advance synthetic voice. Comparisons are inevitable, and you are at a disadvantage being a single developer working for free without any kind of subsidy.

But the more I listen and compare your synth voice to the one Guenter is using, the more impressed I am with yours.

The biggest difference to my ear is that yours has a pronounced artifact where the syllables are joined together, whereas the other engine has figured out a way to blend the syllables more smoothly... which, in a song environment is definitely more musical.

Having dealt with artifacts when blending snippets in a DAW, I have to wonder if it would help to fade each of your separate syllables on both ends so the sharp edge that causes the artifact is less pronounced when they are joined together.

In the final analysis, all comparisons aside, what you have accomplished here is PHENOMEnal! (pun intended)

I wish you much luck with further development and I look forward to hearing more examples as time goes on.


Edited by Pat Marr (06/01/13 03:24 PM)

Top
#205060 - 06/01/13 02:10 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 12/05/11
Posts: 8035
Loc: GA USA
Janice & Bud Offline
Veteran

Registered: 12/05/11
Posts: 8035
Loc: GA USA
The patience of Job must have gone into that. I can't imagine the code (and I used to do a little programming). Gonna be fascinating to see how it plays out. Thanks.
_________________________


Check out our BiaB SoundCloud songs!
J&B YouTube Sampler
J&B Website
Our album "All Over The Map" is on Apple Music, Spotify and others.

Top
#205064 - 06/01/13 03:19 PM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: Pat Marr]
Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
dcuny Online   content
Veteran

Registered: 09/27/10
Posts: 3134
Loc: Sacramento, California
Hi, Pat.

Thanks for the your response. Before getting into details, I should clarify: this project is just re-implementing what others have already done, and much better than me. From text to speech: The MITalk system (Allen) was a great resource for me. The technology is essentially abandoned for other, better approaches. I'll give an explanation why I was unable to go down that route.

Also, my immediate goal is to get Don something usable. If DynaVox finally gets him a better synthesis program, then, Hurrah!. No need for this program.


I'm aware of the Vocaloid software - I've got the Avanna as well, because it's probably got the best English accent of all the current Vocaloids.

I've also spent a lot of time looking at UTAU, a free synthetic singer written along the lines of Vocaloid.

In fact, my initial approach was exactly what you suggested: record various phonemes (using Vowel/Consonant/Vowel patterns), cross-fade them together, and use pitch shifting.

I've actually written a number of tools to do this. The stumbling block was the pitch shifting. The pitch shifting needs to shift some frequencies (the glottal pulse) and keep others fixed (the formants) or you get the "Mickey Mouse" effect.

BiaB uses the astonishingly good elastiq algorithm. I couldn't find any free libraries that gave decent results - even the RubberBand library, which has formant preservation, didn't do an acceptable job.

I tried FFT-based pitch shifting, but didn't have much luck.

I got better results with PSOLA (Pitch Synchronous Overlap and Add), but there were significant artifacts: Here's an example.


The examples I'd heard of formant-based synthesis convinced me that while it lacked realism, it was capable of creating intelligible and musical synthesis. I think you'll agree that, with some tuning, this synthesizer may not create realistic voices, but they can be understandable.


And to be honest, I've been focused on just getting the code to work. I've spent very little time on fine-tuning the phonemes. This is alpha-software, and there's lots of room for improvement.

That said, in Text-to-Speech Synthesis, Paul Taylor argues that formant-based synthesis is intrinsically un-natural because it can't capture the details of real speech, so I don't hold high hopes for it.

I've considered mixing pre-recorded audio with synthesized sounds like eSpeak, but that raises plenty of issues. And there's still the issue of handling sounds like /B/, /D/ and /G/, which are voiced and consonants. So for the moment, I'm sticking with "pure" synthesis.

I hope that somewhat explains that approach I've taken. Despite the many flaws, I figured it was time to move ahead with the project. For the moment, I'll be focusing on creating a UI.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#205129 - 06/02/13 10:36 AM [User Showcase] Re: Loch Lomond (Synthetic Vocals) [Re: dcuny]
Registered: 03/02/13
Posts: 3730
Loc: Helsinki, Finland
dani48 Offline
Veteran

Registered: 03/02/13
Posts: 3730
Loc: Helsinki, Finland
Hi, David !

I am no expert on these matters
which maybe makes me better judging
the end result from a listeners point of view ?
I am referring to the tune. Maybe I am
partial as this tune has a special meaning
for me. You see, once I sang this song to my
dear wife Beni at the shore of this Loch and little
did I know then, that the words held a prophecy
as "me and my true love would never be allowed
to meet again on the bonnie, bonnie banks of Loch Lomond" !
She died of cancer on October the 26 th 2011 !

I think you have done a marvellous job David !
Keep on the good work !

Cheers
Dani

Top


PG Music News
50 New RealStyles included in Loops-with-Style PAK 1 for Band-in-a-Box®!

Loops-with-Style PAK 1 is a new library of 100 original loops that we've created using RealTracks/RealDrums source material, and it includes 50 new RealStyles!

Click here to listen to audio demos of the new RealStyles included with Loops-with-Style PAK 1 for Band-in-a-Box®.

ON SALE UNTIL DECEMBER 31st! Save over 30% when you purchase during our special - add Loops-with-Style PAK 1 to your collection today for just $19! Click here for more information.

New! Loops-with-Style PAK 1 for Band-in-a-Box®!

A brand-new Band-in-a-Box Add-on is HERE!

Loops-with-Style PAK 1 for Band-in-a-Box® 2019 (and higher) expands your loops collection with 100 new loops! Click here to watch our video demonstration.

We've taken the RealDrums and RealTracks source audio and remixed, remastered, and combined it to create these original loops. All of your favorite genres are covered, like jazz, funk, rock, pop, electronic, hip-hop, and more! To show them in action, we've created 50 new RealStyles that each include two of the loops. Plus, if you use the loops in Band-in-a-Box, we've included intelligent versions of the loops that automatically switch at A and B part markers. Each loop has a matching sister loop that works perfectly for another substyle!

ON SALE UNTIL DECEMBER 31st!

Save over 30% when you purchase during our special - add Loops-with-Style PAK 1 to your collection today for just $19!

Click here for more information. Mac user? This new Add-on works for you too! Click here for Mac.

Video - 202 New RealTracks for Band-in-a-Box® 2020 for Windows®!

With the release of Band-in-a-Box® 2020, we've added 202 new RealTracks! These include great Americana, Gospel Vocals, new Blues, Country Pollwinners, Bossa Flute/Horn sections, Jazz Blues and Rhythm Changes, British and Hazy '60s guitars, Modern Funk, Island Grooves with steel drum and marimba, Singer-Songwriter, and more!

Get the newest RealTracks Sets 329-352 when you purchase the PlusPAK upgrade (ideal for 2019 UltraPAK/UltraPAK+ customers) for just $99 during our special (reg. $129).

Want the entire RealTracks collection with over 2,900 RealTracks? They are included with any Band-in-a-Box 2020 for Windows UltraPAK, UltraPAK+, or Audiophile Edition.

Check out this video, which demonstrates the newest RealTracks Sets 329-352.

New Band-in-a-Box® 2019 for Mac Add-ons are Here!

Just released for Band-in-a-Box® 2019 for Mac: Xtra Styles PAK 8 and Loops-with-Style PAK 1!

We've added 164 fantastic new RealStyles in Xtra Styles PAK 8! These are imaginative new combinations of our existing RealTracks library that fall into four categories: Rock/Pop 8, Jazz 8, Country 8, and Celtic 1. To celebrate the release of Xtra Styles PAK 8, ALL of our Xtra Styles PAKs 1-8 are on sale for just $29 each! Learn more & listen to demos here.

Also brand-new is our Loops-with-Style PAK 1 for Band-in-a-Box®, which gives you the chance to expand your loops collection with 100 new loops! We've taken the RealDrums and RealTracks source audio and remixed, remastered, and combined it to create these original loops. All of your favorite genres are covered like jazz, funk, rock, pop, electronic, hip-hop, and more! To show them in action, we've created 50 new RealStyles that each include two of the loops. Plus, if you use the loops in Band-in-a-Box, we've included intelligent versions of the loops that automatically switch at A and B part markers. Each loop has a matching sister loop that works perfectly for another substyle! Get started with Loops-with-Style PAK 1 for the introductory price of just $19 until December 31st (reg: $29). Learn more & watch a great video demonstration here.

Note: Xtra Styles PAK 8 and Loops-with-Style PAK 1 require the Band-in-a-Box® 2019 or higher UltraPAK, UltraPAK+, or Audiophile Edition.

Video - Band-in-a-Box® 2020 - Everything you need to know in under 6 minutes!* (plus the 49-PAK!)

It's HERE! Take a quick tour of the just-released Band-in-a-Box® 2020 for Windows when you check out our latest video:
Band-in-a-Box® 2020 - Everything you need to know in under 6 minutes!* (plus the 49-PAK!)


Read more about this new version here, and check out our amazing upgrade pricing (up to 50% off during our special!) here!

Great New Content in Band-in-a-Box® 2020 for Windows®!

Band-in-a-Box® 2020 for Windows is here! Limited Time Special Offers Until December 31, 2019 - SAVE up to 50% on most Band-in-a-Box® version 2020 for Windows Upgrade packages... and that's not all! When you buy version 2020 for Windows before December 31, 2019, we'll include a Bonus PAK full of great new Add-ons FREE!

Band-in-a-Box® 2020, Band-in-a-Box® VST DAW Plugin 2020 and RealBand 2020 - Over 50 new features!
- We have information about the program here.
- Specific information about the features is here.
- Ordering and info about the pacakages and pricing is available here.

Plus, there is a HUGE amount of new content...
- 202 new RealTracks, description and audio demos: click here.
- Xtra Styles Set 8 for Band-in-a-Box®, (description and video demos): Learn more...
- Loops-with-Style Set 1 for Band-in-a-Box®: product description and video demo.
- Bonus PAK full of great new Add-ons FREE! Learn more...
- Bonus 49 Pak (with 40 RealTracks)! Learn more...
- 2 new MIDI SuperTrack Sets - Learn more...
- 280 Drums RealCharts (previously had only 22, now have transcriptions for half of all of the 600 audio RealDrums)
- 5 Instrumental Studies Sets - Learn more...
- Two new all MIDI styles sets "Look-Ma! More MIDI" Sets 3 and 4 (30 MIDI styles) audio demos: Learn more...

Plus,the Online (or PDF) manual for the full program, or just the new features, are available here.

RealBand 2020 is Here!

RealBand 2020 is here, and is included free with Band-in-a-Box® 2020 for Windows!

There are over 20 additions to RealBand, including intelligent Audio-Harmonies (1-4 voice), Scrub Mode, mixer enhancements, open MusicXML .musicxml and .xml, easier transpose, RealTracks, RealDrums sound improvements, and more!

RealBand 2020 New Features include:

Darker Grid Lines
Grid lines in Tracks and Audio Edit windows are drawn in a darker color instead of white.

Auto-hide Grid Lines
RealBand will auto-hide grid lines when they are within a certain number of pixels of each other.

Audio Scrub Mode
The new audio scrub button in the Audio Edit window lets you click and drag to hear a section of audio.

Shift key for moving multiple volume sliders and pan knobs
If moving a volume slider while pressing the Shift key, then any other selected tracks' sliders will also move. This applies to Tracks and Mixer windows.

...and more!

For the complete list of new features included in RealBand 2020, click here, and choose the "RealBand 2020" Tab.

Forum Stats
30569 Members
63 Forums
61161 Topics
516524 Posts

Max Online: 2434 @ 11/14/17 12:37 AM
Newest Members
Tripdog, roch, sickboy12345678, bobluthier, Don G

30568 Registered Users
Top Posters (30 Days)
VideoTrack 289
Matt Finley 227
Pipeline 185
Noel96 175
MarioD 153
Torrey Bliss 140
PeterGannon 138
Jim 131
jpettit 127
Rob Helms 123
Today's Birthdays
No Birthdays