Page 1 of 2 1 2 >
You need to be logged in to post on the forum
Topic Options
Index
#197215 - 03/06/13 03:55 AM [Off-Topic] Free web-based synthetic singer
dcuny Offline
Veteran

Registered: 09/27/10
Posts: 2386
Loc: Sacramento, California
I ran across Sinsy the other day, and thought it might be a good fit for some forum members who are unable to sing. It's a synthetic singer capable of doing a fairly good job with English.

There are a couple features of Sinsy that make it stand out from other programs. Unlike Vocaloid, it's free. Unlike Utau, it's usable without having to deal with Japanese. It takes MusicXML files as input, and pays attention to various musical markings. It has a fair bit of musical intelligence, and does a good job with putting phrases together in a very musical way.

On the downside, it's got a pretty distinct Japanese accent. It would be nice if they modeled it using a native English speaker, but can you expect for free?

I put together a demo using BiaB: Words Like Smoke

And because I can't stop leave well enough alone, here's a remix: Words Like Smoke (Mix 2)

The music (including the melody) was generated by BiaB using the MIDI PNOMOOD1 style with a couple of added RealTrack guitars, although I tweaked the register of the melody in a couple places. I exported the MIDI file from BiaB and edited the lyrics in Finale NotePad. Sinsy wasn't happy with getting multiple staffs, and I couldn't figure how to remove staffs in NotePad, so I edited it in Notion and did a bit more tweaking of the lyrics.

If BiaB exported MusicXML, it would have been a lot simpler!

The remix was done in Reaper - I started out fixing the guitar strums (they needed to be moved forward) and ended up messing around with the vocal track and Reaper's pitch correction tool. Fun stuff.

Anyway, breaths are automatically added after long phrases - a nice touch. Vibrato is added automatically on long notes, and I'm impressed how well it handled tied notes on the final cadence.

It also works with MuseScore - here's Ave Maria done in MuseScore.

Actually, with the "ze" instead of "the", Sinsy sounds a bit like Natasha Fatale.


Edited by dcuny (03/06/13 05:26 AM)

Top
#197216 - 03/06/13 05:23 AM [Off-Topic] Re: Free web-based synthetic singer [Re: dcuny]
Ryszard Offline
Veteran

Registered: 09/28/07
Posts: 3900
Loc: Kennesaw (Atlanta) GA
That is astonishing--and good. The expressiveness from the Schubert score is especially (ahem) noteworthy. It is unsurprising that it has a bit of an accent, given that it was taking its cues from Kanji (simplified Japanese ideograms). It is hard to tell from your short sample whether it does better with English text. Even the slightly synthetic quality, something like a cross between a human voice and a Theremin, is somewhat appealing, at least to this electronic composer. Excellent work on your part, too. Thanks for sharing.

Richard
_________________________
"My primary musical instrument is the personal computer."

Top
#197217 - 03/06/13 06:55 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Ryszard]
Noel96 Offline
Veteran

Registered: 10/31/08
Posts: 13699
Loc: Australia
Hi David,

Thank you for passing on this information. I'm astounded! It makes me wonder where music will be in 5 years time!

Regards,
Noel
_________________________
LINKS TO MY BIAB/RB SONGS


Top
#197218 - 03/06/13 06:57 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Noel96]
Westside Steve Offline
Journeyman

Registered: 11/17/08
Posts: 389
Loc: Shores of Lake Erie
_________________________
[b]Westside Steve
www.westsidesteve.com
www.YouTube.com/WestsideSteveSimmons

Top
#197219 - 03/06/13 08:07 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Westside Steve]
MountainSide Offline
Expert

Registered: 12/06/11
Posts: 1055
Loc: Lake Keowee, South Carolina
Wow...that's not bad...perhaps could be better used for background vocals huh? Does it also have male vocals? Can it handle multiple vocal parts for harmony and choir?
_________________________
Win10x64, Intel i7 7700k@4.6Ghz, 32Gb RAM, 2x1Tb HD, 500Gb NVMe, BIAB/RB current builds, MOTU 828MK3 audio, MOTU Midi Express, Yamaha DX7II, TX802, Motif XS Rack, Roland Fantom XR Rack, Oberheim Matrix 1000, VoiceLive Touch, Kontakt 5.8, SampleTank 3.7

Top
#197220 - 03/06/13 09:01 AM [Off-Topic] Re: Free web-based synthetic singer [Re: MountainSide]
Mac Offline
Veteran

Registered: 05/29/00
Posts: 38502
Loc: Chesapeake, Virginia USA
Sounds artificial, artifacts and all that.

Bears watching, though, to see if improvements are forthcoming.


--Mac
_________________________
PGmusic FAQs, Tutorials and Updates! click here

You must be Audiominds.
www.audiominds.com

Top
#197221 - 03/06/13 09:42 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Mac]
filkertom Offline
Veteran

Registered: 01/03/06
Posts: 2028
Loc: Ann Arbor, MI
Huh. Okay, that Ave Maria was rather frickin' good.
_________________________
Best,
Tom Smith
My Music at Bandcamp
Win10/64 • i5-3570K • 16 GB RAM
Audiophile 192 • BIAB 2017 • Reaper 5
AKG C1000S • Behringer Xynex1002

Top
#197222 - 03/06/13 09:57 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Mac]
Don Gaynor Offline
Veteran

Registered: 12/10/03
Posts: 8005
Loc: Oklahoma, USA
Mac, the Ave Maria link convinced me. Imagine composing full choral arrangements in the very near future using BIAB and nothing else. Its uncharacteristic of you to poo-poo technological advancements no matter how infantile the current state-of-the-art. Have you had your coffee yet? At least my SLP (Speech Language Pathologist) was fascinated and is sharing this link with her SLP friends and I only imagine that she has access to all the latest advancements in synthesized speech. Have a cupa and revisit the link, my friend.

http://www.youtube.com/watch?v=u7K0-ttUBng

It has fewer phonyms (sp?) than the sample of Beatles' Yesterday (on same page) but the advancements in technology in just three years is phenomenal.

Top
#197223 - 03/06/13 10:01 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Don Gaynor]
Ryszard Offline
Veteran

Registered: 09/28/07
Posts: 3900
Loc: Kennesaw (Atlanta) GA
Quote:

. . . it has fewer phonyms (sp?) than the sample of Beatles' Yesterday . . .




Phonemes (said the professional linguist).

Richard
_________________________
"My primary musical instrument is the personal computer."

Top
#197224 - 03/06/13 12:56 PM [Off-Topic] Re: Free web-based synthetic singer [Re: Ryszard]
PgFantastic Offline
Veteran

Registered: 02/11/05
Posts: 2646
Loc: Kentucky
Makes me want to take singing lessons!
_________________________
My Tunes

Psalm 57:7 My heart, O God, is steadfast, my heart is steadfast; I will sing and make music.

Top
#197225 - 03/06/13 01:14 PM [Off-Topic] Re: Free web-based synthetic singer [Re: Ryszard]
dcuny Offline
Veteran

Registered: 09/27/10
Posts: 2386
Loc: Sacramento, California
As Richard pointed out, a larger phoneme set would make this program phenomenal for English speakers. But it's "good enough" right now, and BIAB user can start working with this program right away.

Having MusicXML export (of the vocal track only) would be ideal, and I'm hoping MusicXML export gets added to BiaB in the near future.

In lieu of that, if you haven't got a music notation program and don't want to download any software, Noteflight is a web-based program that will import MIDI and export MusicXML. So you could get a free account, upload the MIDI into Noteflight, add lyrics, and export MusicXML into Sinsy.

Most synthetic singer programs use a piano roll sort of interface. Because Sinsy generates good performance by default from the score means that it's less cumbersome than other programs. Plus, most of this software needs additional drivers for Japanese language support, even if you're using in English.

There are a number of reasons that this software has taken off in Japan - the limited phoneme set certainly helps. Vocaloid was initially the leading software for synthetic singing, but the free Utau program has started to surpass it. An interesting feature of Utau is that users are able to create their own voice banks instead of being locked down to using commercially released voices.

Instead of just recording single phonemes, voice banks consist of recordings of consonants and vowels (CV) - "sa", "so", "see", and so on. More sophisticated voice banks have more complex combinations - VCV and CVVC. So there are very smooth transitions between phonemes.

Unfortunately, creating equivalent voice banks for English is a much larger undertaking, because there are a lot more combinations. But it's clearly doable.

In any event, it plays nice with BiaB, creates musical results and is free. That's very cool.

Don, interested in using it for any songs?
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#197226 - 03/06/13 02:37 PM [Off-Topic] Re: Free web-based synthetic singer [Re: dcuny]
PeterGannon Offline
PG Music Staff

Registered: 05/29/00
Posts: 13456
Very cool! Thanks for pointing it out David.
_________________________
Have Fun!
Peter Gannon
PG Music Inc.

Top
#197227 - 03/06/13 06:51 PM [Off-Topic] Re: Free web-based synthetic singer [Re: PeterGannon]
Jim Fogle Offline
Veteran

Registered: 08/20/11
Posts: 4128
Loc: Winston-Salem, NC USA
I hear voice synthesis daily listening to NOAA weather radio. An article about the evolution of the voices used is here: http://www.nws.noaa.gov/nwr/newvoice.htm
_________________________
Jim Fogle
2018 BiaB (520) UltraPlusPak RB 2018 (Build 5)
Cakewalk by Bandlab - Sonar Home Studio - Cakewalk Music Creator 6 - Audacity - Zoom MRS-8 recorder
i3 laptop, 64bit Win 7, 8 GB ram, 480GB SSD
Music at: http://fogle622.wix.com/fogle622-audio-home

Top
#197228 - 03/07/13 11:45 AM [Off-Topic] Re: Free web-based synthetic singer [Re: Jim Fogle]
MountainSide Offline
Expert

Registered: 12/06/11
Posts: 1055
Loc: Lake Keowee, South Carolina
Hmmmm....apparently Yamaha is behind Vocaloid and a new English version, albeit by a Japanese vocalist, is now available: http://www.vocaloid.com/en/ and here: http://www.ssw.co.jp/en/products/vocal3/megpoid/index.html .

Wouldn't it be great if PG partnered with these guys to develop an US-based linguistic algorithm to create the ultimate Band in a Box!
_________________________
Win10x64, Intel i7 7700k@4.6Ghz, 32Gb RAM, 2x1Tb HD, 500Gb NVMe, BIAB/RB current builds, MOTU 828MK3 audio, MOTU Midi Express, Yamaha DX7II, TX802, Motif XS Rack, Roland Fantom XR Rack, Oberheim Matrix 1000, VoiceLive Touch, Kontakt 5.8, SampleTank 3.7

Top
#197229 - 03/07/13 01:48 PM [Off-Topic] Re: Free web-based synthetic singer [Re: MountainSide]
Don Gaynor Offline
Veteran

Registered: 12/10/03
Posts: 8005
Loc: Oklahoma, USA
Quote:

Don, interested in using it for any songs?



David, not at the moment, I'm steeped in Songwriting Class, but afterwards, I'll prove Mac true: I'll be "full of it!"

I think we've gotten the good Doctor's attention and that's what we had hoped to accomplish. "Take it, Peter!"

I am so chuffed by all you've done in researching this subject, David. You've acted "above and beyond the call..." Thank you so sincerely.

With Dr Peter carrying the puck to the goal, I'll rest comfortably now.

@David I'll take a look at my current compositions but, unfortunately, I routinely delete my .sgu and .mgu files after rendering to .wav or .mp3. Color me stupid, but please try to stay within the lines.

Top
#197230 - 03/07/13 01:54 PM [Off-Topic] Re: Free web-based synthetic singer [Re: Don Gaynor]
PeterGannon Offline
PG Music Staff

Registered: 05/29/00
Posts: 13456
Hi David,

What controls do you need for a note, other than the time, duration, note number, and the lyric
For example, at bar 1, beat 1, you might want it to say "The" and last for one beat/

How many other parameters can you enter, for example, is there a strength setting (and is it 0-9?) and vibrato information?

Can you be more precise for the timing, for example, to start at bar 1, beat 1, tick 23 (out of 120 PPQ), and last for 87 ticks (120 PPQ).
_________________________
Have Fun!
Peter Gannon
PG Music Inc.

Top
#197231 - 03/07/13 01:59 PM [Off-Topic] Re: Free web-based synthetic singer [Re: MountainSide]
dcuny Offline
Veteran

Registered: 09/27/10
Posts: 2386
Loc: Sacramento, California
There's no doubt that Vocaloid can be quite good. Here's a comparison of all three programs doing "Yesterday":

Yesterday (Vocaloid, "Oliver" voice)
Yesterday (Sinsy)
Yesterday (UTAU, "Camila Melodia" voice)

Vocaloid is a commercial product. The editor runs around $100US, and doesn't include any voice libraries. Voice libraries run about $130US. There are a number of characters that target English, although they all seem to have accents of some sort.

The results can be very good, but it requires spending money and learning a new software program.


Jinriki Vocaloid ("manual Vocaloid") consists of manually cutting and pasting together phonemes of a singer, and creating a new song. (It helps when the language has a limited phoneme set).

UTAU is a free program that started out automating that manual process, and eventually grew into a much more powerful program. Voice banks are created by users, and quality varies dramatically. There are some UTAUoids that rival the quality commercial Vocaloid voice banks.

But... Getting good English results from UTAU can be a challenge, because it requires running a Japanese program with limited translation, and finding a good English voice bank.


Sinsy apparently came out of a research project. I wouldn't be surprised to find that it becomes commercial at some point. There is currently only one English voice for it, and it appears to be built from a Japanese voice, so it's got a pretty strong accent.

Using Sinsy only requires uploading a MusicXML file of the vocals, and just about every music notation program (except for BiaB, hint, hint) does that. It's really easy.


There are other options. For example, MelodyAssistant has a VirtualSinger. It sounds like they're using the free Festival voice synthesizer. Listening to the VirtualSinger demos gives a good comparison to what's changed in the technology.


I think this technology is still very much in development, so adding it to BiaB is an iffy proposition. But adding features (like MusicXML) that make interfacing with tools like Sinsy makes a lot of sense.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#197232 - 03/07/13 02:34 PM [Off-Topic] Re: Free web-based synthetic singer [Re: dcuny]
Don Gaynor Offline
Veteran

Registered: 12/10/03
Posts: 8005
Loc: Oklahoma, USA
I communicate with the R&D folks at Dynavox and, just yesterday, sent them the link to Ave Maria but haven't heard back yet.

In all fairness to Dynavox, their target market is Speech Synthesis, not singing/music. With no one to keep a fire under them and cattle prod them frequently, the project will remain on the proverbial back burner indefinitely. Therefore, I need someone who can get the project moved OVER the burner. I can provide names and email addresses if anyone wants to accept the challenge.

David? You get things accomplished. Would you consider accepting the task?

Thanks.

Top
#197233 - 03/07/13 02:34 PM [Off-Topic] Re: Free web-based synthetic singer [Re: PeterGannon]
dcuny Offline
Veteran

Registered: 09/27/10
Posts: 2386
Loc: Sacramento, California
Quote:

What controls do you need for a note, other than the time, duration, note number, and the lyric



It depends which program you're talking about.

Sinsy takes well-formed MusicXML. There are a lot of elements in MusicXML that aren't required, and it looks like it'll accept a pretty minimal MusicXML file. As I mentioned, just about any music notation will generate MusicXML, and while it's a bit verbose, it's not that hard to generate.

Off tangent for a moment...

I haven't got a Vocaloid, but I know that it's got it's own VSQ file format. It imports MIDI data, and I suspect that's how most people interface with it.

I won't even touch UTAU... There's way too much fiddling to get the Japanese localization working on my machine, and it's only half-translated anyway.

The free Festival system had an old XML file format that looked a lot like what you're describing - a note list of phonemes with pitch and timing. You can see an example of it here. I haven't kept up with Festival, and the document is pretty old (in computer years) - back in 2002.

End of tangent

Quote:

How many other parameters can you enter, for example, is there a strength setting (and is it 0-9?) and vibrato information?



The Sinsy documentation (on the main page) says:

The following musical symbols are supported: tie, slur, staccato, accent, dynamics, crescendo, decrescendo, breath mark.

Quote:

Can you be more precise for the timing, for example, to start at bar 1, beat 1, tick 23 (out of 120 PPQ), and last for 87 ticks (120 PPQ).



Yes, MusicXML will let you get down to that resolution. There are two portions to MusicXML: the displayed notation, and the actual performance. Like MIDI, you specify you base tick value, and you can give the timing relative to that tick. Here's an example (sorry, but the indentation is lost):
Quote:

<measure number="6">
<attributes/>
<note>
<pitch>
<step>B</step>
<octave>5</octave>
</pitch>
<duration>192</duration>
<voice>1</voice>
<type>half</type>
<dot/>
<lyric number="1">
<syllabic>single</syllabic>
<text>this</text>
</lyric>
</note>
</measure>



Really, the simplest thing to do is export a MIDI melody from BiaB, import it into just about any music notation program, add lyrics and export the MusicXML.

While Sinsy is really cool in that it automatically does this stuff, I don't know what their licence is, and I don't know that there's any guarantee that the web site will stay up, and how accessible it will be.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
#197234 - 03/07/13 03:20 PM [Off-Topic] Re: Free web-based synthetic singer [Re: Don Gaynor]
dcuny Offline
Veteran

Registered: 09/27/10
Posts: 2386
Loc: Sacramento, California
Quote:

In all fairness to Dynavox, their target market is Speech Synthesis, not singing/music.



I think this is a key point. Music requires a lot of things that "ordinary" speech synthesis doesn't. Here is a paper that does a good job explaining it.

The frequency envelope of the voice needs to take into account a number of additional factors:
  • Portamento: Notes that belong to the same word or phrase should be smoothly connected, instead of jumping from note to note.
  • Preparation: Before moving to the next note, the pitch may move in the opposite direction first in preparation of the change.
  • Overshoot: Before hitting the target pitch, the pitch may overshoot the target.
  • Vibrato: Sustained notes will typically have an added vibrato.
  • Fluctuation: Holding the pitch perfectly is unnatural, so some low-level fluctuation needs to be added.
All of these are easily observed when looking at the frequency match line from a pitch correction program. (I left off "undershoot" and "scooping", which I see on my own vocal way too much!)

The paper cited above looks like it's got enough information to implement these features. I'd want logic to prevent vibrato on notes less than a particular duration.

There's also the question of how well their voices will translate to singing. I assume the DynaVox model automatically handles formant preservation since they're synthesizing the voice in the first place. The paper cites the addition of a "singing formant" at about 3kHa, along with amplitude modulation based on vibrato (volume changes along with the vibrato).

Quote:

With no one to keep a fire under them and cattle prod them frequently, the project will remain on the proverbial back burner indefinitely. Therefore, I need someone who can get the project moved OVER the burner. I can provide names and email addresses if anyone wants to accept the challenge.



If the DynaVox people are truly interesting in this, I could prod them.

I don't know how difficult this would be to add to their product, but the basic ideas are pretty straight forward. Basically, it's a matter of creating a dynamic frequency envelope. That'll get you a long way to a more realistic singing voice.

There was a project like this for the free Festival system, but it seems to be mostly dead links now. While Festival is legible, it's not really that pleasant to listen to.
_________________________
-- David Cuny
My virtual singer development blog

Vocal control, you say. Never heard of it. Is that some kind of ProTools thing?

Top
Page 1 of 2 1 2 >


PG Music News
#TBT Band-in-a-Box® 2017 Review - Software for Learning Saxophone

McGill Music Sax School posted a video review of Band-in-a-Box® 2017 for Windows last year, stating "I love using software to help learn saxophone. One of my favourite programs is Band-in-a-Box. It's a really powerful tool for quickly making backing tracks I can use for lessons, or my own practice."

Check it out! www.pgmusic.com/?vid=2AgfV7emdLQ

Band-in-a-Box® 2018 for Mac - Importing MIDI Files

Did you know that Standard MIDI files can be read in to the Melodist or Soloist track within Band-in-a-Box? You can use all of the MIDI File, or select channels or a specific range of bars, using these steps:
1. Melody | Import Melody from MIDI File to select a MIDI file from disk using the file dialog.
2. Melody | Import Melody from Clipboard - when the MIDI data is already copied from another program to the
clipboard.

A new window will open, asking for additional criteria - which channels, # of bars to offset..., # of bars to import, include lyrics, include patch changes - and more! Head to our online manual Chapter 10: Working with MIDI to learn more!

YouTube Find - Your First 15 Minutes with Band-in-a-Box

Woody Piano Shack recently got their hands on Band-in-a-Box after a long time without the program - and they're thrilled!!

Check out this video they've shared to their YouTube Channel: https://youtu.be/9dxSfwjBvG0

Thanks Woody Piano Shack!

National Dictionary Day - Guitar & Piano Chord Dictionaries

Today is National Dictionary Day!

We offer 2 dictionaries for just $19 each: the Guitar Chord Dictionary & the Piano Chord Dictionary!

Each of these programs are easy to use! Simply choose a root note, select the chord you want to learn, and the Dictionary will play and display the chord!

An Essential Tool for any Musician!

User Tip - Band-in-a-Box® to Reaper: The Basics

Forum user DeaconBlues09 has created a video to demonstrate how easy the process is for them to streamline their workflow with DAWs, particularly Reaper - check it out: click here to watch.

Join the forum discussion.

Train Your Brain with Band-in-a-Box®

National Train Your Brain Day is October 13th!

Open your Band-in-a-Box® program, and you'll find the [Practice] button in the toolbar - it's a GREAT resource!
Minimized your toolbar? Head to Window | Practice Window or ALT SHIFT L to access

In this window, you'll find many options to help you practice, including Ear Training games, access to the Chord Builder and Chord Subs, some helpful tools within the Playalong Features menu, and more!

Train Your Brain with Band-in-a-Box® today!

Band-in-a-Box Patch Updates - #TechTuesday

Visit our Patches & Updates page, and you will see that with every version released, there are 2+ patch updates applied to that version.

Patch updates are important, and should definitely be installed to your BB folder. They can add new features, update the help resources of the program, fix bugs reported by users, and make your entire Band-in-a-Box experience run smoothly!

Band-in-a-Box update options can be adjusted within the program too - within Band-in-a-Box for Mac, head to Band-in-a-Box | Check for Updates | and then you can select to automatically check for updates by however many days you choose!

Forum Stats
28081 Members
57 Forums
54452 Topics
450237 Posts

Max Online: 2434 @ 11/14/17 12:37 AM
Newest Members
M1S1L, collcoll, Vocaloidgirl9980, boxx.dk, Boto

28080 Registered Users
Top Posters (30 Days)
Noel96 179
David Snyder 142
Matt Finley 138
Janice & Bud 136
Will Josef 134
Tangmo 122
beatmaster 115
MarioD 113
Deryk - PG Music 112
Jim Fogle 105
Today's Birthdays
Jacques Constantineau