In this post I would like to share opinions about Text To Speech (TTS) technology in the context of gaming. I am not a gaming expert – rather, my expertise is in signal and speech processing. I am working at the IBM Research division.
I hope this post will trigger a discussion and opinion sharing. My purpose here is to discuss “quality”. I plan to discuss technology trends in follow on posts.

The potential benefit of “good quality” TTS technology for game developers is clear. But it is still considered as delivering “insufficient quality”. What is “quality” anyway, in our context?

1. The basic quality of modern TTS is good, in general. State of the art machine learning algorithms enable good prediction of the prosody (“intonation”, duration, loudness, emphasis and more) from the text, and the synthesized speech achieves good scores in subjective quality tests. This is no more the “robotic” sound it used to be. It sounds natural and “clean”. For applications such as announcements or commercial question answering, modern TTS provides a good alternative.

2. Natural speech, however, needs correct emphasis of different words across the sentences. Due to the ambiguity of the natural language, the algorithms (and humans as well) cannot always determine “correctly” the emphasis from the text of isolated sentences or utterances, without full knowledge of the entire context. This can limit the quality achievable by modern TTS technology.

3. When we consider gaming applications, additional needs arise. For example, using a formal style for generating the speech for a scene of action would sound weird – where are the emotions? The emotional content in human speech is essential for conveying messages. This is certainly an important aspect of “quality”.

4. Yet another aspect that relates to “quality”, at least in the broader sense, is the variety of voices. Most modern TTS technologies are based on pre-recorded human voices (recording of voice talents uttering a large collection of sentences). As recording, and processing the recorded speech, are expensive and time consuming, the variety of voices in typical TTS products is limited, often a few and less commonly several tens of different voices per “major” language. Moreover, gaming often requires non-human voices, such as “cartoonish” ones, to best support the different characters. To summarize, using repeatedly the same voices across games and characters, amounts to less than optimal experience, or in other words – to lower “quality”.

I hope this provides some initial insights – from the perspective of a speech technology researcher. I hope to get feedback from the gaming experts. Am I right? What have I missed? What would allow game developers to start benefiting from the TTS technology?
All the best, Aharon.

Still trying to decide whether this is a spam or not ?
Please stick to one thread .. You have already posted another ..

Me to ,if it is it’s clever.

Dave

I already apologized to the administrator for my misunderstanding. Spamming is certainly not my intention, and I will not repeat this in the future.

Having said that, I really hope to get feedback on the content - and use it in our ongoing research.

Many thanks! Aharon

Then please accept my apologies Aharon ..
I was also using IE11 when checking it out which did not work so this also gave me doubt to the posts validity.
But I have since tried Chrome and MS Edge and found this to be pretty impressive ...

Definitely worth checking out people but I would start here and the feedback requested could be a good thing
http://www.research.ibm.com/haifa/dept/imt/ivvc.shtml

Surprised, it appears not many have checked out the IBM TTS over in the other thread and simply assumed it was spam, it is actually very impressive and has incredible potential.
Tried it myself and found it awesome. There are little problems with inflection and pronunciation which I solved using a simple comma in the appropriate space. For example the word Madrigal became madr..ai...gal. Using a comma, Madr,igal gave the pronunciation I was looking for. Also helpful for long sentences where a speaker might pause for breath. All unnoticeable to a listener in real terms but stands out when there is no visual indication.Perhaps breath intake at appropriate points would add to realism as well, if you listen closely to news announcers this is very pronounced due to compressor levels and gate used in audio processing.
Be warned if you go looking for this, once heard it can't ever be unheard.

It would be very cool if you could manually add inflection or emotion into specific words or sentences for emphasis rather than rely on an algorithm. Probably never be perfect but for sure this is heading in the right direction. Also needs some way to increase volume on certain words or sentences for emphasis perhaps a color chart from hot to cold for text to speech might help with this.

There is no indication of cost for this app at this point and only an e-mail link if you want to use it in your game, I would be interested in a general idea of cost before I went any further time wise on it.

The mere fact that only a single male and female voice are used and can be tweaked to sound incredibly diverse is pretty impressive and wondering if others could be added to create more of an accent or local dialect.

In general you may find that (particularly around here) voice overs and music/ambience are the very last things most users concern themselves about, even though these are as important as shiny graphics. Just sayin' but there is no real difference between games, movies or announcements when it comes down to quality in audio speech. If you simply set up a mike and record you include all the background noise and even the room ambience can have an effect on the quality of the recording. This might seem trivial to the untrained ear and even seem a bit anal but the comparison between a pro recording and amateur one is pretty distinct if compared side by side.
On the other hand the absence of appropriate background ambience can be just as noticeable so you might want to consider the ability to add this to the mix for those who don't have experience with layering audio and processing.

Unfortunately the original post looked like spam we get every day https://forum.game-guru.com/thread/219204
No " Hi Guys "or etc .. It looked like a default message and went directly to the demo which didn't work for me

Now the original page I posted was far more credible .. had this been posted it would have been a whole different ballgame
http://www.research.ibm.com/haifa/dept/imt/ivvc.shtml

It worked straight off for me hence I was surprised by the lack of response...it really is a very innovative and cool app

Aharon probably did create a generic blurb to copy/paste on as many game dev sites at once as he could, so yeah it would come across as spam to some. Understandable on both sides I guess but his intentions I believe are good since he is going to the source to research.

I was using IE11 and for me it didn't initially but as I said above after using chrome and edge its most impressive ..
A " save as " .wav would be excellent

If I'm not mistaken Sainsburys are using something like this already ( or very similar )
I picked up an synth voice on an announcement last week ... Very good but still just noticeable ..

There is a 'save as' wav, you just need to create an account to enable this

I tried it also, and I think rolfy is right on point. It was very good, and I liked that it had several things that could be adjusted. However, I'm also concerned about what the cost will be.

Yeah, pretty impressive.

Same as rolfy, I found that by putting commas in various places in words themselves, I was able to achieve good pronunciation of made-up words/names.

A few things I immediately thought of:

1) Is there a desktop application for this - many game devs have slow interenet connections, if it was online only then that would instantly prevent many users from making use of the tool. For me, anything that is an on-line tool only, usually gets a thumbs down.

2) Cost - cost is always a factor, especially in the indie scene. Whenever I look at any tool, the first thing I do is look at price - if it is clearly outside of my budget, I look no furthur. If the price is reasonable and within my budget, I will then evaluate the software to see if it is something I can use.

Reliquia....

Have to agree that price is probably the main factor for many including myself ..
I guess thats really the first thing we all want to know .

To answer my own first question:



"The entire solution, including the virtual voice design and synthesis components, is delivered as a cloud service."

I know cloud services are becoming more and more prevelant, but it is not a solution that is useful to many potential users around the globe.

In my experience cloud services tend to be nothing more than a vehicle for extracting ongoing revenue from every customer, and the web is simply not a good dev tool, unless you have super fast connection.

For me this is the first nail in the coffin for this tool - and it has so much potential

Reliquia....

So I can take all my scripted game elements, painstakingly type them in to some web app and spend time fiddling with the settings to get it to sound right, then spend time fiddling with the text itself, adding comma's and such, to get the 'emotional' context I need and finally save each 'speech' away to files for use in my game.

Or I can grab a microphone and power up Logic and do pretty much the same thing in a more organic way using either my own voice or a family members.

I know which is more fun!

Fair Comments

Hello Rolfy, here are my answers:
...it is actually very impressive and has incredible potential.
Thank you!

There are little problems with inflection and pronunciation which I solved using a simple comma in the appropriate space.
In addition, you can manually control the inflection / emphasis manually down to the word level, pls see the help section.

For example the word Madrigal became madr..ai...gal. Using a comma, Madr,igal gave the pronunciation I was looking for. - good idea!

Also helpful for long sentences where a speaker might pause for breath. All unnoticeable to a listener in real terms but stands out when there is no visual indication.
You can manually control the inflection / emphasis manually down to the word level, pls see the help section.

Perhaps breath intake at appropriate points would add to realism as well...
We need to add the option to add pauses, beyond simple commas - in a more controlled way. In progress.

It would be very cool if you could manually add inflection or emotion into specific words or sentences for emphasis rather than rely on an algorithm.
It is supporter, pls see the help in the app.

Probably never be perfect but for sure this is heading in the right direction. Also needs some way to increase volume on certain words or sentences for emphasis perhaps a color chart from hot to cold for text to speech might help with this.
I hope to hear your feedback once you have tries the manual editing capabilities, as I mentioned above.

There is no indication of cost for this app at this point and only an e-mail link if you want to use it in your game, I would be interested in a general idea of cost before I went any further time wise on it.
This is a research project presently, which has not been productized yet. Let me get back with a better answer, within a few days.

Hello Len the man, similarly to my answer to rolfy, I will get back with more answers about pricing if a few days. For now, it is a research project. Thank you! Aharon

Hello reliquia, the service is cloud based, as it allows us online and ongoing updating, as it is a research project.

However, please note that once you have created voice personas (they are saved under your account in the cloud and can be shared across your work group), and once you have uses the voice personas to generate speech from text, the speech files are downloadable. Our intention is to let game developers to embed the speech files in their game software, independently of any cloud.

You see the cloud only during the TTS creation phase. Also, we are working on integrating the tool with common game engines, beginning with Unity. I have no schedule for that.

Another aspect is real-time synthesis, which might be a required feature in certain games. The application you see is offline, but the engine behind the scenes is in fact real-time, and the next step could be exposing APIs to enable real-time functionality. But this would require the games to become cloud-based.

Thank you
Aharon

Thanks for the info, I merely played around a little with the app but time is limited for me to go into it in any depth without a heads up on cost. so apologies if I missed things that are already implemented.

I am of the opinion that there is no point to spending thousands of hours building a game of any quality and not making the effort to ensure audio quality comes up to standards. Unless my game is a sit-com which takes place in my street, I will need to outsource the voice acting to folks with the right tone and accents I would be looking for, I would also expect decent audio quality that I don't need to be processing and cleaning up myself if I have to pay for it.

Voice acting and audio engineering are skills and no less important to a game than quality models, shiny graphics and nifty coding. You will find in any game dev forum some coders who think they don't need Artists, some Artists who think they don't need coders and some in both camps thinking they don't need quality audio. Good luck to 'em with that one, keep us posted on how that works out for you

I don't believe that TTS will ever replace proper voice acting but your app is a sure thing as a decent placeholder for bigger studios and (hopefully) a cheaper alternative to Indie dev's who can't afford the luxury of hiring voice actors.

Let us know something of costs and please take into account the difference between pro studio and Indie license and I will come back to it.

Can't ever see GG becoming cloud based myself so the only way the real-time aspect would be useful is if it came as a plugin speech engine, in the same way that we have a physics engine and a scripting 'engine'.

I feel in some ways the industry is going backwards, in the early 90's I had a speech synth module which sounded quite good and had a small footprint, used it to add speech to Elite easily enough, the voices were mechanical sounding but for sci-fi based games was perfectly adequate. To use it you simply passed the text string to it ahead of time with a small header of config information then sent a trigger when you wanted it to start speaking, the config info contained 'voice', pitch, speed, volume and other parameters that could tweak the 'voice'.

I also had an app which allowed you, easily, to build animated dinosaurs from a kit of parts and then render them with povray into a movie! Both came free on the front of a magazine! (remember those? )

Have you guys seen what Adobe has been up to?



Soon you'll be able to have any celebrity as a voice actor in your game :p

Wow Teabone, the voice sounds good!

The big question is, how much money they will charge for that software, or for monthly and yearly fees?

Lol - pretty cool. Scary stuff though!!

It will only be sub-based and it won't be cheap..... - least that's is my guess.

Reliquia....

Cool and scary at the same time.

It Adobe so itvwill probably another pay for life to use it

Dave

That Adobe thing looks cool.

Yep, lol - the whole world could be using their software if they were not so damn greedy.

I am sure there is some miserly guy with thick black rimmed glasses in the accounts office looking at each and every Adobe product saying gibberishly whilst rubbing his hands together feverishly - must get rich off this one, must get rich off this one.....

Reliquia....

Hello,
We saw the interest in pricing from the responses below. We cannot quote pricing at this point.

We can, though, offer this beta-level service for free for about half a year from now, until Aug 10, 2018. We hope it is attractive enough for game developers to experience the tool and use it, this way. Our purpose is to gain feedback to improve the tool. We encourage you to read the help to get information about the new capabilities and explore them.

Any audio content generated through this tool (https://ivva-tts.sl.haifa.il.ibm.com/) until Aug 10, 2018, can be used for free, forever, including for commercial purpose.

What will happen after Aug 10, 2018? We don’t know yet. Optionally, the term of free use may be extended. Another option – the cloud service will be productized and offered for a fee. Presently, we offer this beta service for free use.
We hope to see productive use of our tool and even more – to learn your insights!

I will post more details about the technology behind the scenes over the next weeks.

Aharon

Synthesized voices that can be created from text is becoming more tenable. however, I'm with AmenMoses for now.



I use my own voice and use MorphVox software to change the way my voice sounds.

AM also said: "I feel in some ways the industry is going backwards, in the early 90's I had a speech synth module which sounded quite good and had a small footprint, "

Actually I cant recall any speech synth during that time but I never looked for one that carefully. When I did start looking before purchasing MorphVox I definitely didn't like how any of them sounded.

Might give this a run and see what it sounds like.

Addendum:

Looked at the on-line demo through IBM.

granny, witch, ms sadness Needs improvement
Almost there: Mr. Allison - the original voice "Allison is good"

None of them sound synthesized; that amazes me. In fact they sound similar to a creation from MorphVox

Yup, another piece of software i won't be using because they're throwing everything they got into sub based software, disgusting in my opinion, they should sell the software then be done with it, if they have updates for new features then charge for the updates, as long as the core remains operational after purchase.



I don't recall any decent speech synth at that time, i do recall the very basic robotic speech synth from windows 95 text to speech, that was always fun to play with lol



That's common in most speech synth software. I've also found that for some reason in most of them a comma has a harder break than a full stop, so ending sentences and paragraphs with a comma sounds more natural than ending with a period.

I have to agree with AmenMoses here, it's much more fun doing it yourself with a mic/amplifier/sampling software. Luckily, my acting skills allow me to do my own voices