HW5 - CS4706 - "TTS Evaluation (100 pts)"

Submission procedure explained below.

A. TTS Evaluation

1.      Propose a set of criteria for evaluating TTS systems and justify this set.  These criteria should be objective and results should be reproducible.  I.e., they should not be based primarily on listener preferences, although you may also include subjective criteria as a secondary set.  Try to be as creative as possible.  Consider many different kinds of inputs, such as equations, addresses, temporal expressions, and inputs appropriate to different types of applications.  Try to exercise each of the components of the system insofar as possible:  text normalization, pronunciation, intonation assignment, unit selection, etc.

2. Develop a set of inputs which will test each criterion.  There should be a sufficient number of input text sentences to demonstrate convincingly how the system performs wrt each.

3.      For two of the following three TTS interactive demo sites: ATT, IBM, Loquendo, test at least 2 voices for languages you know.  Save input text and output .wav files, appropriately marked, together with the conclusions you draw from each.  Note the major errors you discover. (Note that Loquendo will save the file before playing but depending on your OS you may need to find the saved file.  The IBM site has been flakey recently so you may not have the third option.)

4.      Provide an overall assessment of each TTS system and, within the system, of the voices you test.  What are the major strengths and weakness?   Which is the best system/voice?