SLS TELEPHONY APPLICATIONS

What is Jupiter?

Jupiter is a conversational system that provides WSI Corporation's high quality, up-to-date Intellicast weather forecast over the phone. Jupiter features WSI Intellicast forecasts for cities worldwide.

Since it was first publicly deployed in May 1997, Jupiter has fielded over hundreds of thousands of calls, achieving average word accuracies of over 90% on in-domain queries for novice users (over 98% for experienced users), and correct understanding of about 80% of in-domain queries for novice users (over 95% for experienced users). The system has an active vocabulary of just over 2000 words.

Jupiter presents challenges in several interesting research areas, including:

1. Virtual Browsing
   Jupiter provides access to Web weather information to those without PC's, internet providers, etc. All that is needed is a telphone.


2. Displayless Interaction
   Jupiter is a telephone-based conversational system. Our only means can get information to you is via an audio interface. This means that we must worry about issues such as conciseness (to answer user queries in the most efficient, user-friendly way), and the quality of our synthetic speech (to provide the clearest form of audio output).


3. Content Understanding
   Jupiter must understand the content of Web-based weather reports in order to deliver just the information that the user requests (e.g., "What is the temperature today in Boston?"). The challenges here include retrieving, understanding, and synthesizing information from multiple knowledge sources on the Web.


4. Multilinguality
   One of the goals of Jupiter is to understand user queries in multiple languages and answer those queries in the same language. We are currently working on paraphrasing weather content into Spanish, Japanese, and Mandarin Chinese and hope to begin work soon on speech understanding in these languages.


5. Robust Speech Recognition/Understanding
   Jupiter is a telephone-only system, so we must be able to deal with the special challenges telephone speech presents. Additional research issues here include rejection (e.g., for out-of-domain inquiries) and dialogue-dependent language-modelling.

Jupiter can answer questions about general weather forecasts, as well as information on temperature, wind speed, humidity, sunrise time, and advisories. Jupiter can also tell you which cities it knows about in a particular region.

The following are suggested scenarios for you to solve. If you have difficulties getting the information, you can either ask Jupiter for help (e.g. say "help") or consult our hints page [link to a page with sample queries?].

• Find out the temperature in your hometown for tomorrow.
• Find a city in the United States with interesting weather events, e.g, hail, thunderstorms, hurricanes and tornadoes, floods and the like.
• Find a sunny place in the Caribbean.
• Find a windy city.

Note that Jupiter only knows about weather. Jupiter will not be able to tell you about the time, the price of gasoline, traffic conditions, the meaning of life etc. If you ask these questions the system will hopefully tell you it didn't understand, but might give you a forecast for some city! This is a rejection problem which we are working on.

How do I talk to Jupiter?

1. Speaking Style:
   Jupiter is intended to recognize and understand natural, conversational speech. When you talk to Jupiter, it is actually best if you speak naturally to the system, as you would to another person. In other words, you don't need to pause between words, overemphasize words (e.g., pronouncing them one syllable at a time), or speak in computerese (e.g. "weather boston" vs. "what's the weather in boston"). The system also doesn't do as well if you shout, mumble, or speak softly. It is best if you speak clearly, as you might to a young child. Also, if you speak softly (especially at the end of your sentence), or pause extensively, the end point detector might cause your speech to be clipped, which will make it harder for Jupiter to understand you.


2. Noise:
   Jupiter tends to work best on normal telephone lines. It will generally have more problems in noisy environments (e.g., background conversations, loud music, tv, or car noise), or with poor-quality speakerphones or cellular phones. If the system is having trouble understanding you, you might try a normal handset to improve the noise conditions.


3. Correcting Mistakes:
   Jupiter is an experimental prototype that we are constantly improving. When the system makes a mistake, you should try to correct the mistake. These data are useful to us because they help us study (and improve!) dialog corrections. To correct a mistake you can re-ask your query, ask your question in a different way, or simplify your query. Note that if you speak before the beep, or you hear a beep before you finish talking, your speech will be clipped and will potentially cause the system to misunderstand you.

• What cities do you know about in California?
• How about in France?
• What will the temperature be in Boston tomorrow?
• What about the humidity?
• Are there any flood warnings in the United States?
• Where is it sunny in the Caribbean?
• What's the wind speed in Chicago?
• How about London?
• Can you give me the forecast for Seattle?
• Will it rain tomorrow in Denver?

How do I talk to Jupiter?
Currently, the telephone-based Jupiter system has been taken off-line. We have developed a web-based version of Jupiter, and hope to re-deploy it some day. In the meantime, there are successors to Jupiter available at the SLS prototype demonstrations URL.

Here are some basic instructions for talking with Jupiter:

1. When you call, you will be connected with Jupiter (the connection will take about 5 sec.), and the system will greet you with something like "Welcome to Jupiter - the automated weather service from MIT. How may I help you?"


2. At this point, you will hear a brief high pitch beep, indicating that the computer is listening. You can now ask your question, e.g., what's the forecast in Boston, will it rain tomorrow, when will the sun rise, etc.


3. When you are done asking the question, you will hear a brief low pitch beep, indicating that the machine has detected an endpoint. It will entertain you with music and then respond (hopefully with the right answer!). If it doesn't do well, you may want to change your speaking rate a bit, or try not to trail off at the end of a sentence. You might also want to simplify your query, if you are having trouble with longer queries. The system does not do as well if you are in a noisy environment or if you use cell phones.


4. When Jupiter is done, it will say "can I help you with something else" and go to step 2 again. When you are done, simply hang up.


5. If Jupiter doesn't answer the phone, it's because someone else is using it. Wait a couple of minutes and try again. Also, remember that you are being recorded. We are using your voice and queries to improve the system!


6. Please keep in mind that this is not a product, it is a research prototype. We are putting it out there primarily for the purpose of data collection. But it tends to do okay on reasonable queries.

How does Jupiter work?
Jupiter updates its weather forecasts three times a day from a variety of different weather sources. The weather forecasts are parsed by the TINA language understanding system in order to determine and tag the types of weather information available in each forecast.

During a call, Jupiter utilizes a set of core speech technologies developed by the SLS group the recognize, understand, and respond to each request from the user. The meaning of each request is determined and matched against the available weather reports. Jupiter speaks the weather forecast best matching the user's request back to the user.

Comments or questions to bug-galaxy@lists.csail.mit.edu