Computers have been used for language teaching ever since the 1960′s. This 40-year period can be divided into three main stages: behaviorist CALL, communicative CALL, and integrative CALL. Each stage corresponds to a certain level of technology and certain pedagogical theories. The reasons for using Computer-assisted Language Learning include: (a) experiential learning, (b) motivation, (c) enhance student achievement, (d) authentic materials for study, (e) greater interaction, (f) individualization, (g) independence from a single source of information, and (h) global understanding. The barriers inhibiting the practice of Computer-assisted Language Learning can be classified in the following common categories: (a) financial barriers, (b) availability of computer hardware and software, (c) technical and theoretical knowledge, and (d) acceptance of the technology.

Introduction

In the last few years the number of teachers using Computer-assisted Language Learning (CALL) has increased markedly and numerous articles have been written about the role of technology in education in the 21st century. Although the potential of the Internet for educational use has not been fully explored yet and the average school still makes limited use of computers, it is obvious that we have entered a new information age in which the links between technology and TEFL have already been established.

In the early 90′s education started being affected by the introduction of word processors in schools, colleges and universities. This mainly had to do with written assignments. The development of the Internet brought about a revolution in the teachers’ perspective, as the teaching tools offered through the Internet were gradually becoming more reliable. Nowadays, the Internet is gaining immense popularity in foreign language teaching and more and more educators and learners are embracing it.

The History of CALL

Computers have been used for language teaching ever since the 1960′s. According to Warschauer & Healey (1998), this 40-year period can be divided into three main stages: behaviorist CALL, communicative CALL, and integrative CALL. Each stage corresponds to a certain level of technology and certain pedagogical theories.

Behaviorist CALL

In the 1960′s and 1970′s the first form of computer-assisted Language Learning featured repetitive language drills, the so-called drill-and-practice method. It was based on the behaviorist learning model and as such the computer was viewed as little more than a mechanical tutor that never grew tired. Behaviorist CALL was first designed and implemented in the era of the mainframe and the best-known tutorial system, PLATO, ran on its own special hardware. It was mainly used for extensive drills, explicit grammar instruction, and translation tests (Ahmad, et al., 1985).

Communicative CALL

Communicative CALL emerged in the 1970′s and 1980′s as a reaction to the behaviorist approach to language learning. Proponents of communicative CALL rejected behaviorist approaches at both the theoretical and pedagogical level. They stressed that CALL should focus more on using forms rather than on the forms themselves. Grammar should be taught implicitly and students should be encouraged to generate original utterances instead of manipulating prefabricated forms (Jones & Fortescue, 1987; Philips, 1987). This form of computer-based instruction corresponded to cognitive theories which recognized that learning was a creative process of discovery, expression, and development. The mainframe was replaced by personal computers that allowed greater possibilities for individual work. Popular CALL software in this era included text reconstruction programmers and simulations.

Integrative CALL

The last stage of computer-assisted Language Learning is integrative CALL. Communicative CALL was criticized for using the computer in an ad hoc and disconnected fashion and using the computer made ‘a greater contribution to marginal rather than central elements’ of language learning (Kenning & Kenning, 1990: 90). Teachers have moved away from a cognitive view of communicative language teaching to a socio-cognitive view that emphasizes real language use in a meaningful, authentic context. Integrative CALL seeks both to integrate the various skills of language learning (listening, speaking, writing, and reading) and to integrate technology more fully into language teaching (Warschauer & Healey, 1998). To this end the multimedia-networked computer provides a range of informational, communicative, and publishing tools that are potentially available to every student.

Why Use CALL?

Research and practice suggest that, appropriately implemented, network-based technology can contribute significantly to:

Experiential Learning

The World Wide Web makes it possible for students to tackle a huge amount of human experience. In such a way, they can learn by doing things themselves. They become the creators not just the receivers of knowledge. As the way information is presented is not linear, users develop thinking skills and choose what to explore.

Motivation

Computers are most popular among students either because they are associated with fun and games or because they are considered to be fashionable. Student motivation is therefore increased, especially whenever a variety of activities are offered, which make them feel more independent.

Enhanced Student Achievement

Network-based instruction can help pupils strengthen their linguistic skills by positively affecting their learning attitude and by helping them build self-instruction strategies and promote their self-confidence.

Authentic Materials for Study

All students can use various resources of authentic reading materials either at school or from their home. Those materials can be accessed 24 hours a day at a relatively low cost.

Greater Interaction

Random access to Web pages breaks the linear flow of instruction. By sending E-mail and joining newsgroups, EFL students can communicate with people they have never met. They can also interact with their own classmates. Furthermore, some Internet activities give students positive and negative feedback by automatically correcting their on-line exercises.

Individualization

Shy or inhibited students can be greatly benefited by individualized, student-centered collaborative learning. High fliers can also realize their full potential without preventing their peers from working at their own pace.

Independence from a Single Source of Information

Although students can still use their books, they are given the chance to escape from canned knowledge and discover thousands of information sources. As a result, their education fulfils the need for interdisciplinary learning in a multicultural world.

Global Understanding

A foreign language is studied in a cultural context. In a world where the use of the Internet becomes more and more widespread, an English Language teacher’s duty is to facilitate students’ access to the web and make them feel citizens of a global classroom, practicing communication on a global level.

What Can We Do With CALL?

There is a wide range of on-line applications which are already available for use in the foreign language class. These include dictionaries and encyclopedias, links for teachers, chat-rooms, pronunciation tutors, grammar and vocabulary quizzes, games and puzzles, literary extracts. The World Wide Web (WWW) is a virtual library of information that can be accessed by any user around the clock. If someone wants to read or listen to the news, for example, there are a number of sources offering the latest news either printed or recorded. The most important newspapers and magazines in the world are available on-line and the same is the case with radio and TV channels.

Another example is communicating with electronic pen friends, something that most students would enjoy. Teachers should explain how it all works and help students find their keypals. Two EFL classes from different countries can arrange to send E-mail regularly to one another. This can be done quite easily thanks to the web sites providing lists of students looking for communication. It is also possible for two or more students to join a chat-room and talk on-line through E-mail. .

Another network-based EFL activity could be project writing. By working for a project a pupil can construct knowledge rather that only receive it. Students can work on their own, in groups of two or in larger teams, in order to write an assignment, the size of which may vary according to the objectives set by the instructor. A variety of sources can be used besides the Internet such as school libraries, encyclopedias, reference books etc. The Internet itself can provide a lot of food for thought. The final outcome of their research can be typed using a word processor. A word processor can be used in writing compositions, in preparing a class newsletter or in producing a school home page. In such a Web page students can publish their project work so that it can reach a wider audience. That makes them feel more responsible for the final product and consequently makes them work more laboriously.

The Internet and the rise of computer-mediated communication in particular have reshaped the uses of computers for language learning. The recent shift to global information-based economies means that students will need to learn how to deal with large amounts of information and have to be able to communicate across languages and cultures. At the same time, the role of the teacher has changed as well. Teachers are not the only source of information any more, but act as facilitators so that students can actively interpret and organize the information they are given, fitting it into prior knowledge (Dole, et al., 1991). Students have become active participants in learning and are encouraged to be explorers and creators of language rather than passive recipients of it (Brown, 1991). Integrative CALL stresses these issues and additionally lets learners of a language communicate inexpensively with other learners or native speakers. As such, it combines information processing, communication, use of authentic language, and learner autonomy, all of which are of major importance in current language learning theories.

Teachers’ Barriers to the Use of Computer-assisted Language Learning

The barriers inhibiting the practice of Computer-assisted Language Learning can be classified in the following common categories (a) financial barriers, (b) availability of computer hardware and software, (c) technical and theoretical knowledge, and (d) acceptance of the technology.

Financial Barriers

Financial barriers are mentioned most frequently in the literature by language education practitioners. They include the cost of hardware, software, maintenance (particular of the most advanced equipment), and extend to some staff development. Froke (1994b) said, “concerning the money, the challenge was unique because of the nature of the technology.” Existing universities policies and procedures for budgeting and accounting were well advanced for classroom instruction. The costs of media were accounted for in the university as a part of the cost of instruction. Though the initial investment in hardware is high, inhibiting institutions’ introduction of advance technologies; but Hooper (1995) recommends that the cost of computers will be so low that they will be available in most schools and homes in the future.

Lewis et al. (1994) indicate three conditions under which Computer-assisted Learning and other technologies can be cost-effectiveness: Computer-assisted Learning costs the same as conventional instruction but ends up with producing higher achievement in the same amount of instructional time, it results in students achieving the same level but in less time. These authors indicate that in examples where costs of using technologies in education are calculated, they are usually understand because the value of factors, such as faculty time and cost of equipment utilization, is ignored (McClelland, 1996).

Herschbach (1994) argues firmly that new technologies are add-on expenses and will not, in many cases, lower the cost of providing educational services. He stated that that the new technologies probably will not replace the teachers, but will supplement their efforts, as has been the pattern with other technologies. The technologies will not decrease educational costs or increase teacher productivity as currently used. Low usage causes the cost barrier. Computers, interactive instruction TV, and other devices are used very few hours of the day, week, or month. Either the number of learners or the amount of time learners apply the technology must be increased substantially to approach the concept of cost-effectiveness. There are other more quick and less expensive ways of reducing costs, no matter how inexpensive the technology being used (Kincaid, McEachron, & McKinney,1994.

Availability of Computer Hardware and Software

The most significant aspects of computer are hardware and software. Availability of high quality software is the most pressing challenge in applying the new technologies in education (Herschbach, 1994; Miller, 1997; Office of Technology Assessment, 1995; Noreburg & Lundblad, 1997). Underlying this problem is a lack of knowledge of what elements in software will promote different kinds of learning. There are few educators skilled in designing it because software development is costly and time-consuming (McClelland, 1996).

McClelland (1996) indicated having sufficient hardware in locations where learners have access to it problematic and is, of course, partly a financial problem. Computer hardware and software compatibility goes on to be a significant problem. Choosing hardware is difficult because of the many choices of systems to be used in delivering education, the delivery of equipment, and the rapid changes in technology.

Technical and Theoretical Knowledge

A lack of technical and theoretical knowledge is another barrier to the use of Computer-assisted Language Learning technology. Not only is there a shortage of knowledge about developing software to promote learning, as shown above, but many instructors do not understand how to use the new technologies. Furthermore, little is known about integrating these new means of learning into an overall plan. In the communication between McClelland and C. Dede (1995), Dede indicated the more powerful technologies, such as artificial intelligence in computers, might promote learning of higher-order cognitive skills that are difficult to access with today’s evaluation procedures and, therefore, the resulting pedagogical gains may be under-valued. Improper use of technologies can affect both the teacher and learner negatively (Office of Technical Assessment, 1995).

Acceptance of Technologies

We live in a time change. Gelatt (1995) stated that change itself has changed. Change has become so rapid, so turbulent, and so unpredictable that is now called “white water” change (p.10). Murphy & Terry (1998a) indicated the current of change move so quickly that they destroy what was considered the norm in the past, and by doing so, create new opportunities. But, there is a natural tendency for organizations to resist change. Wrong conceptions about the use of technology limit innovation and threaten teachers’ job and security (Zuber-Skerritt, 1994). Instructors are tend not to use technologies that require substantially more preparation time, and it is tough to provide instructors and learners access to technologies that are easy to use (Herschbach, 1994).

Engaging in Computer-assisted Language Learning is a continuing challenge that requires time and commitment. As we approach the 21st century, we realize that technology as such is not the answer to all our problems. What really matters is how we use technology. Computers can/will never substitute teachers but they offer new opportunities for better language practice. They may actually make the process of language learning significantly richer and play a key role in the reform of a country’s educational system. The next generation of students will feel a lot more confident with information technology than we do. As a result, they will also be able to use the Internet to communicate more effectively, practice language skills more thoroughly and solve language learning problems more easily.

Eva Knodt
Sehda, Inc.

ABSTRACT

We investigate the suitability of deploying speech technology in computer-based systems that can be used to teach foreign language skills. In reviewing the current state of speech recognition and speech processing technology and by examining a number of voice-interactive CALL applications, we suggest how to create robust interactive learning environments that exploit the strengths of speech technology while working around its limitations. In the conclusion, we draw on our review of these applications to identify directions of future research that might improve both the design and the overall performance of voice-interactive CALL systems.

INTRODUCTION

During the past two decades, the exercise of spoken language skills has received increasing attention among educators. Foreign language curricula focus on productive skills with special emphasis on communicative competence. Students’ ability to engage in meaningful conversational interaction in the target language is considered an important, if not the most important, goal of second language education. This shift of emphasis has generated a growing need for instructional materials that provide an opportunity for controlled interactive speaking practice outside the classroom.

With recent advances in multimedia technology, computer-aided language learning (CALL) has emerged as a tempting alternative to traditional modes of supplementing or replacing direct student-teacher interaction, such as the language laboratory or audio-tape-based self-study. The integration of sound, voice interaction, text, video, and animation has made it possible to create self-paced interactive learning environments that promise to enhance the classroom model of language learning significantly. A growing number of textbook publishers now offer educational software of some sort, and educators can choose among a large variety of different products. Yet, the practical impact of CALL in the field of foreign language education has been rather modest. Many educators are reluctant to embrace a technology that still seeks acceptance by the language teaching community as a whole (Kenning & Kenning, 1990).

A number of reasons have been cited for the limited practical impact of computer-based language instruction. Among them are the lack of a unified theoretical framework for designing and evaluating CALL systems (Chapelle, 1997; Hubbard, 1988; Ng & Olivier, 1987); the absence of conclusive empirical evidence for the pedagogical benefits of computers in language learning (Chapelle, 1997; Dunkel, 1991; Salaberry, 1996); and finally, the current limitations of the technology itself (Holland, 1995; Warschauer, 1996). The rapid technological advances of the 1980s have raised both the expectations and the demands placed on the computer as a potential learning tool. Educators and second language acquisition (SLA) researchers alike are now demanding intelligent, user-adaptive CALL systems that offer not only sophisticated diagnostic tools, but also effective feedback mechanisms capable of focusing the learner on areas that need remedial practice. As Warschauer puts it, a computerized language teacher should be able to

understand a user’s spoken input and evaluate it not just for correctness but also for appropriateness. It should be able to diagnose a student’s problems with pronunciation, syntax, or usage, and then intelligently decide among a range of options (e.g., repeating, paraphrasing, slowing down, correcting, or directing the student to background explanations). (Warschauer, 1996, p. 6)

Salaberry (1996) demands nothing short of a system capable of simulating the complex socio-communicative competence of a live tutor–in other words, the linguistic intelligence of a human–only to conclude that the attempt to create an “intelligent language tutoring system is a fallacy” (p. 11). Because speech technology isn’t perfect, it is of no use at all. If it “cannot account for the full complexity of human language,” why even bother modeling more constrained aspects of language use (Higgins, 1988, p. vii)? This sort of all-or-nothing reasoning seems symptomatic of much of the latest pedagogical literature on CALL. The quest for a theoretical grounding of CALL system design and evaluation (Chapelle, 1997) tends to lead to exaggerated expectations as to what the technology ought to accomplish. When combined with little or no knowledge of the underlying technology, the inevitable result is disappointment.

-46-

In order to appreciate the potential benefit of using speech technology in CALL, a basic understanding of both the core technology and its limitations–what it can and cannot do–is therefore essential. In the following section, we will present an overview of speech recognition. We will then cover design considerations as they relate to the performance of specific speech applications. An overview of current research trends will help identify the kinds of technological advances that lend themselves to being deployed in computer-based language instruction. Next, to illustrate the potential use of speech technology, we will examine a number of innovative language learning applications that offer voice-interactive capabilities. We will evaluate these applications in view of how they integrate speech technology within an overall technical and pedagogical design, and how effectively they deal with current technological limitations. In the final section of the paper, we will draw on our review of these applications, as well as on our own experience in building a voice-interactive system for learning Japanese, to identify directions of future research that might improve both the design and the overall performance of voice-interactive CALL systems.

PRINCIPLES OF ASR TECHNOLOGY

Consider the following four scenarios:

• 1. A court reporter listens to the opening arguments of the defense and types the words into a steno-machine attached to a word-processor.
  2. A medical doctor activates a dictation device and speaks his or her patient’s name, date of birth, symptoms, and diagnosis into the computer. He or she then pushes “end input” and “print” to produce a written record of the patient’s diagnosis.
  3. A mother tells her three-year old, “Hey Jimmy, get me my slippers, will you?” The toddler smiles, goes to the bedroom, and returns with papa’s hiking boots.
  4. A first-grader reads aloud a sentence displayed by an automated Reading Tutor. When he or she stumbles over a difficult word, the system highlights the word, and a voice reads the word aloud. The student repeats the sentence–this time correctly–and the system responds by displaying the next sentence.

At some level, all four scenarios involve speech recognition. An incoming speech signal elicits a response from a “listener.” In the first two instances, the response consists of a written transcript of the spoken input, whereas in the latter two cases, an action is performed in response to a spoken command. In all four cases, the “success” of the voice interaction is relative to a given task as embodied in a set of expectations that accompany the input. The interaction succeeds when the response–by a machine or human “listener”–matches these expectations.

Recognizing and understanding human speech requires a considerable amount of linguistic knowledge: a command of the phonological, lexical, semantic, grammatical, and pragmatic conventions that constitute a language. The listener’s command of the language must be “up” to the recognition task or else the interaction fails. Jimmy returns with the wrong items, because he cannot yet verbally discriminate between different kinds of shoes. Likewise, the reading tutor would miserably fail in performing the court-reporter’s job or transcribing medical patient information, just as the medical dictation device would be a poor choice for diagnosing a student’s reading errors. On the other hand, the human court reporter–assuming he or she is an adult native speaker–would have no problem performing any of the tasks mentioned under (1) through (4). The linguistic competence of an adult native speaker covers a broad range of recognition tasks and communicative activities. Computers, on the other hand, perform best when designed to operate in clearly circumscribed linguistic sub-domains.

Humans and machines process speech in fundamentally different ways (Bernstein & Franco, 1996). Complex cognitive processes account for the human ability to associate acoustic signals with meanings and intentions. For a computer, on the other hand, speech is essentially a series of digital values. However, despite these differences, the core problem of speech recognition is the same for both humans and machines: namely, of finding the best match between a given speech sound and its corresponding word string. Automatic speech recognition technology attempts to simulate and optimize this process computationally.

Since the early 1970s, a number of different approaches to ASR have been proposed and implemented, including Dynamic Time Warping, template matching, knowledge-based expert systems, neural nets, and Hidden Markov Modeling (HMM) (Levinson & Liberman, 1981; Weinstein, McCandless, Mondshein, & Zue, 1975; for a review, see Bernstein & Franco, 1996). HMM-based modeling applies sophisticated statistical and probabilistic computations to the problem of pattern matching at the sub-word level. The generalized HMM-based approach to speech recognition has proven an effective, if not the most effective, method for creating high-performance speaker-independent recognition engines that can cope with large vocabularies; the vast majority of today’s commercial systems deploy this technique. Therefore, we focus our technical discussion on an explanation of this technique.

-47-

Figure 1. Components of a speech recognition device

A. Signal Analysis

The first step in automatic speech recognition consists of analyzing the incoming speech signal. When a person speaks into an ASR device–usually through a high quality noise-canceling microphone–the computer samples the analog input into a series of 16- or 8-bit values at a particular sampling frequency (ranging from 8 to 22KHz). These values are grouped together in predetermined overlapping temporal intervals called “frames.” These numbers provide a precise description of the speech signal’s amplitude. In a second step, a number of acoustically relevant parameters such as energy, spectral features, and pitch information, are extracted from the speech signal (for a visual representation of some of these parameters, see Figure 2 on page 53). During training, this information is used to model that particular portion of the speech signal. During recognition, this information is matched against the pre-existing model of the signal.

B. Phone Models

Training a machine to recognize spoken language amounts to modeling the basic sounds of speech (phones). Automatic speech recognition strings together these models to form words. Recognizing an incoming speech signal involves matching the observed acoustic sequence with a set of HMM models. An HMM can model either phones or other sub-word units or it can model words or even whole sentences. Phones are either modeled as individual sounds–so-called monophones–or as phone combinations that model several phones and the transitions between them (biphones or triphones). After comparing the incoming acoustic signal with the HMMs representing the sounds of language, the system computes a hypothesis based on the sequence of models that most closely resembles the incoming signal. The HMM model for each linguistic unit (phone or word) contains a probabilistic representation of all the possible pronunciations for that unit–just as the model of the handwritten cursive b would have many different representations.

Building HMMs–a process called training–requires a large amount of speech data of the type the system is expected to recognize. Large-vocabulary speaker-independent continuous dictation systems are typically trained on tens of thousands of read utterances by a cross-section of the population, including members of different dialect regions and age-groups. As a general rule, an automatic speech recognizer cannot correctly process speech that differs in kind from the speech it has been trained on. This is why most commercial dictation systems, when trained on standard American English, perform poorly when encountering accented speech, whether by non-native speakers or by speakers of different dialects. We will return to this point in our discussion of voice-interactive CALL applications.

-48-

C. Lexicon

The lexicon, or dictionary, contains the phonetic spelling for all the words that are expected to be observed by the recognizer. It serves as a reference for converting the phone sequence determined by the search algorithm into a word. It must be carefully designed to cover the entire lexical domain in which the system is expected to perform. If the recognizer encounters a word it does not “know” (i.e., a word not defined in the lexicon), it will either choose the closest match or return an out-of-vocabulary recognition error. Whether a recognition error is registered as a misrecognition or an out-of-vocabulary error depends in part on the vocabulary size. If, for example, the vocabulary is too small for an unrestricted dictation task–let’s say less than 3K–the out-of-vocabulary errors are likely to be very high. If the vocabulary is too large, the chance of misrecognition errors increases because with more similar-sounding words, the confusability increases. The vocabulary size in most commercial dictation systems tends to vary between 5K and 60K.

D. The Language Model

The language model predicts the most likely continuation of an utterance on the basis of statistical information about the frequency in which word sequences occur on average in the language to be recognized. For example, the word sequence A bare attacked him will have a very low probability in any language model based on standard English usage, whereas the sequence A bear attacked him will have a higher probability of occurring. Thus the language model helps constrain the recognition hypothesis produced on the basis of the acoustic decoding just as the context helps decipher an unintelligible word in a handwritten note. Like the HMMs, an efficient language model must be trained on large amounts of data, in this case texts collected from the target domain.

In ASR applications with constrained lexical domain and/or simple task definition, the language model consists of a grammatical network that defines the possible word sequences to be accepted by the system without providing any statistical information. This type of design is suitable for CALL applications in which the possible word combinations and phrases are known in advance and can be easily anticipated (e.g., based on user data collected with a system pre-prototype). Because of the a priori constraining function of a grammar network, applications with clearly defined task grammars tend to perform at much higher accuracy rates than the quality of the acoustic recognition would suggest.

E. Decoder

Simply put, the decoder is an algorithm that tries to find the utterance that maximizes the probability that a given sequence of speech sounds corresponds to that utterance. This is a search problem, and especially in large vocabulary systems careful consideration must be given to questions of efficiency and optimization, for example to whether the decoder should pursue only the most likely hypothesis or a number of them in parallel (Young, 1996). An exhaustive search of all possible completions of an utterance might ultimately be more accurate but of questionable value if one has to wait two days to get a result. Trade-offs are therefore necessary to maximize the search results while at the same time minimizing the amount of CPU and recognition time.

PERFORMANCE AND DESIGN ISSUES IN SPEECH APPLICATIONS

For educators and developers interested in deploying ASR in CALL applications, perhaps the most important consideration is recognition performance: How good is the technology? Is it ready to be deployed in language learning? These questions cannot be answered except with reference to particular applications of the technology, and therefore touch on a key issue in ASR development: the issue of human-machine interface design.

As we recall, speech recognition performance is always domain specific–a machine can only do what it is programmed to do, and a recognizer with models trained to recognize business news dictation under laboratory conditions will be unable to handle spontaneous conversational speech transmitted over noisy telephone channels. The question that needs to be answered is therefore not simply “How good is ASR technology?” but rather, “What do we want to use it for?” and “How do we get it to perform the task?”

In the following section, we will address the issue of system performance as it relates to a number of successful commercial speech applications. By emphasizing the distinction between recognizer performance on the one hand–understood in terms of “raw” recognition accuracy–and system performance on the other; we suggest how the latter can be optimized within an overall design that takes into account not only the factors that affect recognizer performance as such, but also, and perhaps even more importantly, considerations of human-machine interface design.

Historically, basic speech recognition research has focused almost exclusively on optimizing large vocabulary speaker-independent recognition of continuous dictation. A major impetus for this research has come from US government sponsored competitions held annually by the Defense Advanced Research Projects Agency (DARPA). The main emphasis of these competitions has been on improving the “raw” recognition accuracy–calculated in terms of average omissions, insertions, and substitutions–of large-vocabulary continuous speech recognizers (LVCSRs) in the task of recognizing read sentence material from a number of standard sources (e.g., The Wall Street Journal or The New York Times). The best laboratory systems that participated in the WSJ large-vocabulary continuous dictation task have achieved word error rates as low as 5%, that is, on average, one recognition error in every twenty words (Pallet, 1994).

-49-

An important lesson learned in the development of LVCSR systems is that the technology itself is highly adaptable, yielding increasing robustness when tailored to a specific recognition task. This insight has led to the successful commercialization of speech technology in telephony applications with constrained task domains such as voice-dialing, directory assistance, and information retrieval. The key to designing such applications lies in choosing the right task and in optimizing the variables that affect recognition performance. In what follows, we will discuss some of these variables and show how system performance can be maximized when speech technology is integrated within a carefuly designed user interface.

Task Definition

This is the most important step in designing a speech recognizer. Delimiting the performance domain imposes constraints on both the vocabulary size and what is referred to as “perplexity,” which is usually defined as the average branching factor within any given grammar network. A small vocabulary recognizer with limited perplexity (e.g., of the type used in automatic voice dialing), tends to be much more robust than a high-perplexity large-vocabulary dictation system.

In general, recognizers perform faster and more accurately when the incoming speech is enunciated clearly and in a noise-free environment, when the task perplexity is low, and when the dictionary is small. In this case, the system needs less CPU time and memory to process alternative recognition hypotheses, and word error rates tend to be lower. A simple “yes/no” recognition task is trivial compared to a ticket reservation system that uses a natural dialog user interface. The relationship between perplexity and performance can work to our advantage when we are developing voice-interactive instructional materials since words and phrases used by language learners are usually limited to a relatively small set of clearly circumscribed tasks. However, for systems with limited task domain to perform as expected, all potential user responses must be known in advance and anticipated in the system’s grammar and vocabulary. Therefore, it is important to collect authentic user data in the early stages of developing such systems.

Acoustic Models

Recognizers tend to perform best when trained on (or adapted to) the voice characteristics or speaking style of the speaker. Speaker independent recognizers contain acoustic models obtained by averaging over large variations in the speech patterns of large populations of various ages and dialect groups. By contrast, speaker-dependent systems are trained specifically on the voice of the speaker(s) for whom they are designed. A third option is speaker adaptation, a technique in which acoustic parameters obtained from a subset of speakers, or one speaker, are used to augment or modify the generalized models of a speaker-independent system. Speaker adaptation can reduce recognition error rates by 30% to 70% depending on the acoustic environment and the original acoustic models used (Neumeyer, Sankar, & Digalakis, 1995; Woodland, Pye, & Gales, 1996; Zavaliagkos, Schwartz, McDonough, & Makhoul, 1995). The importance for CALL is that native acoustic models can be adapted to recognize the speech of language learners. Doing so involves collecting the appropriate speech data and training non-native models. The resulting acoustic models can be made exclusively from non-native data (Neumeyer et al., 1996), or by adapting native models to the non-native data (Ehsani, 1996).

-50-

Input Modality

Another variable that affects recognition performance is the manner in which the system processes the incoming speech signal. In systems with a discrete speech input modality, the recognizer processes each word separately. Therefore each word must be spoken separately with distinct pauses between them. In systems with continuous input mode, no such pauses are necessary. Continuous Speech Recognition (CSR) systems use more extensive search algorithms in the decoding stage to optimize not only the phone strings, but also the word strings. The trade-off in recognition accuracy can be formidable, and under conditions where system resources are limited and high degrees of accuracy are necessary, discrete input may be the design of choice.

Input Quality

For optimal recognition performance, the incoming speech signal must be of high acoustic quality. A number of standard techniques can remove noise from the signal or adapt acoustical models to noisy data (for a review, see Young, 1996). However, not only noise interference can affect the quality of the speech input. A number of other factors, often overlooked in the literature, play a role as well, such as the type of sound card and microphone used, or whether the speech is run through a pre-amplifier. Sound cards have their own internal amplifiers, but they tend to amplify the noise along with the speech. The amplitude of the speech signal needs to be carefully adjusted for best recognition performance. More specifically, the amplitude needs to be kept within a certain limited range. If the amplitude exceeds an upper limit, the signal is clipped and the signal analyzer cannot extract all of the relevant features. If the amplitude is too low, background noise becomes more prominent and can overpower the signal.

Furthermore, the microphone can make a tremendous difference in recognition performance. Most recognizers perform best when used with a noise-canceling head-mounted microphone. Not only do these microphones filter out extraneous noise, but the head-mounted position ensures that the distance between the speaker’s mouth and the microphone is kept constant and the amplitude remains stable throughout the utterances. Finally, some kind of mechanism for automatically adjusting (or telling the user to adjust) the amplifier or the pre-amp setting is of value. Most commercial voice-interactive CALL systems offer this feature.

Careful consideration of the factors described above must enter into the design of commercial speech applications, if they are to perform well in practical applications.

CURRENT TRENDS IN VOICE-INTERACTIVE CALL

In recent years, an increasing number of speech laboratories have begun deploying speech technology in CALL applications. Results include voice-interactive prototype systems for teaching pronunciation, reading, and limited conversational skills in semi-constrained contexts. Our review of these applications is far from exhaustive. It covers a select number of mostly experimental systems that explore paths we found promising and worth pursuing. We will discuss the range of voice-interactions these systems offer for practicing certain language skills, explain their technical implementation, and comment on the pedagogical value of these implementations. Apart from giving a brief system overview, we report experimental results if available and provide an assessment of how far away the technology is from being deployed in the commercial and educational environments.

Pronunciation Training

A useful and remarkably successful application of speech recognition and processing technology has been demonstrated by a number of research and commercial laboratories in the area of pronunciation training. Voice-interactive pronunciation tutors prompt students to repeat spoken words and phrases or to read aloud sentences in the target language for the purpose of practicing both the sounds and the intonation of the language. The key to teaching pronunciation successfully is corrective feedback, more specifically, a type of feedback that does not rely on the student’s own perception. A number of experimental systems have implemented automatic pronunciation scoring as a means to evaluate spoken learner productions in terms of fluency, segmental quality (phonemes) and supra-segmental features (intonation). The automatically generated proficiency score can then be used as a basis for providing other modes of corrective feedback. We discuss segmental and supra-segmental feedback in more detail below.

Segmental Feedback. Technically, designing a voice-interactive pronunciation tutor goes beyond the state of the art required by commercial dictation systems. While the grammar and vocabulary of a pronunciation tutor is comparatively simple, the underlying speech processing technology tends to be complex since it must be customized to recognize and evaluate the disfluent speech of language learners. A conventional speech recognizer is designed to generate the most charitable reading of a speaker’s utterance. Acoustic models are generalized so as to accept and recognize correctly a wide range of different accents and pronunciations. A pronunciation tutor, by contrast, must be trained to both recognize and correct subtle deviations from standard native pronunciations.

-51-

In a final step, machine generated pronunciation scores are validated by correlating these scores with the judgment of human expert listeners. As one would expect, the accuracy of scores increases with the duration of the utterance to be evaluated. Stanford Research Institute (SRI) has demonstrated a 0.44 correlation between machine scores and human scores at the phone level. At the sentence level, the machine-human correlation was 0.58, and at the speaker level it was 0.72 for a total of 50 utterances per speaker (Franco et al., 1997; Kim et al., 1997). These results compare with 0.55, 0.65, and 0.80 for phone, utterance, and speaker level correlation between human graders. A study conducted at Entropic shows that based on about 20 to 30 utterances per speaker and on a linear combination of the above techniques, it is possible to obtain machine-human grader correlation levels as high as 0.85 (Bernstein, 1997).

Others have used expert knowledge about systematic pronunciation errors made by L2 adult learners in order to diagnose and correct such errors. One such system is the European Community project SPELL for automated assessment and improvement of foreign language pronunciation (Hiller, Rooney, Vaughan, Eckert, Laver, & Jack, 1994). This system uses advanced speech processing and recognition technologies to assess pronunciation errors by L2 learners of English (French or Italian speakers) and provide immediate corrective feedback. One technique for detecting consonant errors induced by inter-language transfer was to include students’ L1 pronunciations into the grammar network. In addition to the English /th/ sound, for example, the grammar network also includes /t/ or /s/, that is, errors typical of non-native Italian speakers of English. This system, although quite simple in the use of ASR technology, can be very effective in diagnosing and correcting known problems of L1 interference. However, it is less effective in detecting rare and more idiosyncratic pronunciation errors. Furthermore, it assumes that the phonetic system of the target language (e.g., English) can be accurately mapped to the learners’ native language (e.g., Italian). While this assumption may work well for an Italian learner of English, it certainly does not for a Chinese learner; that is, there are sounds in Chinese that do not resemble any sounds in English.

A system for teaching the pronunciation of Japanese long vowels, the mora nasal, and mora obstruents was recently built at the University of Tokyo. This system enables students to practice phonemic differences in Japanese that are known to present special challenges to L2 learners. It prompts students to pronounce minimal pairs (e.g., long and short vowels) and returns immediate feedback on segment duration. Based on the limited data, the system seems quite effective at this particular task. Learners quickly mastered the relevant duration cues, and the time spent on learning these pronunciation skills was well within the constraints of Japanese L2 curricula (Kawai & Hirose, 1997). However, the study provides no data on long-term effects of using the system.

Supra-segmental Feedback. Correct usage of supra-segmental features such as intonation and stress has been shown to improve the syntactic and semantic intelligibility of spoken language (Crystal, 1981). In spoken conversation, intonation and stress information not only helps listeners to locate phrase boundaries and word emphasis, but also to identify the pragmatic thrust of the utterance (e.g., interrogative vs. declarative). One of the main acoustical correlates of stress and intonation is fundamental frequency (F0); other acoustical characteristics include loudness, duration, and tempo. Most commercial signal processing software have tools for tracking and visually displaying F0 contours (see Figure 2). Such displays can and have been used to provide valuable pronunciation feedback to students. Experiments have shown that a visual F0 display of supra-segmental features combined with audio feedback is more effective than audio feedback alone (de Bot, 1983; James, 1976), especially if the student’s F0 contour is displayed along with a native model. The feasibility of this type of visual feedback has been demonstrated by a number of simple prototypes (Abberton & Fourcin, 1975; Anderson-Hsieh, 1994; Hiller et al., 1994; Spaai & Hermes, 1993; Stibbard, 1996). We believe that this technology has a good potential for being incorporated into commercial CALL systems.

Other types of visual pronunciation feedback include the graphical display of a native speaker’s face, the vocal tract, spectrum information, and speech waveforms (see Figure 2). Experiments have shown that a visual display of the talker improves not only word identification accuracy (Bernstein & Christian, 1996), but also speech rhythm and timing (Markham & Nagano-Madesen, 1997). A large number of commercial pronunciation tutors on the market today offer this kind of feedback. Yet others have experimented with using a real-time spectrogram or waveform display of speech to provide pronunciation feedback. Molholt (1990) and Manuel (1990) report anecdotal success in using such displays along with guidance on how to interpret the displays to improve the pronunciation of suprasegmental features in L2 learners of English. However, the authors do not provide experimental evidence for the effectiveness of this type of visual feedback. Our own experience with real-time spectrum and waveform displays suggests their potential use as pronunciation feedback provided they are presented along with other types of feedback, as well as with instructions on how to interpret the displays.

-52-

Figure 2. Alternative speech display modes of the phrase He was shot in the back generated with Entropic’s signal processing software. ⁽¹⁾

Reading Aloud

Reading aloud exercises literacy skills in both second language and literacy education. Intensive practice in reading aloud helps students establish the conventional association between sounds and their written form, a skill that requires years of practice in young children and students of languages with non-phonetic writing, such as Japanese or Chinese. Teaching children and students how to read their own native or a foreign language is thus an area where speech recognition technology can make a significant difference. Imagine a reading tutor that not only listens to children and students reading aloud a story presented on the screen, but intervenes to provide help when needed and corrects mistakes.

Designing a basic recognition network for a voice-interactive reading tutor is relatively straightforward. There is only one correct spoken response to any given written prompt, and the system “knows” in advance what the student will be trying to say. However, the technical challenge is to recognize and respond adequately to the disfluencies of inexperienced readers. Such disfluencies include hesitations, mispronunciations, false starts, and self-corrections.

In the early 1990s, Cowan and Jones (1991), McCandless (1992), and Phillips, Zue, and McCandless (1993) among others demonstrated the technical feasibility of a voice-interactive reading tutor, without, however, providing empirical user data. One of the first fielded prototype systems for teaching reading to young children was developed by the Center for Teaching and Learning (CTL) in 1991 (Kantrov, 1991). The simple but robust multimedia application used an isolated, speaker-dependent recognizer and limited reading vocabulary (18+ words). The system was designed to expand children’s reading vocabulary by embedding new words within the context of a goal-oriented game: children are called upon to help a bear overcome obstacles on his way home; reading the word correctly removes the obstacle. Results of three field trials in two Boston-area public schools indicated that the problems with the application were related to the human interface and input mode (microphones), rather than the speech recognition component per se. Ironically, recognition errors, especially misrecognition of correctly read words, contributed positively to the pedagogical effect of the application: the children got additional reading practice, because they had to repeat the words several times until the machine responded appropriately.

One of the most ambitious automated reading coaches currently being developed is the ongoing Project LISTEN at Carnegie Mellon University (CMU). Designed to combat illiteracy, the fully automated prototype uses continuous speech recognition to listen to children read continuous text and automatically trigger pedagogically appropriate interventions (Mostow, Roth, Hauptmann, & Kane, 1994). The system features a personalized agent, “Emily,” who provides feedback and assistance when necessary. The system incorporates expert knowledge on individual reading assistance that is both pedagogically relevant and technically feasible. Emily intervenes when the child misreads one or more words in the current sentence, gets stuck, or clicks on a word to get help. On the other hand, to reduce frustration in children with reading difficulties, the system deliberately refrains from treating false starts, self-corrections, or hesitations as “mistakes.” Instead, errors of this type are modeled and included into the recognition grammar as acceptable.

-53-

An improved version of CMU’s reading coach running real-time on an affordable PC platform was fielded in 1996 among 8 of the poorest third grade readers at Fort Pitt, PA to measure improvements in reading performance over an 8 month period of using the system (Mostow, 1997; Mostow & Aist, 1997). While the earlier study measured reading performance only in terms of student word error rates, the improved system implements algorithms for measuring reading fluency in young children. Relevant performance variables include reading rate, inter-word latency (silence), disfluency (false starts, self-corrections, omissions) and time spent with the assistant. Comparing subjects’ reading fluency levels at the beginning of using the system with those at the end, the experiments suggest an overall improvement in reading accuracy of 16% and a 35% decrease in inter-word latency. After using the system for eight months, students’ reading levels improved by an average of two years. These results are encouraging in that they show how careful system design and evaluation based on user data can lead to useful and practical applications.

Teaching Linguistic Structures and Limited Conversation

Apart from supporting systems for teaching basic pronunciation and literacy skills, ASR technology is being deployed in automated language tutors that offer practice in a variety of higher-level linguistic skills ranging from highly constrained grammar and vocabulary drills to limited conversational skills in simulated real-life situations. Prior to implementing any such system, a choice needs to be made between two fundamentally different system design types: closed response vs. open response design. In both designs, students are prompted for speech input by a combination of written, spoken, or graphical stimuli. However, the designs differ significantly with reference to the type of verbal computer-student interaction they support. In closed response systems, students must choose one response from a limited number of possible responses presented on the screen. Students know exactly what they are allowed to say in response to any given prompt. By contrast, in systems with open response design, the network remains hidden and the student is challenged to generate the appropriate response without any cues from the system.

Closed Response Designs. One of the first implementations of a closed response design was the Voice Interactive Language Instruction System (VILIS) developed at SRI (Bernstein & Rtischev, 1991). This system elicits spoken student responses by presenting queries about graphical displays of maps and charts. Students infer the right answers to a set of multiple-choice questions and produce spoken responses.

A more recent prototype currently under development in SRI is the Voice Interactive Language Training System (VILTS), a system designed to foster speaking and listening skills for beginning through advanced L2 learners of French (Egan, 1996; Neumeyer et al., 1996; Rypa, 1996). The system incorporates authentic, unscripted conversational materials collected from French speakers into an engaging, flexible, and user-centered lesson architecture. The system deploys speech recognition to guide students through the lessons and automatic pronunciation scoring to provide feedback on the fluency of student responses. As far as we know, only the pronunciation scoring aspect of the system has been validated in experimental trials (Neumeyer et al., 1996).

In pedagogically more sophisticated systems, the query-response mode is highly contextualized and presented as part of a simulated conversation with a virtual interlocutor. To stimulate student interest, closed response queries are often presented in the form of games or goal-driven tasks. One commercial system that exploits the full potential of this design is TraciTalk (Courseware Publishing International, Inc., Cupertino, CA), a voice-driven multimedia CALL system aimed at more advanced ESL learners. In a series of loosely connected scenarios, the system engages students in solving a mystery. Prior to each scenario, students are given a task (e.g., eliciting a certain type of information), and they accomplish this task by verbally interacting with characters on the screen. Each voice interaction offers several possible responses, and each spoken response moves the conversation in a slightly different direction. There are many paths through each scenario, and not every path yields the desired information. This motivates students to return to the beginning of the scene and try out a different interrogation strategy. Moreover, TraciTalk features an agent that students can ask for assistance and accepts spoken commands for navigating the system. Apart from being more fun and interesting, games and task-oriented programs implicitly provide positive feedback by giving students the feeling of having solved a problem solely by communicating in the target language.

The speech recognition technology underlying closed response query implementations is very simple, even in the more sophisticated systems. For any given interaction, the task perplexity is low and the vocabulary size is comparatively small. As a result, these systems tend to be very robust. Recognition accuracy rates in the low to upper 90% range can be expected depending on task definition, vocabulary size, and the degree of non-native disfluency.

-54-

Stimulus-response queries

A recent implementation of an open response design for teaching beginning Spanish is The Auto Interactive Tutor (TAIT) by Mitsubishi Research Laboratories (Waters, 1995). The system presents study material in the form of stimulus-response pairs and is organized around a set of primitive items to be learned such as “What is the Spanish word for ‘left’?” It uses speech recognition to process student input and to move forward. What distinguishes this system from the others discussed in this review is the fact that it is user-adaptive. It constructs an evolving model of the user’s knowledge by keeping track of the average error rate, and it presents subsequent material accordingly. Even though the system was never fielded with users, and despite the fact that it uses a rather primitive, small-vocabulary, discrete-input, speaker-dependent speech recognizer, informal evidence suggests that TAIT makes clever use of design in order to get the most out of a simple implementation of speech technology.

Simulated real-life conversation

In the past few years, a number of speech laboratories have tried to build systems that can understand and judge continuous spoken language and maintain a conversation through several turns. The goal is to emulate essential features of human-human communication for the purpose of teaching and practicing conversational skills in the target language. Interactions should work without requiring collateral cues from a mouse or keyboard, operate at an appropriate conversational pace, and incorporate verbal strategies for resolving misunderstandings.

A prototype system for simulating human-human interactions was recently developed at Entropic (Ehsani, Bernstein, Najmi, & Todic, 1997; Ehsani, Bernstein, & Najmi, in press). The system, called Subarashii (Japanese for “wonderful”), offers beginning students of Japanese the opportunity to solve simple problems through (virtual) spoken interactions with monolingual Japanese natives. Subarashii is designed to understand what a student is saying in Japanese (within a constrained context) and to respond in a meaningful way in spoken Japanese.

In a series of loosely connected everyday situations, the system poses problems in written English (e.g., inviting a friend to go to a movie) and offers occasional support to the student in the form of written reminders, but problems can only be solved by speaking and understanding Japanese. Despite the restricted communicative competence of beginning L2 learners, there is a variety of potentially valid utterances that the student can produce in any situation, even if some of these may be grammatically or pragmatically incorrect. Subarashii will not only properly process correct responses, but it will also recognize and reject (with an appropriate message) many incorrect inputs. However, in order to give appropriate feedback on student errors, the system must be able to anticipate such errors along with the expected responses. In other words, they must be included in the recognition network. To create such a network, each encounter was prototyped in a traditional Hypercard environment on a Macintosh with text input. Hypercard provides an effective means of modeling each encounter on the basis of actual input from a test group of students.

The acoustic models for this system were originally built with a low rejection in order to be more forgiving of a student’s accent. However, this approach resulted in a relatively large proportion of misrecognized utterances, and false acceptance. Two trials conducted among 32 students from Silver Creek High School, San Jose, and 13 Stanford students yielded alarmingly low recognition accuracy rates (41.6% and 36.6% respectively) due to insufficient training data. These results imply that only one-third of the students’ responses were correctly recognized. However, the functional accuracy, that is, the percentage of times the system responded appropriately, was significantly higher (66.9% and 71.4% for Silver Creek and Stanford respectively). For example, if the student said, “Hi, how are you doing?” and the question was misrecognized as, “Hello, how are you?,” this is technically a recognition error. Functionally, however, the system will respond appropriately which suggests that near perfect recognition accuracy may not be a necessary requirement for an effective speech dialog system.

-55-

FUTURE TRENDS IN VOICE-INTERACTIVE CALL

In the previous sections, we reviewed the current state of speech technology, discussed some of the factors affecting recognition performance, and introduced a number of research prototypes that illustrate the range of speech-enabled CALL applications that are currently technically and pedagogically feasible. With the exception of a few exploratory open response dialog systems, most of these systems are designed to teach and evaluate linguistic form (pronunciation, fluency, vocabulary study, or grammatical structure). This is no coincidence. Formal features can be clearly identified and integrated into a focused task design. This means that robust performance can be expected. Furthermore, mastering linguistic form remains an important component of L2 instruction, despite the emphasis on communication (Holland, 1995). Prolonged, focused practice of a large number of items is still considered an effective means of expanding and reinforcing linguistic competence (Waters, 1994). However, such practice is time consuming. CALL can automate these aspects of language training, thereby freeing up valuable class time that would otherwise be spent on drills.

While such systems are an important step in the right direction, other more complex and ambitious applications are conceivable and no doubt desirable. Imagine a student being able to access the Internet, find the language of his or her choice, and tap into a comprehensive voice-interactive multimedia language program that would provide the equivalent of an entire first year of college instruction. The computer would evaluate the student’s proficiency level and design a course of study tailored to his or her needs. Or think of using the same Internet resources and a set of high-level authoring tools to put together a series of virtual encounters surrounding the task of finding an apartment in Berlin. As a minimum, one would hope that natural speech input capacity becomes a routine feature of any CALL application.

To many educators, these may still seem like distant goals, and yet we believe that they are not beyond reach. In what follows, we identify four of the most persistent issues in building speech-enabled language learning applications and suggest how they might be resolved to enable a more widespread commercial implementation of speech technology in CALL.

1. More research is necessary on modeling and predicting multi-turn dialogs.

An intelligent open response language tutor must not only correctly recognize a given speech input, but in addition understand what has been said and evaluate the meaning of the utterance for pragmatic appropriateness. Automatic speech understanding requires Natural Language Processing (NLP) capabilities, a technology for extracting grammatical, semantic, and pragmatic information from written or spoken discourse. NLP has been successfully deployed in expert systems and information retrieval. One of the first voice-interactive dialog systems using NLP was the DARPA-sponsored Air Travel Information System (Pallett, 1995), which enables the user to obtain flight information and make ticket reservations over the telephone. Similar commercial systems have been implemented for automatic retrieval of weather and restaurant information, virtual environments, and telephone auto-attendants. Many of the lessons learned in developing such systems can be valuable for designing CALL applications for practicing conversational skills.

2. More and better training data are needed to support basic research on modeling non-native conversational speech.

One of the most needed resources for developing open response conversational CALL applications is large corpora of non-native transcribed speech data, of both read and conversational speech. Since accents vary depending on the student’s first language, separate databases must either be collected for each L1 subgroup, or a representative sample of speakers of different languages must be included in the database. Creating such databases is extremely labor and cost intensive–a phone level transcription of spontaneous conversational data can cost up to one dollar per phone. A number of multilingual conversational databases of telephone speech are publicly available through the Linguistic Data Consortium (LDC), including Switchboard (US English) and CALLHOME (English, Japanese, Spanish, Chinese, Arabic, German). Our own effort in collaboration with John Hopkins University (Byrne, Knodt, Khudanpur, & Bernstein, 1998; Knodt, Bernstein, & Todic,1998) has been to collect and model spontaneous English conversations between Hispanic natives. All of these efforts will improve our understanding of the disfluent speech of language learners and help model this speech type for the purpose of human-machine communication.

-56-

3. Authoring tools and API’s must become more widely available and easier to use.

Speech recognition functionality is more likely to become a standard feature of CALL if it can be easily incorporated into language learning applications. The best way of accomplishing this is by using a standardized speech API. A speech API consists of a set of program modules that allow application developers to access the functionality of a speech decoder without the need for a full understanding of the underlying technology. Most vendors have their own private or publicized speech API’s, and currently there are several competing “standards.” Entropic has recently built a speech API specifically tailored to the needs of language educators. This API integrates a state-of-the-art speech recognizer and a set of high level programming routines with existing authoring environments for incorporating speech recognition into PC-based language training applications. Entropic’s speech API provides a versatile development environment for educational software compatible with other multimedia authoring tools such as Java or Authorware. The API is easy to use and supports powerful functionality for CALL applications, including access to F0 information, timing, confidence scores, and automatic mapping to known non-native pronunciations.

4. As voice-interactive CALL systems become more widely accepted, the quality of commercial systems is likely to improve.

The lion’s share of funding for CALL comes from government sources. Since these funds are modest and production costs high, most of the funded systems remain at an experimental stage; few have been tested with end users and fewer still have entered the commercial market. A common argument assumes that CALL will have a greater practical impact on learning, when more well-designed CALL applications are readily available. No doubt, we do need better and more thoroughly tested systems. Such systems must become more affordable, easy to install, and platform-independent. However, the increasing commercial success of telephone-based voice applications suggests that the technology will rapidly improve once it enters the commercial market on a larger scale. As a result, large amounts of user data will become available to augment and improve the technology even further.

ABOUT THE AUTHORS

Farzad Ehsani is the Chief Technology Officer at Sehda, Inc., a startup focusing on large vocabulary applications for dialogue and captioning. Previously, he was the Head of Language Systems at Entropic Research Laboratory. He holds a Bachelors in Computer Science and a Masters and an Engineers degree in Electrical Engineering all from MIT. Before joining Entropic, Mr. Ehsani worked at NEC, Motorola, MIT, and DEC as a researcher focusing on language modeling and speech recognition.

E-mail: farzad@sehda.com

Eva Knodt is the VP of Marketing at Sehda, Inc., a startup focusing on large vocabulary applications for dialogue and captioning. She holds a PhD in German literature from the University of Virginia, and has taught at Indiana and Stanford Universities. Previously, she was a researcher in Entropic Research Laboratory working in the areas of experimental design and protocol implementation, as well as building language and acoustic models for English and Japanese.

E-mail: knodt@sehda.com

Origins

Computational linguistics as a field predates artificial intelligence, a field under which it is often grouped. Computational linguistics originated with efforts in the United States in the 1950s to use computers to automatically translate texts from foreign languages, particularly Russian scientific journals, into English. ^[1] Since computers can make arithmetic calculations much faster and more accurately than humans, it was thought to be only a short matter of time before the technical details could be taken care of that would allow them the same remarkable capacity to process language.^{[citation needed]}

When machine translation (also known as mechanical translation) failed to yield accurate translations right away, automated processing of human languages was recognized as far more complex than had originally been assumed. Computational linguistics was born as the name of the new field of study devoted to developing algorithms and software for intelligently processing language data. When artificial intelligence came into existence in the 1960s, the field of computational linguistics became that sub-division of artificial intelligence dealing with human-level comprehension and production of natural languages.^{[citation needed]}

In order to translate one language into another, it was observed that one had to understand the grammar of both languages, including both morphology (the grammar of word forms) and syntax (the grammar of sentence structure). In order to understand syntax, one had to also understand the semantics and the lexicon (or ‘vocabulary’), and even to understand something of the pragmatics of language use. Thus, what started as an effort to translate between languages evolved into an entire discipline devoted to understanding how to represent and process natural languages using computers.^{[citation needed]}

Subfields

Computational linguistics can be divided into major areas depending upon the medium of the language being processed, whether spoken or textual; and upon the task being performed, whether analyzing language (recognition) or synthesizing language (generation).

Speech recognition and speech synthesis deal with how spoken language can be understood or created using computers. Parsing and generation are sub-divisions of computational linguistics dealing respectively with taking language apart and putting it together. Machine translation remains the sub-division of computational linguistics dealing with having computers translate between languages.

Some of the areas of research that are studied by computational linguistics include:

• Computational complexity of natural language, largely modeled on automata theory, with the application of context-sensitive grammar and linearly-bounded Turing machines.
• Computational semantics comprises defining suitable logics for linguistic meaning representation, automatically constructing them and reasoning with them
• Computer-aided corpus linguistics
• Design of parsers or chunkers for natural languages
• Design of taggers like POS-taggers (part-of-speech taggers)
• Machine translation as one of the earliest and least successful applications of computational linguistics draws on many subfields.

The Association for Computational Linguistics defines computational linguistics as:

…the scientific study of language from a computational perspective. Computational linguists are interested in providing computational models of various kinds of linguistic phenomena^[2].

It also studies how language varieties differ between groups separated by certain social variables, e.g., ethnicity, religion, status, gender, level of education, age, etc., and how creation and adherence to these rules is used to categorize individuals in social or socioeconomic classes. As the usage of a language varies from place to place (dialect), language usage varies among social classes, and it is these sociolects that sociolinguistics studies.

The social aspects of language were in the modern sense first studied by Indian and Japanese linguists in the 1930s, and also by Gauchat in Switzerland in the early 1900s, but none received much attention in the West until much later. The study of the social motivation of language change, on the other hand, has its foundation in the wave model of the late 19th century. The first attested use of the term sociolinguistics was by Thomas Callan Hodson in the title of a 1939 paper.^[1] Sociolinguistics in the West first appeared in the 1960s and was pioneered by linguists such as William Labov in the US and Basil Bernstein in the UK.

Applications of sociolinguistics

For example, a sociolinguist might determine through study of social attitudes that a particular vernacular would not be considered appropriate language use in a business or professional setting. Sociolinguists might also study the grammar, phonetics, vocabulary, and other aspects of this sociolect much as dialectologists would study the same for a regional dialect.

The study of language variation is concerned with social constraints determining language in its contextual environment. Code-switching is the term given to the use of different varieties of language in different social situations.

William Labov is often regarded as the founder of the study of sociolinguistics. He is especially noted for introducing the quantitative study of language variation and change,^[2] making the sociology of language into a scientific discipline.

Sociolinguistic variables

Studies in the field of sociolinguistics typically take a sample population and interview them, assessing the realisation of certain sociolinguistic variables. Labov specifies the ideal sociolinguistic variable to

• be high in frequency,
• have a certain immunity from conscious suppression,
• be an integral part of larger structures, and
• be easily quantified on a linear scale.

Phonetic variables tend to meet these criteria and are often used, as are grammatical variables and, more rarely, lexical variables. Examples for phonetic variables are: the frequency of the glottal stop, the height or backness of a vowel or the realisation of word-endings. An example of a grammatical variable is the frequency of negative concord (known colloquially as a double negative).

Traditional sociolinguistic interview

Sociolinguistic interviews are an integral part of collecting data for sociolinguistic studies. There is an interviewer, who is conducting the study, and a subject, or informant, who is the interviewee. In order to get a grasp on a specific linguistic form and how it is used in the dialect of the subject, a variety of methods are used to elicit certain registers of speech. There are five different styles, ranging from formal to casual. The most formal style would be elicited by having the subject read a list of minimal pairs (MP). Minimal pairs are pairs of words that differ in only one phoneme, such as cat and bat. Having the subject read a word list (WL) will elicit a formal register, but generally not as formal as MP. The reading passage (RP) style is next down on the formal register, and the interview style (IS) is when an interviewer can finally get into eliciting a more casual speech from the subject. During the IS the interviewer can converse with the subject and try to draw out of him an even more casual sort of speech by asking him to recall childhood memories or maybe a near death experience, in which case the subject will get deeply involved with the story since strong emotions are often attached to these memories. Of course, the most sought after type of speech is the casual style (CS). This type of speech is difficult if not impossible to elicit because of the Observer’s Paradox. The closest one might come to CS in an interview is when the subject is interrupted by a close friend or family member, or perhaps must answer the phone. CS is used in a completely unmonitored environment where the subject feels most comfortable and will use their natural vernacular without overtly thinking about it.

Fundamental Concepts in Sociolinguistics

While the study of sociolinguistics is very broad, there are a few fundamental concepts on which many sociolinguistic inquiries depend.

1.  Speech Community

Speech community is a concept in sociolinguistics that describes a more or less discrete group of people who use language in a unique and mutually accepted way among themselves.

Speech communities can be members of a profession with a specialized jargon, distinct social groups like high school students or hip hop fans, or even tight-knit groups like families and friends. Members of speech communities will often develop slang or jargon to serve the group’s special purposes and priorities.

2.  High prestige and low prestige varieties

Crucial to sociolinguistic analysis is the concept of prestige; certain speech habits are assigned a positive or a negative value which is then applied to the speaker. This can operate on many levels. It can be realised on the level of the individual sound/phoneme, as Labov discovered in investigating pronunciation of the post-vocalic /r/ in the North-Eastern USA, or on the macro scale of language choice, as realised in the various diglossias that exist throughout the world, where Swiss-German/High German is perhaps most well known. An important implication of sociolinguistic theory is that speakers ‘choose’ a variety when making a speech act, whether consciously or subconsciously.

3.  Social network

Understanding language in society means that one also has to understand the social networks in which language is embedded. A social network is another way of describing a particular speech community in terms of relations between individual members in a community. A network could be loose or tight depending on how members interact with each other.^[3] For instance, an office or factory may be considered a tight community because all members interact with each other. A large course with 100+ students be a looser community because students may only interact with the instructor and maybe 1-2 other students. A multiplex community is one in which members have multiple relationships with each other.^[3] For instance, in some neighborhoods, members may live on the same street, work for the same employer and even intermarry.

The looseness or tightness of a social network may affect speech patterns adopted by a speaker. For instance, Dubois and Hovarth (1998:254) found that speakers in one Cajun Louisiana community were more likely to pronounce English “th” [θ] as [t] (or [ð] as [d]) if they participated in a relatively dense social network (i.e. had strong local ties and interacted with many other speakers in the community), and less likely if their networks were looser (i.e. fewer local ties).^[4]

A social network may apply to the macro level of a country or a city, but also to the inter-personal level of neighborhoods or a single family. Recently, social networks have been formed by the Internet, through chat rooms, MySpace groups, organizations, and online dating services.

4.  Internal vs. external language

In Chomskian linguistics, a distinction is drawn between I-language (internal language) and E-language (external language). In this context, internal language applies to the study of syntax and semantics in language on the abstract level; as mentally represented knowledge in a native speaker. External language applies to language in social contexts, i.e. behavioral habits shared by a community. Internal language analyses operate on the assumption that all native speakers of a language are quite homogeneous in how they process and perceive language.^{[citation needed]} External language fields, such as sociolinguistics, attempt to explain why this is in fact not the case. Many sociolinguists reject the distinction between I- and E-language on the grounds that it is based on a mentalist view of language. On this view, grammar is first and foremost an interactional (social) phenomenon (e.g. Elinor Ochs, Emanuel Schegloff, Sandra Thompson).

Differences according to class

Sociolinguistics as a field distinct from dialectology was pioneered through the study of language variation in urban areas. Whereas dialectology studies the geographic distribution of language variation, sociolinguistics focuses on other sources of variation, among them class. Class and occupation are among the most important linguistic markers found in society. One of the fundamental findings of sociolinguistics, which has been hard to disprove, is that class and language variety are related. Members of the working class tend to speak less standard language, while the lower, middle, and upper middle class will in turn speak closer to the standard. However, the upper class, even members of the upper middle class, may often speak ‘less’ standard than the middle class. This is because not only class, but class aspirations, are important.

1.  Class aspiration

Studies, such as those by William Labov in the 1960s, have shown that social aspirations influence speech patterns. This is also true of class aspirations. In the process of wishing to be associated with a certain class (usually the upper class and upper middle class) people who are moving in that direction socio-economically will adjust their speech patterns to sound like them. However, not being native upper class speakers, they often hypercorrect, which involves overcorrecting their speech to the point of introducing new errors. The same is true for individuals moving down in socio-economic status.

2.  Social language codes

Basil Bernstein, a well-known British socio-linguist, devised in his book, ‘Elaborated and restricted codes: their social origins and some consequences,’ a social code system which he used to classify the various speech patterns for different social classes. He claimed that members of the middle class have ways of organizing their speech which are fundamentally very different from the ways adopted by the working class.

3.  Restricted code

In Basil Bernstein’s theory, the restricted code was an example of the speech patterns used by the working-class. He stated that this type of code allows strong bonds between group members, who tend to behave largely on the basis of distinctions such as ‘male’, ‘female’, ‘older’, and ‘younger’. This social group also uses language in a way which brings unity between people, and members often do not need to be explicit about meaning, as their shared knowledge and common understanding often bring them together in a way which other social language groups do not experience. The difference with the restricted code is the emphasis on ‘we’ as a social group, which fosters greater solidarity than an emphasis on ‘I’.

4.  Elaborated code

Basil Bernstein also studied what he named the ‘elaborated code’ explaining that in this type of speech pattern the middle and upper classes use this language style to gain access to education and career advancement. Bonds within this social group are not as well defined and people achieve their social identity largely on the basis of individual disposition and temperament. There is no obvious division of tasks according to sex or age and generally, within this social formation members negotiate and achieve their roles, rather than have them there ready-made in advance. Due to the lack of solidarity the elaborated social language code requires individual intentions and viewpoints to be made explicit as the ‘I’ has a greater emphasis with this social group than the working class.

5. Deviation from standard language varieties

A diagram showing variation in the English language by region (the bottom axis) and by social class (the side axis). The higher the social class, the less variation.

The existence of differences in language between social classes can be illustrated by the following table:

Any native speaker of English would immediately be able to guess that speaker 1 was likely of a different social class than speaker 2, namely from a lower social class, probably from a working class pedigree. The differences in grammar between the two examples of speech is referred to as differences between social class dialects or sociolects.

It is also notable that, at least in England and Australia, the closer to standard English a dialect gets, the less the lexicon varies by region, and vice-versa.

6.  Covert prestige

It is generally assumed that non-standard language is low-prestige language. However, in certain groups, such as traditional working class neighborhoods, standard language may be considered undesirable in many contexts. This is because the working class dialect is a powerful in-group marker, and especially for non-mobile individuals, the use of non-standard varieties (even exaggeratedly so) expresses neighborhood pride and group and class solidarity. There will thus be a considerable difference in use of non-standard varieties when going to the pub or having a neighborhood barbecue (high), and going to the bank (lower) for the same individual.

Differences according to age groups

There are several different types of age-based variation one may see within a population. They are: vernacular of a subgroup with membership typically characterized by a specific age range, age-graded variation, and indications of linguistic change in progress.

One example of subgroup vernacular is the speech of street youth. Just as street youth dress differently from the “norm”, they also often have their own “language”. The reasons for this are the following: (1) To enhance their own cultural identity (2) To identify with each other, (3) To exclude others, and (4) To invoke feelings of fear or admiration from the outside world. Strictly speaking, this is not truly age-based, since it does not apply to all individuals of that age bracket within the community.

Age-graded variation is a stable variation which varies within a population based on age. That is, speakers of a particular age will use a specific linguistic form in successive generations. This is relatively rare. Chambers (1995) cites an example from southern Ontario, Canada where the pronunciation of the letter ‘Z’ varies. Most of the English-speaking world pronounces it ‘zed’; however, in the United States, it is pronounced ‘zee’. A linguistic survey found that in 1979 two-thirds of the 12 year olds in Toronto ended the recitation of the alphabet with the letter ‘zee’ where only 8% of the adults did so. Then in 1991, (when those 12 year olds were in their mid-20s) a survey showed only 39% of the 20-25 year olds used ‘zee’. In fact, the survey showed that only 12% of those over 30 used the form ‘zee’. This seems to be tied to an American children’s song frequently used to teach the alphabet. In this song, the rhyme scheme matches the letter Z with V ‘vee’, prompting the use of the American pronunciation. As the individual grows older, this marked form ‘zee’ is dropped in favor of the standard form ‘zed’.^[5]

People tend to use linguistic forms that were prevalent when they reached adulthood. So, in the case of linguistic change in progress, one would expect to see variation over a broader range of ages. Bright (1997) provides an example taken from American English where there is an on-going merger of the vowel sounds in such pairs of words as ‘caught’ and ‘cot’.^[6] Examining the speech across several generations of a single family, one would find the grandparents’ generation would never or rarely merge these two vowel sounds; their children’s generation may on occasion, particularly in quick or informal speech; while their grandchildren’s generation would merge these two vowels uniformly. This is the basis of the apparent-time hypothesis where age-based variation is taken as an indication of linguistic change in progress.

Differences according to geography

Main article: Dialectology

Differences according to gender

Men and women, on average, tend to use slightly different language styles. These differences tend to be quantitative rather than qualitative. That is, to say that women make more minimal responses (see below) than men is akin to saying that men are taller than women (i.e., men are on average taller than women, but some women are taller than some men). The initial identification of a women’s register was by Robin Lakoff in 1975, who argued that the style of language served to maintain women’s (inferior) role in society (“female deficit approach”).^[7] A later refinement of this argument was that gender differences in language reflected a power difference (O’Barr & Atkins, 1980) (“dominance theory”). However, both these perspectives have the language style of men as normative, implying that women’s style is inferior.

More recently, Deborah Tannen has compared gender differences in language as more similar to ‘cultural’ differences (“cultural difference approach”). Comparing conversational goals, she argued that men have a report style, aiming to communicate factual information, whereas women have a rapport style, more concerned with building and maintaining relationships.^[8] Such differences are pervasive across media, including face-to-face conversation (e.g., Fitzpatrick, Mulac, & Dindia, 1995: Hannah & Murachver, 1999), written essays of primary school children (Mulac, Studley, & Blau, 1990), email (Thomson & Murachver, 2001), and even toilet graffiti (Green, 2003).^{[9][10][11][12]}

Communication styles are always a product of context, and as such, gender differences tend to be most pronounced in single-gender groups. One explanation for this, is that people accommodate their language towards the style of the person they are interacting with. Thus, in a mixed-gender group, gender differences tend to be less pronounced. A similarly important observation is that this accommodation is usually towards the language style, not the gender of the person (Thomson, Murachver, & Green, 2001). That is, a polite and empathic male will tend to be accommodated to on the basis of their being polite and empathic, rather than their being male.^[13]

Minimal responses

One of the ways in which the communicative competence of men and women differ is in their use of minimal responses, i.e., paralinguistic features such as ‘mhm’ and ‘yeah’, which is behaviour associated with collaborative language use (Carli, 1990).^[14] Men, on the other hand, generally use them less frequently and where they do, it is usually to show agreement, as Zimmerman and West’s (1975) study of turn-taking in conversation indicates.^[15]

Questions

Men and women differ in their use of questions in conversations. For men, a question is usually a genuine request for information whereas with women it can often be a rhetorical means of engaging the other’s conversational contribution or of acquiring attention from others conversationally involved, techniques associated with a collaborative approach to language use (Barnes, 1971).^[16] Therefore women use questions more frequently (Fitzpatrick, et al., 1995; Todd, 1983).^[9][17][18] In writing, however, both genders use rhetorical questions as literary devices. For example, Mark Twain used them in “A War Prayer” to provoke the reader to question his actions and beliefs.

Turn-taking

As the work of DeFrancisco (1991) shows, female linguistic behaviour characteristically encompasses a desire to take turns in conversation with others, which is opposed to men’s tendency towards centering on their own point or remaining silent when presented with such implicit offers of conversational turn-taking as are provided by hedges such as “y’ know” and “isn’t it”.^[19] This desire for turn-taking gives rise to complex forms of interaction in relation to the more regimented form of turn-taking commonly exhibited by men (Sacks et al., 1974).^[20]

Changing the topic of conversation

According to Dorval (1990), in his study of same-sex friend interaction, males tend to change subject more frequently than females. This difference may well be at the root of the conception that women chatter and talk too much, and may still trigger the same thinking in some males. In this way lowered estimation of women may arise.^[21] Incidentally, this androcentric attitude towards women as chatterers arguably arose from the idea that any female conversation was too much talking according to the patriarchal consideration of silence as a womanly virtue common to many cultures.

Self-disclosure

Female tendencies toward self-disclosure, i.e., sharing their problems and experiences with others, often to offer sympathy (Dindia & Allen, 1992; Tannen, 1991:49), contrasts with male tendencies to non-self disclosure and professing advice or offering a solution when confronted with another’s problems.^[8][22]

Verbal aggression

Men tend to be more verbally aggressive in conversing (Labov, 1972), frequently using threats, profanities, yelling and name-calling.^[23] Women, on the whole, deem this to disrupt the flow of conversation and not as a means of upholding one’s hierarchical status in the conversation. Where women swear, it is usually to demonstrate to others what is normal behaviour for them.^[24]

Listening and attentiveness

It appears that women attach more weight than men to the importance of listening in conversation, with its connotations of power to the listener as confidant of the speaker. This attachment of import by women to listening is inferred by women’s normally lower rate of interruption — i.e., disrupting the flow of conversation with a topic unrelated to the previous one (Fishman, 1980) — and by their largely increased use of minimal responses in relation to men (Zimmerman and West, 1975).^[15][25] Men, however, interrupt far more frequently with non-related topics, especially in the mixed sex setting (Zimmerman and West,1975) and, far from rendering a female speaker’s responses minimal, are apt to greet her conversational spotlights with silence, as the work of DeFrancisco (1991) demonstrates.^[19]

Dominance versus subjection

This, in turn, suggests a dichotomy between a male desire for conversational dominance – noted by Leet-Pellegrini (1980) with reference to male experts speaking more verbosely than their female counterparts – and a female aspiration to group conversational participation.^[26] One corollary of this is, according to Coates (1993: 202), that males are afforded more attention in the context of the classroom and that this can lead to their gaining more attention in scientific and technical subjects, which in turn can lead to their achieving better success in those areas, ultimately leading to their having more power in a technocratic society.^[27]

Politeness

Politeness in speech is described in terms of positive and negative face.^[28] Positive face refers to one’s desire to be liked and admired, while negative face refers to one’s wish to remain autonomous and not to suffer imposition. Both forms, according to Brown’s study of the Tzeltal language (1980), are used more frequently by women whether in mixed or single-sex pairs, suggesting for Brown a greater sensitivity in women than have men to face the needs of others.^[29] In short, women are to all intents and purposes largely more polite than men. However, negative face politeness can be potentially viewed as weak language because of its associated hedges and tag questions, a view propounded by O’Barr and Atkins (1980) in their work on courtroom interaction.^[30]

Structural ambiguity

The sentence “You have a green light” is ambiguous. Without knowing the context, the identity of the speaker, and their intent, it is not possible to infer the meaning with confidence. For example:

• It could mean you are holding a green light bulb.
• Or that you have a green light to drive your car.
• Or it could be indicating that you can go ahead with the project.
• Or that your body has a green glow

Similarly, the sentence “Sherlock saw the man with binoculars” could mean that Sherlock observed the man by using binoculars; or it could mean that Sherlock observed a man who was holding binoculars.^[3] The meaning of the sentence depends on an understanding of the context and the speaker’s intent. As defined in linguistics, a sentence is an abstract entity — a string of words divorced from non-linguistic context — as opposed to an utterance, which is a concrete example of a speech act in a specific context. The cat sat on the mat is a sentence of English; if you say to your sister on Tuesday afternoon: “The cat sat on the mat”, this is an example of an utterance. Thus, there is no such thing as a sentence with a single true meaning; it is underspecified (which cat sat on which mat?) and potentially ambiguous. The meaning of an utterance, on the other hand, is inferred based on linguistic knowledge and knowledge of the non-linguistic context of the utterance (which may or may not be sufficient to resolve ambiguity).

Origins

Pragmatics was a reaction to structuralist linguistics as outlined by Ferdinand de Saussure. In many cases, it expanded upon his idea that language has an analyzable structure, composed of parts that can be defined in relation to others. Pragmatics first engaged only in synchronic study, as opposed to examining the historical development of language. However, it rejected the notion that all meaning comes from signs existing purely in the abstract space of langue. Meanwhile, historical pragmatics has also come into being.

While Chomskyan linguistics famously repudiated Bloomfieldian anthropological linguistics^{[clarification needed]}, pragmatics continues its tradition. Also influential were Franz Boas, Edward Sapir and Benjamin Whorf.

Areas of interest

• The study of the speaker’s meaning, not focusing on the phonetic or grammatical form of an utterance, but instead on what the speaker’s intentions and beliefs are.

• The study of the meaning in context, and the influence that a given context can have on the message. It requires knowledge of the speaker’s identities, and the place and time of the utterance.

• The study of implicatures, i.e. the things that are communicated even though they are not explicitly expressed.

• The study of relative distance, both social and physical, between speakers in order to understand what determines the choice of what is said and what is not said.

• The study of what is not meant, as opposed to the intended meaning, i.e. that which is unsaid and unintended, or unintentional.

Referential uses of language

When we speak of the referential uses of language we are talking about how we use signs to refer to certain items. Below is an explanation of, first, what a sign is, second, how meanings are accomplished through its usage.

A sign is the link or relationship between a signified and the signifier as defined by Saussure and Huguenin. The signified is some entity or concept in the world. The signifier represents the signified. An example would be:
Signified: the concept cat
Signifier: the word ‘cat’”
The relationship between the two gives the sign meaning. This relationship can be further explained by considering what we mean by “meaning.” In pragmatics, there are two different types of meaning to consider: semantico-referential meaning and indexical meaning. Semantico-referential meaning refers to the aspect of meaning, which describes events in the world that are independent of the circumstance they are uttered in. An example would be propositions such as:

“Santa Claus eats cookies.”

In this case, the proposition is describing that Santa Claus eats cookies. The meaning of this proposition does not rely on whether or not Santa Claus is eating cookies at the time of its utterance. Santa Claus could be eating cookies at any time and the meaning of the proposition would remain the same. The meaning is simply describing something that is the case in the world. In contrast, the proposition, “Santa Claus is eating a cookie right now,” describes events that are happening at the time the proposition is uttered.

Semantico-referential meaning is also present in meta-semantical statements such as:

Tiger: omnivorous, a mammal

If someone were to say that a tiger is an omnivorous animal in one context and a mammal in another, the definition of tiger would still be the same. The meaning of the sign tiger is describing some animal in the world, which does not change in either circumstance.

Indexical meaning, on the other hand, is dependent on the context of the utterance and has rules of use. By rules of use, it is meant that indexicals can tell you when they are used, but not what they actually mean.

Example: “I”

Whom “I” refers to depends on the context and the person uttering it.

As mentioned, these meanings are brought about through the relationship between the signified and the signifier. One way to define the relationship is by placing signs in two categories: referential indexical signs, also called “shifters,” and pure indexical signs.

Referential indexical signs are signs where the meaning shifts depending on the context hence the nickname “shifters.” ‘I’ would be considered a referential indexical sign. The referential aspect of its meaning would be ’1st person singular’ while the indexical aspect would be the person who is speaking (refer above for definitions of semantico-referential and indexical meaning). Another example would be:

“This”
Referential: singular count
Indexical: Close by

A pure indexical sign does not contribute to the meaning of the propositions at all. It is an example of a “”non-referential use of language.”"

A second way to define the signified and signifier relationship is C.S. Peirce‘s Peircean Trichotomy. The components of the trichotomy are the following:

1. Icon: the signified resembles the signifier (signified: a dog’s barking noise, signifier: bow-wow)
2. Index: the signified and signifier are linked by proximity or the signifier has meaning only because it is pointing to the signified
3. Symbol: the signified and signifier are arbitrarily linked (signified: a cat, signifier: the word cat)

These relationships allow us to use signs to convey what we want to say. If two people were in a room and one of them wanted to refer to a characteristic of a chair in the room he would say “this chair has four legs” instead of “a chair has four legs.” The former relies on context (indexical and referential meaning) by referring to a chair specifically in the room at that moment while the latter is independent of the context (semantico-referential meaning), meaning the concept chair.

Non-referential uses of language

1.  Silverstein’s “Pure” Indexes

Michael Silverstein has argued that “nonreferential” or “pure” indexes do not contribute to an utterance’s referential meaning but instead “signal some particular value of one or more contextual variables.”^[4] Although nonreferential indexes are devoid of semantico-referential meaning, they do encode “pragmatic” meaning.

The sorts of contexts that such indexes can mark are varied. Examples include:

• Sex indexes are affixes or inflections that index the sex of the speaker, e.g. the verb forms of female Koasati speakers take the suffix “-s”.
• Deference indexes are words that signal social differences (usually related to status or age) between the speaker and the addressee. The most common example of a deference index is the V form in a language with a T-V distinction, the widespread phenomenon in which there are multiple second-person pronouns that correspond to the addressee’s relative status or familiarity to the speaker. Honorifics are another common form of deference index and demonstrate the speaker’s respect or esteem for the addressee via special forms of address and/or self-humbling first-person pronouns.
• An Affinal taboo index is an example of avoidance speech and produces and reinforces sociological distance, as seen in the Aboriginal Dyirbal language of Australia. In this language and some others, there is a social taboo against the use of the everyday lexicon in the presence of certain relatives (mother-in-law, child-in-law, paternal aunt’s child, and maternal uncle’s child). If any of those relatives are present, a Dyirbal speaker has to switch to a completely separate lexicon reserved for that purpose.

In all of these cases, the semantico-referential meaning of the utterances is unchanged from that of the other possible (but often impermissible) forms, but the pragmatic meaning is vastly different.

2.  The Performative

Main articles: Performative utterance, Speech act theory

J.L. Austin introduced the concept of the Performative, contrasted in his writing with “constative” (i.e. descriptive) utterances. According to Austin’s original formulation, a performative is a type of utterance characterized by two distinctive features:

• It is not truth-evaluable (i.e. it is neither true nor false)
• Its uttering performs an action rather than simply describing one

However, a performative utterance must also conform to a set of felicity conditions.

Examples:

• “I hereby pronounce you man and wife.”
• “I accept your apology.”
• “This meeting is now adjourned.”

3.  Jakobson’s Six Functions of Language

Roman Jakobson, expanding on the work of Karl Bühler, described six “constitutive factors” of a speech event, each of which represents the privileging of a corresponding function, and only one of which is the referential (which corresponds to the context of the speech event). The six constitutive factors and their corresponding functions are diagrammed below.

a.    The Six Constitutive Factors of a Speech Event

Context

Message

Addresser———————Addressee

Contact

Code

b.    The Six Functions of Language

Referential

Poetic

Emotive———————–Conative

Phatic

Metalingual

• The Referential Function corresponds to the factor of Context and describes a situation, object or mental state. The descriptive statements of the referential function can consist of both definite descriptions and deictic words, e.g. “The autumn leaves have all fallen now.”
• The Expressive (alternatively called “emotive” or “affective”) Function relates to the Addresser and is best exemplified by interjections and other sound changes that do not alter the denotative meaning of an utterance but do add information about the Addresser’s (speaker’s) internal state, e.g. “Wow, what a view!”
• The Conative Function engages the Addressee directly and is best illustrated by vocatives and imperatives, e.g. “Tom! Come inside and eat!”
• The Poetic Function focuses on “the message for its own sake”^[5] and is the operative function in poetry as well as slogans.
• The Phatic Function is language for the sake of interaction and is therefore associated with the Contact factor. The Phatic Function can be observed in greetings and casual discussions of the weather, particularly with strangers.
• The Metalingual (alternatively called “metalinguistic” or “reflexive”) Function is the use of language (what Jakobson calls “Code”) to discuss or describe itself.

Related fields

There is considerable overlap between pragmatics and sociolinguistics, since both share an interest in linguistic meaning as determined by usage in a speech community. However, sociolinguists tend to be more interested in variations in language within such communities.

Pragmatics helps anthropologists relate elements of language to broader social phenomena; it thus pervades the field of linguistic anthropology. Because pragmatics describes generally the forces in play for a given utterance, it includes the study of power, gender, race, identity, and their interactions with individual speech acts. For example, the study of code switching directly relates to pragmatics, since a switch in code effects a shift in pragmatic force.^[5]

According to Charles W. Morris, pragmatics tries to understand the relationship between signs and their users, while semantics tends to focus on the actual objects or ideas to which a word refers, and syntax (or “syntactics”) examines relationships among signs or symbols. Semantics is the literal meaning of an idea whereas pragmatics is the implied meaning of the given idea.

Speech Act Theory, pioneered by J.L. Austin and further developed by John Searle, centers around the idea of the performative, a type of utterance that performs the very action it describes. Speech Act Theory’s examination of Illocutionary Acts has many of the same goals as pragmatics, as outlined above.

Pragmatics in philosophy

Pragmatics (more specifically, Speech Act Theory’s notion of the performative) underpins Judith Butler’s theory of gender performativity. In Gender Trouble, she claims that gender and sex are not natural categories, but socially constructed roles produced by “reiterative acting.”

In Excitable Speech she extends her theory of performativity to hate speech and censorship, arguing that censorship necessarily strengthens any discourse it tries to suppress and therefore, since the state has sole power to define hate speech legally, it is the state that makes hate speech performative.

Jaques Derrida remarked that some work done under Pragmatics aligned well with the program he outlined in his book Of Grammatology.

Émile Benveniste argued that the pronouns “I” and “you” are fundamentally distinct from other pronouns because of their role in creating the subject.

Gilles Deleuze and Felix Guattari discuss linguistic pragmatics in the fourth chapter of A Thousand Plateaus (“November 20, 1923–Postulates of Linguistics”). They draw three conclusions from Austin: (1) A performative utterance doesn’t communicate information about an act second-hand—it is the act; (2) Every aspect of language (“semantics, syntactics, or even phonematics”) functionally interacts with pragmatics; (3) There is no distinction between language and speech. This last conclusion attempts to refute Saussure’s division between langue and parole and Chomsky’s distinction between surface structure and deep structure simultaneously. ^[6]