CHAPTER I

ASSESSING LISTENING

In earlier chapters, a number of foundational principles of language assessment were introduced. Before focusing on listening itself, think about the two interacting concepts of performance and observation. All language users perform the acts of listening, speaking, reading, and writing. They of course rely on their underlying competence in order to accomplish these performances.

The listening performance itself is the invisible, inaudible process of internalizing meaning from the auditory signals being transmitted to  the ear and brain. Or you may argue that the product of listening is a spoken or written response from the student that indicates correct (or incorrect) auditory processing. Again, the product of listening and reading is not the spoken or written response. The product is within the structure of  the brain and until teachers carry with them little portable MRI scanners to detect  meaningful intake, it is impossible to observe the product. You observe only the result of the meaningful input in the form of spoken or written output, just as you observe the result of the wind by noticing trees waving back and forth.

·         THE IMPORTANCE OF LISTENING

            Listening has often played second fiddle to its counterpart, speaking. In the standardized testing industry, a number of   separate oral production tests  are available (test of spoken English, oral Proficiency Inventory, and Phone Pass), but it is rare to find just a listening test. One reason for this emphasis is that listening is often implied as a component   of speaking. How could you speak a language without also listening? In addition, the overtly observable nature of speaking renders it more empirically measurable then listening. But perhaps a deeper cause  lies in universal biases toward speaking. A good speaker is often (unwisely) valued more highly than a good listener. To determine if someone is a proficient user of a language, people customarily ask, “do you   speak Spanish?” people rarely ask, “do you understand and speak Spanish?”.

            Every teacher of language knows that one’s oral production ability-other than monologues, speeches, reading aloud, and the like – is only as good as one’s listening comprehension ability. But of even further impact is the likelihood that input in the aural – oral mode accounts for a large proportion of successful  language acquisition. In a typical day, we do measurably more listening than speaking (with the exception of one or two of your friends who may be nonstop chatterboxes!). Whether in the workplace, educational, or home contexts, aural comprehension far outstrips oral production in quantifiable terms of time, number of words, effort, and attention.

            We therefore need to pay close  attention to listening as a  mode of performance for assessment in the classroom. In this chapter,  we will  begin with basic principles  and types of  listening, then move to a survey of tasks that can  be used  to assess listening.

·         BASIC  TYPES OF LISTENING

            As with all effectives’ tests, designing appropriate assessment tasks in listening begins with the specification of objectives, or criteria. Those objectives may be classified in terms of several types of listening performance. Think about what you do when you listen. Literally in nanoseconds, the following processes flash through your brain:

1.      You recognize speech sounds   and hold a temporary “imprint” of them in short-term memory.

2.      You simultaneously determine the type of speech  event (monologue, interpersonal dialogue, transactional  dialogue) that is being processed  and attend to its context (who       the speaker is,  location, purpose) and  the content of the message.

3.      You use (bottom-up) linguistic decoding skills  and/or (top-down) background  schemata to bring a plausible interpretation to the message,  and assign a literal  and  intended meaning to the utterance.

4.      In most cases (except  for repetition tasks, which involve shot-term  memory only), you delete the exact linguistic form in which the message was originally received in favor  of  conceptually  retaining  important or relevant  information in long-term memory.

Each  of these stages  represents a  potential assessment objective:

Ø  Comprehending of surface structure elements such as phonemes, words, intonation, or a grammatical category.

Ø  Understanding of pragmatic context.

Ø  Determining  meaning  of  auditory input

Ø  Developing the gist, a global or comprehensive understanding.

From these stages we can derive four commonly identified types of listening performance, each of which comprises a category within which to consider assessment tasks and procedures.

1.      Intensive.  Listening for perception of the components (phonemes, words, intonation, discourse markers, etc) of a larger stretch of language.

2.      Responsive. Listening to a relatively short stretch of language (a greeting, question, command, comprehension check, etc) in order to make  an equally short response.

3.      selective. Processing stretches of discourse such as short monologues for several minutes in order  to “scan” for certain information.

4.      Extensive. Listening to develop a top-down, global understanding of spoken language.

·         MICRO – MACROSKILLS OF LISTENING

            A useful way of synthesizing the above two lists is  to consider a finite number  of micro-and macro skills implied in the performance  of listening  comprehension. The micro-and macros kills provide 17 different objectives to asses in listening.

Micro-and macro skills of listening (adapted from Richards, 1983)

Micro skills

1.       Discriminate among the distinctive sounds of English.

2.       Retain chunks of language of different lengths in short-term memory.

3.      Recognize English stress patterns,  word in stressed and unstressed positions, rhythmic structure, intonation contours, and their role in signaling information.

4.      Recognize reduced forms of words.

5.      Distinguish word boundaries, recognize a core    of words, and interpret   word order patterns and their significance.

6.      Process speech at different  rates of delivery.

7.      Process speech containing pauses, errors, corrections, and  other performance variables.

8.      Recognize grammatical word classes (nouns, verbs, etc), systems  (e.g. tense, agreement, pluralization), patterns, rules, and elliptical  forms.

9.      Detect sentence constituents and distinguish between major and minor constituents.

10.  Recognize that a particular meaning may be expressed in differed  grammatical  forms.

11.  Recognize cohesive devices in spoken discourse.

Macroskills

12.  Recognize the communicative functions of utterances, according to situations, participants, goals.

13.  Infer situations, participants,  goals using real-word knowledge.

14.  From events, ideas, and so on,  described, predict  outcomes, infer links and connections     between events, deduce  causes and  effects,  and detect  such relations  as main idea, supporting  idea, new  information, given  information, generalization,  and exemplification.

15.  Distinguish between literal and implied meanings.

16.  Use facial, kinesics, body language, and other nonverbal clues to decipher meanings.

17.  Develop and uses a battery of listening strategies, such as detecting key  words, guessing  the meaning  of  words from context, appealing for help, and signaling comprehension or lack thereof.

   

·         DESIGNING  ASSESSMENT  TASKS: INTENSIVE LISTENING

            Once you have determined objectives, your next  step is to design  the tasks, including making decisions about how you will elicit performance and how you will  expect the test-taker to respond.  We will look at  tasks that range from intensive listening performance, such as minimal phonemic pair recognition, to extensive comprehension of language in  communicative contexts.  The focus in this section is on the microskills of intensive listening.

·         RECOGNIZING  PHONOLOGICAL and MORPHOGICAL ELEMENTS

            A typical form of  intensive listening at this level  is the  assissment of recognition of  phonological and morphological elements of  language. A  classic  test task gives a  spoken stimulus and asks test-takers to identify the stimulus from two more choices, as in the following two examples:

Phonemic  pair, consonants

Test-takers  hear:                    He’s  from  California Test-takers  read:                    (a) He’s from  C alifornia                                                 (b)  She’s from  California

Phonemic  pair, vowel

Test-takers  hear:                    Is he living? Test-takers read:                     (a) Is  he  leaving?                                                 (b) Is he  living?

In both cases above, minimal phonemic distinctions are the target. If you are testing recognition of morphology, you can use the same format:

Morphological pair,    -ed ending

Test-takers hear:                     I  missed you very much Test-takers  read:                    (a) I missed you very much                                                 (b) I miss you very much

Hearing the past tense morpheme in this sentence challenges  even advanced learners, especially if no context is provided. Stressed  and unstressed words may also be tested with the same rubric. In the following example, the reduced form (contraction) of can not is tested:

Stress pattern in can’t

Test-takers hear:                     My girlfriend  can’t go to the party  Test-taker read:                      (a) My  girlfriend can’t go to  the party                                                 (b) My girlfriend can go to the party

Because they are decontextualized, the kinds of tasks leave something desired in their authenticity. But they are a step better than items that simply vide a one-word stimulus:

One –word stimulus

Test-takers hear:                     vine Test-takers read:                     (a) vine                                                 (b) wine

·         PARAPHRASE RECOGNITION

            The next step up on the scale of listening comprehension micros kills is word, phrases, and sentences, which are frequently assessed by providing a stimulus sentence and asking the test-taker to choose the correct paraphrase  from a  number of choices.

   Sentence paraphrase

Test-takers hear:                     hellow,  my name’s Keiko. I come from Japan. Test-takers read:                     (a) Keiko is comfortable in Japan.                                                 (b) Keiko wants to come to Japan                                                 (c) Keiko is Japanese                                                 (d) Keiko likes Japan

Dialogue paraphrase

Test-takers hear:         man     :Hi, Maria, my names george                                     Woman            :Nice  to meet you, George. Are you American?                                     Man     : No, I’m canadian Test-takers read:                     (a) George lives in the United States.                                                 (b) George is American                                                 (c) George comes from Canada                                                 (d) Maria is Canadian

·         DESIGNING ASSESSMENT TASKS: RESPONSIVE LISTENING

            A question and answer Fermat can provide some interactivity in these lower-end listening tasks.  The test –takers response is  the appropriate  answer to  a question.

Appropriate response to a question

Test-takers hear:                     how much time did  you take to do your homework? test-takers read:                       (a) In about an hour                                                 (b) About  an hour                                                 (c) about  $10                                                 (d) yes,  I did

Open-ended response to a question

Test-takers hear:                     how much  time did you take to  do your homework? test-takers  write or speak:

If open-ended  response  formats gain a  small amount of authenticity and  creativity,  they of course suffer some  in their practicality, as  teachers  must then  read  students’  responses and judge their appropriateness, which takes time.

·         DESIGNING  ASSESSMENT TASKS:  SELECTIVE LISTENING

A third type of listening performance is selective listening, in which the test-taker listens to a limited quantity of aural input and must discern within  it some  specific  information. A number of techniques have been used that require selective listening.

·         LISTENING CLOZE

Listening cloze tasks (sometimes called cloze dictations or partial dictations) require the test-taker to listen to a a story, monologue, or  conversation  and simultaneously.

One potential of weakness of listening cloze techniques is that they may simply become reading comprehension tasks. Who are asked to listen to a story with periodic deletions in the written version may not need to listen at all, yet may still be able to respond with the appropriate word or phrase. You can guard against this eventually if the blanks are items with high information load that cannot be easily predicted simply by reading the passage. In the example below (adapted from Bailey, 1998, p. 16), such a shortcoming was avoided by focusing only on the criterion of numbers. Test-takers hear an announcement from an airline agent and see the transcript with the underlined words deleted:

a.       Listening Cloze

Test-takers hear: Ladies and gentlemen, I now have some connecting gate information for those of you making connections to other flight out of San Francisco. Flight seven-oh-six to Portland will depart from gate seventy-three at nine-thirty P.M.  Flight ten-forty-five to Reno will depart at nine-fifty P.M. from gate seventeen.  Flight four-forty to Monterey will depart at nine-thirty-five P.M. from gate sixty. And flight sixteen-on-three to Sacramento will depart from gate nineteen at ten-fifteen P.M. Test-takers write the missing words or phrase in the blanks.

 

b.      Information transfer.

Selective listening can also be accessed through an information technique in aurally processed information must be transferred to a visual representation, such as labeling a diagram, identifying an element in a picture, completing a form, showing routes on a map.

c.       Sentence Repetition.

The task of simply repeating a sentence or a phrase, or sentence repetition, is also used as assessment of listening comprehension. As in a dictation (discussed below), the test-taker must retain a stretch of language long enough to reproduce it, and then must respond with an oral repetition of that stimulus incorrect listening comprehension, weather of the phonemic or discourse level, may be manifested in the correctness of the repetition.

·         DESIGNING ASSESSMENT TASKS : EXTENSIVE LISTENING

a.      Dictation

Ø  First reading (natural speed , no pauses, test takes listen for gist):

Larry suddenly woke up from a deep sleep. The sun was dazzling his half-open eyes, and he couldn’t figure out what time it was. The door to his room was closed; the house was immersed in some sort of reckless silence. He slowly got out of his bed and approached the bench right next to the window. For a moment, he thought, he heard a tapping sound coming from the attic. Then again he heard the sound, only this time it seemed to be somewhat closer. He looked outside the window and saw a man going by the left side of the road. On seeing Larry, the man approached his garden’s fence and whistled. At this point, Larry recognized Nick and waved his hand. He quickly got dressed and was about the get down to open the gate, but he again heard someone murmuring in the other part of the house. Larry decided to go to the attic and see what was causing this, now buzzing, sound. He got to the second floor of his house and looked toward the attic. He quickly opened its door and looked inside. Nothing was found. He was about to turn back and attend to his guest when he, suddenly, slipped on the stairs and fell. He called out to Nick to help him get up.

Ø  Second reading (slowed speed, pause at each // break, test takes write

Larry suddenly woke up // from a deep sleep. The sun // was dazzling his half-open eyes, and // he couldn’t figure out what time it was. The door to his room was closed; // the house was immersed in some sort of reckless silence. He slowly got out of // his bed and approached the bench right next to the window. For a moment, he thought, // he heard a tapping sound coming from the attic. Then // again he heard the sound, only // this time it seemed to be somewhat closer. He looked// outside the window and saw a man going by the left side of the road. On seeing Larry, // the man approached his garden’s fence and whistled. At this point, // Larry recognized Nick and waved his hand. // He quickly got dressed and was about the get down to open the gate, but he again heard someone murmuring in the other part of the house.// Larry decided to go to the attic and // see what was causing this, now buzzing, sound. // He got to the second floor of his house and looked toward the attic.// He quickly opened its door and looked inside.// Nothing was found. He was // about to turn back and // attend to his guest when he, suddenly, // slipped on the stairs and fell. // He called out to Nick to help him get up.

Ø  Third reading (natural speed, test takes check their work).

• Communicative stimulus Response Tasks.

o Dialogue and multiple choice comprehension items.

Test takers hear:

Direction: now you will hear a conversation between Antonius and Lica. You will hear a conversation two times. After you hear the conversation the second time, choose the correct answer below. Mark your answer sheet provided.

Antonius: Hello, Lica. I haven’t seen you for a long time. Where have you been?

Lica : Hello, antonius, I have been to Bogor to visit my father.

Antonius : Great. Did you visit the Botanical Garden too?

Lica : Yes, I did. Besides the giant old tropical trees. I also saw the Rafflesia Arnoldi and the giant Water Lily. Do you know who discovered the Rafflesia Arnoldi?

Antonius : Of course, Becarry discovered it in Sumatra in 1928. Why was it named Rafflesia?

Lica : Rafflesia Arnoldi derived from the name of the British Government General, SirThomas Stanford Raffles.

Antonius : Is it a protected plant?

Lica : Yes, all of the plats in Botanical Garden are protected, because they are very rare plants.

Test takes read:

1. From the dialogue we know that………

a. Lica met Antonius in the Bogor Botanical Garden

b. Lica did not only visit her father but also the Botanical Garden in Bogor

c. Antonius visited Lica in her father’s house

d. Antonius went to Bogor with Lica to visit the Botanical Garden.

2. Why are all of the plats in Bogor Botanical Garden protected?
Because they are very………

a. Expensive

b. Numerous

c. Unusual

d. beautiful

3. “Becarry Discovered it in Sumatra in 1928”. The word “it” refers to?

a. Water Lily

b. Tropical plant

c. Botanical Garden

d. Rafflesia Arnoldi

b. Authentic Listening Tasks.

Ideally, the language assessment field would have a stockpile of listening test types that are cognitively demanding, communicative and authentic, not to mention interactive by means of integration with speaking. However, the nature of the test  as sample of performance and set of task with limited time frames implies an equally limited capacity to mirror all the real-world contexts of listening performance.

Beyond the rubrics of intensive, responsive, selective, and quasi-extensive communicative contexts describe above, can we assess aural comprehension in a truly communicative contexts? The answer is cautious yes, but not without some concessions to practicality. And the answer the answer as more certain yes if we take the library of stretching the concept of assessment to extend beyond tests and into a broader framework alternatives. Here are some possibilities.

1.      Not to taking, in the academic world, classroom lecturer by professor as communicative features of a non-native English-user’s experience. One form of midterm examination at the American Language Institute at San Francisco State University (Khan, 2002) uses a 15-minutes lecture as a stimulus. One among several response formats includes note-taking y the test-takers. These notes are evaluated by the teacher on a 30-points system, as follows:

Scoring system for lecturer notes

0-15 points Visual representation: Are you notes clear and easy to read? Can you easily find and retrieve information from them? Do you use the space on the paper to visually represent ideas? Do you use indentation, headers, numbers, etc.? 0-10 points Accuracy: do you accurately indicate main ideas from lectures? Do you note important details and supporting information and example? Do you leave out unimportant information and tangents? 0-5 points Symbols and abbreviation: do you use symbols and abbreviations as much as possible to save time? Do you avoid writing out whole words, and do you, avoid writing down every single word the lecturer says?

 

The purpose of scoring is time consuming (a loss of practicality), and because of subjectivity of the point system, it lacks some reliability. But the gain is to offering student an authentic task that mirrors exactly what they have been focusing on in the classroom.

  

2.      Editing, another, authentic task provides both a written and a spoken stimulus, and requires the test-takers to listen for discrepancies. Scoring achieves relatively high reliability as there are usually a small number of specific differences that must be identified. Here is the way task proceeds.

Editing a written version of an aural stimulus.

Test-taker read: the written stimulus material ( a news report, an email form a friend, notes from a lecture, or an editorial in a newspaper). Test-taker hears: a spoken version of the stimulus that deviates, in a finite numbers of facts or opinions, from the original written form. Test-taker mark: The written stimulus by circling any words, phrase, facts, or opinions that a discrepancy between the two version.

 

One potential interesting set of stimuli for such a task is the description of a political first from newspaper with a political bias, and then from a radio broadcast from an “alternative” news station.

3.      Interpretive task. One of the intensive listening tasks described above was paraphrasing a story or conversation. An interpretative task extends the stimulus material to a longer stretch of discourse and forces the test-taker to inter  a response. Potential stimuli include.

·         Song lyrics.

·         (Recited) poetry.

·         Radio / television news reports, and

·         An oral account of an experience.

Test-takers are then directed to interpret the stimulus by answering a few questions (in open-ended form). Question might be:

·         ”why was the singer feeling sad?”

·         “what events might have led up to reciting or this poem, etc.

This kind of task moves us away from what might traditionally be considered a test toward an informal assessment, or possibly even a pedagogical technique or activity. But the task conforms to certain time limitations, and the question can be quite specific, even though they ask the test-taker to use inference.

4.      Retelling. In a related task, test-takers listen to a story or news event and simply retell it, or summarize it, either orally (on an audiotape) or in writing. In doing, test-takers must identify the gist, main idea, purpose, supporting points and / or conclusion to show full comprehension. Scoring is partially predetermined by specifying a minimum number of elements that must appear in the retelling.

ASSESSING LISTENING

C R E A T E D

BY

The fourth Group

Ø Nurmaisury

Ø Nur ariska Syam.

Ø Nurlaelatul Qadri.

SEKOLAH TINGGI AGAMA ISLAM NEGERI (STAIN) PAREPARE

TAHUN AJARAN 2012-2013