Widgepedia Widgepedia

Like most teachers today, we take it as a given that communicative ability in a second language must be considered holistically. That is, communicative ability includes not only vocabulary, pronunciation and grammar skills, but also the capacity to use these in real-world contexts. It is this last point which is often missed by traditional assessment tools, largely because it is considered too time-consuming and subjective to test them.

Here, we hope to demonstrate that task-based assessment is neither time-consuming nor any more subjective than other methods, and in fact includes other advantages as well.

Task-based assessment can be easy, straightforward and meaningful for learners if one primarily looks at the appropriate outcome of a given task, and secondarily looks at the accuracy of the language used. In other words, if the learner can achieve the task appropriately, then the learner passes. Conversely, if they cannot achieve the task in a generally acceptable manner, they fail. This presupposes two important factors, which are discussed later: 1) that the task was well selected to begin with; and 2) what "appropriate" means.

Here is a hypothetical task, including some specific parameters to be met:

Task:

Tell a story

Parameters:

On a simple, familiar topic (e.g., family trip)

To a single sympathetic listener (e.g., a friend)

For a certain length of time (e.g., two minutes)

Note that parameters are important in order to keep tasks at a specific level. For example, if you were to change the “single sympathetic listener” to “an audience of English teachers during a high-stakes university entrance interview” it would make the same basic task far more challenging.

Now, let us say you are grading the task outcome on a 10-point scale. If the learner appropriately completes the task (i.e. they tell a story according to these parameters, regardless of how “good” it actually was), then they pass. Their score is somewhere between 6 and 10.

If they could not accomplish the task (i.e. they could not be understood at all, or if what they produced would not reasonably be called “a story”) then they fail. Also, if they spoke reasonably well but did not stay within the parameters—for example if they spoke for only one minute or spoke about an entirely different topic—then they did not complete the task and they fail. Their score is between 1 and 5.

The next step is to assess how well the task was achieved. Now—secondarily—we can look at things such as pronunciation, vocabulary and grammar. Bear in mind that, if an appropriate outcome has been achieved, then by definition we already know that the learner's pronunciation, vocabulary and grammar are at least at an acceptable level for the task. They could not have managed to complete the task otherwise.

In short, task-based assessment works like this:

Step 1

Was the task appropriately completed?

Would the outcome be reasonably recognizable by an “average” native speaker of English as an example of its “type”?

In this case, Was it a story? Was there a beginning, a middle and an end? Were the events in the story linked to each other coherently? Was it clear enough to understand the meaning, despite any possible language problems such as poor pronunciation or grammar mistakes?

If yes, PASS.

If no, FAIL.

Step 2

If PASS, then how good was the story? Read the descriptors below and assign a grade from 6 to 10.

If FAIL, what were the issues? Were there at least some redeeming qualities? Assign a grade from 1 to 5.

Example scoring criteria for a pass:

10 No grammar mistakes worth mentioning. Vocabulary use was very appropriate. Pronunciation was exceptionally clear. Speech was very smooth and fluent. Gestures, facial expressions and manner were effective and natural. Speaker was very confident and engaging.

9 A few small grammar, vocabulary or pronunciation mistakes that did not affect meaning. Speech was smooth, near-fluent, and very easy to understand. Gestures, facial expressions and manner were effective and natural, with perhaps a minor slip. Speaker was confident and engaging.

8 A considerable number of small grammar, vocabulary or pronunciation difficulties sometimes affected meaning. Despite these, the overall speech was easy to follow and understand. Gestures, facial expressions and manner were generally appropriate. Speaker was mostly confident and engaging.

7 Occasional serious difficulties with grammar, vocabulary or pronunciation. Speech was not always smooth and clear, but overall quite understandable. Speaker did not revert to L1. Gestures, facial expressions or manner may have been marked or distracting.

6 Serious difficulties with basic grammar, vocabulary or pronunciation. Required some support from the listener. Occasionally reverted to L1. Gestures, facial expressions or manner may have been distracting. Speech not always clear, but overall complete and understandable.

Example scoring criteria for a fail:

5 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1; nevertheless, short sections of the speech could sometimes be understandable, or contained other redeeming features.

4 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1. Understandable only to a very sympathetic listener who is familiar with the learner's L1.

3 Did not display an ability to use basic grammar structures. Spoke in two- or three-word utterances using limited but appropriate vocabulary. Used other means to support speech, including heavy use of L1. Difficult to understand even for a very sympathetic listener.

2 Did not display an ability to use basic grammar structures at all. Spoke in one- or two-word utterances using basic vocabulary. Used other means to support speech, including heavy use of L1. Extremely difficult to understand even for a very sympathetic listener.

1 Could not respond to or use basic set greetings such as "How are you?" Made only single-word utterances in basic vocabulary, or none at all. Used L1 almost exclusively, perhaps making an attempt to anglicize some words. Impossible for an average listener to understand.

Task Selection

The real trick in TBL assessment is to select tasks and task parameters in a thoughtful way. [to be continued...]

What Does "Appropriate" Mean?

One of the more difficult points in task-based assessment is arriving at a good definition of what "appropriate" means. For example, when we say that the point of a task such as Order a pizza by telephone is that a hypothetical pizza restaurant employee would either understand the order "well enough" to deliver the pizza, what do we mean?

At what point do we draw the line between clear and unclear, appropriate or inappropriate, etc? Do we assume that the typical interlocutor knows the learner's L1? Is familiar with other learners of the L1? Is not familiar with those learners, but is experienced with language learners in general? What about attitude—is the interlocutor patient and sympathetic the learner? These questions can go on and on, and there is not easy answer.

One way of solving the issue is to arrive at a clear definition of one's own, and accept that these factors may be interpreted differently by different teachers. For example, some teachers imagine a person they know to be a "reasonable" interlocutor. A teacher in Japan who is quite familiar with Japanese L1 learners could ask themselves the question, Would my brother/mother/friend understand this? That is, instead of relying on a hypothetical "pizza guy", we could imagine a more specific person in that role as we assess.

Feedback

In a task-based approach, specific language forms are never be the primary focus of feedback, because it is important that learners be allowed to make meaning as they see fit, at least at first try. Teachers may assist and even correct learners when asked, of course, but should never restrict the learners' choice of which forms to use by explicitly teaching, say, the present continuous before the task is attempted.

A post-task language feedback phase, on the other hand, is recognized by many task-based practitioners as useful. During this segment of the lesson, after the learners have attempted the task, the teacher may choose to go over the language used, correcting specific errors and/or highlighting particularly well-suited forms that learners did not use, but could have chosen to.