TBL Assessment

General Guidelines

Task-based assessment can be easy, straightforward, and meaningful for learners. This section explains how.

Consider the following hypothetical task, including some specific set parameters:

Task:

Tell a story

Parameters:

On a simple, familiar topic (e.g., family trip)

To a single sympathetic listener (e.g., a friend)

For a certain length of time (e.g., two minutes)

With visual support (e.g., a photo album)

Note that parameters are important in order to keep tasks at a specific level. For example, if we were to remove the visual support, or change the “single sympathetic listener” to “an audience of English teachers during a high-stakes university entrance interview”, this would make the same basic task far more challenging.

Now, let us say we are grading the task on a 10-point scale. If the learner appropriately completes the task (i.e. they tell a story within these parameters, regardless of how “good” it actually was), then they pass. Their grade will be somewhere between 6 and 10.

If they could not accomplish the task (i.e. they could barely be understood, or if what they produced would not reasonably be called “a story”) then they fail. Also, if they spoke reasonably well but did not stay within the parameters—for example if they spoke for only one minute, or spoke about an entirely different topic—then they did not complete the task and they fail. Their grade will be between 1 and 5.

The next step is to assess how well the task was achieved or not achieved. Now, secondarily, we can look at things such as pronunciation, vocabulary and grammar. Bear in mind that, if an appropriate task outcome has been achieved, then by definition we already know that the learner's pronunciation, vocabulary and grammar are at least at an acceptable level for the task; they could not have managed to complete the task otherwise.

In short, task-based assessment works like this:

Step 1: Decide PASS or FAIL based on task outcome.

Ask yourself: Was the task appropriately completed?

Would the task outcome be reasonably acceptable by an “average” fluent speaker of English as an example of its type?

In this case, Was it a story? (i.e., was it a series of events linked to each other in a reasonably coherent way?) At this stage, avoid judging how good the language is, but rather judge only if the learner manages to convey their intended meaning in a way that an "average" listener would understand.

If yes, PASS.

If no, FAIL.

Step 2: Decide how well the task was performed.

At this point, i.e. secondarily, consider various aspects of language use.

If PASS, then how good was the perfomance? Read the descriptors below and assign a grade from 6 to 10.

If FAIL, what were the issues? Were there nevertheless some redeeming qualities? Assign a grade from 1 to 5.

Example scoring criteria for a PASS:

10 No grammar mistakes worth mentioning. Vocabulary use was very appropriate. Pronunciation was exceptionally clear. Speech was very smooth and fluent. Gestures, facial expressions and manner were effective and natural. Speaker was very confident and engaging.

9 A few small grammar, vocabulary or pronunciation mistakes that did not affect meaning. Speech was smooth, near-fluent, and very easy to understand. Gestures, facial expressions and manner were effective and natural, with perhaps one or two minor slips. Speaker was confident and engaging.

8 A small number of small grammar, vocabulary or pronunciation difficulties sometimes affected meaning. Despite these, the overall speech was easy to follow and understand. Gestures, facial expressions and manner were generally appropriate. Speaker was mostly confident and engaging.

7 Occasional serious difficulties with grammar, vocabulary or pronunciation. Speech was not always smooth and clear, but overall quite understandable. Speaker did not revert to L1. Gestures, facial expressions or manner may have been marked or distracting.

6 Serious difficulties with basic grammar, vocabulary or pronunciation. Required some support from the listener. Occasionally reverted to L1. Gestures, facial expressions or manner may have been distracting. Speech not always clear, but overall it was complete and understandable.

Example scoring criteria for a FAIL:

5 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1; nevertheless, short sections of the speech could sometimes be understandable, or contained other redeeming features.

4 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1. Understandable only to a very sympathetic listener who is familiar with the learner's L1.

3 Did not display an ability to use basic grammar structures. Spoke in two- or three-word utterances using limited but appropriate vocabulary. Used other means to support speech, including heavy use of L1. Difficult to understand even for a very sympathetic listener.

2 Did not display an ability to use basic grammar structures at all. Spoke in one- or two-word utterances using basic vocabulary. Used other means to support speech, including heavy use of L1. Extremely difficult to understand even for a very sympathetic listener.

1 Could not respond to or use basic set greetings such as "How are you?" Made only single-word utterances in basic vocabulary, or none at all. Used L1 almost exclusively, perhaps making an attempt to anglicize some words. Impossible for an average listener to understand.

The example above is for a speaking task, and on a 10-point scale; however, the same basic principles apply for writing, listening, or reading tasks, and to other measurement scale: First, decide pass or fail based on task outcome; then, decide how well the task outcome was achieved or not achieved.

One other useful tip: Whenever possible, use an even-numbered scale so that there is no "maybe", option. Force yourself and other graders to always make that first, crucial, yes-or-no decision. If necessary, get a second or third opinion from colleagues and go with consensus.

Sequencing tasks and setting parameters

The real trick in TBL assessment is to select tasks appropriately, and then to set parameters which apply to your context. [to be continued...]

What Does "Appropriate" Mean?

One of the more difficult points in task-based assessment is arriving at a good definition of what "appropriate" means. For example, when we say that the point of a task such as Order a pizza by telephone is that a hypothetical pizza restaurant employee would understand the order well enough to successfully deliver the pizza, what do we mean by "well enough"?

At what point do we draw the line between clear and unclear, appropriate or inappropriate, etc? Do we assume that a typical interlocutor knows the learner's L1? Is familiar with other learners of the L1? Is not familiar with those learners at all, but is experienced with language learners in general? What about attitude—is the interlocutor patient and sympathetic to learners? Are they even in a position to be sympathetic; e.g. a waiter vs. a police officer? These questions can go on and on, and there is no easy answer.

One way of solving the issue is to simply arrive at a clear definition of one's own, and to accept that these variables will be interpreted differently by different teachers. Some teachers prefer to imagine a person whom they know to be a "reasonable" interlocutor. For example, a teacher in Japan who is too familiar with Japanese L1 learners could distance themselves somewhat by asking the question, Would my brother/mother/friend understand this? That is, instead of trying to establish what a hypothetical "pizza guy" might do, they might imagine a more specific person in that role, at least for the purpose of making that crucial initial pass/fail distinction.

Assessment in Widgets

Many self and peer assessment pages are built into the student book (for example, the "Paperwork" pages at the end of each stage). These can be filled in and then a copy made to submit to the teacher, or they can be downloaded and copies made for students to fill out:

Stage 1

Self assessment

Peer assessment

Teacher assessment

Stage 2

Self assessment

Peer assessment

Teacher assessment

Stage 3

Self assessment

Peer assessment

Teacher assessment

Stage 4

Self-assessment

Peer assessment

Teacher assessment

Stage 5

Self assessment

Peer assessment

Teacher assessment

Stage 6

Self assessment

Peer assessment

Teacher assessment