|
1. Anatomy of Assessment by
John Langille and Eldon Friesen 2009
The primary purpose of
assessment is to inform better teaching and more effective
learning.
-Validity in Testing:
A test should test what the
writer wants it to test with clear and explicit picture
about what is to be tested.
-Reliability in Testing:
A test should have an
actual agreement between the results of one test/assessment
with itself and consistency of test scores.
-Why do we test?
1. To measure an ability.
2. To discover whether
students have achieved the goals.
3. To diagnose students'
strengths and weaknesses.
4. To evaluate
teacher/curriculum/program.
(Remember: Many variables
affect students when doing the test.)
(Remember: Testing should
support good teaching and good teaching should support good
testing.)
-The testing cycle:
curriculum learning
outcomes - test specifications - tasks/items -
administration of test- task/item analysis
(students/teachers/program)
-Test specifications:
Generative explanatory
documents for the creation of test tasks
1. General description:
purpose/ construct/learning objective addressed.
2. Prompt attribute:
description of what the student encounters.
3. Response attribute:
description of the way student provides an answer.
(Remember: It is good to
have a sample test in order to reflect the test
specification.)
-Why do we need test
specifications?
1. Test equivalence from
one semester to the next.
2. Issues of test security.
3. Issues of validity and
reliability (marking consistency).
-Who needs/uses test
specifications?
1. Test constructors
2. Those editing or
moderating the test
3. Those responsible for
establishing the validity of a test
4. Test users
5. The program
-Norm-referenced Test
and Criterion-referenced Test:
NRT features:
1. It relates students'
performance to that of other students.
2. Interpretation of the
test results is relative.
3. Acceptable standard is
decided after the test.
4. Items discriminate
between good and poor students.
5. There is no direct
relationship between the results and students’ actual
proficiency.
CRT features:
1. It classifies people to
whether they can or cannot perform a task satisfactorily.
2. The cut-off score is set
before the test.
3. Decisions are absolute.
4. It is meaningful without
reference to other students' scores.
5. Distribution of scores
may be highly skewed
- Points to consider:
1. Students should know
about the test structure beforehand.
2. Test should be
referenced and tied to an ability level and teaching
objectives.
3. Teachers should clarify
teaching objectives and to what degree they have been met in
tests.
- Remember the
following:
We should address fairness
to students when designing tests. Piloting a test based on
specifications ensures reliability and validity. Some
learning objectives are too difficult to assess. We need to
analyze test specifications in order to improve our
curriculum.
2. Assessing Language
Learners' pragmatic knowledge by Dr. Ahmed AL Issa, 2009
Pragmatics:
It goes beyond the literal
meaning of the words. How words are interpreted. What people
mean not what the words mean by themselves.
Pragmaticlinguistics:
It is linguistic resources
for conveying communicative acts and interpersonal meanings.
Socio-pragmatics:
It is the social
perceptions of what the speaker means/ contextual factors
Pragmatics within the
construct of communicative competence
L2 pragmatics:
The study of nonnative
speaker's production
Dimensions of L2
Pragmatic Competence:
1. Knowledge dimension:
1. Knowledge of L2
sociolinguistics
2. Knowledge of L2
pragmatics
(Mapping pragmatics
conventions on socio-pragmatic norms)
2. Process dimension:
ability for use
1. Speech acts
2. Implications
3. Situational routines
4. Conversational
management
(Whether grammatical
development guarantees a corresponding level of pragmatic
development)
(Whether pragmatic
knowledge is teachable?)
(Yes)
(The role of explicit and
implicit teaching of pragmatics)
(Issue of pragmatic
transfer)
Factors determining
teaching L2 pragmatic competence:
Teachers' attitude:
Difficult/ time
consuming/limited resources/lack of training on teaching
pragmatics/many teachers nonnative speakers/assessment
policies don't help/ focus on form/lack of some valid
methods for testing interlanguage pragmatic knowledge
Assessment instruments
for L2 pragmatics
Lack of tests: difficulty
and practicality of pragmatic tests
Pragmatic tests:
1. Completion tasks based
on situations with missing utterance from one speaker
(students should write an appropriate utterance)
2. Role-play
3. Multiple-choice
discourse completion test
4. Discourse
self-assessment test
5. Metapragmatic assessment
questionnaire
6. Self-assessment on video
tapes
(Roever's pilot test 2006)
(
www.carla.umn.edu/speechacts )
Challenges:
1. Defining the construct
of Pragmatic Competence
2. Providing sufficient
context makes test design very complex
3. How best to present
contextual information and how much risk of
construct-irrelevant
variance
4. Practicality
(time-consuming, expensive to administer)
5. Estimating
6. Validity
7. Variety of English
Suggestions for L2
classroom-based assessment strategies:
1. Assessing knowledge or
performance
2. Assessing productive or
receptive abilities
3. Low-stake testing
4. Understanding how
learners of different ability levels approach various kinds
of pragmatics assessment
tasks
5. Use of films, pictures,
sound
6. Realistic situations
7. Follow up discussion
3. Materials
Development for Test Development by Rachel Lange 2009
- Characteristics of
good materials:
1. Materials should achieve
impact.
They should help learners
to develop confidence: balance/iteration/predictable
format/progression. Also, materials should take into account
that learners differ in affective attitudes and learning
styles. They should require and facilitate learner
self-investment using simple or familiar tasks and
instructions.
2. Materials should be
attractive and have appealing content.
They should help learners
to feel at ease: lots of white space/ images/ people from
own culture/helpful/informal discourse/ active/concreteness:
examples and anecdotes/inclusiveness
3. Materials should have
novelty.
They should not rely too
much on controlled practice. They should provide learners
with freedom to create their own output. Also, they should
provide them with opportunities for outcome feedback.
4. Materials should target
learning outcomes.
They should provide the
learners with opportunities to use the target language to
achieve communicative processes. The learners' attention
should be drawn to the target linguistic features of the
input. What is being taught should be perceived by learners
as relevant and useful.
5. Materials should have
variety.
They should expose learners
to language in different authentic contexts and uses. Topics
should be different related to learners' interests.
6. Materials should have
easy-to-follow structure.
Vocabulary list according
to chapters and pages/ unit list with target language/ new
vocabulary 1st exposure in context then repeated throughout
materials/ grammar presented in the same format/ writing
processed as planning - writing - reflection
- Special challenges in
test preparation materials:
1. Content Validity:
Tests should assess what
learners have already studied. Topics in tests should be
similar to those presented in materials.
2. Face validity:
The layout in test
questions should be presented in materials, so it is
familiar to learners. The order of test sections should be
logical, for example, writing sections should be the last
part.
|
