Rating EFL written performance / / Katalin Bukta; managing editor, Anna Borowska; language editor, Carl Becker.
Saved in:
Superior document: | Versita Discipline: Language, Literature |
---|---|
VerfasserIn: | |
TeilnehmendeR: | |
Place / Publishing House: | London, England : : Versita, Versita Limited,, 2013. 2013 |
Year of Publication: | 2013 |
Edition: | 1st ed. |
Language: | English |
Series: | Versita discipline. Language, literature.
|
Physical Description: | 1 online resource (424 pages). |
Notes: | Bibliographic Level Mode of Issuance: Monograph |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- Intro
- Introduction
- Overview of the Book
- PART I
- An Overview of the Literature on L2 Writing and Assessment
- Chapter 1
- Writing Ability and L2 Writing Instruction
- Introduction
- 1.1 The Writing Skill in L1 and L2
- 1.1.1 Theoretical Models of Written Text Production
- 1.1.2 Theoretical Frameworks of Communicative Competence
- 1.2 Writing Ability in L1
- 1.3 Writing Ability in L2
- 1.3.1 The Relationship Between L1 and L2 Writing Ability
- 1.3.2 Skilled and Unskilled L2 Writers
- 1.4 L2 Writing Instruction
- 1.4.1 Controlled Composition Approach to Writing
- 1.4.2 Process-Oriented Approach to Writing
- 1.4.3 The Role of Writing in Communicative Language Teaching
- 1.5 Conclusion
- Chapter 2
- Assessing Language Ability
- Introduction
- 2.1 Assessing Language Ability
- 2.1.1 History of Language Testing
- 2.1.2 Recent Developments in Language Testing Research
- 2.2 L2 Ability Assessment
- 2.2.1 Language Ability and Language Performance
- 2.2.2 Characteristics of Performance Assessment
- 2.3 Test Features
- 2.3.1 Test Purpose
- 2.3.2 Testing Methods
- 2.4 Qualities of Language Tests
- 2.4.1 Reliability
- 2.4.2 Validity
- 2.4.3 Authenticity
- 2.4.4 Impact
- 2.4.5 Washback
- 2.4.6 Practicality
- 2.5 Test Method Characteristics
- 2.6 Assessing the Four Language Skills
- 2.7 Scoring Methods
- 2.7.1 Characteristics of Rating Scales
- 2.7.2 Rater Variables
- 2.7.3 Score Interpretation
- 2.8 Test-Taker Characteristics
- 2.9 Language Test Construction
- 2.10 Conclusion
- Chapter 3
- Assessing Writing Ability
- Introduction
- 3.1 Written Performance Assessment
- 3.2 Alternative Ways of Writing Ability Assessment
- 3.3 Nature of Written Performance Assessment
- 3.3.1 Validity of Writing Assessment
- 3.3.2 Task Characteristics
- 3.3.3 Definition of Audience
- 3.3.4 Test Taker Characteristics.
- 3.4 Conclusion
- Chapter 4
- Rating Written Performance
- Introduction
- 4.1 Scoring Procedures
- 4.2 Rater Variables
- 4.3 Rating as a Problem-Solving Activity
- 4.4 Frameworks of Scoring Processes
- 4.4.1 Milanovic, Saville and Shuhong's Framework of the Scoring Process
- 4.4.2 Wolfe's Framework of the Scoring Process
- 4.4.3 Lumley's Framework of the Scoring Process
- 4.4.4 Cumming, Kantor and Powers' Framework of the Scoring Process
- 4.5 Rater Stability
- 4.6 Rater Training
- 4.7 Conclusion
- Chapter 5
- Verbal Protocol Analysis as a Research Method in Written Performance Assessment
- Introduction
- 5.1 Research in L2 Acquisition Studies
- 5.2 Introspective Reports as Data Collection
- 5.3 Categorisation of Verbal Protocols
- 5.4 Concurrent Think-Aloud Protocols in Written Performance Assessment
- 5.5 Verbal Protocol Analysis Procedure
- 5.5.1 Data Preparation and Collection Procedures
- 5.5.2 Verbal Report Procedure Preparation
- 5.5.3 Data Transcription
- 5.5.4 Segmenting the Protocols for Coding
- 5.5.5 Developing a Coding Scheme
- 5.5.6 Analysis of the Coded Data
- 5.6 Advantages and Limitations of Verbal Protocol Analysis as Research Methodology
- 5.7 Conclusion
- PART II
- Investigating Raters' Decision-Making Processes and Awarded Scores in Rating Hungarian Efl Learners' Compositions
- Chapter 6
- A Pilot Study: Tracing Raters' Decision-Making Processes
- Introduction
- 6.1 Background to the Pilot Study on Assessment of Written Performance
- 6.2 Research Questions
- 6.3 Research Design
- 6.3.1 Participants: the Raters
- 6.3.2 Procedures for Data Collection
- 6.3.3 Test of Written Performance: the Task
- 6.3.4 The Assessment Scale
- 6.3.5 The Coding Scheme
- 6.4 Results and Discussion
- 6.4.1 Distribution of Comments During Rating
- 6.4.2 Comments on Rating Technicalities.
- 6.4.3 General Comments on the Scripts
- 6.4.4 The Way Raters Arrived at a Score
- 6.4.5 Assessment of the Communicative Goal
- 6.4.6 Assessment of Vocabulary
- 6.4.7 Assessment of accuracy and spelling
- 6.4.8 Assessment of the Text Organisation
- 6.5 The Rating Process
- 6.6 Conclusion
- Chapter 7
- The Main Study: Processes and Outcomes in Rating L2 English Written Performance in a Pre-Service Tefl Course
- Introduction
- 7.1 Background to Main Study
- 7.2 Design of the Study
- 7.3 Research Questions
- 7.4 Participants
- 7.5 Data Collection Instruments
- 7.5.1 The Scripts
- 7.5.2 The Rating Scale
- 7.5.3 The Rating Task Assignment Package
- 7.6 Data Collection Procedures
- 7.6.1 Script Characteristics
- 7.6.2 Script Selection
- 7.6.3 Training Raters
- 7.6.4 The Rating Task
- 7.7 Processing Verbal Data
- 7.7.1 Transcript Segmentation
- 7.7.2 Coding Scheme Production
- 7.8 Rater Characteristics: Grouping Raters
- 7.9 Summary
- Chapter 8
- Features of Raters' Rating Patterns
- Introduction
- 8.1 Raters' Gender Distribution
- 8.2 Language of the Verbal Protocols
- 8.3 Sequencing the Scripts for Rating
- 8.4 Length of Verbal Protocols
- 8.5 Raters' Rating Sequences
- 8.6 Raters' Rating Patterns
- 8.7 Conclusion
- Chapter 9
- Raters' Rating Focus
- Introduction
- 9.1 Raters' Rating Foci
- 9.1.1 Management Focus: Management Strategies
- 9.1.2 Rating Focus: Rating Strategies
- 9.1.3 Reading Focus: Reading Strategies
- 9.1.4 Raters' Own Focus: Other Comments
- 9.2 Conclusion
- Chapter 10
- Raters' Focus on the Four Rating Criteria
- Introduction
- 10.1 Raters' Focus on the Four Rating Criteria
- 10.2 Raters' Focus When Rating Task Achievement
- 10.3 Raters' Focus When Rating Vocabulary
- 10.4 Raters' Focus When Rating Grammar
- 10.5 Raters' Focus When Rating Organisation
- 10.6 Conclusion
- Chapter 11.
- Raters' Script Interpretation on the Weakest and Top Script
- Introduction
- 11.1 Rating Script N2: Benchmarks and Total Scores
- 11.1.1 Comments on Script N2 in the Pre-Scoring Stage
- 11.1.2 Rating Task Achievement of Script N2
- 11.1.3 Rating Vocabulary of Script N2
- 1.1.4 Rating Grammar of Script N2
- 11.1.5 Rating Organisation of Script N2
- 11.2 Ratings of Script N6: Benchmarks and Total Scores
- 11.2.1 Comments on Script N6 in the Pre-Scoring Stage
- 11.2.2 Rating Task Achievement of Script N6
- 11.2.4 Rating Grammar of Script N6
- 11.2.5 Rating Organisation of Script N6
- 11.3 Conclusion
- Chapter 12
- Raters' Perception of the Rating Task and Thinking Aloud
- Introduction
- 12.1 The Feedback Sheet
- 12.1.1 Raters' Feedback on the Language-Testing Course
- 12.1.2 Raters' Feedback on Training for Rating Written Performance
- 12.1.3 Raters' Feedback on the Rating Task
- 12.2 Conclusion
- General Conclusions
- Introduction
- Background to Written Performance Assessment
- Features of Rating Processes
- Raters' Rating Foci
- Interpretation of the Rating Criteria
- Raters' Script Interpretation
- Raters' Perceptions of the Rating Task
- Placing Empirical Findings on the Rating Processes into Theoretical Frameworks
- Verbal Protocol Analysis as Data Collection Method
- Implications of the Findings
- Limitations of the Study
- Further Research
- References
- Appendices
- Appendix 6.1 The letter writing task in the pilot study
- Appendix 6.2 The rating scale in the pilot study
- Appendix 6.3 The coding scheme for the pilot study
- Appendix 6.4 Number of utterances made during the rating process in the pilot study
- Appendix 6.5 A sample from EngR1 transcript in the pilot study (translated from Hungarian)
- Appendix 7.1 The writing task in the main study
- Appendix 7.2 The rating scale in the main study.
- Appendix 7.3 The letter to the students in the main study
- Appendix 7.4 The ten scripts in the main study
- Appendix 7.5 The score sheet in the main study
- Appendix 7.6 The feedback sheet in the main study
- Appendix 7.7 The course description for the elective seminar course in Testing in ELT
- Appendix 7.8 A sample verbal protocol transcript in the main study
- Appendix 7.9 The coding scheme in the main study
- Appendix 7.10 A sample of a coded protocol in the main study
- Appendix 7.11 Competent raters' total scores and rankings of the ten scripts in parentheses in the main study
- Appendix 7.12 Proficient raters' total scores and rankings in parentheses in the main study
- Index.