Rating EFL written performance / / Katalin Bukta; managing editor, Anna Borowska; language editor, Carl Becker.

• Superior document: Versita Discipline: Language, Literature
• VerfasserIn: Bukta, Katalin,
• TeilnehmendeR: Borowska, Anna,; Becker, Carl,
• Place / Publishing House: London, England : : Versita, Versita Limited,, 2013.; 2013
• Year of Publication: 2013
• Edition: 1st ed.
• Language: English
• Series: Versita discipline. Language, literature.
• Physical Description: 1 online resource (424 pages).
• Notes: Bibliographic Level Mode of Issuance: Monograph

Table of Contents

• Intro
• Introduction
• Overview of the Book
• PART I
• An Overview of the Literature on L2 Writing and Assessment
• Chapter 1
• Writing Ability and L2 Writing Instruction
• Introduction
• 1.1 The Writing Skill in L1 and L2
• 1.1.1 Theoretical Models of Written Text Production
• 1.1.2 Theoretical Frameworks of Communicative Competence
• 1.2 Writing Ability in L1
• 1.3 Writing Ability in L2
• 1.3.1 The Relationship Between L1 and L2 Writing Ability
• 1.3.2 Skilled and Unskilled L2 Writers
• 1.4 L2 Writing Instruction
• 1.4.1 Controlled Composition Approach to Writing
• 1.4.2 Process-Oriented Approach to Writing
• 1.4.3 The Role of Writing in Communicative Language Teaching
• 1.5 Conclusion
• Chapter 2
• Assessing Language Ability
• Introduction
• 2.1 Assessing Language Ability
• 2.1.1 History of Language Testing
• 2.1.2 Recent Developments in Language Testing Research
• 2.2 L2 Ability Assessment
• 2.2.1 Language Ability and Language Performance
• 2.2.2 Characteristics of Performance Assessment
• 2.3 Test Features
• 2.3.1 Test Purpose
• 2.3.2 Testing Methods
• 2.4 Qualities of Language Tests
• 2.4.1 Reliability
• 2.4.2 Validity
• 2.4.3 Authenticity
• 2.4.4 Impact
• 2.4.5 Washback
• 2.4.6 Practicality
• 2.5 Test Method Characteristics
• 2.6 Assessing the Four Language Skills
• 2.7 Scoring Methods
• 2.7.1 Characteristics of Rating Scales
• 2.7.2 Rater Variables
• 2.7.3 Score Interpretation
• 2.8 Test-Taker Characteristics
• 2.9 Language Test Construction
• 2.10 Conclusion
• Chapter 3
• Assessing Writing Ability
• Introduction
• 3.1 Written Performance Assessment
• 3.2 Alternative Ways of Writing Ability Assessment
• 3.3 Nature of Written Performance Assessment
• 3.3.1 Validity of Writing Assessment
• 3.3.2 Task Characteristics
• 3.3.3 Definition of Audience
• 3.3.4 Test Taker Characteristics.
• 3.4 Conclusion
• Chapter 4
• Rating Written Performance
• Introduction
• 4.1 Scoring Procedures
• 4.2 Rater Variables
• 4.3 Rating as a Problem-Solving Activity
• 4.4 Frameworks of Scoring Processes
• 4.4.1 Milanovic, Saville and Shuhong's Framework of the Scoring Process
• 4.4.2 Wolfe's Framework of the Scoring Process
• 4.4.3 Lumley's Framework of the Scoring Process
• 4.4.4 Cumming, Kantor and Powers' Framework of the Scoring Process
• 4.5 Rater Stability
• 4.6 Rater Training
• 4.7 Conclusion
• Chapter 5
• Verbal Protocol Analysis as a Research Method in Written Performance Assessment
• Introduction
• 5.1 Research in L2 Acquisition Studies
• 5.2 Introspective Reports as Data Collection
• 5.3 Categorisation of Verbal Protocols
• 5.4 Concurrent Think-Aloud Protocols in Written Performance Assessment
• 5.5 Verbal Protocol Analysis Procedure
• 5.5.1 Data Preparation and Collection Procedures
• 5.5.2 Verbal Report Procedure Preparation
• 5.5.3 Data Transcription
• 5.5.4 Segmenting the Protocols for Coding
• 5.5.5 Developing a Coding Scheme
• 5.5.6 Analysis of the Coded Data
• 5.6 Advantages and Limitations of Verbal Protocol Analysis as Research Methodology
• 5.7 Conclusion
• PART II
• Investigating Raters' Decision-Making Processes and Awarded Scores in Rating Hungarian Efl Learners' Compositions
• Chapter 6
• A Pilot Study: Tracing Raters' Decision-Making Processes
• Introduction
• 6.1 Background to the Pilot Study on Assessment of Written Performance
• 6.2 Research Questions
• 6.3 Research Design
• 6.3.1 Participants: the Raters
• 6.3.2 Procedures for Data Collection
• 6.3.3 Test of Written Performance: the Task
• 6.3.4 The Assessment Scale
• 6.3.5 The Coding Scheme
• 6.4 Results and Discussion
• 6.4.1 Distribution of Comments During Rating
• 6.4.2 Comments on Rating Technicalities.
• 6.4.3 General Comments on the Scripts
• 6.4.4 The Way Raters Arrived at a Score
• 6.4.5 Assessment of the Communicative Goal
• 6.4.6 Assessment of Vocabulary
• 6.4.7 Assessment of accuracy and spelling
• 6.4.8 Assessment of the Text Organisation
• 6.5 The Rating Process
• 6.6 Conclusion
• Chapter 7
• The Main Study: Processes and Outcomes in Rating L2 English Written Performance in a Pre-Service Tefl Course
• Introduction
• 7.1 Background to Main Study
• 7.2 Design of the Study
• 7.3 Research Questions
• 7.4 Participants
• 7.5 Data Collection Instruments
• 7.5.1 The Scripts
• 7.5.2 The Rating Scale
• 7.5.3 The Rating Task Assignment Package
• 7.6 Data Collection Procedures
• 7.6.1 Script Characteristics
• 7.6.2 Script Selection
• 7.6.3 Training Raters
• 7.6.4 The Rating Task
• 7.7 Processing Verbal Data
• 7.7.1 Transcript Segmentation
• 7.7.2 Coding Scheme Production
• 7.8 Rater Characteristics: Grouping Raters
• 7.9 Summary
• Chapter 8
• Features of Raters' Rating Patterns
• Introduction
• 8.1 Raters' Gender Distribution
• 8.2 Language of the Verbal Protocols
• 8.3 Sequencing the Scripts for Rating
• 8.4 Length of Verbal Protocols
• 8.5 Raters' Rating Sequences
• 8.6 Raters' Rating Patterns
• 8.7 Conclusion
• Chapter 9
• Raters' Rating Focus
• Introduction
• 9.1 Raters' Rating Foci
• 9.1.1 Management Focus: Management Strategies
• 9.1.2 Rating Focus: Rating Strategies
• 9.1.3 Reading Focus: Reading Strategies
• 9.1.4 Raters' Own Focus: Other Comments
• 9.2 Conclusion
• Chapter 10
• Raters' Focus on the Four Rating Criteria
• Introduction
• 10.1 Raters' Focus on the Four Rating Criteria
• 10.2 Raters' Focus When Rating Task Achievement
• 10.3 Raters' Focus When Rating Vocabulary
• 10.4 Raters' Focus When Rating Grammar
• 10.5 Raters' Focus When Rating Organisation
• 10.6 Conclusion
• Chapter 11.
• Raters' Script Interpretation on the Weakest and Top Script
• Introduction
• 11.1 Rating Script N2: Benchmarks and Total Scores
• 11.1.1 Comments on Script N2 in the Pre-Scoring Stage
• 11.1.2 Rating Task Achievement of Script N2
• 11.1.3 Rating Vocabulary of Script N2
• 1.1.4 Rating Grammar of Script N2
• 11.1.5 Rating Organisation of Script N2
• 11.2 Ratings of Script N6: Benchmarks and Total Scores
• 11.2.1 Comments on Script N6 in the Pre-Scoring Stage
• 11.2.2 Rating Task Achievement of Script N6
• 11.2.4 Rating Grammar of Script N6
• 11.2.5 Rating Organisation of Script N6
• 11.3 Conclusion
• Chapter 12
• Raters' Perception of the Rating Task and Thinking Aloud
• Introduction
• 12.1 The Feedback Sheet
• 12.1.1 Raters' Feedback on the Language-Testing Course
• 12.1.2 Raters' Feedback on Training for Rating Written Performance
• 12.1.3 Raters' Feedback on the Rating Task
• 12.2 Conclusion
• General Conclusions
• Introduction
• Background to Written Performance Assessment
• Features of Rating Processes
• Raters' Rating Foci
• Interpretation of the Rating Criteria
• Raters' Script Interpretation
• Raters' Perceptions of the Rating Task
• Placing Empirical Findings on the Rating Processes into Theoretical Frameworks
• Verbal Protocol Analysis as Data Collection Method
• Implications of the Findings
• Limitations of the Study
• Further Research
• References
• Appendices
• Appendix 6.1 The letter writing task in the pilot study
• Appendix 6.2 The rating scale in the pilot study
• Appendix 6.3 The coding scheme for the pilot study
• Appendix 6.4 Number of utterances made during the rating process in the pilot study
• Appendix 6.5 A sample from EngR1 transcript in the pilot study (translated from Hungarian)
• Appendix 7.1 The writing task in the main study
• Appendix 7.2 The rating scale in the main study.
• Appendix 7.3 The letter to the students in the main study
• Appendix 7.4 The ten scripts in the main study
• Appendix 7.5 The score sheet in the main study
• Appendix 7.6 The feedback sheet in the main study
• Appendix 7.7 The course description for the elective seminar course in Testing in ELT
• Appendix 7.8 A sample verbal protocol transcript in the main  study
• Appendix 7.9 The coding scheme in the main study
• Appendix 7.10 A sample of a coded protocol in the main study
• Appendix 7.11   Competent raters' total scores and rankings of the ten scripts in parentheses in the main study
• Appendix 7.12 Proficient raters' total scores and rankings in parentheses in the main study
• Index.