Rating EFL written performance / / Katalin Bukta; managing editor, Anna Borowska; language editor, Carl Becker.

• Superior document: Versita Discipline: Language, Literature
• VerfasserIn: Bukta, Katalin,
• TeilnehmendeR: Borowska, Anna,; Becker, Carl,
• Place / Publishing House: London, England : : Versita, Versita Limited,, 2013.; 2013
• Year of Publication: 2013
• Edition: 1st ed.
• Language: English
• Series: Versita discipline. Language, literature.
• Physical Description: 1 online resource (424 pages).
• Notes: Bibliographic Level Mode of Issuance: Monograph

Table of Contents

Intro -- Introduction -- Overview of the Book -- PART I -- An Overview of the Literature on L2 Writing and Assessment -- Chapter 1 -- Writing Ability and L2 Writing Instruction -- Introduction -- 1.1 The Writing Skill in L1 and L2 -- 1.1.1 Theoretical Models of Written Text Production -- 1.1.2 Theoretical Frameworks of Communicative Competence -- 1.2 Writing Ability in L1 -- 1.3 Writing Ability in L2 -- 1.3.1 The Relationship Between L1 and L2 Writing Ability -- 1.3.2 Skilled and Unskilled L2 Writers -- 1.4 L2 Writing Instruction -- 1.4.1 Controlled Composition Approach to Writing -- 1.4.2 Process-Oriented Approach to Writing -- 1.4.3 The Role of Writing in Communicative Language Teaching -- 1.5 Conclusion -- Chapter 2 -- Assessing Language Ability -- Introduction -- 2.1 Assessing Language Ability -- 2.1.1 History of Language Testing -- 2.1.2 Recent Developments in Language Testing Research -- 2.2 L2 Ability Assessment -- 2.2.1 Language Ability and Language Performance -- 2.2.2 Characteristics of Performance Assessment -- 2.3 Test Features -- 2.3.1 Test Purpose -- 2.3.2 Testing Methods -- 2.4 Qualities of Language Tests -- 2.4.1 Reliability -- 2.4.2 Validity -- 2.4.3 Authenticity -- 2.4.4 Impact -- 2.4.5 Washback -- 2.4.6 Practicality -- 2.5 Test Method Characteristics -- 2.6 Assessing the Four Language Skills -- 2.7 Scoring Methods -- 2.7.1 Characteristics of Rating Scales -- 2.7.2 Rater Variables -- 2.7.3 Score Interpretation -- 2.8 Test-Taker Characteristics -- 2.9 Language Test Construction -- 2.10 Conclusion -- Chapter 3 -- Assessing Writing Ability -- Introduction -- 3.1 Written Performance Assessment -- 3.2 Alternative Ways of Writing Ability Assessment -- 3.3 Nature of Written Performance Assessment -- 3.3.1 Validity of Writing Assessment -- 3.3.2 Task Characteristics -- 3.3.3 Definition of Audience -- 3.3.4 Test Taker Characteristics.

3.4 Conclusion -- Chapter 4 -- Rating Written Performance -- Introduction -- 4.1 Scoring Procedures -- 4.2 Rater Variables -- 4.3 Rating as a Problem-Solving Activity -- 4.4 Frameworks of Scoring Processes -- 4.4.1 Milanovic, Saville and Shuhong's Framework of the Scoring Process -- 4.4.2 Wolfe's Framework of the Scoring Process -- 4.4.3 Lumley's Framework of the Scoring Process -- 4.4.4 Cumming, Kantor and Powers' Framework of the Scoring Process -- 4.5 Rater Stability -- 4.6 Rater Training -- 4.7 Conclusion -- Chapter 5 -- Verbal Protocol Analysis as a Research Method in Written Performance Assessment -- Introduction -- 5.1 Research in L2 Acquisition Studies -- 5.2 Introspective Reports as Data Collection -- 5.3 Categorisation of Verbal Protocols -- 5.4 Concurrent Think-Aloud Protocols in Written Performance Assessment -- 5.5 Verbal Protocol Analysis Procedure -- 5.5.1 Data Preparation and Collection Procedures -- 5.5.2 Verbal Report Procedure Preparation -- 5.5.3 Data Transcription -- 5.5.4 Segmenting the Protocols for Coding -- 5.5.5 Developing a Coding Scheme -- 5.5.6 Analysis of the Coded Data -- 5.6 Advantages and Limitations of Verbal Protocol Analysis as Research Methodology -- 5.7 Conclusion -- PART II -- Investigating Raters' Decision-Making Processes and Awarded Scores in Rating Hungarian Efl Learners' Compositions -- Chapter 6 -- A Pilot Study: Tracing Raters' Decision-Making Processes -- Introduction -- 6.1 Background to the Pilot Study on Assessment of Written Performance -- 6.2 Research Questions -- 6.3 Research Design -- 6.3.1 Participants: the Raters -- 6.3.2 Procedures for Data Collection -- 6.3.3 Test of Written Performance: the Task -- 6.3.4 The Assessment Scale -- 6.3.5 The Coding Scheme -- 6.4 Results and Discussion -- 6.4.1 Distribution of Comments During Rating -- 6.4.2 Comments on Rating Technicalities.

6.4.3 General Comments on the Scripts -- 6.4.4 The Way Raters Arrived at a Score -- 6.4.5 Assessment of the Communicative Goal -- 6.4.6 Assessment of Vocabulary -- 6.4.7 Assessment of accuracy and spelling -- 6.4.8 Assessment of the Text Organisation -- 6.5 The Rating Process -- 6.6 Conclusion -- Chapter 7 -- The Main Study: Processes and Outcomes in Rating L2 English Written Performance in a Pre-Service Tefl Course -- Introduction -- 7.1 Background to Main Study -- 7.2 Design of the Study -- 7.3 Research Questions -- 7.4 Participants -- 7.5 Data Collection Instruments -- 7.5.1 The Scripts -- 7.5.2 The Rating Scale -- 7.5.3 The Rating Task Assignment Package -- 7.6 Data Collection Procedures -- 7.6.1 Script Characteristics -- 7.6.2 Script Selection -- 7.6.3 Training Raters -- 7.6.4 The Rating Task -- 7.7 Processing Verbal Data -- 7.7.1 Transcript Segmentation -- 7.7.2 Coding Scheme Production -- 7.8 Rater Characteristics: Grouping Raters -- 7.9 Summary -- Chapter 8 -- Features of Raters' Rating Patterns -- Introduction -- 8.1 Raters' Gender Distribution -- 8.2 Language of the Verbal Protocols -- 8.3 Sequencing the Scripts for Rating -- 8.4 Length of Verbal Protocols -- 8.5 Raters' Rating Sequences -- 8.6 Raters' Rating Patterns -- 8.7 Conclusion -- Chapter 9 -- Raters' Rating Focus -- Introduction -- 9.1 Raters' Rating Foci -- 9.1.1 Management Focus: Management Strategies -- 9.1.2 Rating Focus: Rating Strategies -- 9.1.3 Reading Focus: Reading Strategies -- 9.1.4 Raters' Own Focus: Other Comments -- 9.2 Conclusion -- Chapter 10 -- Raters' Focus on the Four Rating Criteria -- Introduction -- 10.1 Raters' Focus on the Four Rating Criteria -- 10.2 Raters' Focus When Rating Task Achievement -- 10.3 Raters' Focus When Rating Vocabulary -- 10.4 Raters' Focus When Rating Grammar -- 10.5 Raters' Focus When Rating Organisation -- 10.6 Conclusion -- Chapter 11.

Raters' Script Interpretation on the Weakest and Top Script -- Introduction -- 11.1 Rating Script N2: Benchmarks and Total Scores -- 11.1.1 Comments on Script N2 in the Pre-Scoring Stage -- 11.1.2 Rating Task Achievement of Script N2 -- 11.1.3 Rating Vocabulary of Script N2 -- 1.1.4 Rating Grammar of Script N2 -- 11.1.5 Rating Organisation of Script N2 -- 11.2 Ratings of Script N6: Benchmarks and Total Scores -- 11.2.1 Comments on Script N6 in the Pre-Scoring Stage -- 11.2.2 Rating Task Achievement of Script N6 -- 11.2.4 Rating Grammar of Script N6 -- 11.2.5 Rating Organisation of Script N6 -- 11.3 Conclusion -- Chapter 12 -- Raters' Perception of the Rating Task and Thinking Aloud -- Introduction -- 12.1 The Feedback Sheet -- 12.1.1 Raters' Feedback on the Language-Testing Course -- 12.1.2 Raters' Feedback on Training for Rating Written Performance -- 12.1.3 Raters' Feedback on the Rating Task -- 12.2 Conclusion -- General Conclusions -- Introduction -- Background to Written Performance Assessment -- Features of Rating Processes -- Raters' Rating Foci -- Interpretation of the Rating Criteria -- Raters' Script Interpretation -- Raters' Perceptions of the Rating Task -- Placing Empirical Findings on the Rating Processes into Theoretical Frameworks -- Verbal Protocol Analysis as Data Collection Method -- Implications of the Findings -- Limitations of the Study -- Further Research -- References -- Appendices -- Appendix 6.1 The letter writing task in the pilot study -- Appendix 6.2 The rating scale in the pilot study -- Appendix 6.3 The coding scheme for the pilot study -- Appendix 6.4 Number of utterances made during the rating process in the pilot study -- Appendix 6.5 A sample from EngR1 transcript in the pilot study (translated from Hungarian) -- Appendix 7.1 The writing task in the main study -- Appendix 7.2 The rating scale in the main study.

Appendix 7.3 The letter to the students in the main study -- Appendix 7.4 The ten scripts in the main study -- Appendix 7.5 The score sheet in the main study -- Appendix 7.6 The feedback sheet in the main study -- Appendix 7.7 The course description for the elective seminar course in Testing in ELT -- Appendix 7.8 A sample verbal protocol transcript in the main study -- Appendix 7.9 The coding scheme in the main study -- Appendix 7.10 A sample of a coded protocol in the main study -- Appendix 7.11 Competent raters' total scores and rankings of the ten scripts in parentheses in the main study -- Appendix 7.12 Proficient raters' total scores and rankings in parentheses in the main study -- Index.