Your search found 5611 results.

Descriptors:
English (Second Language); Language Tests; Essay Tests; Factor Structure; Academic Discourse; Reading Comprehension; Listening Comprehension; Factor Analysis; Correlation; Scores; Listening Comprehension Tests; Reading Tests

Abstract:
The present study examined the factor structures across features of 446 examinees' responses to a writing task that integrates reading and listening modalities as well as reading and listening comprehension items of the TOEFL iBT[R] (Internet-based test). Both human and automated scores obtained for the integrated essays were utilized. Based on a series of preliminary factor analyses, a confirmatory factor analysis (CFA) identified a model that specified a higher order factor for comprehension. In the model, the Comprehension factor underlay factors representing content of the written essay as well as reading and listening comprehension. The Comprehension factor correlated with two writing factors--Productive Vocabulary and Sentence Conventions. Furthermore, follow-up CFA models with covariates (multiple indicators multiple causes models, or MIMIC models) were tested to compare performance between a group of 190 examinees scoring above a frequently used TOEFL iBT Total score requirement for international student admission and the other group of 128 examinees scoring below the requirement. The higher ability group performed significantly better than the lower ability group on all three constructs: Comprehension, Productive Vocabulary, and Sentence Conventions. The identification of the multiple distinct factors in this study may hold promise for obtaining writing profiles that inform instruction in contexts such as test preparation. (Contains 9 tables, 4 figure and 1 footnote.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Reading; Essay Tests; Language Tests; Integrated Activities; English (Second Language); Undergraduate Students; Language Usage; Syntax; Grammar; Accuracy; Language Fluency; Information Sources; Statistical Analysis; Foreign Countries

Abstract:
As a growing number of testing programs use integrated writing tasks, more validation research is needed to inform stakeholders about score use and interpretation. The current study investigates the relationship between writing proficiency and discourse features in an integrated reading-writing task. At a Middle Eastern university, 136 undergraduate students completed a reading-based writing task. The essays were holistically scored by two raters and then classified into three proficiency levels. In addition, the essays were analyzed for a number of discourse features, including fluency, lexical sophistication, syntactic complexity, grammatical accuracy, verbatim source use, and direct and indirect source use. A one-way analysis of variance was employed to look into the relationship between writing proficiency and the discourse features of interest. The results yielded significant differences across proficiency levels for a number of discourse features. Nonetheless, follow-up comparisons indicated that the differences were greater between the lowest level and the two upper levels. As for the upper levels, no statistically significant differences were found between these two levels for most of the discourse features. The implications of the study suggest that the selected discourse features play a major role at lower levels, whereas other textual features, such as cohesion, content, and organization, are more critical at higher level writing. The results also support the need in a construct of integrated writing for the inclusion of reading proficiency and knowledge about discourse synthesis. (Contains 7 tables, 1 figure and 3 footnotes.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Documentation; Writing Tests; Language Usage; Language Tests; English (Second Language); Expertise; Educational Research; Evaluation Criteria

Abstract:
This article reports the lexical diversity of summaries written by experts and test takers in an empirical study and then interrogates the (in)congruity between the conceptualisations of "summary" and "summarize" in the literature of educational research and the operationalization of summarization tasks in three international English language tests in relation to their task directions and assessment criteria. These analyses show that summarization is not a uniform construct; rather, it is multidimensional in terms of its purposes, functions, and practices in real-life contexts, and as a consequence not particularly well operationalized in language tests and so in need of precise definitions for specific assessment contexts. This conclusion has implications for designing summarization tasks and assessing task performance. First, clear and transparent task directions are essential to ensure that test takers and evaluators share a common understanding of a test task; test takers need to be instructed what kind of summary they are expected to produce, particularly what information to include and to exclude. Second, because summarization may well be a unique type of writing process, it is important to employ parameters different from and additional to those for independent composition writing in order to measure the quality of a summary effectively. (Contains 4 tables and 3 footnotes.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Language Tests; Reading Processes; Reading Tests; Test Format; Integrated Activities; Reading Comprehension; Language Usage; Writing (Composition); English for Academic Purposes; Test Validity; Cognitive Processes; Foreign Students; English Language Learners; College Students

Abstract:
Integrated reading/writing tasks are becoming more common in large-scale language tests. Much of the research on these tasks has focused on writing through reading; assessing reading through writing is a less explored area. In this article we describe a reading-into-writing task that is intended to measure both reading comprehension and language use on an academic English test. The task involves responding to short-answer questions (SAQs) that require examinees to use their own words to state the main idea of a text, draw inferences, or synthesize information across multiple texts. The article presents results of a two-part study addressing the validity of this method of assessing reading by investigating the cognitive processes involved in responding to SAQs. First, we present the results of a qualitative study of five nonnative English-speaking students, who provided verbal protocols as they read the texts and responded to the SAQs. Next, we present data from a larger sample of students focusing specifically on the cognitive processes used when reading the texts for the purpose of responding to SAQs. Implications of the study for the validity of this method of testing are discussed. (Contains 4 tables and 2 figures.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Language Tests; Integrated Activities; Reading; Writing (Composition); Information Sources; Writing Tests; Academic Discourse; Evaluation Research

Abstract:
The five studies presented in this special issue offer unique evidence, analyses, and theoretical rationales for assessment tasks that involve writing in reference to information from source material with substantial content. I review the five studies in respect to five "promises" and five "perils," concluding that, collectively, the promises were mostly fulfilled, but so were most of the perils. The promises are that these task types (a) provide realistic, challenging literacy activities; (b) engage test takers in writing that is responsible to specific content; (c) countertest method or practice effects associated with conventional item types; (d) evaluate language abilities consistent with construction-integration or multiliteracies models of literacy; and (e) offer diagnostic value for instruction or self-assessment. The perils of these task types, however, are that they (a) confound the measurement of writing abilities with abilities to comprehend source materials; (b) muddle assessment and diagnostic information together; (c) involve genres that are ill-defined and so difficult to score; (d) require threshold levels of abilities for competent performance, producing test results that may not compare neatly across different ability levels; and (e) elicit texts in which the language from source materials is hard to distinguish from examinees' own language production. Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Listening Comprehension Tests; Language Variation; Second Language Learning; English (Second Language); Foreign Countries; Listening Comprehension; Language Usage; Language Tests; English for Academic Purposes; Native Speakers; Testing

Abstract:
The existence of different varieties of English in target language use (TLU) domains calls into question the usefulness of listening comprehension tests whose input is limited only to a native speaker variety. This study investigated the impact of non-native varieties or accented English speech on test takers from three different English use contexts: Korea, Sri Lanka and Brazil. The findings showed that the variety of English or accented speech used had no impact on test taker's performance on a listening test for academic purposes. Also test takers from the three different countries performed similarly even when the speakers shared the same native languages as the test takers. Despite these findings students still perceived that a native variety of English should be used in listening comprehension tests. Though the study suggests the use of non-native varieties as test input, it also raises questions of fairness in the use of such varieties. (Contains 4 tables and 1 figure.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Grammar; Nouns; Language Processing; Second Language Learning; Syntax; Advanced Students; Form Classes (Languages); Computational Linguistics; Morphology (Languages); Adults; English; Native Speakers; German; Language Research; Control Groups; Eye Movements; Task Analysis; Language Tests; Placement

Abstract:
In order to identify the causes of inflectional variability in adult second-language (L2) acquisition, this study investigates lexical and syntactic aspects of gender processing in real-time L2 production and comprehension. Twenty advanced to near-native adult first language (L1) English speakers of L2 German and 20 native controls were tested in a study comprising two experiments. In elicited production, we probe accuracy in lexical gender assignment. In a visual-world eye tracking task, we test the predictive processing of syntactic gender agreement between determiners and nouns. The findings show clear contingencies (1) between overall accuracy in lexical gender assignment in production and target predictive processing of syntactic gender agreement in comprehension and (2) between the speed of lexical access and predictive syntactic gender agreement. These findings support lexical and computational accounts of L2 inflectional variability and argue against models positing representational deficits in morphosyntax in late L2 acquisition and processing. (Contains 7 tables, 3 figures and 2 notes.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Performance Based Assessment; Reliability; Evaluation Methods; Outsourcing; Case Studies; Oral Language; Language Proficiency; Longitudinal Studies; Telecommunications; Foreign Countries; Language Tests; Language Usage; Rating Scales; English for Special Purposes; Measures (Individuals); Intercultural Communication

Abstract:
This case study discusses the development and use of an oral performance assessment instrument intended to evaluate Filipino agents' customer service transactions with callers from the United States (US). The design and applications of the instrument were based on a longitudinal, qualitative observation of language training and customer service support practices of Philippine-based agents employed by a US-owned call centre company. Although language training in Philippine call centers continues to improve (Lockwood, 2012), there are still clear limitations to how the oral performance of Filipino agents is evaluated internally by call centre companies. Specialized assessment instruments, following ESP/EOP norms, broadly used by the industry are still relatively untested and many call centers maintain their own metrics that often measure agents' language use and service quality separately (Friginal, 2007, 2009). In this study, the assessment instrument was adapted from the Melbourne Medical Students' Diagnostic Speaking Scale (Grove & Brown, 2001) and further developed to include ESP/EOP approaches in this context of inter-cultural communication. A conveniently sampled set of recorded calls (N = 100) across different task categories (e.g., troubleshooting interactions, product inquiry) was used to test the instrument for initial reliability measures. Results and analysis of the instrument's context suitability and limitations are discussed below. (Contains 4 tables.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Scoring; Second Language Learning; Second Languages; English Language Learners; English (Second Language); Second Language Instruction; Validity; Language Tests; Writing Evaluation; Writing Instruction; Essays; Essay Tests; Computer Assisted Testing

Abstract:
This article presents considerations for using automated scoring systems to evaluate second language writing. A distinction is made between English language learners in English-medium educational systems and those studying English in their own countries for a variety of purposes, and between learning-to-write and writing-to-learn in a second language (Manchon, 2011a), extending Manchon's framework from instruction to assessment and drawing implications for construct definition. Next, an approach to validity based on articulating an interpretive argument is presented and discussed with reference to a recent study of the use of e-rater on the TOEFL. Challenges and opportunities for the use of automated scoring system are presented. (Contains 6 tables.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website

Descriptors:
Standard Setting (Scoring); Validity; Models; Language Tests; English (Second Language); Second Language Learning; Cutting Scores

Abstract:
Standard setting theory has largely developed with reference to a typical situation, determining a level or levels of performance for one exam for one context. However, standard setting is now being used with international reference frameworks, where some parameters and assumptions of classical standard setting do not hold. We consider the challenges standard setting poses to reference frameworks and vice versa, focusing on the acceptance within standard setting theory of divergent outcomes. We argue that the justification for it does not hold in the context of reference frameworks; convergent outcomes should be expected and divergences investigated. The argument is illustrated using work relating the International English Language Testing System, an English language proficiency examination, to the Common European Framework of Reference for Languages (CEFR), a reference framework of language ability. We describe a standard setting study and a criterion validation study, show how their results agree, and reconcile findings with those from other studies. Implications for standard setting and for the CEFR are discussed. (Contains 7 tables.) Note:The following two links are not-applicable for text-based browsers or screen-reading software. Show Hide Full Abstract

Related Items: Show Related Items

Full-Text Availability Options:

More Info:
Help | Tutorial Help Finding Full Text |  More Info:
Help Find in a Library |  Publisher's website