NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1238389
Record Type: Journal
Publication Date: 2019-Dec
Pages: 28
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
A Proof-of-Concept Study on Scoring Oral Presentation Videos in Higher Education. Research Report. ETS RR-19-22
Feng, Gary; Joe, Jilliam; Kitchen, Christopher; Mao, Liyang; Roohr, Katrina Crotts; Chen, Lei
ETS Research Report Series, Dec 2019
This proof-of-concept study examined the feasibility of a new scoring procedure designed to reduce the time of scoring a video-based public speaking assessment task. Instead of scoring the video in its entirety, the performance was evaluated based on content-related (e.g., speech organization, word choice) and delivery-related (e.g., vocal expression, nonverbal behaviors) dimensions. Content-related dimensions were scored based on speech transcripts, while delivery dimensions were scored using a video thin-slicing technique, where scores were assigned based on brief vignettes of a video rather than the complete performance. Initial feasibility data were collected from 4 novice raters scoring 10 video performances. Results indicated that for delivery dimensions, four 10 second thin slices yielded interrater consistency reliability somewhat similar to that of full-video scoring, and additional slices resulted in only small improvements. For transcription-based scoring, while raters were consistent, their scores had low correlations with criterion scores, which was likely due to differences in the modality (video vs. text). Video thin-slicing appears to be a promising scoring technique for relevant constructs. Further testing of a combination of audio and transcript is recommended for scoring content-related constructs.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A