NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1231442
Record Type: Journal
Publication Date: 2019-Oct
Pages: 33
Abstractor: As Provided
ISSN: ISSN-0305-0009
The Cross-Linguistic Performance of Word Segmentation Models over Time
Caines, Andrew; Altmann-Richer, Emma; Buttery, Paula
Journal of Child Language, v46 n6 p1169-1201 Nov 2019
We select three word segmentation models with psycholinguistic foundations -- transitional probabilities, the diphone-based segmenter, and PUDDLE -- which track phoneme co-occurrence and positional frequencies in input strings, and in the case of PUDDLE build lexical and diphone inventories. The models are evaluated on caregiver utterances in 132 CHILDES corpora representing 28 languages and 11.9 m words. PUDDLE shows the best performance overall, albeit with wide cross-linguistic variation. We explore the reasons for this variation, fitting regression models to performance scores with linguistic properties which capture lexico-phonological characteristics of the input: word length, utterance length, diversity in the lexicon, the frequency of one-word utterances, the regularity of phoneme patterns at word boundaries, and the distribution of diphones in each language. These properties together explain four-tenths of the observed variation in segmentation performance, a strong outcome and a solid foundation for studying further variables which make the segmentation task difficult.
Cambridge University Press. 100 Brook Hill Drive, West Nyack, NY 10994. Tel: 800-872-7423; Tel: 845-353-7500; Fax: 845-353-4141; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A