Background: Deficits in the ability to use language in social contexts, including storytelling skills, are observed across the autism spectrum. Development in machine-learning approaches may contribute to clinical psychology and psychiatry, given its potential to support decisions concerning the diagnosis and treatment of psychiatric conditions and disorders. Aims: To evaluate the usefulness of deep neural networks for detecting autism spectrum disorder (ASD) from textual utterances, specifically from narrations produced by individuals with ASD. Methods & Procedures: We examined two text encoders: Embeddings from Language Models (ELMo) and Universal Sentence Encoder (USE), and three classification algorithms: XGBoost, support vector machines, and dense neural network layer. We aimed to classify 25 participants with ASD and 25 participants with typical development (TD) based on their narrations produced during the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) picture book task. The results of computational approaches were compared with the results of standardized testing and classifications made by two psychiatrists (raters). The raters were asked to read utterances produced by a participant (without an examiner's statements and additional information) and assign a participant to one of the two groups: ASD or with typical development (TD). Outcomes & Results: The computer-based models had higher sensitivity, specificity, positive predictive values and negative predictive values than the raters, and lower than the two standardized instruments: ADOS-2 and Social Communication Questionnaire (SCQ). Conclusions & Implications: Our findings lay the groundwork for future studies involving deep neural network-based text representation models as tools for augmenting the ASD diagnosis or screening. Both ELMo and USE text encoders provided promising specificities, sensitivities, positive predictive values and negative predictive values. Our results indicate the usefulness of page-level embeddings for utterance representation in ADOS-2 picture book task.