Where the veracity of a statement is in question, listeners tend to interpret disfluency as signaling dishonesty. Previous research in deception suggests that this results from a speaker model, linking lying to cognitive effort and effort to disfluency. However, the disfluency-lying bias occurs very quickly: Might listeners instead simply heuristically associate disfluency with lying? To investigate this, we look at whether listeners' disfluency-lying biases are sensitive to context. Participants listened to a potentially dishonest speaker describe treasure as being behind a named object while viewing scenes comprising the referent (the named object) and a distractor. Their task was to click on the treasure's suspected true location. In line with previous work, participants clicked on the distractor more following disfluent descriptions, and this effect corresponded to an early fixation bias, demonstrating the online nature of the pragmatic judgment. The present study, however, also manipulated the presence of an alternative, local cause of speaker disfluency: the speaker momentarily distracted by a car horn. When disfluency could be attributed to speaker distraction, participants initially fixated more on the referent, only later fixating on and selecting the distractor. These findings support the speaker modeling view, showing that listeners can take momentary contextual causes of disfluency into account.