In the psycholinguistic literature, reaction times and accuracy can be analyzed separately using mixed (logistic) effects models with crossed random effects for item and subject. Given the potential correlation between these two outcomes, a joint model for the reaction time and accuracy may provide further insight. In this paper, a Bayesian hierarchical framework is proposed that allows estimation of the correlation between time intensity and difficulty at the item level, and between speed and ability at the subject level. The framework is shown to be flexible in that reaction times can follow a (log-) normal or (shifted) Weibull distribution. A simulation study reveals the reduction in bias gains possible when using joint models, and an analysis of an example from a Dutch-English word recognition study illustrates the proposed method.