Latent variable models exist with continuous, categorical, or both types of latent variables. The role of latent variables is to account for systematic patterns in the observed responses. This article has two goals: (a) to establish whether, based on observed responses, it can be decided that an underlying latent variable is continuous or categorical, and (b) to quantify the effect of sample size and class proportions on making this distinction. Latent variable models with categorical, continuous, or both types of latent variables are fitted to simulated data generated under different types of latent variable models. If an analysis is restricted to fitting continuous latent variable models assuming a homogeneous population and data stem from a heterogeneous population, overextraction of factors may occur. Similarly, if an analysis is restricted to fitting latent class models, overextraction of classes may occur if covariation between observed variables is due to continuous factors. For the data-generating models used in this study, comparing the fit of different exploratory factor mixture models usually allows one to distinguish correctly between categorical and/or continuous latent variables. Correct model choice depends on class separation and within-class sample size.