Bayesian structural equation modeling (BSEM) is a flexible tool for the exploration and estimation of sparse factor loading structures; that is, most cross-loading entries are zero and only a few important cross-loadings are nonzero. The current investigation was focused on the BSEM with small-variance normal distribution priors (BSEM-N) for both variable selection and model estimation. The prior sensitivity in BSEM-N was explored in factor analysis models with sparse loading structures through a simulation study (Study 1) and an empirical example (Study 2). Study 1 examined the prior sensitivity in BSEM-N based on the model fit, population model recovery, true and false positive rates, and parameter estimation. Seven shrinkage priors on cross-loadings and five noninformative/vague priors on other model parameters were examined. Study 2 provided a real data example to illustrate the impact of various priors on model fit and parameter selection and estimation. Results indicated that when the 95% credible intervals of shrinkage priors barely covered the population cross-loading values, it resulted in the best balance between true and false positives. If the goal is to perform variable selection, a sparse cross-loading structure is required, preferably with a minimal number of nontrivial cross-loadings and relatively high primary loading values. To improve parameter estimates, a relatively large prior variance is preferred. When cross-loadings are relatively large, BSEM-N with zero-mean priors is not recommended for the estimation of cross-loadings and factor correlations.