Memories for spatial locations often show systematic errors toward the central value of the surrounding region. The Category Adjustment (CA) model suggests that this bias is due to a Bayesian combination of categorical and metric information, which offers an optimal solution under conditions of uncertainty (Huttenlocher, Hedges, & Duncan, 1991). A fundamental assumption of this model is that representations of locations are unbiased but uncertain; during combination, greater metric uncertainty results in relatively greater emphasis on categorical information, ultimately leading to increased bias (but also minimizing error across multiple estimates). Sampaio and Wang (2009) have demonstrated that metric information is not lost during this combination process, supporting the CA model's assumption that underlying spatial representations are undistorted. Here, we examine the 2nd half of the CA model's central assumption: that increasing metric uncertainty drives the combination process. Participants recognized point locations within visually complex images in a 4-choice task. Our results indicate that individuals recognized the correct location over other, biased alternatives, confirming that metric information is unbiased at the time of retrieval. In addition, we found that, when participants make errors, they are more likely to select locations that are biased toward the category prototype. In Experiment 2, we demonstrate that categorically biased locations are most likely to be chosen under conditions of uncertainty. Indeed, under these conditions, categorically biased locations were chosen more frequently than the correct location. These results suggest that systematic errors are the result of combination across multiple levels of spatial representations that are undistorted but somewhat uncertain.