This paper presents the results from two studies on anaphoric reference to quantifying expressions (QEs) in Swedish, contributing to the current cross-linguistic discussion on this issue. For English it has been shown that the polarity of the QE (positive vs negative) determines the anaphoric set reference (to the referens set, REFSET, or to the complement set, COMPSET), while for Spanish it has been claimed that while REFSET interpretation is the default, the relative sizes of the two sets (REFSET and COMPSET) also matters. In Experiment 1, a semantic plausibility study. The results showed that for positive QEs, anaphoric reference can only be to the REFSET, while for negative QEs, it can only be to the COMPSET. Unlike in English and Spanish, REFSET continuations were categorically ruled out for negative QEs. To investigate whether the internal differences between QEs could be explained in terms of set size, we conducted Experiment 2, an estimation task. The results from this experiment showed that the size of the REFSET relative to the COMPSET was not a determining factor.