It is widely assumed that the short-term retention of information is accomplished via maintenance of an active neural trace. However, we demonstrate that memory can be preserved across a brief delay despite the apparent loss of sustained representations. Delay period activity may, in fact, reflect the focus of attention, rather than STM. We unconfounded attention and memory by causing external and internal shifts of attention away from items that were being actively retained. Multivariate pattern analysis of fMRI indicated that only items within the focus of attention elicited an active neural trace. Activity corresponding to representations of items outside the focus quickly dropped to baseline. Nevertheless, this information was remembered after a brief delay. Our data also show that refocusing attention toward a previously unattended memory item can reactivate its neural signature. The loss of sustained activity has long been thought to indicate a disruption of STM, but our results suggest that, even for small memory loads not exceeding the capacity limits of STM, the active maintenance of a stimulus representation may not be necessary for its short-term retention.