Psychologists have long been intrigued with the rationales that underlie our decisions. Similarly, the concept of conditioned reinforcement has a venerable history, particularly in accounting for behavior not obviously maintained by primary reinforcers. The studies of choice and of conditioned reinforcement have often developed in lockstep. Many contemporary approaches to these fundamental topics share an emphasis on context and on relative value. We trace the evolution of thinking about the potency of conditioned reinforcers from stimuli that were thought to acquire their value from pairings with more fundamental reinforcers to stimuli that acquire their value by being differentially correlated with these more fundamental reinforcers. We discuss some seminal experiments (including several that have been underappreciated) and some ongoing data, all of which have propelled us to the conclusion that the strength of conditioned reinforcers is determined by their signaling a relative improvement in the organism's relation to reinforcement. (Contains 13 figures.)