This paper presents a formulation of the second law of thermodynamics couched in terms of Shannon's measure of information. This formulation has an advantage over other formulations of the second law. First, it shows explicitly what is the thing that changes in a spontaneous process in an isolated system, which is traditionally referred to as the entropy. Second, and more importantly, this formulation offers a probabilistic answer to the question of why that thing changes in one direction only. (Contains 7 figures.)