Women are underrepresented in physics. Because of the structure of the Japanese educational system, more women must choose physics as a subject for university entrance exams to increase the number of women studying advanced physics at university. In this study, we investigated the factors influencing girls' choice of physics for university entrance examination in Japan, focusing on preference and self-efficacy for science subjects. We investigated two Japanese populations (members of the public who graduated from university with a degree in science, and professional physicists) to identify characteristics of physicists. We conducted online retrospective questionnaires. First, we found that the preference for physics at junior high school and the first year of high school were positively related to the choice of physics for university entrance exams in both female and male university science graduates. Second, we found that preferences for museums and science magazines as well as the recognition of the importance of physics and mathematics at elementary or junior high school were significantly related to the choice of physics for female university science graduates. Third, we found that professional physicists, especially women, had a lower mathematical stereotype than male and female university science graduates. Our results suggest that initiatives to prevent girls from disliking physics at junior high school or high school may be important for encouraging them to choose physics for university entrance examination in Japan.