Albert Einstein had famously said that Nature is subtle but not malicious. He should know better because he had unravelled some of the secrets of Nature at a relatively young age as an obscure patent clerk working in isolation. This paper tells of stories of other scientists who had also made ground-breaking discoveries in forced or self-imposed isolation. Their passion and devotion to physics was largely driven by what Einstein himself called "the hunger of the soul".