Administrators at many colleges and universities have had online courses at their institutions for many years, now. One of the hidden challenges about online courses is that they tend to be observed and evaluated far less frequently than their face-to-face course counterparts. This is party due to the fact that many of us administrators today never taught online courses ourselves when we were teaching. This article provides six "secrets" to performing meaningful observations and evaluations of online teaching, including how to use data analytics, avoid biases, and produce useful results even if observers have never taught online themselves.