In the support of individuals with profound intellectual and multiple disabilities (PIMD), assessing the level of alertness is a recurring issue for parents and other direct support persons. Although observations show clear advantages above and beyond other assessment methods, there are problems related to this method as well. Subjectivity of interpretation and low reliability results have been described as the main problems. In the present study, our aim was to estimate the reliability of the Alertness Observation List (AOL) while, at the same time, minimizing the problems entailed in observations. We calculated both the inter-observer agreement and intra-observer agreement for 39 situations. Since the results exceeded the formulated 80%-criterion, we concluded that the AOL was a reliable instrument. However, the large range found in the results was striking. Moreover, observers with different information about the observed individuals came up with different reliability scores. To determine the value of observation of individuals with PIMD, it might well be necessary to judge the actual usefulness that the instrument has in clinical practice, besides the reliability of the results. (Contains 3 tables.)