Social media websites such as Facebook and Twitter provide an unprecedented amount of qualitative data about organizations and collective behavior. Yet these new data sources lack critical information about the broader social context of collective behavior--or protect it behind strict privacy barriers. In this article, I introduce social media survey apps (SMSAs) that adjoin computational social science methods with conventional survey techniques in order to enable more comprehensive analysis of collective behavior online. SMSAs (1) request large amounts of public and non-public data from organizations that maintain social media pages, (2) survey these organizations to collect additional data of interest to a researcher, and (3) return the results of a scholarly analysis back to these organizations as incentive for them to participate in social science research. SMSAs thus provide a highly efficient, cost-effective, and secure method for extracting detailed data from very large samples of organizations that use social media sites. This article describes how to design and implement SMSAs and discusses an application of this new method to study how nonprofit organizations attract public attention to their cause on Facebook. I conclude by evaluating the quality of the sample derived from this application of SMSAs and discussing the potential of this new method to study non-organizational populations on social media sites as well.