We examine the respective roles of substantive understanding (i.e., understanding of factual knowledge, concepts, laws and theories) and procedural understanding (an understanding of ideas about evidence; concepts such as reliability and validity, measurement and calibration, data collection, measurement error, the ability to interpret evidence and the like) required to carry out an open-ended science investigation. Our chosen method of analysis is Charles Ragin's Fuzzy Set Qualitative Comparative Analysis which we introduce in the paper. Comparing the performance of undergraduate students on two investigation tasks which differ with regard to the amount of substantive content, we demonstrate that both substantive understanding and an understanding of ideas about evidence are jointly involved in carrying out such tasks competently. It might be expected that substantive knowledge is less important when carrying out an investigation with little substantive demand. However, we find that the contribution of substantive understanding and an understanding of ideas about evidence is remarkably similar for both tasks. We discuss possible reasons for our findings.