Reading comprehension tests are supposed to measure students' understanding of what they have read; answering a question correctly should depend entirely on information contained in the associated reading passage. However, both teachers and students suspect that general knowledge and sound guessing may enable one to perform substantially better than chance, even with little knowledge of the reading passage. Studies of “passage dependency” in tests for native speakers of English confirm this suspicion.This paper reports an experiment which measured passage dependency in one major ESL test: the reading comprehension section of the Michigan Test of English Language Proficiency (MTELP). We administered one form of this section (20 items) to 60 adult subjects: 30 native speakers of English and 30 non‐native speakers of varied language backgrounds. Half of the subjects in each group received an ordinary version of the test, with a reading passage preceding each set of five questions; the other half received only the questions, with appropriate instructions (the “passage‐out” condition).The results indicate the extent to which the test is passage‐dependent for both native and non‐native students. Test items varied considerably in their passage dependency. We suggest that passage‐dependency is one measure which can be used to select good reading compreh