Exploring the psychometric properties of the Testwiseness Rating Scale using a measurement modeling approach

Testwiseness refers to the ability of a test-taker or examinee to use strategic knowledge and cues within a test in order to improve their performance. These skills, cues, and knowledge are often independent of the content being assessed. Testwiseness includes skills such as using the process of elimination, recognizing patterns in answer choices, paying attention to grammatical errors, and managing time effectively. Testwiseness is important because it influences test outcomes, and potentially gives advantages to students with high testwiseness skills, rather than solely reflecting their true ability or knowledge level. Thus, understanding testwiseness is crucial for ensuring meaningful interpretation and use of test scores. The current study describes the development and psychometric evaluation of the Testwiseness Rating Scale (TWRS). We collected pilot data for the TWRS from a sample of 300 students. We evaluated the psychometric properties using a combination of factor analysis and Rasch measurement theory techniques, including differential item functioning analysis related to student gender. Overall, we found evidence to support the interpretation and use of scores from the TWRS to measure students’ testwiseness. We consider the implications of our findings for research and practice and discuss directions for future research related to this instrument.