Response processes are accepted as evidence of validity in the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999). Hence, the studies related to individuals’ response processes tend to enhance the validity of survey items or test items. In this context, the cognitive interview method plays an important and beneficial role in better understanding the cognitive processes leading to individuals’ responses. In this way, cognitive interviews are able to reveal what individuals are actually thinking, their expectations regarding items and the strategies they utilize in responding to items. The aim of this study is to reveal that use of cognitive interviews can be valuable in gaining validity evidence as well as utilizing students’ responses to further develop item features. For this purpose, 28 Turkish fourth-grade students were chosen as research participants, half were higher achieving students and the remaining half were lower achieving students. Items were selected by taking into consideration the item statistics, cognitive domains and the content of the items from the Turkish version of the Science Test of the Trends in International Mathematics and Science Study [TIMSS]. Before cognitive interviews, an interview guide was prepared by the authors. The questions in the interview guide for this study were linked to the features of items, such as; item length, language simplicity, visual materials, strategies which students utilized while responding to the items and reflections of students towards the items. Content analysis was used to analyze the collected data so that similar response processes were placed into to specific categories. In order to determine whether students’ categorized responses were appropriate for providing validity evidence, experts in the field of educational measurement and evaluation assessed categorized data. Findings were analysed acorrding to both gender and academic achivement level. Conclusions and suggestions were based on analysis findings.