Article Sidebar

Submitted
Oct 27, 2020
Published
Oct 26, 2020
Download
PDF
Statistic
Read Counter : 325 Download : 295

Main Article Content

Abstract

Student evaluation of teaching is a global predominant practice in higher education institutions. Therefore, a major university in Jordan developed a questionnaire for students’ use in evaluating their instructors’ teaching effectiveness.  Since student evaluation of teaching is an important process, the present study tried to examine the psychometric properties of the instrument. Item-total correlations showed acceptable internal consistency. In addition, a two-factor structure of the scale (teaching effectiveness and course attributes) was supported by exploratory factor analysis and confirmatory factor analysis from two independent samples. Convergent validity was supported by a moderate correlation coefficient between course averages of students’ ratings on the first factor and course averages of students’ final grades in each course. Finally, students’ responses on the factor that captures teaching effectiveness were found to have very high internal consistency (Cronbach’s alpha of 0.96). However, this instrument lacks evidences of content validity and convergent validity. Therefore, it is important to be cautious in evaluating faculty members and making promotion decisions that is based solely on the scores obtained using this instrument. 

Keywords

Students’ ratings validity reliability teaching effectiveness

Article Details

References

  1. Aleamoni, L. M. (1978). Development and factorial validation of the Arizona course/instructor evaluation questionnaire. Educational and Psychological Measurement, 38(4), 1063-1067. doi:10.1177/001316447803800426.
  2. Balam, E. M., & Shannon, D. M. (2009). Student ratings of college teaching: A comparison of faculty and their students. Assessment & Evaluation in Higher Education, 35(2), 209-221. doi:10.1080/02602930902795901.
  3. Bassi, F., Clerci, R., & Aquario, D. (2017). Students’ evaluation of teaching at a large Italian university: validation of measurement scale. Electronic Journal of Applied Statistical Analysis, 10(1), 93-117. Doi: 10.1285/i20705948v10n1p93.
  4. Benton S.& Cashin W. (2014) Student Ratings of Instruction in College and University Courses. In: Paulsen M. (eds) Higher Education: Handbook of Theory and Research (vol. 29, pp. 279-326). Dordrecht: Springer,. https://doi.org/10.1007/978-94-017-8005-6_7.
  5. Catano, V. M., & Harvey, S. (2011). Student perception of teaching effectiveness: development and validation of the Evaluation of Teaching Competencies Scale (ETCS). Assessment & Evaluation in Higher Education, 36(6), 701-717. doi:10.1080/02602938.2010.484879.
  6. Chen, G. H., & Watkins, D. (2010). Stability and correlates of student evaluations of teaching at a Chinese university. Assessment & Evaluation in Higher Education, 35(6), 675-685. doi:10.1080/02602930902977715.
  7. d'Apollonia, S., & Abrami, P. C. (1997). Navigating student ratings of instruction. American Psychologist, 52(11), 1198-1208. doi:10.1037/0003-066X.52.11.1198.
  8. Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London: Sage Publications Ltd.
  9. Greenwald, A. G., & Gillmore, G. M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52(11), 1209-1217. doi:http://dx.doi.org/10.1037/0003-066X.52.11.1209.
  10. Griffin, B. W. (2001). Instructor reputation and student ratings of instruction. Contemporary Educational Psychology, 26(4), 534-552. doi:http://dx.doi.org/10.1006/ceps.2000.1075.
  11. Haertel, E. H. (2006). Reliability. In R. L. Brennan (Ed.), Educational Measurement (4th ed., pp. 65-110). Westport, CT: American Council on Education and Praeger Publishers.
  12. Jöreskog, K., & Sörbom, D. (2006). LISREL 8.80. Lincolnwood, IL: Scientific Software International, Inc.
  13. Mulaik, S. (2010). Foundations of facotr analysis (2nd ed.). FL:CPC Press.
  14. Kane, M. (2006). Validation. In R. L. Brennan (Ed.), Educational Measurement (4th ed., pp. 17-64). Westport, CT: American Council on Education and Praeger Publishers.
  15. Kelloway, K. E. (1998). Using LISREL for structural equation modelling: A researcher's guide. Thousand Oaks, CA: Sage.
  16. Kogan, L. R., Schoenfeld-Tacher, R., & Hellyer, P. W. (2010). Student evaluations of teaching: Perceptions of faculty based on gender, position, and rank. Teaching in Higher Education, 15(6), 623-636. doi:10.1080/13562517.2010.491911.
  17. Lemos, M. S., Queirós, C., Teixeira, P. M., & Menezes, I. (2011). Development and validation of a theoretically based, multidimensional questionnaire of student evaluation of university teaching. Assessment & Evaluation in Higher Education, 36(7), 843-864. doi:10.1080/02602938.2010.493969.
  18. Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students' evaluations of university teaching. British Journal of Educational Psychology, 52(1), 77-95. doi:10.1111/j.2044-8279.1982.tb02505.x.
  19. Martínez-Gómez, M., Sierra, J. M. C., Jabaloyes, J., & Zarzo, M. (2010). A multivariate method for analyzing and improving the use of student evaluation of teaching questionnaires: A case study. Quality & Quantity, 45(6), 1415-1427. doi:10.1007/s11135-010-9345-5
  20. Morley, D. (2014). Assessing the reliability of student evaluations of teaching: Choosing the right coefficient. Assessment & Evaluation in Higher Education, 39(2), 127-139. doi:10.1080/02602938.2013.796508.
  21. Nemec, E. C., Baker, D. M., Zhang, D., & Dintzner, M. (2018). Development of valid and reliable tools for student evaluation of teaching. Currents in Pharmacy Teaching and Learning, 10(5), 549-557. doi:10.1016/j.cptl.2018.02.009.
  22. Nunnally, J., & Bernstein, I. (1994). Psycho-metric theory (3rd ed.). NewYork: McGraw-Hill.
  23. Onwuegbuzie, A. J., Daniel, L. G., & Collins, K. M. T. (2007). A meta-validation model for assessing the score-validity of student teaching evaluations. Quality & Quantity, 43(2), 197-209. doi:10.1007/s11135-007-9112-4.
  24. Oon, P.-T., Spencer, B., & Kam, C. C. S. (2016). Psychometric quality of a student evaluation of teaching survey in higher education. Assessment & Evaluation in Higher Education, 42(5), 788-800. doi:10.1080/02602938.2016.1193119.
  25. Ory, J. C. (2001). Faculty thoughts and concerns about student ratings. New Directions for Teaching and Learning, 2001(87), 3-15. doi:10.1002/tl.23.
  26. Patrick, J., & Smart, R. M. (1998). An empirical evaluation of teacher effectiveness: The emergence of three critical factors. Assessment & Evaluation in Higher Education, 23(2), 165-178. doi:10.1080/0260293980230205.
  27. Penny, A. R. (2003). Changing the agenda for research into students' views about university teaching: Four shortcomings of SRT research. Teaching in Higher Education, 8(3), 399-411. doi:10.1080/13562510309396.
  28. Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The course experience questionnaire. Studies in Higher Education, 16(2), 129-150. doi:10.1080/03075079112331382944.
  29. Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387-415. doi:10.1080/02602930500099193.
  30. Schumacker, R. E., & Lomax, R. G. (2010). A beginner’s guide to structural equation modeling (3rd ed.). New York: Routledge.
  31. Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397-405. doi:10.1080/713611436.
  32. Simpson, P. M., & Siguaw, J. A. (2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22(3), 199-213. doi:10.1177/0273475300223004.
  33. Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642. doi:10.3102/0034654313496870.
  34. Stehle, S., Spinath, B., & Kadmon, M. (2012). Measuring teaching effectiveness: Correspondence between students’ evaluations of teaching and different measures of student learning. Research in Higher Education, 53(8), 888-904. doi:10.1007/s11162-012-9260-9.
  35. Toland, M. D., & De Ayala, R. J. (2005). A multilevel factor analysis of students’ evaluations of teaching. Educational and Psychological Measurement, 65(2), 272-296. doi:10.1177/0013164404268667.
  36. Willits, F. & Brennan, M. (2017). Another look at college student’s ratings of course quality: data from Penn State student surveys in three settings. As-sessment & Evaluation in Higher Educa-tion, 42(3), 443-462. DOI: http://dx.doi.org/10.1080/02602938.2015.1120858.
  37. Wilson, K. L., Lizzio, A., & Ramsden, P. (1997). The development, validation and application of the course experience questionnaire. Studies in Higher Education, 22(1), 33-53. doi:10.1080/03075079712331381121.
  38. Zhao, J., & Gallant, D. J. (2012). Student evaluation of instruction in higher education: Exploring issues of validity and reliability. Assessment & Evaluation in Higher Education, 37(2), 227-235. doi:10.1080/02602938.2010.523819.