Using semantic iterative keyword sampling for polyonymous concepts in interlingual educational research by example of ‘formative evaluation’ in German

Lists

Tools

Cookson, C. ORCID: https://orcid.org/0000-0002-7488-3016 (2024) Using semantic iterative keyword sampling for polyonymous concepts in interlingual educational research by example of ‘formative evaluation’ in German. International Journal of Research & Method in Education. ISSN 1743-7288

Text - Accepted Version
· Restricted to Repository staff only until 26 August 2025.
514kB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1080/1743727X.2024.2315552

Abstract/Summary

This paper details the methodology originally used for a cross-sector literature review of formative evaluation in Germany. The study involved an online keyword search using academic databases and Google Search to uncover documents containing and meaningfully discussing formative evaluation as well as alternative forms of this term in German that matched a predefined conceptualization, a process termed here ‘semantic iterative keyword sampling’ (SIKS). Formative evaluation was recognized from the outset as a polyonym (defined as each of at least four different terms having the same meaning) in German, but the search revealed that German authors use no fewer than 47 terms to refer to this concept – far more than expected. Formative evaluation appears to be a polyonymous concept even in English. This demonstrates the utility of a more comprehensive and structured approach to online keyword searches for lexically flexible concepts. Educational researchers studying other polyonymous concepts may find SIKS to be useful for collecting more data within not only the target context but also other disciplinary and linguistic domains.

Item Type:	Article
Refereed:	Yes
Divisions:	No Reading authors. Back catalogue items Arts, Humanities and Social Science > International Study and Language Institute (ISLI)
ID Code:	118620
Uncontrolled Keywords:	Formative evaluation, Google Search, interlingual research, keyword search, polyonym, semantic search
Publisher:	Routledge

Altmetric

Deposit Details

References

Altheide, D. L. & Schneider, C. J. (2013). Qualitative Media Analysis. Thousand Oaks, CA: Sage Publications. Atkinson, R. and Flint, J. (2001). Accessing hidden and hard-to-reach populations: Snowball research strategies. Social Research Update. Vol. 33. Berners-Lee, T., Hendler, J. & Lassila, O. (2001). The Semantic Web. Scientific American. 284 (5), pp. 34-43. Biernacki, P. and Waldorf, D. (1981). Snowball Sampling: Problems and Techniques of Chain Referral Sampling. Sociological Methods & Research. 10 (2): 141-163. Black, P. & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice. 5 (1): 7-71. Braun, V. and Clarke, V. (2012). Thematic analysis. In: Cooper H. (ed). APA Handbook of Research Methods in Psychology (pp. 57-71). Vol. 2, Research designs. Washington (D.C.): American Psychological Association. Bryman, A. (2012). Social Research Methods. 4th Ed. Oxford: Oxford University Press. Charmaz, K. (2000). Grounded Theory: Objectivist and Constructivist Methods. In: Denzin, N. K. & Y. S. Lincoln (eds.). Handbook of Qualitative Research (pp. 509-535). 2nd Ed. Thousand Oaks, California: Sage Publications. Clark, I. (2011). Formative Assessment: Policy, Perspectives and Practice. Florida Journal of Educational Administration & Policy. 4 (2), pp. 158-180. Coates, D. (2022). Semantic Search: How It Works & Who It’s For. Accessed August 1, 2023. https://www.searchenginejournal.com/semantic-search-how-it-works-who-its-for/438960. Cohen, L., Manion, L., and Morrison, K. (2018). Research Methods in Education. 8th Ed. London: Routledge. Cookson, C. J. (2018). Assessment terms half a century in the making and unmaking: from conceptual ingenuity to definitional anarchy. Assessment & Evaluation in Higher Education. 43 (6), pp. 930-942. Cruse, A. (2000). Meaning in Language: An Introduction to Semantics and Pragmatics. Oxford: Oxford University Press. Dunn, K. E. & Mulvenon, S. W. (2009). A Critical Review of Research on Formative Assessment: The Limited Scientific Evidence of the Impact of Formative Assessment in Education. Practical Assessment, Research & Evaluation. 14 (7), pp. 1-11. Gipps, C. (2012). Beyond Testing: Towards a Theory of Educational Assessment. Classic Ed. London: Routledge. Glaser, B. G. & Strauss, A. L. (1967). The Discovery of Grounded Theory: strategies for qualitative research. Hawthorne, New York: Aldine de Gruyter. Google (2023). How Google Search organizes information. Accessed November 22, 2020. https://www.google.com/search/howsearchworks/how-search-works/organizing-information. Google Search Central (2023). In-depth guide to how Google Search works. Accessed July 30, 2023. https://developers.google.com/search/docs/fundamentals/how-search-works. Grotlüschen, A. and F. Bonna (2008). German-language Literature Review. In Organisation for Economic Co-operation and Development. Teaching, Learning and Assessment for Adults: Improving Foundation Skills (pp. 1-59). Paris: OECD Publishing. Ac-cessed August 26, 2018. http://www.oecd.org/education/ceri/40046802.pdf. Gübür, K. T. (2021). What is Semantic Search? How Does Semantic Search Affect SEO? Accessed August 1, 2023. https://www.holisticseo.digital/theoretical-seo/semantic-search. Guha, R., McCool, R., and Miller, E. (2003). Semantic search. Proceedings of the 12th international conference on World Wide Web, Budapest, May 20-24 (700-709). ACM Press: NY, New York. Haspelmath, M. and Sims, A. (2010). Understanding Morphology. 2nd Ed. London: Routledge. Hsieh, H. and Shannon, S. E. (2015). Three Approaches to Qualitative Content Analysis. Qualitative Health Research. 15(9): 1277-1288. Kiger, M. E. and Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher. 42(8), 846-854. Kopp, O. (2022a). What is semantic search: A deep dive into entity-based search. Accessed July 30, 2023. https://searchengineland.com/semantic-search-entity-based-search-388221. Kopp, O. (2022b). Google MUM update: What can SEOs expect in the future? Accessed July 30, 2023. https://searchengineland.com/google-mum-update-seo-future-383551. Köller, O. (2005). Formative Assessment in Classrooms: A Review of the Empirical German Literature. In Organisation for Economic Co-operation and Development (ed). Formative Assessment: Improving Learning in Secondary Classrooms (pp. 265-279). Paris: OECD Publishing. Accessed 05 October, 2018. http://www.edra.gr/pdf/9605021E.pdf Kracauer, S. (1952). The challenge of qualitative content analysis. Public Opinion Quarterly. 16 (4), pp. 631-642. Kromrey, H. (2001). Evaluation von Lehre und Studium – Anforderungen an Methodik und Design [Evaluation of teaching and studies – requirements for methodology and design]. In Spiel, C. (ed). Evaluation univeristärer Lehre – zwischen Qualitätsmanagement und Selbstzweck [Evaluation of university teaching – between quality management and autotelic] (pp. 21-60). Münster: Waxmann. Maier, U. (2011). Formative Leistungsmessung: Von einer Noten- zu einer Diagnosekultur [Formative performance measurement: from a culture of grades to one of diagnosis]. Schulmanagement. 3, pp. 22-24. McMillan, J. H., Hellsten, L., Kelly, I. W., Noonan, B. & Klinger, D. (2000). Classroom Assessment: Principles and Practice for Effective Standards-Based Instruction. Toronto: Pearson. Middlewood, D. & Cardno, C. (eds) (2001). Managing Teacher Appraisal and Performance: A Comparative Approach. London: Routledge/Falmer. Negi, Y. S. and Kumar, S. (2014). A Comparative Analysis of Keyword- and Semantic-Based Search Engines. In Mohapatra, D. P. and Patnaik, S. (eds). Intelligent Computing, Networking, and Informatics: Proceedings of the International Conference on Advanced Computing, Networking, and Informatics, Raipur, June 12-14 (727-736). Springer: New Delhi. Nowell, L. S, Norris, J. M., White, D. E. & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods. 16 (1), pp. 1-13. Online Etymology Dictionary (2023). Upgrade. Accessed 27 November, 2023. https://www.etymonline.com/word/upgrade. Organisation for Economic Co-operation and Development (2013). OECD Reviews of Evaluation and Assessment in Education. Synergies for Better Learning: An International Perspective on Evaluation and Assessment. Accessed 10 April, 2022. https://www.oecd.org/education/school/Evaluation_and_Assessment_Synthesis_Report.pdf. Popham, W. J. (2008). Transformative assessment. Alexandria, VA: ASCD. Robertson, C., Al-Moasseb, Z., Noonan, Z. & Boyle, J. G. (2021). The 3-D Skills Model: a Randomised Controlled Pilot Study Comparing a Novel 1–1 Near-Peer Teaching Model to a Formative OSCE with Self-regulated Practice. Medical Science Educator. 31, 1789-1801. Ross, M. (2011). Cultivating the Arts in Education and Therapy. Abingdon: Routledge. Roy, S., Modak, A., Barik, D. and Goon, S. (2019). An Overview of Semantic Search Engines. International Journal of Research and Review. 10 (6): 73-85. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science 18 (2): 118-144. Schmidt, C. (2018). Formatives Assessment als Aspekt professioneller diagnostischer Kompetenz im Kontext des Grundschulunterrichts [Formative assessment as aspect of professional diagnostic competency in primary school education]. In Miller, S., Holler-Nowitzki, B., Kottmann, B., Lesemann, S., Letmathe-Henkel, B., Meyer, N., Schroeder, R. & Velten, K. (eds). Profession und Disziplin: Grundschulpädagogik im Diskurs [Profession and discipline: discourse in primary school pedagogy] (pp. 158-163). Wiesbaden: Springer VS. Schmidt, C. (2019). Formatives Assessment in der Grundschule: Konzept, Einschätzungen der Lehrkräfte und Zusammenhänge [Formative Assessment at the Elementary Level: Concept, Considerations and Connections]. Wiesbaden: Springer VS. Scriven, M. (1967). The Methodology of Evaluation. In Tyler, R. W., Gagné, R. M. & Scriven, M. (eds). Perspectives of Curriculum Evaluation (pp. 39-83). Chicago, IL: Rand McNally. Schwartz, B. (2016). Google’s search knows about over 130 trillion pages. Accessed May 02, 2020. https://searchengineland.com/googles-search-indexes-hits-130-trillion-pages-documents-263378. Stahl, S. (2011). Investitionen in Fortbildung sind Investitionen in die Zukunft: Interview mit dem Kasseler Erziehungswissenschaftler Prof. Dr. Lipowsky [Investments in professional development are investments in the future: interview with Kassel-based educationist Prof. Dr. Lipowsky]. Bildung Bewegt. No. 13 (June), pp. 10-14. Stiggins, R. (2005). Assessment for learning defined. ETS/Assessment Training Institute's International Conference: Promoting Sound Assessment in Every Classroom. Port-land, Oregon, September 2005. Accessed 19 September, 2020. http://downloads.pearsonassessments.com/ati/downloads/afldefined.pdf. Straumberger, W. (2017). Wirksamkeit von Selbstdiagnose beim Üben [Effectiveness of Self-Assessment during Practice]. In Kortenkamp, U. & Kuzle, A. (eds). Beiträge zum Mathematikunterricht 2017 [2017 papers on mathematics instruction] (pp. 957-960). Münster: WTM-Verlag. Torrance, H. (2012). Formative assessment at the crossroads: conformative, deformative and transformative assessment. Oxford Review of Education. 38 (3), pp. 323-342. Treve, M. (2021). English for Academic Purposes (EAP) Lecturers' perceptions of formative assessment integration in the Thai EAP context. Journal of Language and Linguistic Studies. 17(2), 1096-1113. United Nations Population Fund (2020). Assessing the quality of developmental evaluations at UNFPA. UNFPA Evaluation Office. Accessed 10 April, 2022. https://www.unfpa.org/sites/default/files/admin-resource/DE_EQA_FINAL.pdf Waters, J. (2015). Snowball sampling: a cautionary tale involving a study of older drug users. International Journal of Social Research Methodology. 18 (4): 367-380. Williamson, K., Given, L. M. & Scifleet, P. (2018). Qualitative data analysis, 2nd Ed. In Williamson, K. & Johanson, G. (eds). Research Methods: Information, Systems, and Contexts (pp. 453-476). Cambridge, MA: Chandos Publishing.

University Staff: Request a correction | Centaur Editors: Update this record

University of Reading

CentAUR: Central Archive at the University of Reading

Accessibility navigation

Using semantic iterative keyword sampling for polyonymous concepts in interlingual educational research by example of ‘formative evaluation’ in German

Abstract/Summary

Page navigation

See also

Footer navigation