Improvement of the CT Puzzle Test for computational thinking assessment
DOI:
https://doi.org/10.18222/eae.v33.8938Keywords:
Problem-Solving, Learning Assessment, Assessment Instruments, Test ValidityAbstract
Computational thinking is a problem-solving skill that involves algorithmic thinking, pattern identification, abstraction, and the modularization of problems and their solutions. This subject has been incorporated into school curricula in several countries, including Brazil, but there are few instruments validated to measure it. The present article deals with searches for evidence of the validity of the CT Puzzle Test, a puzzle-based tool to diagnose computational thinking. Convergent validity was verified with another test that already has the evidence of its validity established. The results confirm the convergence between the instruments. In addition, the score calculation for the CT Puzzle Test items was improved and the validity of adopting puzzles as an assessment tool was evidenced.
Downloads
References
ARAUJO, N. de; PRIMO, T. T.; PERNAS, A. M. OntoScratch: ontologias para a avaliação do ensino de Pensamento Computacional através do Scratch. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 31., 2020, on-line. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2020. p. 1823-1832. DOI: https://doi.org/10.5753/cbie.sbie.2020.1823 DOI: https://doi.org/10.5753/cbie.sbie.2020.1823
BALL, M. A.; GARCIA, D. D. Autograding and feedback for snap! In: TECHNICAL SYMPOSIUM ON COMPUTING SCIENCE EDUCATION, 47., 2016, Memphis. Proceedings […]. New York: ACM, 2016. DOI: https://doi.org/10.1145/2839509.2850572
BITTENCOURT, H. R.; CREUTZBERG, M.; RODRIGUES, A. C. de M.; CASARTELLI, A. de O.; FREITAS, A. L. S de. Desenvolvimento e validação de um instrumento para avaliação de disciplinas na educação superior. Estudos em Avaliação Educacional, São Paulo, v. 22, n. 48, p. 91-113, jan./abr. 2011. DOI: https://doi.org/10.18222/eae224820111994 DOI: https://doi.org/10.18222/eae224820111994
BOCCONI, S.; CHIOCCARIELLO, A.; DETTORI, G.; FERRARI, A.; ENGELHARDT, K.; KAMPYLIS, P.; PUNIE, Y. Developing computational thinking in compulsory education: implications for policy and practice. Luxembourg: Publications Office of the European Union, 2016.
BRACKMANN, C. P. Desenvolvimento do pensamento computacional através de atividades desplugadas na educação básica. 2017. Tese (Doutorado em Informática na Educação) – Centro Interdisciplinar de Novas Tecnologias na Educação, Universidade Federal do Rio Grande do Sul, Porto Alegre, 2017.
BRACKMANN, C.; BOUCINHA, R.; ROMÁN-GONZÁLEZ, M.; BARONE, D.; CASALI, A. Pensamento computacional desplugado: ensino e avaliação na educação primária espanhola. In: CONGRESSO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 6., 2017, Recife. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2017. DOI: http://dx.doi.org/10.5753/cbie.wcbie.2017.982 DOI: https://doi.org/10.5753/cbie.wcbie.2017.982
BRENNAN, K.; RESNICK, M. New frameworks for studying and assessing the development of computational thinking. Vancouver: American Educational Research Association, 2012.
CORDENONZI, W. H.; DEL PINO, J. C.; CARDOSO, V. M. Analisando o desenvolvimento do Pensamento Computacional na disciplina Matemática Discreta. Revista Brasileira de Informática na Educação, Porto Alegre, v. 29, p. 880-902, 2021. DOI: http://dx.doi.org/10.5753/rbie.2021.29.0.880 DOI: https://doi.org/10.5753/rbie.2021.29.0.880
COUTO, N. E. R. Impacto da implantação de uma disciplina de pensamento computacional em estudantes. 2018. Dissertação (Mestrado em Computação Aplicada) – Universidade do Vale de Itajaí, Itajaí, SC, 2018.
CSTA. K–12 Computer Science Standards. New York: ACM, 2011.
DAGIENE, V.; FUTSCHEK, G. Bebras international contest on informatics and computer literacy: criteria for good tasks. Lecture notes in computer science. [S. l.]: Springer Berlin Heidelberg, 2008. p. 19-30. DOI: https://doi.org/10.1007/978-3-540-69924-8_2
FALKNER, N.; SOORIAMURTHI, R.; MICHALEWICZ, Z. Puzzle-based learning for engineering and computer science. Computer, [S. l.], v. 43, n. 4, p. 20-28, abr. 2010. DOI: https://doi.org/10.1109/ MC.2010.113 DOI: https://doi.org/10.1109/MC.2010.113
FEUERSTEIN, R. The Feuerstein Instrumental Enrichment Program. Jerusalem: ICELP Publications, 2006.
FRANÇA, C.; SILVA, C. G. Identificação de critérios para avaliação do pensamento computacional aplicado. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 31., 2020, on-line. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2020. p. 1493-1502. DOI: https://doi. org/10.5753/cbie.sbie.2020.1493 DOI: https://doi.org/10.5753/cbie.sbie.2020.1493
GONÇALVES, F. A. Um instrumento para diagnóstico do pensamento computacional. 2015. Dissertação (Mestrado em Computação Aplicada) – Universidade do Vale de Itajaí, Itajaí, SC, 2015.
GROVER, S. Systems of assessments for deeper learning of computational thinking in k-12. In: ANNUAL MEETING OF THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, 2015, Chicago, IL. Proceedings […]. Washington, DC: American Educational Research Association, 2015.
HINKLE, D.; WIERSMA, W.; JURS, S. Applied statistics for the behavioral sciences. Boston, MA: Houghton Mifflin, 2003.
INTERNATIONAL SOCIETY FOR TECHNOLOGY IN EDUCATION – ISTE; COMPUTER SCIENCE TEACHERS ASSOCIATION – CSTA. Computational thinking in k-12 teachers resources. 2nd. ed. [S. l]: ISTE: CSTA, 2011. Disponível em: http://code.intef.es/wp-content/uploads/2017/10/ct-teacher-resources_2ed-pdf.pdf. Acesso em: 2 ago. 2021.
KALELIOGLU, F.; GÜLBAHAR, Y.; KUKUL, V. A framework for computational thinking based on a systematic research review. Baltic Journal of Modern Computing, Riga, v. 4, n. 3, p. 583-596, 2016.
KONG, S. C.; ABELSON, H.; LAI, M. Introduction to computational thinking education. In: KONG, S. C.; ABELSON, H. (ed.). Computational Thinking Education. [S. l]: Springer Singapore, 2019. p. 1-10. DOI: https://doi.org/10.1007/978-981-13-6528-7_1
LANDIS, J. R.; KOCH, G. G. The measurement of observer agreement for categorical data. Biometrics, Bethesda, v. 33, n. 1, p. 159-174, 1977. DOI: https://doi.org/10.2307/2529310
MAIORANA, F.; GIORDANO, D.; MORELLI, R. Quizly: a live coding assessment platform for app inventor. In: IEEE BLOCKS AND BEYOND WORKSHOP, 2015, [S. l.]. Proceedings […]. [S. l.]: IEEE, out. 2015. p. 25-30. DOI: https://doi.org/10.1109/BLOCKS.2015.7368995 DOI: https://doi.org/10.1109/BLOCKS.2015.7368995
MERRICK, K. E. An empirical evaluation of puzzle-based learning as an interest approach for teaching introductory computer science. [S. l.]: IEEE Transactions on Education, 2010. DOI: https://doi.org/10.1109/TE.2009.2039217
MICHALEWICZ, Z.; MICHALEWICZ, M. Puzzle-based learning: introduction to critical thinking, Mathematics, and problem solving. Melbourne: Hybrid Publishers, 2008.
MORENO-LEÓN, J.; ROBLES, G.; ROMÁN-GONZÁLEZ, M. Dr. Scratch: automatic analysis of scratch projects to assess and foster computational thinking. RED-Revista de Educación a Distancia, Murcia, n. 46, p. 1-23, 2015.
MÜHLING, A.; RUF, A.; HUBWIESER, P. Design and first results of a psychometric test for measuring basic programming abilities. In: THE WORKSHOP IN PRIMARY AND SECONDARY COMPUTING EDUCATION, 2015, London. Proceedings […]. New York: ACM, 2015. DOI: https://doi.org/10.1145/2818314.2818320
OTA, G.; MORIMOTO, Y.; KATO, H. Ninja code village for scratch: Function samples/function analyser and automatic assessment of computational thinking concepts. In: IEEE SYMPOSIUM ON VISUAL LANGUAGES AND HUMAN-CENTRIC COMPUTING (VL/HCC), set. 2016, Cambridge, UK. Proceedings [...]. Cambridge, UK: IEEE, 2016. DOI: https://doi.org/10.1109/VLHCC.2016.7739695
PAPERT, S. Teaching children thinking. Cambridge, MA: Massachusetts Institute of Technology, 1971. Disponível em: https://dspace.mit.edu/handle/1721.1/5835. Acesso em: 2 ago. 2021.
PASQUALI, L. Validade dos Testes. Examen: Política, Gestão e Avaliação da Educação, v. 1, n. 1, p. 36, 1 set. 2017.
RAABE, A.; SANTANA, A. L. M.; ELLERY, N.; GONÇALVES, F. Um instrumento para diagnóstico do pensamento computacional. In: CONGRESSO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO, 6., 2017, Recife. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2017. DOI: http://dx.doi.org/10.5753/cbie.wcbie.2017.1172 DOI: https://doi.org/10.5753/cbie.wcbie.2017.1172
RAABE, A.; ZORZO, A.; BLIKSTEIN, P. Computação na educação básica: fundamentos e experiências. Porto Alegre: Penso, 2020.
ROBERTS, P.; PRIEST, H.; TRAYNOR, M. Reliability and validity in research. Nursing Standard, v. 20, n. 44, p. 41–45, 12 jul. 2006. DOI: https://doi.org/10.7748/ns2006.07.20.44.41.c6560
ROBLES, G.; HAUCK, J.; ROMÁN-GONZÁLEZ, M.; NOMBELA ALONSO, R.; GRESSE VON WANGENHEIM, C.; MORENO-LEÓN, J. On tools that support the development of computational thinking skills: Some thoughts and future vision. In: INTERNATIONAL CONFERENCE ON COMPUTATIONAL THINKING EDUCATION, 2018, Hong Kong. Proceedings […]. Hong Kong: The Education University of Hong Kong, 2018. p. 125-129.
RODRIGUES, M. M. M. Proposta de análise de itens das provas do Saeb sob a perspectiva pedagógica e psicométrica. Estudos em Avaliação Educacional, São Paulo, v. 17, n. 34, p. 43-78, maio/ago. 2006. DOI: https://doi.org/10.18222/eae173420062117 DOI: https://doi.org/10.18222/eae173420062117
RODRIGUEZ, C.; REIS, R. C. D.; ISOTANI, S. Recursos e estratégias para desenvolvimento e avaliação do Pensamento Computacional na escola. Tecnologias, Sociedade e Conhecimento, Campinas, SP, v. 4, n. 1, p. 59-81, dez. 2017. DOI: https://doi.org/10.20396/tsc.v4i1.14485 DOI: https://doi.org/10.20396/tsc.v4i1.14485
ROMÁN-GONZÁLEZ, M. Computational thinking test: Design guidelines and content validation. In: ANNUAL INTERNATIONAL CONFERENCE ON EDUCATION AND NEW LEARNING TECHNOLOGIES, 7., 2015, Barcelona. Proceedings […]. Barcelona: IATED, 2015.
ROMÁN-GONZÁLEZ, M. Codigoalfabetización y pensamiento computacional en educación primaria y secundaria: validación de un instrumento y evaluación de programas. 2016. Tese (Programa de Doctorado en Educación) – Universidad Nacional de Educación a Distancia, Madrid, 2016.
ROMÁN-GONZÁLEZ, M., MORENO-LEÓN, J., ROBLES, G. Complementary tools for computational thinking assessment. In: INTERNACIONAL CONFERENCE ON COMPUTATIONAL THINKING EDUCATION, 2017, Hong Kong. Proceedings […]. Hong Kong: The Education University of Hong Kong, p. 154-159, 2017.
ROMÁN-GONZÁLEZ, M.; MORENO-LEÓN, J.; ROBLES, G. Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In: KONG, S. C.; Abelson, H. (ed.). Computational Thinking Education. [S. l]: Springer Singapore, 2019. p. 79-98. DOI: https://doi.org/10.1007/978-981-13-6528-7_6
ROMÁN-GONZÁLEZ, M.; PÉREZ-GONZÁLEZ, J.-C.; JIMÉNEZ-FERNÁNDEZ, C. Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior, Amsterdam, v. 72, p. 678-691, 2017. DOI: https://doi.org/10.1016/j.chb.2016.08.047
ROMÁN-GONZÁLEZ, M.; PÉREZ-GONZÁLEZ, J.-C.; MORENO-LEÓN, J.; ROBLES, G. Can computational talent be detected? Predictive validity of the Computational Thinking Test. International Journal of Child-Computer Interaction, v. 18, p. 47-58, nov. 2018. DOI: https://doi. org/10.1016/j.ijcci.2018.06.004. DOI: https://doi.org/10.1016/j.ijcci.2018.06.004
SEITER, L.; FOREMAN, B. Modeling the learning progressions of computational thinking of primary grade students. In: ANNUAL INTERNATIONAL ACM CONFERENCE ON INTERNATIONAL COMPUTING EDUCATION RESEARCH, 9., 2013, San Diego. Proceedings […]. New York: ACM, 2013. p. 59-66. DOI: https://doi.org/10.1145/2493394.2493403
SMITH, L.; CORDOVA, J. Weighted primary trait analysis for computer program evaluation. Journal of Computing Sciences in Colleges, Evansville, v. 20, n. 6, p. 14-19, June 2005.
SRIKANT, S.; AGGARWAL, V. A system to grade computer programming skills using machine learning. In: ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, 20., 2014, New York. Proceedings […]. New York: ACM, 2014. p. 1887-1896. DOI: https://doi.org/10.1145/2623330.2623377 DOI: https://doi.org/10.1145/2623330.2623377
WANGENHEIM, C. G. V.; HAUCK, J. C. R.; DEMETRIO, M. F.; PELLE, R.; ALVES, N. D. C.; BARBOSA, H.; AZEVEDO, L. F. CodeMaster – Automatic assessment and grading of app inventor and snap! programs. Informatics in Education, Vilnius, v. 17, n. 1, p. 117-150, 2018. DOI: https://doi.org/10.15388/infedu.2018.08
WEINTROP, D.; WILENSKY, U. Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs. In: THE ELEVENTH ANNUAL INTERNATIONAL CONFERENCE, 2015, Omaha. Proceedings [...]. New York: ACM, 2015. p. 101-110. DOI: https://doi.org/10.1145/2787622.2787721 DOI: https://doi.org/10.1145/2787622.2787721
WING, J. M. Computational thinking. Communications of the ACM, New York, v. 49, n. 3, p. 33-35, 2006. DOI: https://doi.org/10.1145/1118178.1118215
ZUR-BARGURY, I.; PÂRV, B.; LANZBERG, D. A nationwide exam as a tool for improving a new curriculum. In: INNOVATION AND TECHNOLOGY IN COMPUTER SCIENCE EDUCATION CONFERENCE, 18., 2013, Canterbury. Proceedings [...]. New York: ACM, 2013. p. 267-272. DOI: https://doi.org/10.1145/2462476.2462479 DOI: https://doi.org/10.1145/2462476.2462479
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Estudos em Avaliação Educacional
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish in this journal agree to the following terms:
a. Authors retain the copyright and grant the journal the right to first publication, with the paper simultaneously licensed under the Creative Commons Attribution License (CC BY 4.0) license that allows the sharing of the paper with acknowledgment of authorship and initial publication in this journal.
b. Authors are authorized to assume additional contracts separately, for non-exclusive distribution of the version of the paper published in this journal (for example publishing in institutional repository or as a book chapter), with acknowledgment of authorship and initial publication in this journal.
c. Authors are allowed and encouraged to publish and distribute their paper on-line (for example in institutional repositories or on their personal page) at any moment before or during the editorial process, as this can generate productive changes, as well as increase the impact and citation of the published paper (See The Effect of Open Access).
