Differential functioning of PISA science items in Brazil and Japan
DOI:
https://doi.org/10.18222/eae.v28i68.4584Keywords:
Differential Item Functioning – DIFF, PISA, Scientific Literacy, Sciences.Abstract
The Programme for International Student Assessment (PISA) is an international comparative assessment program applied to samples of 15-year-old students. Together with other countries, Brazil and Japan have participated in this program since its first edition in 2000. DIF (Differential Item Functioning) analysis was used to identify factors that could explain performance differences in scientific literacy between Brazilian and Japanese students, in the 2006 edition. Based on the test results, this analysis showed patterns of differentiated effects. To identify the items that showed differential functioning between Brazil and Japan, we used the Bayesian integrated model. In addition to confirming this occurrence, this model may also explain the DIF. DIF was found in all covariates selected. However, differential functioning did not always favor either of the two countries. There are competencies that discriminate more against the Brazilian students and areas of application of the items that sometimes were easier for Brazilian and sometimes for Japanese students.Downloads
References
AGUIAR, Glauco. Estudo comparativo entre Brasil e Portugal, sobre diferenças nas ênfases curriculares de Matemática, a partir da análise do Funcionamento Diferencial do Item (DIF) do PISA 2003.
246f. Tese (Doutorado em Educação) – Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, 2008.
ANASTASI, Anne. Psychological testing. New York: MacMillan. 1988.
ANDRIOLA, Wagner. Descrição dos principais métodos para detectar o funcionamento diferencial dos itens (DIF). Psicologia: Refl e Crítica, Rio Grande do Sul, v. 14, n. 3, p. 643-652, 2001.
BARROSO, Marta; FRANCO, Creso. Avaliações educacionais: o PISA e o ensino de ciências. In: ENCONTRO DE PESQUISA EM ENSINO DE FÍSICA, 11.,2008, Curitiba. Anais... Curitiba, 2008. Disponível em: <http://www.if.ufrj.br/~marta/artigosetal/2008-epef11-PISA.pdf>. Acesso em: 27 jul. 2014.
BERBEROGLU, Giray. Differential item functioning (DIF) analysis of computation, word problem and geometry questions across gender and SES groups. Studies in Educational Evaluation, Great Britain, v. 21, n. 4, p. 439-456, 1995.
BRASIL. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Resultados nacionais – PISA 2006: Programa Internacional de Avaliação de Alunos (PISA). Brasília, DF: Inep, 2008.
CLAUSER, Brian; MAZOR, Kathleen. Using statistical procedures to identify differential item functioning test items. Educational Measurement: Issues and Practice, Philadelphia, v. 17, n. 1, p. 31-44, 1998.
COLE, Nancy. History and development of DIF. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential item functioning. Hilldsdale, NJ: Lawrence Erlbaum, 1993.
FERRER, Alejandro Tiana. Que variáveis explicam os melhores resultados nos estudos internacionais? In: AZEVEDO, Joaquim. Avaliação dos resultados escolares. Porto: ASA, 2003.
GAMERMAN, Dani; LOPES, Hedibert. Markov Chain Monte Carlo: Stochastic simulation for Bayesian inference. New York: Chapman & Hall / CRC, 2006.
GAMERMAN, Dani; SOARES, Tufi GONÇALVES, Flávio. Bayesian analysis in item response theory applied to a large-scale educational assessment. In: O’HAGAN, Anthony; WEST, Mike. The Oxford handbook of applied Bayesian analysis. New York: Oxford University, 2010. p. 624-652.
GIERL, Mark; BISANZ, Jeffrey; BISANZ, Gay; BOUGHTON, Keith. Identifying content and cognitive skills that produce gender differences in mathematics: a demonstration of the DIF analysis framework. Journal of Educational Measurement, Philadelphia, v. 40, n. 4, p. 281-306, 2003.
HAMBLETON, Ronald; SWAMINATHAN, H.; ROGERS, Jane. Fundamentals of Item Response Theory. Newburry Parks: Sage, 1991.
HOLLAND, Paul; THAYER, Dorothy. Differential item performance and the Mantel–Haenszel procedure. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential item functioning. Hilldsdale, NJ: Lawrence Erlbaum, 1988. p. 129-145.
LINN, Robert; DRASGOW, Fritz. Implications of the golden rule settiemernt for test construction. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential item functioning. Hilldsdale, NJ: Lawrence Erlbaum, 1993.
O’NEIL, Kathleen; McPEEK, Miles. Item and test characteristics that are associated with differential item functioning. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential item functioning. Hilldsdale, NJ: Lawrence Erlbaum Associates, 1993. p. 255-276.
PASQUALI, Luiz. Psicometria: teoria dos testes psicológicos. Brasília, DF: Prática, 2000.
SCHMITT, Alicia P.; BLEISTEIN, Carole A. Factors affecting differential item functioning for black examinees on scholastic aptitude test analogy items (ETS RR-87-23). Princeton, NJ: Educational Testing Service, 1987.
SCHMITT, Alicia; HOLLAND, Paul; DORANS, Neil. Evaluating hypotheses about differential item functioning. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential Item Functioning. Hilldsdale, NJ: Lawrence Erlbaum, 1993. p. 281-316.
SOARES, Tufi Utilização da Teoria de Resposta ao Item na produção de indicadores sócio-econômicos. Pesquisa Operacional, Rio de Janeiro, v. 25, n. 1, p. 83-112, jan./abr. 2005.
SOARES, Tufi GENOVEZ, Silene; GALVÃO, Ailton. Análise do Comportamento Diferencial dos Itens de Geografi estudo da 4a série avaliada no Proeb/Simave, 2001. Estudos em Avaliação Educacional, São Paulo, v. 16, n. 32, p. 81-110, jul./dez. 2005.
SOARES, Tufi GONÇALVES, Flávio; GAMERMAN, Dani. Na integrated Bayesian model for DIF analysis. Journal of Educational ad Behavioral Statistics, Washington, v. 34, n. 3, p. 348-377, Sep. 2009.
SWAMINATHAN, Hariharam; ROGERS, Jane. Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, Philadelphia, v. 27, p. 361-370, 1990.
THISSEN, David. IRTLRDIF v.2.0.b: software for the computation of the statistics involved in item response theory Likelihood-Ratio Tests for differential item functioning. 2001.
THISSEN, David; STEINBERG, Lynne; WAINER, Howard. Detection of differential item functioning using the parameters of item response models. In: HOLLAND, Paul W.; WAINER, Howard (Ed.). Differential Item Functioning. Hilldsdale, NJ: Lawrence Erlbaum, 1993. p. 67-114.
ZIMOWSKI, Michele F.; MURAKI, Eiji; MISLEVY, Robert J.; BOCK, R. Darrell. BILOG-MG: Multiple Group IRT Analysis and test maintenance for binary items. [Computer software]. Chicago: Scientifi Software International, 1996.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2017 Estudos em Avaliação Educacional
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish in this journal agree to the following terms:
a. Authors retain the copyright and grant the journal the right to first publication, with the paper simultaneously licensed under the Creative Commons Attribution license that allows the sharing of the paper with acknowledgment of authorship and initial publication in this journal.
b. Authors are authorized to assume additional contracts separately, for non-exclusive distribution of the version of the paper published in this journal (for example publishing in institutional repository or as a book chapter), with acknowledgment of authorship and initial publication in this journal.
c. Authors are allowed and encouraged to publish and distribute their paper on-line (for example in institutional repositories or on their personal page) at any moment before or during the editorial process, as this can generate productive changes, as well as increase the impact and citation of the published paper (See The Effect of Open Access).
