Theory of Change and public policies: An integrative review
DOI:
https://doi.org/10.18222/eae.v34.10019Keywords:
Education, Health, Evaluation Methods, Education PolicyAbstract
In order to investigate the use of the Theory of Change tool, this article collected data from the EbscoHost and Web of Science platforms. The results show a large number of studies on health and in the European continent. The results were diverse as far as definition, objective, construction, components, functions, advantages and limitations of using the tool. The best practices involve stakeholder participation, testing the Theory of Change, temporal division of results, causal links supported by evidence and the development of indicators. It is recommended that the tool is periodically updated since it is comprehensive and useful for assuring quality in the monitoring and evaluation of interventions.
Downloads
References
Abayneh, S., Lempp, H., Alem, A., Kohrt, B. A., Fekadu, A., & Hanlon, C. (2020). Developing a Theory of Change model of service user and caregiver involvement in mental health system strengthening in primary health care in rural Ethiopia. International Journal of Mental Health Systems, 14(1), Article 51. https://doi.org/10.1186/s13033-020-00383-6 DOI: https://doi.org/10.1186/s13033-020-00383-6
Agum, R., Riscado, P., & Menezes, M. (2015). Políticas públicas: Conceitos e análise em revisão. Agenda Política, 3(2), 12-42. https://doi.org/10.31990/10.31990/agenda.ano.volume.numero DOI: https://doi.org/10.31990/10.31990/agenda.ano.volume.numero
Aromatario, O., Van Hoye, A., Vuillemin, A., Foucaut, A., Pommier, J., & Cambon, L. (2019). Using theory of change to develop an intervention theory for designing and evaluating behavior change SDApps for healthy eating and physical exercise: The OCAPREV theory. BMC Public Health, 19(1), Article 1435. https://doi.org/10.1186/s12889-019-7828-4 DOI: https://doi.org/10.1186/s12889-019-7828-4
Balayah, Z., Khadjesari, Z., Keohane, A., To, W., Green, J. S. A., & Sevdalis, N. (2021). National implementation of a pragmatic quality improvement skills curriculum for urology residents in the UK: Application and results of ‘theory-of-change’ methodology. American Journal of Surgery, 221(2), 401-409. https://doi.org/10.1016/j.amjsurg.2020.12.007 DOI: https://doi.org/10.1016/j.amjsurg.2020.12.007
Barkat, S. (2019). Evaluating the impact of the Academic Enrichment Programme on widening access to selective universities: Application of the Theory of Change framework. British Educational Research Journal, 45(6), 1160-1185. https://doi.org/10.1002/berj.3556 DOI: https://doi.org/10.1002/berj.3556
Bauer, A., & Fernandes, F. S. (2022). Avaliação de projetos, programas e políticas educacionais. Estudos em Avaliação Educacional, 33, Artigo e08673. https://doi.org/10.18222/eae.v33.8673 DOI: https://doi.org/10.18222/eae.v33.8673
Breuer, E., De Silva, M. J., Shidaye, R., Petersen, I., Nakku, J., Jordans, M. J. D., Fekadu, A., & Lund, C. (2016). Planning and evaluating mental health services in low- and middle-income countries using theory of change. British Journal of Psychiatry, 208, 55-62. https://doi.org/10.1192/bjp.bp.114.153841 DOI: https://doi.org/10.1192/bjp.bp.114.153841
Breuer, E., Subba, P., Luitel, N., Jordans, M., De Silva, M., Marchal, B., & Lund, C. (2018). Using qualitative comparative analysis and theory of change to unravel the effects of a mental health intervention on service utilisation in Nepal. BMJ Global Health, 3(6). https://doi.org/10.1136/bmjgh-2018-001023 DOI: https://doi.org/10.1136/bmjgh-2018-001023
Chesniak, O. M., Drane, D., Young, C., Hokanson, S. C., & Goldberg, B. B. (2021). Theory of Change models deepen online learning evaluation. Evaluation and Program Planning, 88, Article 101945. https://doi.org/10.1016/j.evalprogplan.2021.101945 DOI: https://doi.org/10.1016/j.evalprogplan.2021.101945
Clarke, B., Doabler, C. T., Cary, M. S., Kosty, D., Baker, S., Fien, H., & Smolkowski, K. (2014). Preliminary evaluation of a Tier 2 mathematics intervention for first-grade students: Using a theory of change to guide formative evaluation activities. School Psychology Review, 43(2), 160-177. https://doi.org/10.1080/02796015.2014.12087442 DOI: https://doi.org/10.1080/02796015.2014.12087442
Daruwalla, N., Jaswal, S., Fernandes, P., Pinto, P., Hate, K., Ambavkar, G., Kakad, B., Gram, L., & Osrin, D. (2019). A theory of change for community interventions to prevent domestic violence against women and girls in Mumbai, India. Wellcome Open Research, 4, Article 54. https://doi.org/10.12688/wellcomeopenres.15128.1 DOI: https://doi.org/10.12688/wellcomeopenres.15128.1
De Buck, E., Hannes, K., Cargo, M., Van Remoortel, H., Vande Veegaete, A., Mosler, H., Govender, T., Vandekerckhove, P., & Young, T. (2018). Engagement of stakeholders in the development of a Theory of Change for handwashing and sanitation behaviour change. International Journal of Environmental Health Research, 28(1), 8-22. https://doi.org/10.1080/09603123.2017.1415306 DOI: https://doi.org/10.1080/09603123.2017.1415306
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014). Theory of Change: A theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials, 15(1). https://doi.org/10.1186/1745-6215-15-267 DOI: https://doi.org/10.1186/1745-6215-15-267
Dolan, J. E., Lonsdale, H., Ahumada, L. M., Patel, A., Samuel, J., Jalali, A., Peck, J., Derosa, J. C., Rehman, M., Varughese, A. M., & Fernandez, A. M. (2019). Quality initiative using theory of change and visual analytics to improve controlled substance documentation discrepancies in the operating room. Applied Clinical Informatics, 10(3), 543-551. https://doi.org/10.1055/s-0039-1693688 DOI: https://doi.org/10.1055/s-0039-1693688
Dugand, A. G., & Brandão, A. A. P. (2017). A Teoria da Mudança como ferramenta avaliativa do desenho dos programas sociais: O caso das ações estruturantes para Comunidades Quilombolas. Meta: Avaliação, 9(25), 110-140. https://revistas.cesgranrio.org.br/index.php/metaavaliacao/article/view/1214 DOI: https://doi.org/10.22347/2175-2753v9i25.1214
Esponda, G. M., Ryan, G. K., Estrin, G. L., Usmani, S., Lee, L., Murphy, J., Qureshi, O., Endale, T., Regan, M., Eaton, J., & De Silva, M. (2021). Lessons from a theory of change-driven evaluation of a global mental health funding portfolio. International Journal of Mental Health Systems, 15(1), Article 18. https://doi.org/10.1186/s13033-021-00442-6 DOI: https://doi.org/10.1186/s13033-021-00442-6
Fuhr, D. C., Acarturk, C., Sijbrandij, M., Brown, F. L., Jordans, M. J. D., Woodward, A., Mcgrath, M., Sondorp, E., Ventevogel, P., Ikkursun, Z., El Chammy, R., Cuijpers, P., & Roberts, B. (2020). Planning the scale up of brief psychological interventions using theory of change. BMC Health Services Research, 20(1), Article 801. https://doi.org/10.1186/s12913-020-05677-6 DOI: https://doi.org/10.1186/s12913-020-05677-6
Fuhr, D. C., Acarturk, C., Uygun, E., Mcgrath, M., Ilkkursun, Z., Kaykha, S., Sondorp, E., Sijbrandij, M., Ventevogel, P., Cuijpers, P., & Roberts, B. (2020). Pathways towards scaling up Problem Management Plus in Turkey: A theory of change workshop. Conflict and Health, 14(1), Article 22. https://doi.org/10.1186/s13031-020-00278-w DOI: https://doi.org/10.1186/s13031-020-00278-w
Gertler, P. J., Martínez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2018). Avaliação de impacto na prática (2a ed.). Grupo Banco Mundial. https://openknowledge.worldbank.org/bitstream/handle/10986/25030/9781464808890.pdf
Gilissen, J., Pivodic, L., Gastmans, C., Stichele, R. V., Deliens, L., Breuer, E., & Van den Block, L. (2018). How to achieve the desired outcomes of advance care planning in nursing homes: A theory of change. BMC Geriatrics, 18(1). https://doi.org/10.1186/s12877-018-0723-5 DOI: https://doi.org/10.1186/s12877-018-0723-5
Glasgow, L., Adams, E., Joshi, S., Curry, L., Schmitt, C. L., Rogers, T., Willett, J., & Van Hersh, D. (2017). Using a theory of change to guide grant monitoring and grantmaking. Journal of Public Health Management and Practice, 23(2), 126-130. https://doi.org/10.1097/PHH.0000000000000421 DOI: https://doi.org/10.1097/PHH.0000000000000421
Hailemariam, M., Fekadu, A., Selamu, M., Alem, A., Medhin, G., Giorgis, T. W., De Silva, M., & Breuer, E. (2015). Developing a mental health care plan in a low resource setting: The theory of change approach. BMC Health Services Research, 15(1), Article 429. https://doi.org/10.1186/s12913-015-1097-4 DOI: https://doi.org/10.1186/s12913-015-1097-4
Hamdani, S. U., Huma, Z., Suleman, N., Warraitch, A., Muzzafar, N., Farzeen, M., Minhas, F. A., Rahman, A., & Wissow, L. S. (2021). Scaling‐up school mental health services in low resource public schools of rural Pakistan: The Theory of Change (ToC) approach. International Journal of Mental Health Systems, 15(1), Article 8. https://doi.org/10.1186/s13033-021-00435-5 DOI: https://doi.org/10.1186/s13033-021-00435-5
Insper Metricis. (2022). Guia de avaliação de impacto socioambiental para utilização em projetos e investimentos de impacto: Guia geral com foco em monitoramento e verificação de adicionalidade (5a ed.). Insper. https://www.insper.edu.br/wp-content/uploads/2022/05/GUIA-AVALIACAO-DE-IMPACTO-SOCIOAMBIENTAL_PT.pdf
Jannuzzi, P. M. (2014). Avaliação de programas sociais: Conceitos e referenciais de quem a realiza. Estudos em Avaliação Educacional, 25(58), 22-42. https://doi.org/10.18222/eae255820142916 DOI: https://doi.org/10.18222/eae255820142916
Kabongo, E. M., Mukumbang, F. C., Delobelle, P., & Nicol, E. (2020). Combining the theory of change and realist evaluation approaches to elicit an initial program theory of the MomConnect program in South Africa. BMC Medical Research Methodology, 20(1), Article 196. https://doi.org/10.1186/s12874-020-01164-y DOI: https://doi.org/10.1186/s12874-020-01164-y
Krishnan, A. (2017). Theory of Change [PowerPoint slides]. J-Pal. https://www.povertyactionlab.org/sites/default/files/Lecture%202-THEORY%20OF%20CHANGE.pdf
Lalli, M., Ruysen, H., Blencowe, H., Yee, K., Clune, K., De Silva, M., Leffler, M., Hillman, E., El-Noush, H., Mulligan, J., Murray, J. C., Silver, K., & Lawn, J. E. (2018). Saving lives at birth; Development of a retrospective theory of change, impact framework and prioritised metrics. Globalization and Health, 14(1). https://doi.org/10.1186/s12992-018-0327-z DOI: https://doi.org/10.1186/s12992-018-0327-z
Levay, A. V., Chapman, G. E., Seed, B., & Wittman, H. (2018). It’s just the right thing to do: Conceptualizing a theory of change for a school food and beverage sales environment intervention and implications for implementation evaluation. Evaluation and Program Planning, 70, 73-82. https://doi.org/10.1016/j.evalprogplan.2018.04.011 DOI: https://doi.org/10.1016/j.evalprogplan.2018.04.011
MacQueen, K. M., Eley, N. T., Frick, M., & Hamilton, C. (2018). Using theory of change frameworks to develop evaluation strategies for research engagement: Results of a pre-pilot study. Journal of the International AIDS Society, 21, Article e25181. https://doi.org/10.1002/jia2.25181 DOI: https://doi.org/10.1002/jia2.25181
Makowiecka, K., Marchant, T., Betemariam, W., Chaturvedi, A., Jana, L., Liman, A., Mathewos, B., Muhammad, F. B., Semrau, K., Wunnava, S. S., Sibley, L. M., Berhanu, D., Gautham, M., Umar, N., Spicer, N., & Schellenberg, J. (2019). Characterising innovations in maternal and newborn health based on a common theory of change: Lessons from developing and applying a characterisation framework in Nigeria, Ethiopia and India. BMJ Global Health, 4(4). https://doi.org/10.1136/bmjgh-2019-001405 DOI: https://doi.org/10.1136/bmjgh-2019-001405
Mendes, G. S. C. V., & Sordi, M. R. L. de. (2013). Metodologia de avaliação de implementação de programas e políticas públicas. EccoS – Revista Científica, 30, 93-112. https://doi.org/10.5585/eccos.n30.3697 DOI: https://doi.org/10.5585/eccos.n30.3697
Mendes, K. D. S., Silveira, R. C. C. P., & Galvão, M. C. (2008). Revisão integrativa: Método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto & Contexto: Enfermagem, 17(4), 758-764. https://doi.org/10.1590/S0104-07072008000400018 DOI: https://doi.org/10.1590/S0104-07072008000400018
Nimpagaritse, M., Korachais, C., & Meessen, B. (2020). Effects in spite of tough constraints – A theory of change based investigation of contextual and implementation factors affecting the results of a performance based financing scheme extended to malnutrition in Burundi. PLoS ONE, 15(1). https://doi.org/10.1371/journal.pone.0226376 DOI: https://doi.org/10.1371/journal.pone.0226376
Rezel-Potts, E., Free, C., Syred, J., & Baraitser, P. (2020). Expanding choice through online contraception: A theory of change to inform service development and evaluation. BMJ Sexual and Reproductive Health, 46(2), 108-115. https://doi.org/10.1136/bmjsrh-2018-200297 DOI: https://doi.org/10.1136/bmjsrh-2018-200297
Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6(2), 134-147. https://doi.org/10.1177/1524839904273387 DOI: https://doi.org/10.1177/1524839904273387
Sloan, S., Gildea, A., Miller, S., Poulton, L., Egar, C., & Thurston, A. (2017). Evaluation of Zippy’s Friends for improving socio-emotional and academic outcomes in six to seven year olds: Protocol for a cluster randomised controlled trial and process evaluation. International Journal of Educational Research, 82, 200-209. https://doi.org/10.1016/j.ijer.2017.01.003 DOI: https://doi.org/10.1016/j.ijer.2017.01.003
Vieira, J. E. C., Ponti, M. A., Versuti, F. M., & Santos, D. D. dos. (2022). O uso da Teoria de Mudança para avaliação de programas educacionais e intersetoriais: Resultados preliminares de uma revisão integrativa. Brazilian Journal of Development, 8(4), 25183-25190. https://doi.org/10.34117/bjdv8n4-168 DOI: https://doi.org/10.34117/bjdv8n4-168
Walker, J. S. (2015). A Theory of Change for positive developmental approaches to improving outcomes among emerging adults with serious mental health conditions. Journal of Behavioral Health Services and Research, 42(2), 131-149. https://doi.org/10.1007/s11414-015-9455-x DOI: https://doi.org/10.1007/s11414-015-9455-x
Walker, J. S., & Matarese, M. (2011). Using a theory of change to drive human resource development for wraparound. Journal of Child and Family Studies, 20(6), 791-803. https://doi.org/10.1007/s10826-011-9532-6 DOI: https://doi.org/10.1007/s10826-011-9532-6
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating community initiatives: Concepts, methods and contexts (pp. 65-92). The Aspen Institute. https://canvas.harvard.edu/files/1453087/download?download_frd=1&verifier=IVZpf0ynt3iriSXpb8lE7WirRBXUHfbceDQUHleG
Yearwood, A. C. (2018). Applying a logical theory of change for strengthening research uptake in policy: A case study of the Evidence Informed Decision Making Network of the Caribbean. Revista Panamericana de Salud Publica, 42, Article e91. https://doi.org/10.26633/RPSP.2018.91 DOI: https://doi.org/10.26633/RPSP.2018.91
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Estudos em Avaliação Educacional
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish in this journal agree to the following terms:
a. Authors retain the copyright and grant the journal the right to first publication, with the paper simultaneously licensed under the Creative Commons Attribution license that allows the sharing of the paper with acknowledgment of authorship and initial publication in this journal.
b. Authors are authorized to assume additional contracts separately, for non-exclusive distribution of the version of the paper published in this journal (for example publishing in institutional repository or as a book chapter), with acknowledgment of authorship and initial publication in this journal.
c. Authors are allowed and encouraged to publish and distribute their paper on-line (for example in institutional repositories or on their personal page) at any moment before or during the editorial process, as this can generate productive changes, as well as increase the impact and citation of the published paper (See The Effect of Open Access).
