This work is increasingly important as the global development community works to reduce poverty and achieve the. Research process quantitative methods designs validity and research designs. Samad 2010 the international bank for reconstruction and development. Section iv addresses some general concerns about impact evaluation, introducing them as myths and explaining why impact evaluation is less complicated than often assumed. Qualitative and quantitative methods can usefully be combined in mixed methods research to provide more comprehensive information for an evaluation. Data collection and analysis methods in impact evaluation page 2 outputs and desired outcomes and impacts see brief no. The tools, while designed specifically for each type.
Impact assessment methods this document outlines several common impact assessment methods, including. The choice of methods and designs for impact evaluation of policies and. While quantitative research is based on numerical data analysed statistically, qual. Additional attention was drawn to the issue recently by a. The impact of the full range of development interventions can be assessed projects, programmes, sector wide support and donor country strategies and macro economic growth and programme aid support using qualitative or quantitative approaches or a mix. Utilising quantitative and qualitative approaches in impact evaluation.
Students will learn to critically analyze evaluation research and to gauge how convincing the research is in identifying a causal impact. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. The proper analysis of impact requires a counterfactual of what those outcomes would have been in the absence of the intervention. The book incorporates realworld examples to present practical guidelines for designing and implementing evaluations. Realism, subjectivism and the paradigm wars now we have defined quantitative research, it is a good idea to compare it with qualitative research, to which it is usually put in opposition. In response to this growing consensus there are a number of international initiatives on impact evaluation. Quantitative research, is defined as a the systematic investigation of phenomena by gathering quantifiable data and performing statistical, mathematical or computational techniques. Analyze quantitative and qualitative data using biostatistics, informatics, computerbased programming and software, as. Handbook on impact evaluation foro consultivo cientifico. Usaid definition of impact evaluation part 1 impact evaluations measure the change in a development outcome focused on higher level results outcomes rather than inputs and outputs e. Quantitative and qualitative methods in impact evaluation and measuring results 1 1 introduction there has been a renewed interest in impact evaluation1 in recent years amongst development agencies and donors. How to evaluate the programme world health organization.
Impact evaluations and development no nie guidance on impact evaluation. Research is a process of steps used to collect and analyze information in order to increase our understanding of a topic or issue. In particular it examines how qualitative methods could address some of the limitations of randomized trials and other quantitative impact evaluation methods. Department of education, institute of education sciences, national center for education evaluation and regional assistance, analytic technical assistance and development. By identifying if development assistance is working or not, impact evaluation is also serving the accountability function. Introduction to mixed methods in impact evaluation.
Using recent literature on the characteristics of such practices, we developed two tools to assess the methodological rigour and mixed methods integration of 40 ies and 7 srs, drawing upon previous approaches. The formal literature on impact evaluation methods and practices is large, with a few. Outline of principles of impact evaluation part i key concepts definition impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended. Quantitative and qualitative methods in impact evaluation. Hence impact evaluation is aligned with results based management3 and monitoring the contribution of development assistance toward meeting the. A particular concern is the accessibility of health services to poor and vulnerable sectors of the population. Although both experimental and nonexperimental methods are grounded in quantitative approach to evaluation, incorporating qualitative methods enriches the quality of the evaluation results. Choosing appropriate designs and methods for impact. Handbook on impact evaluation quantitative methods and practice.
Impact evaluation in practice is an essential resource for evaluators, social programs, ministries, and others committed to making decisions using good evidence. The methods used will depend on the aim and the budget for the evaluation. Using qualitative methods to assess impact jean ellis, ncvo charities evaluation services associate july 2015 1. Students do not have to actually evaluate, but must be able to identify the correct. The theory of change should also take into account any unintended positive or negative results. While impact evaluations can be based on both quantitative and qualitative methods, this note focuses primarily on quantitative methods and introduces.
Critically analyze impact evaluation research in economics and public health and gauge the validity of causal estimates understand evaluation design, including methods for designing randomizedcontrolled field trials learn how to develop meaningful hypotheses that are amenable to evaluation and test them. This note describes a number of commonly used impact evaluation. Needs assessment process evaluation impact evaluation to address the low literacy levels, the government will deliver free text books to schools to improve passing rates and increase employment prospects of school leavers. This book also details challenges and goals in other realms of evaluation. The formal literature on impact evaluation methods and practices is. Decision tree for selecting quantitative evaluation designs to deal. Understanding variation in treatment effects in education impact evaluations. Matching methods are gaining visibility in impact evaluation because they use rigorous statistical techniques to build a control unprotected group of sites for comparison that are as similar as. Our simplified and aggregate list includes 9 potential impacts of a food safety regulation.
Impact evaluation seeks to determine the longer term results that are generated by policy decisions, often through interventions, projects or programmes. Introduction voluntary sector organisations are being urged to provide more convincing evidence of their impact and with good reason, given that funders and commissioners want. Using an experimental or quasiexperimental design to demonstrate a change or not is the most powerful programme evaluation for detecting changes in outcome. C2960p november 19, 2018 evaluation briefs analyzing quantitative data for evaluation no. Descriptive statistics are performed by analyzing one variable at a time univariate analysis. The impact evaluation in practice handbook is a comprehensive and accessible introduction to impact evaluation for policymakers and development practitioners. There are several types of evaluations that can be conducted. Quantitative and qualitative methods in impact evaluation and measuring results. This guidance note provides an outline of a mixed methods impact evaluation with particular reference to the difference between this approach and qualitative and quantitative impact evaluation designs. Pdf on apr 2, 2017, abhisek mishra and others published methods of impact evaluation. Formative evaluation ensures that a program or program activity is feasible, appropriate, and acceptable before it. Impact evaluation methods for youth employment interventions. Learn more about quantitative research methods along with types and examples, characteristics and advantages. Outline of principles of impact evalation discussion dra.
Handbook on impact evaluation quantitative methods and. Research question evaluation design methods data to be collected, when. Measurement of impact of any developmental programme is a difficult task. Yet there is a need to put the theory into practice in a handson fashion for practitioners. A basic understanding of quantitative research methods would be helpful. All researchers perform these descriptive statistics before beginning any type of data analysis. Understanding variation in treatment effects in education. Impact evaluation methodologies impact evaluation workshop, lilongwe, dec. Quantitative and qualitative methods in impact evaluation gsdrc. Includes qualitative and quantitative data collection does not require advanced statistical methods.
Quantitative and qualitative methods in impact evaluation and measuring results this paper from the governance and social development resource centre grdrc provides a detailed discussion of using both qualitative and quantitative methods in impact evaluation. Impact evaluations in low, moderate, or heterogeneoustransmission settings will likely use plausibility designs, and methods highlighted by the framework include interrupted time series. Expert judgment quantitative physical and mathematical models cumulative impact assessment matrices and interaction diagrams rapid impact assessment matrix riam battelle environmental evaluation system. Impact evaluation provides the causal effect of a programme on the outcome. Types of evaluation centers for disease control and.
Impacts may be positive or negative, intended or unintended, direct or indirect. The problem with seeking to advance a qualitative method of impact analysis is that impact is a matter of causation and a non quantitative approach to design is apparently not well suited to the task of establishing causal relations. This approach to evaluation focuses on the practicalities of defining successful outcomes and success cases brinkerhoff, 2003 and uses some of the processes from theorydriven evaluation to determine the linkages, which may take the form of a logic model, an impact. As you move across this gradient, the usefulness, potential questions answered, and results garnered change categorically. Quantitative research methods nova southeastern university. In particular, qualitative methods not only provide qualitative measures of impact, but. Evaluations using quantitative impact evaluation approaches 71 6. Note 5 impact evaluation methods for youth employment. Also learn about primary and secondary quantitative research along with techniques and types of.
1486 1064 508 944 1167 471 905 496 396 618 1445 1395 390 1321 967 171 1492 1375 67 799 268 89 1387 865 107 886 1150 680 755 1496 813 34 918 977 1067 574 216 1248 607 1341 668