Evaluation Research

Dr. David Garson of North Carolina State University provides an informative website at ncsu.edu. One of the topics he included in Multivariate Analysis is the Evaluation Research. He started with an overview stating that, “Evaluation research seeks to provide objective assessments of past, present, or proposed programs of action”. Here is a summary of Dr. Garson’s article.

There are three basic types of question asked in evaluation research – descriptive, normative, and impact. Descriptive studies are used to describe the goals, objectives, start-up procedures, implementation processes, and anticipated outcomes of a program. Normative studies evaluate programs by multiple values while impact studies evaluate in terms of outcomes.

The four major types of evaluation designs are survey research, case study, field experiment, and secondary analysis of archival data. Both survey and case studies are primarily used in descriptive and normative questions. Field experiments, which are used primarily around impact questions, have three main types – true, nonequivalent comparison, and before-and-after studies. Secondary data analysis is useful for all three types of research questions.

Some research examples cited are pilot studies, cost-benefit analysis, performance reference model, program assessment rating tool, needs assessment, and feasibility studies. Pilot studies are “reality check” trial runs of a full blown research. Cost-benefit analysis deals with measuring costs or benefits. It may sound straight-forward and simple, but a lot of questions arise from this analysis. Benefits may be tangible or intangible. The return of investment is traditionally used for the cost-benefit analysis of tangible benefits. On the other hand, intangible benefits cannot be adequately measured. There is also performance reference model, which is a framework for evaluation developed by the Office of Management. It uses four types of evaluation measure – mission and business results, customer results, processes and activities, and technology.

Another approach to evaluation developed by the Office of Management is Program Assessment Rating Tool, which is designed to assess and improve program performance through a formal review process which was meant to identify a program's strengths and weaknesses.

Needs assessment serves to identify organizational problems (both internal and arising from the organization's environment), to investigate opportunities for change, and to document and defend projects which are proposed to address problems. Needs assessments assure that projects are directed to real problems faced by the organization rather than wasting scarce resources. Needs assessment process is implemented in four stages – collecting information, identifying and prioritizing problems researching alternative possible solutions, and seeking consensus on a proposed solution.

Lastly, there is feasibility study. It is normally undertaken only after the organization has identified and defined a proposed project, and has a written requirements analysis and a general design statement for it. It has three dimensions – operational, economic, and technical.


In general, the article provides substantial information on evaluation research especially for the business sector. It provides specific examples such as questions regarding measurement of benefits and questions regarding measurement of costs. Furthermore, it also compares and contrasts one type of evaluation research from the other. For example, aside from describing feasibility studies, it makes a distinction between feasibility studies from needs assessments.

There are technical business terms used such as cost-benefit, potential value, and present value that might hinder a non-business oriented person from understanding the text. Nevertheless, educators can relate to some of the evaluation research designs such as pilot studies, survey research, and field experiment. An educator, however, will find the article more helpful if the examples given are centered on education research.

Furthermore some organizations cited are not found locally; thus, their standards may not be applicable in our field. It would be beneficial if there is an academic site featuring evaluation research standards in the Philippines.

On the bright side, the article opens a new window for us to expand our knowledge on research. Dr. Garson has done a commendable website, which is informative not only in North Carolina State University but in the research communities worldwide.

Reference:
Garson, G. David (n.d.). "Evaluation Research", from Statnotes: Topics in
Multivariate Analysis. Retrieved 12/28/2008 from
http://www2.chass.ncsu.edu/garson/pa765/statnote.htm.