Search Reviews Become a Reviewer Suggest a book for review About Political ReviewNet Go back to Home Page

Review of: Program Evaluation. Forms and Approaches by John M. Owen and Patricia Rogers
Sage Publications, 1999.
312 pages. £47.50.
Click here to see all the reviews for this journal
  Reviewed by: Inger Marie Stigen
Norwegian Institute for Urban and Regional Research, Oslo
 
  Reviewed in: Public Administration  
  Date accepted online: 14/11/2001
Published in print: Volume 79, Issue 1, Pages 223-248
 

Reviews

In this new international edition of Program Evaluation. Forms and Approaches, Owen and Rogers seek to provide a conceptual framework for anchoring existing evaluation knowledge and practice. According to the authors (p. 2), they have adopted an eclectic view of the field in order to give the readers new directions for evaluation work.

Chapter 1 introduces a working description of evaluation and the logic of evaluation. One major idea of this chapter is that evaluation covers a broad range of activities, from negotiating an evaluation plan, via collecting and analysing evidence to produce findings, to the dissemination of findings, judgements and recommendations. The other major idea is that these key elements in the evaluation chain are linked together in a logic of evaluation. In the first chapter this logic is discussed with reference to impact evaluations. The major objective is to relate this logic to the other approaches of evaluation outlined in the book (p. 20). The ambition of the authors is not only to present an overview of up-to-date evaluation approaches, but also to offer an expanded and more integrated framework.

In chapter 2, Owens and Rogers focus on the ‘what’ question, as they classify possible objects for an evaluation into five categories: programmes, policies, organizations, products and individuals. Although it might seem obvious, experience shows that it is necessary to identify and define the evaluation object, the ‘what’ question, before the evaluator decides on appropriate models and approaches for the evaluation. The ‘why’ question, i.e. different evaluation forms and approaches, is outlined in chapter 3. Here Owen and Rogers classify evaluations into five categories or forms: proactive, clarificative, interactive, monitoring, and impact evaluation. The authors give a brief review of the different forms and link them to different approaches, such as objective-based needs-based or goal-free evaluation.

Chapters 4, 5 and 6 deal with the planning, data-collecting and analysing, and disseminating stages of the evaluation process. The authors devote special attention to the need for planning and communication skills in addition to data management skills in evaluations. Here they underline the difference between evaluations and other kinds of research. Evaluations are generally commissioned enquiries. This implies that evaluators have to communicate with stakeholders who have a special and legitimate interest in the project. Other kinds of research are usually motivated by the creation of new knowledge and the advancement of frontiers in a discipline or field, and the researchers are first and foremost accountable to the scientific community at large (pp. 64–5). This is a very important point, with both practical, professional and ethical implications for applied research which should not be underestimated. Obviously, a well-prepared tender from the commissioner and a well-negotiated plan which specify the various elements of the evaluation (i.e. the object, purpose, client/audiences, resources, key questions, data management, dissemination of findings, codes of behaviour, budget and timetable) are necessary in order to make realistic expectations of what to achieve and how to use the products of the evaluation. This is important because it helps to clarify the professional role of the evaluator. In chapter 5, Owen and Rogers discuss the question development and data-management stages of evaluations. They show how different styles of questions can be linked to different evaluation forms, and give a review of data management techniques and methodology. Data-techniques and methodology are, however, only superficially and summarily treated in this book. Interested readers will have to follow the authors’ advice to consult other texts.

In chapter 7, Owen and Rogers turn their attention to the financial and human resources needed in an evaluation process. They illustrate how resources provided for the evaluation could be wasted if the identification of key players, the working relations between them, their contributions within the evaluation and their information needs are not clarified during the negotiation and planning stage of an evaluation (p. 133ff). Key players are conceptually distinguished into evaluators and audiences/clients on the one hand, and insiders and outsiders on the other. The authors then suggest four configurations of insiders and outsiders taking into account the status of the evaluator and client (insiders for insiders, insiders for outsiders, outsiders for insiders, and outsiders for outsiders). I find this chapter, considered in isolation, to be the least interesting one in the book. Considered together with chapter 8, however, where Owen and Rogers present and discuss ethical codes of behaviour for evaluators, the configurations of insiders and outsiders in chapter 7 are useful and interesting. An American study, referred to in the book (p. 167), shows that almost two-thirds of the respondents had encountered ethical problems in their work, and most problems were located in the post-data management phase. These included conflicts over presentations of findings being at odds with the expectation of the clients, and disputes concerning disclosure of findings, ownership and misuse of the final report. These findings show that ethical standards represent a challenge to external evaluators, whether they are consultants or researchers. Commissioners, stakeholders and clients should, however, also pay attention to and discuss the ethical dilemmas evaluations may create.

In the last part of the book (chapters 9–13), the different forms of evaluation outlined in chapter 3 are discussed in greater detail, illustrated with examples from case studies of evaluations. These chapters represent an integrated discussion of orientation, typical issues, key approaches and methodologies linked to the different evaluation forms, presented in a well-structured and informative manner.

The book adresses both beginners and practioners, students as well as consultants and researchers. This aim is an ambitious one, but in my opinion, Owen and Rogers are quite successful in fulfilling their ambition. Readers with no or little knowledge of evaluation theory and evaluation practice get a well-structured textbook in evaluation forms, strategies and stages, illustrated with a lot of examples from evaluation projects in education and social services. Working at an institute dedicated to applied research in the social sciences, I find the authors’ broad and integrated perspective on evaluations interesting and stimulating, both on a practical and theoretical level.


Search Reviews Become a Reviewer Suggest a book for review About Political ReviewNet Go back to Home Page