LEE E. SECHREST

Book Review

ENVIRONMENT AND BEHAVIOR

 

Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability. Edited by David M. Fetterman, Shakeh J. Kaftarian, & Abraham Wandersman. Thousand Oaks, CA: Sage, 1996, pp. xii + 411.

This book, Empowerment Evaluation, is of considerable merit. It is replete with excellent ideas about evaluation that should be of interest to, and taken seriously by, almost any program evaluator. Editors of the book have distinguished records in the field of program evaluation, and Fetterman is primarily responsible for the term empowerment evaluation. The book is a collection of chapters written by numerous authors who are abundantly redundant in their assurances of concern for "the people," the disadvantaged and downtrodden, and in their convictions that people can solve their own problems it they are simply empowered to do so and that conventional, or "traditional" approaches to evaluation are unlikely to be of much help.

Any reasonably thoughtful evaluator of whatever persuasion who reads Empowerment Evaluation will come away from it with increased awareness of the need to get program recipients directly involved in program planning and evaluation. The evaluatar, even if traditional, will also benefit from a range of useful ideas about how, specifically, to get program recipients involved. The book even includes materials such as "toolboxes," worksheets, and assessment instruments. In general, however, the book is considerably more inspirational than prescriptive, and an evaluator who is not already part of the empowerment evaluation movement will need a great deal more reading or tutoring even to plan initially, let alone to carry out, an empowerment evaluation enterprise, at least as set forth.

Make no mistake about it&emdash;empowerment evaluation is a movement, not simply an approach or strategy. It is frankly ideological: Several chapters refer specifically to the disadvantaged as the focus of concern for empowerment, and Mithaug's chapter, "Fairness, Liberty, and Empowerment Evaluation," is an attempt to provide a philosophical justification for it. There is nothing about empowerment evaluation as an idea or an approach that necessarily constrains its use in such a way. For example, one of my acquaintances was once approached by a business executive who wanted help because he said he felt "powerless" in having to deal with all the people he was having to fire in the downsizing of his company. He wanted, in effect, to be empowered to fire them without remorse despite any of their demands for decency or pleas for mercy. I, myself, once had dealings with a police organization that wanted

BOOK REVIEW 422

 

BOOK REVIEW 423

 

help because they felt powerless to deal with civilian review boards that were looking into allegations of police misconduct. Empowerment evaluation could be employed on behalf of almost any group, even those that might ordinarily be regarded as privileged, a term used several times in the book. It seems scarcely likely that the authors would be interested. They are advocates for the disadvantaged, and empowerment evaluation is seen by them as a way for enhancing the power of those groups. In one chapter (Fawcett et al.) in fact, the question of who should be empowered is made explicit and is couched in terms of ethics. A specific contrast is drawn between violent youths and their families, on one hand, and those who fear and resent them on the other. That is, to make it clear, some empowerment evaluators might regard working on behalf of some clients as inherently unethical. Perhaps not everyone would agree on whether it is more ethical to work to empower violent youths or those who fear them, although the context of the example makes plain the authors' antipathy toward those who are so misguided as to resent violent youths.

Thus the book is largely silent on the interests of stakeholders, whose interests might be seen by the disadvantaged as at odds with their own. For example, except for the chapter by Millett, who represents a foundation that supports social programs, there is scarcely a mention of those who pay for programs for the disadvantaged, for example, agencies and taxpayers. The book is written as if it were sufficient that program recipients be satisfied with what is happening to them whether those who provide the funds to make things happen are satisfied or not. No mention is made of what action is required if a program is satisfying to recipients because they like the immediate benefits it brings but is dissatisfying to sponsors because there is no evidence of any longer range benefit.

Empowerment Evaluation makes virtually no mention of what "traditional" evaluators would think of as program outcomes or impact. In fact, save for one reference to an evaluation of a program called Accelerated Schools (Levin), I found no mention of any outcome in the form of a demonstration that people were better off in their lives in any way at all, that crime decreased, that housing actually improved, and so on. The story of empowerment evaluation, at least as represented in this book, seems to end at the point at which people are empowered to decide whether what is happening to them is something they like and then to go on to evaluate other programs in the same way. The Accelerated Schools example is an interesting one because Levin states that "Evaluations of Accelerated Schools have shown substantial gains in student achievement and attendance ....These evaluations have included multiyear assessments in which Accelerated Schools have been compared to control schools. (p. 52). That is, the Accelerated Schools program seems to have been evaluated by quite traditional methods, which is the substantiation of its claim to be worthy of emulation. Fetterman appears to have designated the Accelerated Schools program as a model of empow-

 

424 ENVIRONMENT AND BEHAVIOR / May 1997

erment evaluation, even though it was not in any explicit way guided by the philosophy of empowerment evaluation.

Although Fetterman claims in his opening chapter that empowerment evaluation employs both qualitative and quantitative methods and that it does not require "significant compromises . . . in the rigor required to conduct evaluations," (p. 21), Empowerment Evaluation is virtually devoid of discussions of specific research methodology such as might be"required to conduct evaluations," and the very few references that do occur are disconcerting. There is a statement about an "emergent design," but with no indication what the design emerged to be (p. 281). Mention is made of the fact that an Indian group rejected survey methods as too intrusive, leading to a substitute approach, but no indication is given of the alternative so that one might judge whether it was a reasonable one. (Empowerment evaluators might well say that the alternative was reasonable if it was acceptable to the tribe.) Readers who are not committed to the empowerment approach or who do not understand the basis for the conclusions it produces will want to how what methods are used to what end if those of traditional evaluation are abandoned. Where is the evaluation?

Unfortunately, the empowerment evaluation movement at least appears to be at least disdainful of, if not actively hostile to, traditional evaluation, which is to say, science and the scientific method. In Empowerment Evaluation the term scientific nor anything related to it is scarcely ever used without some pejorative modifier such as strict, rigid, or inflexible. For example, consider the following items:

"strict methodologists" (p. 71 )

"rigid standards of reliability and validity" (p. 71 )

"heavy-handed, prescriptive, inflexible approach to evaluation" (p. 72)

"rather than imposing evaluation and remediation from the outside, empowerment evaluation encourages self-evaluation and self-adjustment from the inside. It creates user-friendly environments by motivating improvement rather than fomenting user-hostile environments that create defensiveness" (p. 235)

"validity anxiety" (p. 280)

"alleged precision that counting or scoring events that form a dependent variable allegedly provides" (p. 333) (Two "alleges" in the same sentence!)

In several places the authors make a point of the idea that empowerment evaluation "builds on strengths rather than focusing on weaknesses," which many readers may take as implying that other evaluation does focus on weaknesses and ignore strengths.

Traditional evaluators may not recognize themselves as they are portrayed in Empowerment Evaluation. For example, it is said that "Traditional evalu-

BOOK REVIEW 425

 

ation tends not to pay much attention to the real voices of real people" (p. 333). Traditional evaluation is conducted not "with the community, (but) on the community" and is "paternalistic and patronizing." The contrast to empowerment evaluation appears to be an outside evaluation group that arrives on the scene with a rigid (rigorous) prepost randomized experimental design requiring measures that are unacceptable to the persons subjected to the evaluation but that are irrelevant anyway and yielding results that are analyzed by a significance-test crucible with the end result that program recipients lose some valuable benefits. It has been a long time since I have seen an evaluation done in that way if, indeed, I ever saw one.

One at the chapters notes that it is not always clear whether empowerment is a process (becoming empowered) or a state (having power). The various chapters discussing and illustrating empowerment are very much oriented toward the process of empowerment. Apparently that process is one that requires guidance from outsiders, that is, evaluators. Although the various authors point out repeatedly that the people know best, seemingly the people can bring out that knowledge only with assistance from empowerment evaluators.

One has to ask just what it is people are being empowered to do. Implicit in much of the discussion is the idea that people are being empowered to solve their own problems, to take responsibility for their own actions. It seems scarcely likely, however, that these authors would want to press that idea too far, for in many places, the authors also imply, or even state directly, that empowerment is aimed at disadvantaged groups. They go on to suggest, even if very often indirectly, that empowerment is to include the capacity to demand that other people, for example, the government or broader society, take responsibility for the problems of the disadvantaged. Empowerment, then, consists in part of gaining the strength to demand help. (Nowhere is there even a suggestion that people might be empowered to endure hardship.)

In fact, aside from the ideological cant that pervades Empowerment Evaluation and the movement that appears to undergird it, I like the ideas in the book very much. A principal reason is that I find, like Moliere's doctor, that my students and I have been doing empowerment evaluation for quite a long while without ever knowing it. In fact, we appear to go even beyond empowerment evaluation in some respects because we usually try, in addition, to find out whether the program and the evaluation have actually made any difference in people's lives.

Which leads to a final comment that lies at the heart of some of my reservations about empowerment. Specifically, empowerment is never really defined, and it admits of diverse interpretations. As the writers admit, empowerment may sometimes be a process or it may be an outcome. Empowerment may sometimes amount to little more than granting of permission; for example, "You are empowered to solve your own problems, so go do it." It appears in some instances to involve gaining power to demand resources from others,

426 ENVIRONMENT AND BEHAVIOR / May 1997

that is, to demand that other persons solve one's problems, perhaps in the name of "fairness," but empowered dependency nonetheless. In still other instances, empowerment is discussed in terms of capacity to conduct evaluations and arrive at conclusions concerning program improvements or embellishments, to be brought about in unspecified ways.

The book contains statements that can only lead a careful reader to assume that it is more important to be empowered than to be right. Running through the various chapters is the idea that less than "rigorous" research should produce information "good enough" for the disadvantaged groups that are to be empowered. So, presumably, it may be sufficient for a disadvantaged set of parents to believe that their school program is effective, based on "evaluation methods that fit their nature and level of sophistication," (p. 66) whether such a conclusion might be more "objectively (a term unpleasant to empowerment evaluators) warranted. Two of the chapter authors (Fetterman and Millett), at least, recognize the problem that may exist in adjudicating between the desires of people for quick and easy answers and the more stringent demands imposed by the imperative to be right. Empowerment evaluation has some work ahead of it.

 

-- Lee E. Sechrest

University of Arizona

 

 

ENVIRONMENT AND BEHAVIOR, Vol. 29 No. 3, May 1997, 422-426

© 1997 Sage Publications, Inc.