Analyze Fiona’s case listed below. What are the ethical issues that you believe she is facing?
What are the “benefits and costs” from Fiona’s perspectives?
What are the “benefits and costs” from the agencies’ perspectives?
If you were Fiona, what would be your decision (conduct the evaluation in-house or contract out)? What is your rationale for this decision?
Appendix A: Fiona’s Choice: An Ethical Dilemma for a Program Evaluator
Fiona Barnes did not feel well as the deputy commissioner’s office door closed behind her. She walked back to her office wondering why bad news seems to come on Friday afternoons. Sitting at her desk, she went over the events of the past several days and the decision that lay ahead of her. This was clearly the most difficult situation that she had encountered since her promotion to the position of director of evaluation in the Department of Human Services.
Fiona’s predicament had begun the day before, when the new commissioner, Fran Atkin, had called a meeting with Fiona and the deputy commissioner. The governor was in a difficult position: In his recent election campaign, he had made potentially conflicting campaign promises. He had promised to reduce taxes and had also promised to maintain existing health and social programs, while balancing the state budget.
The week before, a loud and lengthy meeting of the commissioners in the state government had resulted in a course of action intended to resolve the issue of conflicting election promises. Fran Atkin had been persuaded by the governor that she should meet with the senior staff in her department, and after the meeting, a major evaluation of the department’s programs would be announced. The evaluation would provide the governor with some post-election breathing space. But the evaluation results were predetermined—they would be used to justify program cuts. In sum, a “compassionate” but substantial reduction in the department’s social programs would be made to ensure the department’s contribution to a balanced budget.
As the new commissioner, Fran Atkin relied on her deputy commissioner, Elinor Ames. Elinor had been one of several deputies to continue on under the new administration and had been heavily committed to developing and implementing key programs in the department, under the previous administration. Her success in doing that had been a principal reason why she had been promoted to deputy commissioner.
On Wednesday, the day before the meeting with Fiona, Fran Atkin had met with Elinor Ames to explain the decision reached by the governor, downplaying the contentiousness of the discussion. Fran had acknowledged some discomfort with her position, but she believed her department now had a mandate. Proceeding with it was in the public’s interest.
Elinor was upset with the governor’s decision. She had fought hard over the years to build the programs in question. Now she was being told to dismantle her legacy—programs she believed in that made up a considerable part of her budget and person-year allocations.
In her meeting with Fiona on Friday afternoon, Elinor had filled Fiona in on the political rationale for the decision to cut human service programs. She also made clear what Fiona had suspected when they had met with the commissioner earlier that week—the outcomes of the evaluation were predetermined: They would show that key programs where substantial resources were tied up were not effective and would be used to justify cuts to the department’s programs.
Fiona was upset with the commissioner’s intended use of her branch. Elinor, watching Fiona’s reactions closely, had expressed some regret over the situation. After some hesitation, she suggested that she and Fiona could work on the evaluation together, “to ensure that it meets our needs and is done according to our standards.” After pausing once more, Elinor added, “Of course, Fiona, if you do not feel that the branch has the capabilities needed to undertake this project, we can contract it out. I know some good people in this area.”
Fiona was shown to the door and asked to think about it over the weekend.
Fiona Barnes took pride in her growing reputation as a competent and serious director of a good evaluation shop. Her people did good work that was viewed as being honest, and they prided themselves on being able to handle any work that came their way. Elinor Ames had appointed Fiona to the job, and now this.
Analyze this case and offer a resolution to Fiona’s dilemma. Should Fiona undertake the evaluation project? Should she agree to have the work contracted out? Why?
In responding to this case, consider the issues on two levels: (1) look at the issues taking into account Fiona’s personal situation and the “benefits and costs” of the options available to her and (2) look at the issues from an organizational standpoint, again weighing the “benefits and the costs.” Ultimately, you will have to decide how to weigh the benefits and costs from both Fiona’s and the department’s standpoints.
Abercrombie, M. L. J. (1960). The anatomy of judgment: An investigation into the processes of perception and reasoning. New York: Basic Books.
Altschuld, J. (1999). The certification of evaluators: Highlights from a report submitted to the Board of Directors of the American Evaluation Association. American Journal of Evaluation, 20(3), 481–493.
American Evaluation Association. (1995). Guiding principles for evaluators. New Directions for Program Evaluation, 66, 19–26.
American Evaluation Association. (2004). Guiding principles for evaluators. Retrieved from http://www.eval.org/Publications/GuidingPrinciples…
Ayton, P. (1998). How bad is human judgment? In G. Wright & P. Goodwin (Eds.), Forecasting with judgement (pp. 237–267). Chichester, West Sussex, UK: John Wiley.
Bamberger, M., Rugh, J., & Mabry, L. (2012). Real world evaluation: Working under budget, time, data, and political constraints (2nd ed.). Thousand Oaks, CA: Sage.
Basilevsky, A., & Hum, D. (1984). Experimental social programs and analytic methods: An evaluation of the U.S. income maintenance projects. Orlando, FL: Academic Press.
Berk, R. A., & Rossi, P. H. (1999). Thinking about program evaluation (2nd ed.). Thousand Oaks, CA: Sage.
Bickman, L. (1997). Evaluating evaluation: Where do we go from here? Evaluation Practice, 18(1), 1–16.
Brandon, P., Smith, N., & Hwalek, M. (2011). Aspects of successful evaluation practice at an established private evaluation firm. American Journal of Evaluation, 32(2), 295–307.
Campbell Collaboration. (2010). About us. Retrieved from http://www.campbellcollaboration.org/about_us/inde…
Campbell, D. T. (1991). Methods for the experimenting society. Evaluation Practice, 12(3), 223–260.
Canadian Evaluation Society. (2012a). CES guidelines for ethical conduct. Retrieved from http://www.evaluationcanada.ca/site.cgi?s=5&ss=4&_…
Canadian Evaluation Society. (2012b). Program evaluation standards. Retrieved from http://www.evaluationcanada.ca/site.cgi?s=6&ss=10&…
Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, & Social Sciences and Humanities Research Council of Canada. (2010). Tri-council policy statement: Ethical conduct for research involving humans, December 2010. Retrieved from http://www.pre.ethics.gc.ca/pdf/eng/tcps2/TCPS_2_F…
Chen, H. T., Donaldson, S. I., & Mark, M. M. (2011). Validity frameworks for outcome evaluation. New Directions for Evaluation, 2011(130), 5–16.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Chicago, IL: Rand McNally.
Cook, T. D., Scriven, M., Coryn, C. L., & Evergreen, S. D. (2010). Contemporary thinking about causation in evaluation: A dialogue with Tom Cook and Michael Scriven. American Journal of Evaluation, 31(1), 105–117.
Cooksy, L. J. (2008). Challenges and opportunities in experiential learning. American Journal of Evaluation, 29(3), 340–342.
Cronbach, L. J. (1980). Toward reform of program evaluation (1st ed.). San Francisco, CA: Jossey-Bass.
Cronbach, L. J. (1982). Designing evaluations of educational and social programs (1st ed.). San Francisco, CA: Jossey-Bass.
Epstein, R. M. (1999). Mindful practice. Journal of the American Medical Association, 282(9), 833–839.