PROGRAM PLANNING, IMPLEMENTATION, & EVALUATION

• An underlying theoretical and logical rationale that is the foundation for the program’s design.
• A process of implementation that makes the program operational.
• An expected impact.

Course Objectives.

• Understand and apply the theory and logic of evaluation research.
• Review analytic methods and apply them to assessing implementation.
• Extend familiarity with using social science methods to assess impact.
• Expand skills of critical thinking and analysis.

Required Books:

• Berk, Richard A. & Peter H. Rossi (1999).  Thinking About Program Evaluation (2nd Edition).  Newbury Park:  Sage.
• Patton, Michael Quinn (1997).  Utilization-Focused Evaluation (3rd Edition).  Thousand Oaks:  Sage.
• Rossi, Peter H., Howard E. Freeman, Mark W. Lipsey  (1999).  Evaluation:  A Systematic Approach (6th Edition).  Thousand Oaks:  Sage.

 Grades will be assessed according to the following scale (established MCJ scale):

A = 95-100 C = 75-79 B+ = 90-94 D+ = 70-74 B = 85-89 F = 69 and below C+ = 80-84

Course Requirements:  Students will be graded on the following assignments:
 

Class preparation and participation	15%
Evaluation Exercises	10%
Journal article review (Separate Handout)	20%
Evaluation Plan (Separate Handout)	35%
Class Presentation	20%

Attendance and participation.  These are essential aspects of any graduate seminar, with attendance being particularly important in the compressed format being used for this class.  Attendance at every class is required and every one is expected to be an active participant in the seminar discussions.

Evaluation exercises.  At the end of each Saturday session I give you a set of questions and problems that cover that weekend’s topics.  These exercises will be due in writing approximately one week later.  These assignments may be turned in one of three ways:  fax, as an email attachment (please let me know if you use a word processing program other than Word or WordPerfect), or in hard copy delivered to my office.

There will be separate handouts explaining the article review (this handout also covers the presentation) and evaluation plan that will be discussed in detail during the first class meeting.  Nevertheless, let me explain briefly what each entails.

Journal Article Summary.  This assignment requires that you find an article published in a peer reviewed journal that presents program evaluation results.  You will then write a 4-5 page essay that reviews and critiques the article by using the information covered in the class.

Evaluation Plan.  This is the major seminar paper for the class.  You will need to find a description of a specific program and write an evaluation plan that explains how you would design an evaluation of the program.  {Please note that you will NOT actually carry out the evaluation.  Rather you will describe how you would evaluate the program if you had the opportunity.}

Late Assignments.  Late assignments will be accepted only if prior arrangements are agreed upon or the lateness resulted from an emergency beyond your control.  Computer problems such as, among other things, accidentally erased computer files, damaged disks, malfunctioning printers, careless roommates who delete files, etc. are not considered emergencies.  The computer age we all live in requires planning and caution, so take steps to protect your work by making back-up files, keeping computer disks away from harsh elements, and not waiting until you are on your way to class to print an assignment.

Conduct.  Please refer to your Student Handbook for Loyola University’s rules of conduct.  Academic study rests on principles of respect for people, their ideas, and their right to learn.  Behavior that is rude, disrespectful, or disruptive not only violates school rules, but also violates individual dignity and interferes with the educational process.  I encourage disagreement with and the questioning of ideas, but these must be done respectfully in the spirit of furthering everyone’s learning and promoting open discussion.  This is the case whether the idea expressed comes from the reading, the professor, a guest speaker, or a student in the class.
In addition, please turn off cell phones and pagers before class begins.  If you work at a profession that requires you to keep your beeper or phone on, please sit near the door, and if you get called, please take and complete the call in the hallway.
Since universities rely on individual honesty misdeeds such as cheating, plagiarism, or other misrepresentations and fabrications cannot be tolerated.  Dishonesty in any form is unfair to others, embarrasses the individual who behaves in such a manner, and disrespects the educational process.  Please do not do something that will sully the semester for you and for me.  Violations of Loyola’s honesty and behavior code will result in an F for the course and will be handled according to procedures explained in the Handbook.

Disabilities.  I recognize that some students might have disabilities that do not reflect on their intelligence or capacity to learn, but present special challenges to learning.  I am very willing to work with students who need some kind of accommodation.  Please understand that if you confront a situation that requires accommodation because of a disability, it is your responsibility to notify Loyola’s Disability Resource Center (Monroe Hall, 4th floor)) and me to make suitable arrangements at the beginning of the semester.

Teaching Philosophy & Approach.  I incorporate several principles into my teaching.  Each is briefly explained below:

1. Multiple methods facilitate learning.  The class combines lecture, discussion, reading, writing, and independent research to enhance learning.  Individuals learn differently, but whatever our strengths and learning styles, multiple learning methods reinforce each other to amplify learning.
2. Collaboration.  I encourage students to study together if they wish and use each other as resources.  Also, I encourage students to talk with me in class as well as during my office hours (8 hours are set aside each week) about the course and their work outside of class.  If necessary, appointments also are available.  Furthermore, I suggest students use email and the phone to communicate with me.  Problems and difficulties in learning do not go away if left unattended, so please address any confusion or other difficulty you experience as soon as it arises.
3. Understanding & Critical Analysis.  Lectures, discussions, and writing exercises are intended to promote understanding, link of ideas together, and integrate wide-ranging knowledge.  The class structure and assignments are designed to assess knowledge, facilitate understanding, improve analysis, and promote critical thinking. Used in this way, the term critical thinking refers to using sound analytical processes (e.g., logic, theory, and evidence) to examine ideas.

---

COURSE OUTLINE

• How do social science and evaluation research combine?
• How do we define theory?
• How do we apply theory to specific programs and policies?
• How do we understand and analyze the logic that underlies a program?

 Date                               Topic                                                                                   Reading

Friday (2/22)                     Review course
                                        Discuss handout on journal article summary
                                        Review handout on evaluation plan
                                        Setting the Stage
                                        Evaluation and Social Science
                                        The Nature and Role of Theory                                              Berk & Rossi (1999)
                                        Evaluation Theory and Concepts  Rossi et al.(1999) Ch. 1

Saturday A.M.  (2/23)        Program Design and Process Evaluation                                Rossi, et al. Ch.2-4
                                        Constructing a Logic Model                                                   Classroom  Exercise

Saturday P.M (2/23)          Program Theory                                                                   Rossi, et al., Ch. 5
                                        Program Theory and Evaluation                                             Patton (1997), Ch. 1-5
                                        Program theory exercises given in class.

Assignments Due between Sections I & II:

• Written answers to Section I exercises due by 5:00 P.M. on March 4th via fax, email attachment, or hard copy in my office.
• Full citation (see handout for details on how to cite) for journal article due by 5:00 P.M. on March 8th via fax, email, or hard copy in my office.

• How does social and political context affect evaluation?
• How do we combine our knowledge of the program’s theory and structure to analyze implementation?
• What is the basic structure of a process evaluation, including measures?
• How do we link the process evaluation with the impact evaluation?

 Date                              Topic                                                                               Reading

Friday (3/15)                    Return and Review Exercises
                                       Context & Evaluation                                                          Rossi, et al., Ch. 12
                                                                                                                               Patton, Ch. 14
                                       Focusing an Evaluation                                                       Patton, Ch. 6-8
                                      Copy of program description for Evaluation Plan due in class.

Saturday A.M. (3/16)        Establishing a Process Evaluation                                       Patton, Ch. 9 &10
                                       Review Evaluation Plan assignment
                                       Review Article Review assignment

Saturday P.M. (3/16)        Implementing a Process Evaluation                                    Rossi, et al., Ch. 6
                                       Bridge between Process and Impact
                                       Process evaluation exercises given out in class.

Assignments Due between Sections II & III:

• Written answers to Section II exercises due by 5:00 P.M. on March 22nd via fax, email attachment, or hard copy in my office.
• Article reviews due by 5:00 P.M. on March 29th via fax, email attachment, or hard copy in my office.
• Outline of Evaluation Plan due by 5:00 P.M. on April 5th via fax, email attachment, or hard copy in my office.

SECTION III (4/19 & 4/20).  Evaluating Impact

• How can we be sure that the evaluation results are attributable to the program?
• How differentiate between short- and long-term results?
• How do we develop sound impact measures?

 Date                              Topic                                                                                 Reading

Friday (4/19)                    Return & Review Exercises
                                       Regression Model in Evaluation  Lecture
                                       Designing an Impact Assessment
                                       Design and Quantitative Analysis                                           Rossi & Freeman, Ch. 7-10

Saturday A.M. (4/20)        Design and Qualitative Analysis                                             Patton, Ch 11-13; 15
                                        Impact evaluation exercises given out in class.

Saturday P.M) (4/20)        STUDENT PRESENTATIONS

Due dates between Section III and end of semester:

• Written answers to Section III exercises due by 5:00 P.M. on April 29th via fax, email attachment, or hardcopy in my office.
• Evaluation Plan due by 5:00 P.M. on May 15th via fax, email attachment, or hard copy in my office.

  ---

  CITY COLLEGE HOME PAGE(http://www.loyno.edu/citycollege/)                                  LOYOLA UNIVERSITY HOME PAGE (http://www.loyno.edu)